<
Single Day: Oct 24▾
Pick a day
>
View JSON API
↗ Open Failure patterns for this day

Environment: INT

Time (UTC)RuntimeJobRegionFailed atResultPRFailed testsDetails
1h 8m 28s branch-ci-Azure-ARO-HCP-main-e2e-integration-e2e-parallel-all
post-goodmerged PR
No additional run metadata captured
westus3 unknown failed n/a 4
unknown (4)
ERROR CODE: DeadlineExceeded; detail message cluster creation did not complete before the deadline; [clusterServiceClusterStatus] cluster service is installing; [hypershiftHostedCluster] hosted cluster is not available: KubeconfigWaitingForCreate: Waiting for hosted control plane kubeconfig to be created; hosted cluster degraded: UnavailableReplicas: [capi-provider]; provider Microsoft.RedHatOpenShift
22:15:18 UTC · failed at unknown · test Service Provider should upgrade the control plane z-stream automatically on behalf of the customer for 4.22 · suite integration/parallel/all
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:125]: failed to create HCP cluster "cluster-zstream-4-22-njchjb" with version 4.22.18 on candidate channel
Unexpected error:
    <*fmt.wrapError | 0x2294f905a340>: 
    failed to create HCP cluster cluster-zstream-4-22-njchjb: failed waiting for cluster="cluster-zstream-4-22-njchjb" in resourcegroup="rg-zstream-upgrade-4-22-5bksdkff6tvq" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westus3/hcpOperationStatuses/b7f42e30-e443-47e9-982e-3fe8abc06d9a
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: DeadlineExceeded
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westus3/hcpOperationStatuses/b7f42e30-e443-47e9-982e-3fe8abc06d9a",
      "name": "b7f42e30-e443-47e9-982e-3fe8abc06d9a",
      "status": "Failed",
      "startTime": "2026-10-24T22:19:20.094796575Z",
      "endTime": "2026-10-24T22:48:29.265123983Z",
      "error": {
        "code": "DeadlineExceeded",
        "message": "cluster creation did not complete before the deadline; [clusterServiceClusterStatus] cluster service is installing; [hypershiftHostedCluster] hosted cluster is not available: KubeconfigWaitingForCreate: Waiting for hosted control plane kubeconfig to be created; hosted cluster degraded: UnavailableReplicas: capi-provider deployment has 1 unavailable replicas"
      }
    }
    --------------------------------------------------------------------------------
    
    <Azure error internals omitted>
        Azure request IDs:
          GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westus3/hcpOperationStatuses/b7f42e30-e443-47e9-982e-3fe8abc06d9a
            correlation ID: c6c76fd1-9a00-45d4-a377-4b7ed8ffb192
            client request ID: unavailable
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:125]: failed to create HCP cluster "cluster-zstream-4-22-njchjb" with version 4.22.18 on candidate channel
Unexpected error:
    <*fmt.wrapError | 0x2294f905a340>: 
    failed to create HCP cluster cluster-zstream-4-22-njchjb: failed waiting for cluster="cluster-zstream-4-22-njchjb" in resourcegroup="rg-zstream-upgrade-4-22-5bksdkff6tvq" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westus3/hcpOperationStatuses/b7f42e30-e443-47e9-982e-3fe8abc06d9a
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: DeadlineExceeded
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westus3/hcpOperationStatuses/b7f42e30-e443-47e9-982e-3fe8abc06d9a",
      "name": "b7f42e30-e443-47e9-982e-3fe8abc06d9a",
      "status": "Failed",
      "startTime": "2026-10-24T22:19:20.094796575Z",
      "endTime": "2026-10-24T22:48:29.265123983Z",
      "error": {
        "code": "DeadlineExceeded",
        "message": "cluster creation did not complete before the deadline; [clusterServiceClusterStatus] cluster service is installing; [hypershiftHostedCluster] hosted cluster is not available: KubeconfigWaitingForCreate: Waiting for hosted control plane kubeconfig to be created; hosted cluster degraded: UnavailableReplicas: capi-provider deployment has 1 unavailable replicas"
      }
    }
    --------------------------------------------------------------------------------
    
    <Azure error internals omitted>
        Azure request IDs:
          GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westus3/hcpOperationStatuses/b7f42e30-e443-47e9-982e-3fe8abc06d9a
            correlation ID: c6c76fd1-9a00-45d4-a377-4b7ed8ffb192
            client request ID: unavailable
occurred
tls: failed to verify certificate: x509: certificate signed by unknown authority
22:15:18 UTC · failed at unknown · test ARO-HCP should be able to perform a control plane and node pool install with OCP candidate channel for 4.23 · suite integration/parallel/all
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:194]: failed to verify simple web app runs on cluster "cluster-candidate-4-23-l96rfk"
Unexpected error:
    <*fmt.wrapError | 0x11cfecaf8760>: 
    [strict TLS verification] route was never reachable: Get "https://agnhost-e2e-sample-app-4vg4t.apps.aro.o6e4m4u5u4t4c4u.sbp8.westus3.aroapp-hcp.azure-test.net": tls: failed to verify certificate: x509: certificate signed by unknown authority
    {
        msg: "[strict TLS verification] route was never reachable: Get \"https://agnhost-e2e-sample-app-4vg4t.apps.aro.o6e4m4u5u4t4c4u.sbp8.westus3.aroapp-hcp.azure-test.net\": tls: failed to verify certificate: x509: certificate signed by unknown authority",
        err: <*url.Error | 0x11cfee5e70e0>{
            Op: "Get",
            URL: "https://agnhost-e2e-sample-app-4vg4t.apps.aro.o6e4m4u5u4t4c4u.sbp8.westus3.aroapp-hcp.azure-test.net",
            Err: <*tls.CertificateVerificationError | 0x11cfee5e70b0>{
                UnverifiedCertificates: [
                    {
                        Raw: [48, 130, 4, 1, 48, 130, 2, 233, 160, 3, 2, 1, 2, 2, 8, 122, 194, 216, 171, 111, 107, 55, 112, 48, 13, 6, 9, 42, 134, 72, 134, 247, 13, 1, 1, 11, 5, 0, 48, 38, 49, 18, 48, 16, 6, 3, 85, 4, 11, 19, 9, 111, 112, 101, 110, 115, 104, 105, 102, 116, 49, 16, 48, 14, 6, 3, 85, 4, 3, 19, 7, 114, 111, 111, 116, 45, 99, 97, 48, 30, 23, 13, 50, 54, 49, 48, 50, 52, 50, 50, 50, 56, 48, 49, 90, 23, 13, 50, 55, 49, 48, 50, 52, 50, 50, 50, 56, 48, 49, 90, 48, 48, 49, 18, 48, 16, 6, 3, 85, 4, 10, 19, 9, 111, 112, 101, 110, 115, 104, 105, 102, 116, 49, 26, 48, 24, 6, 3, 85, 4, 3, 19, 17, 111, 112, 101, 110, 115, 104, 105, 102, 116, 45, 105, 110, 103, 114, 101, 115, 115, 48, 130, 1, 34, 48, 13, 6, 9, 42, 134, 72, 134, 247, 13, 1, 1, 1, 5, 0, 3, 130, 1, 15, 0, 48, 130, 1, 10, 2, 130, 1, 1, 0, 178, 55, 149, 119, 219, 63, 47, 5, 249, 2, 142, 64, 226, 181, 25, 19, 163, 170, 217, 26, 124, 161, 29, 250, 162, 212, 197, 31, 10, 179, 100, 47, 54, 215, 80, 7, 78, 121, 50, 101, 254, 19, 69, 126, 231, 240, 34, 4, 205, 235, 198, 28, 58, 139, 209, 113, 230, 105, 161, 163, 221, 192, 72, 27, 82, 64, 156, 231, 230, 222, 216, 170, 66, 81, 109, 14, 218, 138, 180, 42, 46, 37, 12, 20, 42, 46, 254, 75, 14, 239, 200, 146, 1, 44, 249, 109, 167, 6, 174, 142, 209, 55, 87, 205, 253, 244, 165, 78, 181, 174, 211, 88, 111, 195, 22, 44, 169, 29, 62, 197, 215, 209, 73, 64, 144, 182, 37, 230, 141, 3, 209, 58, 41, 137, 165, 79, 15, 206, 195, 232, 245, 252, 140, 232, 188, 179, 203, 150, 101, 191, 186, 47, 212, 214, 37, 240, 83, 70, 206, 45, 54, 33, 211, 136, 55, 189, 43, 255, 208, 60, 224, 146, 245, 105, 10, 7, 134, 154, 110, 155, 81, 215, 26, 170, 120, 141, 123, 32, 172, 156, 94, 207, 62, 225, 195, 61, 155, 50, 127, 235, 92, 32, 207, 220, 201, 84, 27, 1, 117, 152, 204, 160, 80, 33, 0, 199, 209, 234, 168, 90, 184, 171, 178, 224, 8, 13, 41, 49, 94, 136, 52, 225, 159, 183, 109, 30, 220, 54, 37, 247, 46, 253, 174, 26, 197, 35, 142, 122, 171, 24, 218, 246, 148, 31, 58, 235, 2, 3, 1, 0, 1, 163, 130, 1, 39, 48, 130, 1, 35, 48, 14, 6, 3, 85, 29, 15, 1, 1, 255, 4, 4, 3, 2, 5, 160, 48, 29, 6, 3, 85, 29, 37, 4, 22, 48, 20, 6, 8, 43, 6, 1, 5, 5, 7, 3, 2, 6, 8, 43, 6, 1, 5, 5, 7, 3, 1, 48, 12, 6, 3, 85, 29, 19, 1, 1, 255, 4, 2, 48, 0, 48, 73, 6, 3, 85, 29, 14, 4, 66, 4, 64, 197, 98, 86, 250, 137, 217, 107, 201, 120, 11, 250, 28, 8, 159, 50, 58, 147, 2, 127, 27, 39, 46, 176, 228, 89, 113, 119, 156, 18, 254, 186, 225, 239, 236, 201, 198, 127, 141, 38, 73, 33, 245, 173, 2, 193, 175, 61, 245, 132, 223, 97, 52, 79, 55, 120, 201, 63, 76, 209, 109, 45, 189, 34, 122, 48, 75, 6, 3, 85, 29, 35, 4, 68, 48, 66, 128, 64, 248, 50, 11, 119, 242, 227, 194, 167, 149, 215, 190, 179, 176, 143, 74, 56, 221, 252, 111, 212, 224, 237, 204, 6, 0, 197, 97, 180, 119, 93, 170, 15, 222, 152, 153, 8, 168, 38, 140, 186, 14, 89, 130, 247, 23, 168, 143, 191, 156, 56, 163, 177, 216, 127, 102, 115, 15, 90, 73, 84, 30, 110, 35, 172, 48, 76, 6, 3, 85, 29, 17, 4, 69, 48, 67, 130, 65, 42, 46, 97, 112, 112, 115, 46, 97, 114, 111, 46, 111, 54, 101, 52, 109, 52, 117, 53, 117, 52, 116, 52, 99, 52, 117, 46, 115, 98, 112, 56, 46, 119, 101, 115, 116, 117, 115, 51, 46, 97, 114, 111, 97, 112, 112, 45, 104, 99, 112, 46, 97, 122, 117, 114, 101, 45, 116, 101, 115, 116, 46, 110, 101, 116, 48, 13, 6, 9, 42, 134, 72, 134, 247, 13, 1, 1, 11, 5, 0, 3, 130, 1, 1, 0, 50, 25, 231, 199, 177, 69, 58, 16, 51, 91, 243, 219, 210, 199, 138, 155, 53, 245, 245, 106, 127, 215, 113, 116, 53, 212, 199, 147, 106, 103, 247, 189, 33, 171, 252, 21, 143, 150, 56, 31, 85, 0, 41, 147, 65, 24, 201, 140, 203, 150, 3, 31, 110, 137, 35, 75, 171, 175, 235, 209, 21, 26, 164, 179, 71, 67, 21, 133, 105, 14, 253, 71, 51, 45, 32, 43, 100, 94, 84, 47, 64, 244, 56, 58, 136, 157, 46, 87, 211, 40, 31, 180, 2, 165, 196, 210, 222, 204, 152, 251, 130, 232, 9, 182, 120, 239, 235, 22, 89, 58, 198, 78, 82, 139, 120, 162, 101, 8, 223, 107, 59, 223, 193, 151, 252, 30, 79, 0, 15, 40, 112, 104, 59, 157, 68, 154, 197, 178, 218, 209, 164, 56, 94, 93, 22, 22, 98, 191, 126, 106, 4, 194, 233, 190, 68, 193, 123, 232, 86, 187, 65, 26, 135, 65, 91, 129, 105, 54, 92, 78, 247, 245, 172, 13, 38, 206, 186, 5, 136, 153, 211, 200, 140, 153, 205, 176, 93, 247, 111, 230, 227, 13, 56, 73, 251, 143, 236, 121, 113, 183, 124, 61, 169, 206, 40, 146, 37, 166, 252, 132, 4, 153, 168, 84, 39, 23, 232, 218, 136, 80, 210, 143, 97, 106, 58, 193, 56, 253, 19, 202, 243, 199, 49, 235, 83, 136, 105, 217, 57, 123, 238, 192, 105, 226, 211, 224, 255, 115, 130, 221, 177, 40, 19, 51, 31, 249],
                        RawTBSCertificate: [48, 130, 2, 233, 160, 3, 2, 1, 2, 2, 8, 122, 194, 216, 171, 111, 107, 55, 112, 48, 13, 6, 9, 42, 134, 72, 134, 247, 13, 1, 1, 11, 5, 0, 48, 38, 49, 18, 48, 16, 6, 3, 85, 4, 11, 19, 9, 111, 112, 101, 110, 115, 104, 105, 102, 116, 49, 16, 48, 14, 6, 3, 85, 4, 3, 19, 7, 114, 111, 111, 116, 45, 99, 97, 48, 30, 23, 13, 50, 54, 49, 48, 50, 52, 50, 50, 50, 56, 48, 49, 90, 23, 13, 50, 55, 49, 48, 50, 52, 50, 50, 50, 56, 48, 49, 90, 48, 48, 49, 18, 48, 16, 6, 3, 85, 4, 10, 19, 9, 111, 112, 101, 110, 115, 104, 105, 102, 116, 49, 26, 48, 24, 6, 3, 85, 4, 3, 19, 17, 111, 112, 101, 110, 115, 104, 105, 102, 116, 45, 105, 110, 103, 114, 101, 115, 115, 48, 130, 1, 34, 48, 13, 6, 9, 42, 134, 72, 134, 247, 13, 1, 1, 1, 5, 0, 3, 130, 1, 15, 0, 48, 130, 1, 10, 2, 130, 1, 1, 0, 178, 55, 149, 119, 219, 63, 47, 5, 249, 2, 142, 64, 226, 181, 25, 19, 163, 170, 217, 26, 124, 161, 29, 250, 162, 212, 197, 31, 10, 179, 100, 47, 54, 215, 80, 7, 78, 121, 50, 101, 254, 19, 69, 126, 231, 240, 34, 4, 205, 235, 198, 28, 58, 139, 209, 113, 230, 105, 161, 163, 221, 192, 72, 27, 82, 64, 156, 231, 230, 222, 216, 170, 66, 81, 109, 14, 218, 138, 180, 42, 46, 37, 12, 20, 42, 46, 254, 75, 14, 239, 200, 146, 1, 44, 249, 109, 167, 6, 174, 142, 209, 55, 87, 205, 253, 244, 165, 78, 181, 174, 211, 88, 111, 195, 22, 44, 169, 29, 62, 197, 215, 209, 73, 64, 144, 182, 37, 230, 141, 3, 209, 58, 41, 137, 165, 79, 15, 206, 195, 232, 245, 252, 140, 232, 188, 179, 203, 150, 101, 191, 186, 47, 212, 214, 37, 240, 83, 70, 206, 45, 54, 33, 211, 136, 55, 189, 43, 255, 208, 60, 224, 146, 245, 105, 10, 7, 134, 154, 110, 155, 81, 215, 26, 170, 120, 141, 123, 32, 172, 156, 94, 207, 62, 225, 195, 61, 155, 50, 127, 235, 92, 32, 207, 220, 201, 84, 27, 1, 117, 152, 204, 160, 80, 33, 0, 199, 209, 234, 168, 90, 184, 171, 178, 224, 8, 13, 41, 49, 94, 136, 52, 225, 159, 183, 109, 30, 220, 54, 37, 247, 46, 253, 174, 26, 197, 35, 142, 122, 171, 24, 218, 246, 148, 31, 58, 235, 2, 3, 1, 0, 1, 163, 130, 1, 39, 48, 130, 1, 35, 48, 14, 6, 3, 85, 29, 15, 1, 1, 255, 4, 4, 3, 2, 5, 160, 48, 29, 6, 3, 85, 29, 37, 4, 22, 48, 20, 6, 8, 43, 6, 1, 5, 5, 7, 3, 2, 6, 8, 43, 6, 1, 5, 5, 7, 3, 1, 48, 12, 6, 3, 85, 29, 19, 1, 1, 255, 4, 2, 48, 0, 48, 73, 6, 3, 85, 29, 14, 4, 66, 4, 64, 197, 98, 86, 250, 137, 217, 107, 201, 120, 11, 250, 28, 8, 159, 50, 58, 147, 2, 127, 27, 39, 46, 176, 228, 89, 113, 119, 156, 18, 254, 186, 225, 239, 236, 201, 198, 127, 141, 38, 73, 33, 245, 173, 2, 193, 175, 61, 245, 132, 223, 97, 52, 79, 55, 120, 201, 63, 76, 209, 109, 45, 189, 34, 122, 48, 75, 6, 3, 85, 29, 35, 4, 68, 48, 66, 128, 64, 248, 50, 11, 119, 242, 227, 194, 167, 149, 215, 190, 179, 176, 143, 74, 56, 221, 252, 111, 212, 224, 237, 204, 6, 0, 197, 97, 180, 119, 93, 170, 15, 222, 152, 153, 8, 168, 38, 140, 186, 14, 89, 130, 247, 23, 168, 143, 191, 156, 56, 163, 177, 216, 127, 102, 115, 15, 90, 73, 84, 30, 110, 35, 172, 48, 76, 6, 3, 85, 29, 17, 4, 69, 48, 67, 130, 65, 42, 46, 97, 112, 112, 115, 46, 97, 114, 111, 46, 111, 54, 101, 52, 109, 52, 117, 53, 117, 52, 116, 52, 99, 52, 117, 46, 115, 98, 112, 56, 46, 119, 101, 115, 116, 117, 115, 51, 46, 97, 114, 111, 97, 112, 112, 45, 104, 99, 112, 46, 97, 122, 117, 114, 101, 45, 116, 101, 115, 116, 46, 110, 101, 116],
                        RawSubjectPublicKeyInfo: [48, 130, 1, 34, 48, 13, 6, 9, 42, 134, 72, 134, 247, 13, 1, 1, 1, 5, 0, 3, 130, 1, 15, 0, 48, 130, 1, 10, 2, 130, 1, 1, 0, 178, 55, 149, 119, 219, 63, 47, 5, 249, 2, 142, 64, 226, 181, 25, 19, 163, 170, 217, 26, 124, 161, 29, 250, 162, 212, 197, 31, 10, 179, 100, 47, 54, 215, 80, 7, 78, 121, 50, 101, 254, 19, 69, 126, 231, 240, 34, 4, 205, 235, 198, 28, 58, 139, 209, 113, 230, 105, 161, 163, 221, 192, 72, 27, 82, 64, 156, 231, 230, 222, 216, 170, 66, 81, 109, 14, 218, 138, 180, 42, 46, 37, 12, 20, 42, 46, 254, 75, 14, 239, 200, 146, 1, 44, 249, 109, 167, 6, 174, 142, 209, 55, 87, 205, 253, 244, 165, 78, 181, 174, 211, 88, 111, 195, 22, 44, 169, 29, 62, 197, 215, 209, 73, 64, 144, 182, 37, 230, 141, 3, 209, 58, 41, 137, 165, 79, 15, 206, 195, 232, 245, 252, 140, 232, 188, 179, 203, 150, 101, 191, 186, 47, 212, 214, 37, 240, 83, 70, 206, 45, 54, 33, 211, 136, 55, 189, 43, 255, 208, 60, 224, 146, 245, 105, 10, 7, 134, 154, 110, 155, 81, 215, 26, 170, 120, 141, 123, 32, 172, 156, 94, 207, 62, 225, 195, 61, 155, 50, 127, 235, 92, 32, 207, 220, 201, 84, 27, 1, 117, 152, 204, 160, 80, 33, 0, 199, 209, 234, 168, 90, 184, 171, 178, 224, 8, 13, 41, 49, 94, 136, 52, 225, 159, 183, 109, 30, 220, 54, 37, 247, 46, 253, 174, 26, 197, 35, 142, 122, 171, 24, 218, 246, 148, 31, 58, 235, 2, 3, 1, 0, 1],
                        RawSubject: [48, 48, 49, 18, 48, 16, 6, 3, 85, 4, 10, 19, 9, 111, 112, 101, 110, 115, 104, 105, 102, 116, 49, 26, 48, 24, 6, 3, 85, 4, 3, 19, 17, 111, 112, 101, 110, 115, 104, 105, 102, 116, 45, 105, 110, 103, 114, 101, 115, 115],
                        RawIssuer: [48, 38, 49, 18, 48, 16, 6, 3, 85, 4, 11, 19, 9, 111, 112, 101, 110, 115, 104, 105, 102, 116, 49, 16, 48, 14, 6, 3, 85, 4, 3, 19, 7, 114, 111, 111, 116, 45, 99, 97],
                        Signature: [50, 25, 231, 199, 177, 69, 58, 16, 51, 91, 243, 219, 210, 199, 138, 155, 53, 245, 245, 106, 127, 215, 113, 116, 53, 212, 199, 147, 106, 103, 247, 189, 33, 171, 252, 21, 143, 150, 56, 31, 85, 0, 41, 147, 65, 24, 201, 140, 203, 150, 3, 31, 110, 137, 35, 75, 171, 175, 235, 209, 21, 26, 164, 179, 71, 67, 21, 133, 105, 14, 253, 71, 51, 45, 32, 43, 100, 94, 84, 47, 64, 244, 56, 58, 136, 157, 46, 87, 211, 40, 31, 180, 2, 165, 196, 210, 222, 204, 152, 251, 130, 232, 9, 182, 120, 239, 235, 22, 89, 58, 198, 78, 82, 139, 120, 162, 101, 8, 223, 107, 59, 223, 193, 151, 252, 30, 79, 0, 15, 40, 112, 104, 59, 157, 68, 154, 197, 178, 218, 209, 164, 56, 94, 93, 22, 22, 98, 191, 126, 106, 4, 194, 233, 190, 68, 193, 123, 232, 86, 187, 65, 26, 135, 65, 91, 129, 105, 54, 92, 78, 247, 245, 172, 13, 38, 206, 186, 5, 136, 153, 211, 200, 140, 153, 205, 176, 93, 247, 111, 230, 227, 13, 56, 73, 251, 143, 236, 121, 113, 183, 124, 61, 169, 206, 40, 146, 37, 166, 252, 132, 4, 153, 168, 84, 39, 23, 232, 218, 136, 80, 210, 143, 97, 106, 58, 193, 56, 253, 19, 202, 243, 199, 49, 235, 83, 136, 105, 217, 57, 123, 238, 192, 105, 226, 211, 224, 255, 115, 130, 221, 177, 40, 19, 51, 31, 249],
                        SignatureAlgorithm: 4,
                        PublicKeyAlgorithm: 1,
                        PublicKey: <*rsa.PublicKey | 0x11cfee823170>{
                            N: {
                                neg: false,
                                abs: [12328844732508617451, 3386053924852305530, 11508787352079574519, 580165219016783073, 15126087406076015328, 8473747687808762055, 6638534397921598209, 4531117369475629035, 8686734636532653775, 9699186360867822250, 15005114980278798855, 3900631334283717631, 15336487295176330797, 13597435546774256175, 1139063161098112232, 10161195232013165903, 15551291529941624294, 8053304939740217029, 18299432943387333464, 12035498982612883405, 1076299365375342957, 3325077180629581387, 4778720590180299818, 5926909629444315306, 16602979226450151451, 14838171221845856625, 18308053321550668292, 3951715190352458341, 11733219666321171503, 11793477282006375930, 17943060274730047763, 12841897204482256645],
                            },
                            E: 65537,
                        },
                        Version: 3,
                        SerialNumber: {
                            neg: false,
                            abs: [8845870848929380208],
                        },
                        Issuer: {
                            Country: nil,
                            Organization: nil,
                            OrganizationalUnit: ["openshift"],
                            Locality: nil,
                            Province: nil,
                            StreetAddress: nil,
                            PostalCode: nil,
                            SerialNumber: "",
                            CommonName: "root-ca",
                            Names: [
                                {
                                    Type: [2, 5, 4, 11],
                                    Value: <string>"openshift",
                                },
                                {
                                    Type: [2, 5, 4, 3],
                                    Value: <string>"root-ca",
                                },
                            ],
                            ExtraNames: nil,
                        },
                        Subject: {
                            Country: nil,
                            Organization: ["openshift"],
                            OrganizationalUnit: nil,
                            Locality: nil,
                            Province: nil,
                            StreetAddress: nil,
                            PostalCode: nil,
                            SerialNumber: "",
                            CommonName: "openshift-ingress",
                            Names: [
                                {
                                    Type: [2, 5, 4, 10],
                                    Value: <string>"openshift",
                                },
                                {
                                    Type: [2, 5, 4, 3],
                                    Value: <string>"openshift-ingress",
                                },
                            ],
                            ExtraNames: nil,
                        },
                        NotBefore: {wall: 0, ext: 63928477681, loc: nil},
                        NotAfter: {wall: 0, ext: 63960013681, loc: nil},
                        KeyUsage: 5,
                        Extensions: [
                            {
                                Id: [2, 5, 29, 15],
                                Critical: true,
                                Value: [3, 2, 5, 160],
                            },
                            {
                                Id: [2, 5, 29, 37],
                                Critical: false,
                                Value: [48, 20, 6, 8, 43, 6, 1, 5, 5, 7, 3, 2, 6, 8, 43, 6, 1, 5, 5, 7, 3, 1],
                            },
                            {Id: [2, 5, 29, 19], Critical: true, Value: [48, 0]},
                            {
                                Id: [2, 5, 29, 14],
                                Critical: false,
                                Value: [4, 64, 197, 98, 86, 250, 137, 217, 107, 201, 120, 11, 250, 28, 8, 159, 50, 58, 147, 2, 127, 27, 39, 46, 176, 228, 89, 113, 119, 156, 18, 254, 186, 225, 239, 236, 201, 198, 127, 141, 38, 73, 33, 245, 173, 2, 193, 175, 61, 245, 132, 223, 97, 52, 79, 55, 120, 201, 63, 76, 209, 109, 45, 189, 34, 122],
                            },
                            {
                                Id: [2, 5, 29, 35],
                                Critical: false,
                                Value: [48, 66, 128, 64, 248, 50, 11, 119, 242, 227, 194, 167, 149, 215, 190, 179, 176, 143, 74, 56, 221, 252, 111, 212, 224, 237, 204, 6, 0, 197, 97, 180, 119, 93, 170, 15, 222, 152, 153, 8, 168, 38, 140, 186, 14, 89, 130, 247, 23, 168, 143, 191, 156, 56, 163, 177, 216, 127, 102, 115, 15, 90, 73, 84, 30, 110, 35, 172],
                            },
                            {
                                Id: [2, 5, 29, 17],
                                Critical: false,
                                Value: "0C\x82A*.apps.aro.o6e4m4u5u4t4c4u.sbp8.westus3.aroapp-hcp.azure-test.net",
                            },
                        ],
                        ExtraExtensions: nil,
                        UnhandledCriticalExtensions: nil,
                        ExtKeyUsage: [2, 1],
                        UnknownExtKeyUsage: nil,
                        BasicConstraintsValid: true,
                        IsCA: false,
                        MaxPathLen: -1,
                        MaxPathLenZero: false,
                        SubjectKeyId: [197, 98, 86, 250, 137, 217, 107, 201, 120, 11, 250, 28, 8, 159, 50, 58, 147, 2, 127, 27, 39, 46, 176, 228, 89, 113, 119, 156, 18, 254, 186, 225, 239, 236, 201, 198, 127, 141, 38, 73, 33, 245, 173, 2, 193, 175, 61, 245, 132, 223, 97, 52, 79, 55, 120, 201, 63, 76, 209, 109, 45, 189, 34, 122],
                        AuthorityKeyId: [248, 50, 11, 119, 242, 227, 194, 167, 149, 215, 190, 179, 176, 143, 74, 56, 221, 252, 111, 212, 224, 237, 204, 6, 0, 197, 97, 180, 119, 93, 170, 15, 222, 152, 153, 8, 168, 38, 140, 186, 14, 89, 130, 247, 23, 168, 143, 191, 156, 56, 163, 177, 216, 127, 102, 115, 15, 90, 73, 84, 30, 110, 35, 172],
                        OCSPServer: nil,
                        IssuingCertificateURL: nil,
                        DNSNames: [
                            "*.apps.aro.o6e4m4u5u4t4c4u.sbp8.westus3.aroapp-hcp.azure-test.net",
                        ],
                        EmailAddresses: nil,
                        IPAddresses: nil,
                        URIs: nil,
                        PermittedDNSDomainsCritical: false,
                        PermittedDNSDomains: nil,
                        ExcludedDNSDomains: nil,
                        PermittedIPRanges: nil,
                        ExcludedIPRanges: nil,
                        PermittedEmailAddresses: nil,
                        ExcludedEmailAddresses: nil,
                        PermittedURIDomains: nil,
                        ExcludedURIDomains: nil,
                        CRLDistributionPoints: nil,
                        PolicyIdentifiers: nil,
                        Policies: nil,
                        InhibitAnyPolicy: 0,
                        InhibitAnyPolicyZero: false,
                        InhibitPolicyMapping: 0,
                        InhibitPolicyMappingZero: false,
                        RequireExplicitPolicy: 0,
                        RequireExplicitPolicyZero: false,
                        PolicyMappings: nil,
                    },
                ],
                Err: <x509.UnknownAuthorityError>{
                    Cert: 0x11cfed040008 (cyclic reference),
                    hintErr: nil,
                    hintCert: nil,
                },
            },
        },
    }
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:194]: failed to verify simple web app runs on cluster "cluster-candidate-4-23-l96rfk"
Unexpected error:
    <*fmt.wrapError | 0x11cfecaf8760>: 
    [strict TLS verification] route was never reachable: Get "https://agnhost-e2e-sample-app-4vg4t.apps.aro.o6e4m4u5u4t4c4u.sbp8.westus3.aroapp-hcp.azure-test.net": tls: failed to verify certificate: x509: certificate signed by unknown authority
    {
        msg: "[strict TLS verification] route was never reachable: Get \"https://agnhost-e2e-sample-app-4vg4t.apps.aro.o6e4m4u5u4t4c4u.sbp8.westus3.aroapp-hcp.azure-test.net\": tls: failed to verify certificate: x509: certificate signed by unknown authority",
        err: <*url.Error | 0x11cfee5e70e0>{
            Op: "Get",
            URL: "https://agnhost-e2e-sample-app-4vg4t.apps.aro.o6e4m4u5u4t4c4u.sbp8.westus3.aroapp-hcp.azure-test.net",
            Err: <*tls.CertificateVerificationError | 0x11cfee5e70b0>{
                UnverifiedCertificates: [
                    {
                        Raw: [48, 130, 4, 1, 48, 130, 2, 233, 160, 3, 2, 1, 2, 2, 8, 122, 194, 216, 171, 111, 107, 55, 112, 48, 13, 6, 9, 42, 134, 72, 134, 247, 13, 1, 1, 11, 5, 0, 48, 38, 49, 18, 48, 16, 6, 3, 85, 4, 11, 19, 9, 111, 112, 101, 110, 115, 104, 105, 102, 116, 49, 16, 48, 14, 6, 3, 85, 4, 3, 19, 7, 114, 111, 111, 116, 45, 99, 97, 48, 30, 23, 13, 50, 54, 49, 48, 50, 52, 50, 50, 50, 56, 48, 49, 90, 23, 13, 50, 55, 49, 48, 50, 52, 50, 50, 50, 56, 48, 49, 90, 48, 48, 49, 18, 48, 16, 6, 3, 85, 4, 10, 19, 9, 111, 112, 101, 110, 115, 104, 105, 102, 116, 49, 26, 48, 24, 6, 3, 85, 4, 3, 19, 17, 111, 112, 101, 110, 115, 104, 105, 102, 116, 45, 105, 110, 103, 114, 101, 115, 115, 48, 130, 1, 34, 48, 13, 6, 9, 42, 134, 72, 134, 247, 13, 1, 1, 1, 5, 0, 3, 130, 1, 15, 0, 48, 130, 1, 10, 2, 130, 1, 1, 0, 178, 55, 149, 119, 219, 63, 47, 5, 249, 2, 142, 64, 226, 181, 25, 19, 163, 170, 217, 26, 124, 161, 29, 250, 162, 212, 197, 31, 10, 179, 100, 47, 54, 215, 80, 7, 78, 121, 50, 101, 254, 19, 69, 126, 231, 240, 34, 4, 205, 235, 198, 28, 58, 139, 209, 113, 230, 105, 161, 163, 221, 192, 72, 27, 82, 64, 156, 231, 230, 222, 216, 170, 66, 81, 109, 14, 218, 138, 180, 42, 46, 37, 12, 20, 42, 46, 254, 75, 14, 239, 200, 146, 1, 44, 249, 109, 167, 6, 174, 142, 209, 55, 87, 205, 253, 244, 165, 78, 181, 174, 211, 88, 111, 195, 22, 44, 169, 29, 62, 197, 215, 209, 73, 64, 144, 182, 37, 230, 141, 3, 209, 58, 41, 137, 165, 79, 15, 206, 195, 232, 245, 252, 140, 232, 188, 179, 203, 150, 101, 191, 186, 47, 212, 214, 37, 240, 83, 70, 206, 45, 54, 33, 211, 136, 55, 189, 43, 255, 208, 60, 224, 146, 245, 105, 10, 7, 134, 154, 110, 155, 81, 215, 26, 170, 120, 141, 123, 32, 172, 156, 94, 207, 62, 225, 195, 61, 155, 50, 127, 235, 92, 32, 207, 220, 201, 84, 27, 1, 117, 152, 204, 160, 80, 33, 0, 199, 209, 234, 168, 90, 184, 171, 178, 224, 8, 13, 41, 49, 94, 136, 52, 225, 159, 183, 109, 30, 220, 54, 37, 247, 46, 253, 174, 26, 197, 35, 142, 122, 171, 24, 218, 246, 148, 31, 58, 235, 2, 3, 1, 0, 1, 163, 130, 1, 39, 48, 130, 1, 35, 48, 14, 6, 3, 85, 29, 15, 1, 1, 255, 4, 4, 3, 2, 5, 160, 48, 29, 6, 3, 85, 29, 37, 4, 22, 48, 20, 6, 8, 43, 6, 1, 5, 5, 7, 3, 2, 6, 8, 43, 6, 1, 5, 5, 7, 3, 1, 48, 12, 6, 3, 85, 29, 19, 1, 1, 255, 4, 2, 48, 0, 48, 73, 6, 3, 85, 29, 14, 4, 66, 4, 64, 197, 98, 86, 250, 137, 217, 107, 201, 120, 11, 250, 28, 8, 159, 50, 58, 147, 2, 127, 27, 39, 46, 176, 228, 89, 113, 119, 156, 18, 254, 186, 225, 239, 236, 201, 198, 127, 141, 38, 73, 33, 245, 173, 2, 193, 175, 61, 245, 132, 223, 97, 52, 79, 55, 120, 201, 63, 76, 209, 109, 45, 189, 34, 122, 48, 75, 6, 3, 85, 29, 35, 4, 68, 48, 66, 128, 64, 248, 50, 11, 119, 242, 227, 194, 167, 149, 215, 190, 179, 176, 143, 74, 56, 221, 252, 111, 212, 224, 237, 204, 6, 0, 197, 97, 180, 119, 93, 170, 15, 222, 152, 153, 8, 168, 38, 140, 186, 14, 89, 130, 247, 23, 168, 143, 191, 156, 56, 163, 177, 216, 127, 102, 115, 15, 90, 73, 84, 30, 110, 35, 172, 48, 76, 6, 3, 85, 29, 17, 4, 69, 48, 67, 130, 65, 42, 46, 97, 112, 112, 115, 46, 97, 114, 111, 46, 111, 54, 101, 52, 109, 52, 117, 53, 117, 52, 116, 52, 99, 52, 117, 46, 115, 98, 112, 56, 46, 119, 101, 115, 116, 117, 115, 51, 46, 97, 114, 111, 97, 112, 112, 45, 104, 99, 112, 46, 97, 122, 117, 114, 101, 45, 116, 101, 115, 116, 46, 110, 101, 116, 48, 13, 6, 9, 42, 134, 72, 134, 247, 13, 1, 1, 11, 5, 0, 3, 130, 1, 1, 0, 50, 25, 231, 199, 177, 69, 58, 16, 51, 91, 243, 219, 210, 199, 138, 155, 53, 245, 245, 106, 127, 215, 113, 116, 53, 212, 199, 147, 106, 103, 247, 189, 33, 171, 252, 21, 143, 150, 56, 31, 85, 0, 41, 147, 65, 24, 201, 140, 203, 150, 3, 31, 110, 137, 35, 75, 171, 175, 235, 209, 21, 26, 164, 179, 71, 67, 21, 133, 105, 14, 253, 71, 51, 45, 32, 43, 100, 94, 84, 47, 64, 244, 56, 58, 136, 157, 46, 87, 211, 40, 31, 180, 2, 165, 196, 210, 222, 204, 152, 251, 130, 232, 9, 182, 120, 239, 235, 22, 89, 58, 198, 78, 82, 139, 120, 162, 101, 8, 223, 107, 59, 223, 193, 151, 252, 30, 79, 0, 15, 40, 112, 104, 59, 157, 68, 154, 197, 178, 218, 209, 164, 56, 94, 93, 22, 22, 98, 191, 126, 106, 4, 194, 233, 190, 68, 193, 123, 232, 86, 187, 65, 26, 135, 65, 91, 129, 105, 54, 92, 78, 247, 245, 172, 13, 38, 206, 186, 5, 136, 153, 211, 200, 140, 153, 205, 176, 93, 247, 111, 230, 227, 13, 56, 73, 251, 143, 236, 121, 113, 183, 124, 61, 169, 206, 40, 146, 37, 166, 252, 132, 4, 153, 168, 84, 39, 23, 232, 218, 136, 80, 210, 143, 97, 106, 58, 193, 56, 253, 19, 202, 243, 199, 49, 235, 83, 136, 105, 217, 57, 123, 238, 192, 105, 226, 211, 224, 255, 115, 130, 221, 177, 40, 19, 51, 31, 249],
                        RawTBSCertificate: [48, 130, 2, 233, 160, 3, 2, 1, 2, 2, 8, 122, 194, 216, 171, 111, 107, 55, 112, 48, 13, 6, 9, 42, 134, 72, 134, 247, 13, 1, 1, 11, 5, 0, 48, 38, 49, 18, 48, 16, 6, 3, 85, 4, 11, 19, 9, 111, 112, 101, 110, 115, 104, 105, 102, 116, 49, 16, 48, 14, 6, 3, 85, 4, 3, 19, 7, 114, 111, 111, 116, 45, 99, 97, 48, 30, 23, 13, 50, 54, 49, 48, 50, 52, 50, 50, 50, 56, 48, 49, 90, 23, 13, 50, 55, 49, 48, 50, 52, 50, 50, 50, 56, 48, 49, 90, 48, 48, 49, 18, 48, 16, 6, 3, 85, 4, 10, 19, 9, 111, 112, 101, 110, 115, 104, 105, 102, 116, 49, 26, 48, 24, 6, 3, 85, 4, 3, 19, 17, 111, 112, 101, 110, 115, 104, 105, 102, 116, 45, 105, 110, 103, 114, 101, 115, 115, 48, 130, 1, 34, 48, 13, 6, 9, 42, 134, 72, 134, 247, 13, 1, 1, 1, 5, 0, 3, 130, 1, 15, 0, 48, 130, 1, 10, 2, 130, 1, 1, 0, 178, 55, 149, 119, 219, 63, 47, 5, 249, 2, 142, 64, 226, 181, 25, 19, 163, 170, 217, 26, 124, 161, 29, 250, 162, 212, 197, 31, 10, 179, 100, 47, 54, 215, 80, 7, 78, 121, 50, 101, 254, 19, 69, 126, 231, 240, 34, 4, 205, 235, 198, 28, 58, 139, 209, 113, 230, 105, 161, 163, 221, 192, 72, 27, 82, 64, 156, 231, 230, 222, 216, 170, 66, 81, 109, 14, 218, 138, 180, 42, 46, 37, 12, 20, 42, 46, 254, 75, 14, 239, 200, 146, 1, 44, 249, 109, 167, 6, 174, 142, 209, 55, 87, 205, 253, 244, 165, 78, 181, 174, 211, 88, 111, 195, 22, 44, 169, 29, 62, 197, 215, 209, 73, 64, 144, 182, 37, 230, 141, 3, 209, 58, 41, 137, 165, 79, 15, 206, 195, 232, 245, 252, 140, 232, 188, 179, 203, 150, 101, 191, 186, 47, 212, 214, 37, 240, 83, 70, 206, 45, 54, 33, 211, 136, 55, 189, 43, 255, 208, 60, 224, 146, 245, 105, 10, 7, 134, 154, 110, 155, 81, 215, 26, 170, 120, 141, 123, 32, 172, 156, 94, 207, 62, 225, 195, 61, 155, 50, 127, 235, 92, 32, 207, 220, 201, 84, 27, 1, 117, 152, 204, 160, 80, 33, 0, 199, 209, 234, 168, 90, 184, 171, 178, 224, 8, 13, 41, 49, 94, 136, 52, 225, 159, 183, 109, 30, 220, 54, 37, 247, 46, 253, 174, 26, 197, 35, 142, 122, 171, 24, 218, 246, 148, 31, 58, 235, 2, 3, 1, 0, 1, 163, 130, 1, 39, 48, 130, 1, 35, 48, 14, 6, 3, 85, 29, 15, 1, 1, 255, 4, 4, 3, 2, 5, 160, 48, 29, 6, 3, 85, 29, 37, 4, 22, 48, 20, 6, 8, 43, 6, 1, 5, 5, 7, 3, 2, 6, 8, 43, 6, 1, 5, 5, 7, 3, 1, 48, 12, 6, 3, 85, 29, 19, 1, 1, 255, 4, 2, 48, 0, 48, 73, 6, 3, 85, 29, 14, 4, 66, 4, 64, 197, 98, 86, 250, 137, 217, 107, 201, 120, 11, 250, 28, 8, 159, 50, 58, 147, 2, 127, 27, 39, 46, 176, 228, 89, 113, 119, 156, 18, 254, 186, 225, 239, 236, 201, 198, 127, 141, 38, 73, 33, 245, 173, 2, 193, 175, 61, 245, 132, 223, 97, 52, 79, 55, 120, 201, 63, 76, 209, 109, 45, 189, 34, 122, 48, 75, 6, 3, 85, 29, 35, 4, 68, 48, 66, 128, 64, 248, 50, 11, 119, 242, 227, 194, 167, 149, 215, 190, 179, 176, 143, 74, 56, 221, 252, 111, 212, 224, 237, 204, 6, 0, 197, 97, 180, 119, 93, 170, 15, 222, 152, 153, 8, 168, 38, 140, 186, 14, 89, 130, 247, 23, 168, 143, 191, 156, 56, 163, 177, 216, 127, 102, 115, 15, 90, 73, 84, 30, 110, 35, 172, 48, 76, 6, 3, 85, 29, 17, 4, 69, 48, 67, 130, 65, 42, 46, 97, 112, 112, 115, 46, 97, 114, 111, 46, 111, 54, 101, 52, 109, 52, 117, 53, 117, 52, 116, 52, 99, 52, 117, 46, 115, 98, 112, 56, 46, 119, 101, 115, 116, 117, 115, 51, 46, 97, 114, 111, 97, 112, 112, 45, 104, 99, 112, 46, 97, 122, 117, 114, 101, 45, 116, 101, 115, 116, 46, 110, 101, 116],
                        RawSubjectPublicKeyInfo: [48, 130, 1, 34, 48, 13, 6, 9, 42, 134, 72, 134, 247, 13, 1, 1, 1, 5, 0, 3, 130, 1, 15, 0, 48, 130, 1, 10, 2, 130, 1, 1, 0, 178, 55, 149, 119, 219, 63, 47, 5, 249, 2, 142, 64, 226, 181, 25, 19, 163, 170, 217, 26, 124, 161, 29, 250, 162, 212, 197, 31, 10, 179, 100, 47, 54, 215, 80, 7, 78, 121, 50, 101, 254, 19, 69, 126, 231, 240, 34, 4, 205, 235, 198, 28, 58, 139, 209, 113, 230, 105, 161, 163, 221, 192, 72, 27, 82, 64, 156, 231, 230, 222, 216, 170, 66, 81, 109, 14, 218, 138, 180, 42, 46, 37, 12, 20, 42, 46, 254, 75, 14, 239, 200, 146, 1, 44, 249, 109, 167, 6, 174, 142, 209, 55, 87, 205, 253, 244, 165, 78, 181, 174, 211, 88, 111, 195, 22, 44, 169, 29, 62, 197, 215, 209, 73, 64, 144, 182, 37, 230, 141, 3, 209, 58, 41, 137, 165, 79, 15, 206, 195, 232, 245, 252, 140, 232, 188, 179, 203, 150, 101, 191, 186, 47, 212, 214, 37, 240, 83, 70, 206, 45, 54, 33, 211, 136, 55, 189, 43, 255, 208, 60, 224, 146, 245, 105, 10, 7, 134, 154, 110, 155, 81, 215, 26, 170, 120, 141, 123, 32, 172, 156, 94, 207, 62, 225, 195, 61, 155, 50, 127, 235, 92, 32, 207, 220, 201, 84, 27, 1, 117, 152, 204, 160, 80, 33, 0, 199, 209, 234, 168, 90, 184, 171, 178, 224, 8, 13, 41, 49, 94, 136, 52, 225, 159, 183, 109, 30, 220, 54, 37, 247, 46, 253, 174, 26, 197, 35, 142, 122, 171, 24, 218, 246, 148, 31, 58, 235, 2, 3, 1, 0, 1],
                        RawSubject: [48, 48, 49, 18, 48, 16, 6, 3, 85, 4, 10, 19, 9, 111, 112, 101, 110, 115, 104, 105, 102, 116, 49, 26, 48, 24, 6, 3, 85, 4, 3, 19, 17, 111, 112, 101, 110, 115, 104, 105, 102, 116, 45, 105, 110, 103, 114, 101, 115, 115],
                        RawIssuer: [48, 38, 49, 18, 48, 16, 6, 3, 85, 4, 11, 19, 9, 111, 112, 101, 110, 115, 104, 105, 102, 116, 49, 16, 48, 14, 6, 3, 85, 4, 3, 19, 7, 114, 111, 111, 116, 45, 99, 97],
                        Signature: [50, 25, 231, 199, 177, 69, 58, 16, 51, 91, 243, 219, 210, 199, 138, 155, 53, 245, 245, 106, 127, 215, 113, 116, 53, 212, 199, 147, 106, 103, 247, 189, 33, 171, 252, 21, 143, 150, 56, 31, 85, 0, 41, 147, 65, 24, 201, 140, 203, 150, 3, 31, 110, 137, 35, 75, 171, 175, 235, 209, 21, 26, 164, 179, 71, 67, 21, 133, 105, 14, 253, 71, 51, 45, 32, 43, 100, 94, 84, 47, 64, 244, 56, 58, 136, 157, 46, 87, 211, 40, 31, 180, 2, 165, 196, 210, 222, 204, 152, 251, 130, 232, 9, 182, 120, 239, 235, 22, 89, 58, 198, 78, 82, 139, 120, 162, 101, 8, 223, 107, 59, 223, 193, 151, 252, 30, 79, 0, 15, 40, 112, 104, 59, 157, 68, 154, 197, 178, 218, 209, 164, 56, 94, 93, 22, 22, 98, 191, 126, 106, 4, 194, 233, 190, 68, 193, 123, 232, 86, 187, 65, 26, 135, 65, 91, 129, 105, 54, 92, 78, 247, 245, 172, 13, 38, 206, 186, 5, 136, 153, 211, 200, 140, 153, 205, 176, 93, 247, 111, 230, 227, 13, 56, 73, 251, 143, 236, 121, 113, 183, 124, 61, 169, 206, 40, 146, 37, 166, 252, 132, 4, 153, 168, 84, 39, 23, 232, 218, 136, 80, 210, 143, 97, 106, 58, 193, 56, 253, 19, 202, 243, 199, 49, 235, 83, 136, 105, 217, 57, 123, 238, 192, 105, 226, 211, 224, 255, 115, 130, 221, 177, 40, 19, 51, 31, 249],
                        SignatureAlgorithm: 4,
                        PublicKeyAlgorithm: 1,
                        PublicKey: <*rsa.PublicKey | 0x11cfee823170>{
                            N: {
                                neg: false,
                                abs: [12328844732508617451, 3386053924852305530, 11508787352079574519, 580165219016783073, 15126087406076015328, 8473747687808762055, 6638534397921598209, 4531117369475629035, 8686734636532653775, 9699186360867822250, 15005114980278798855, 3900631334283717631, 15336487295176330797, 13597435546774256175, 1139063161098112232, 10161195232013165903, 15551291529941624294, 8053304939740217029, 18299432943387333464, 12035498982612883405, 1076299365375342957, 3325077180629581387, 4778720590180299818, 5926909629444315306, 16602979226450151451, 14838171221845856625, 18308053321550668292, 3951715190352458341, 11733219666321171503, 11793477282006375930, 17943060274730047763, 12841897204482256645],
                            },
                            E: 65537,
                        },
                        Version: 3,
                        SerialNumber: {
                            neg: false,
                            abs: [8845870848929380208],
                        },
                        Issuer: {
                            Country: nil,
                            Organization: nil,
                            OrganizationalUnit: ["openshift"],
                            Locality: nil,
                            Province: nil,
                            StreetAddress: nil,
                            PostalCode: nil,
                            SerialNumber: "",
                            CommonName: "root-ca",
                            Names: [
                                {
                                    Type: [2, 5, 4, 11],
                                    Value: <string>"openshift",
                                },
                                {
                                    Type: [2, 5, 4, 3],
                                    Value: <string>"root-ca",
                                },
                            ],
                            ExtraNames: nil,
                        },
                        Subject: {
                            Country: nil,
                            Organization: ["openshift"],
                            OrganizationalUnit: nil,
                            Locality: nil,
                            Province: nil,
                            StreetAddress: nil,
                            PostalCode: nil,
                            SerialNumber: "",
                            CommonName: "openshift-ingress",
                            Names: [
                                {
                                    Type: [2, 5, 4, 10],
                                    Value: <string>"openshift",
                                },
                                {
                                    Type: [2, 5, 4, 3],
                                    Value: <string>"openshift-ingress",
                                },
                            ],
                            ExtraNames: nil,
                        },
                        NotBefore: {wall: 0, ext: 63928477681, loc: nil},
                        NotAfter: {wall: 0, ext: 63960013681, loc: nil},
                        KeyUsage: 5,
                        Extensions: [
                            {
                                Id: [2, 5, 29, 15],
                                Critical: true,
                                Value: [3, 2, 5, 160],
                            },
                            {
                                Id: [2, 5, 29, 37],
                                Critical: false,
                                Value: [48, 20, 6, 8, 43, 6, 1, 5, 5, 7, 3, 2, 6, 8, 43, 6, 1, 5, 5, 7, 3, 1],
                            },
                            {Id: [2, 5, 29, 19], Critical: true, Value: [48, 0]},
                            {
                                Id: [2, 5, 29, 14],
                                Critical: false,
                                Value: [4, 64, 197, 98, 86, 250, 137, 217, 107, 201, 120, 11, 250, 28, 8, 159, 50, 58, 147, 2, 127, 27, 39, 46, 176, 228, 89, 113, 119, 156, 18, 254, 186, 225, 239, 236, 201, 198, 127, 141, 38, 73, 33, 245, 173, 2, 193, 175, 61, 245, 132, 223, 97, 52, 79, 55, 120, 201, 63, 76, 209, 109, 45, 189, 34, 122],
                            },
                            {
                                Id: [2, 5, 29, 35],
                                Critical: false,
                                Value: [48, 66, 128, 64, 248, 50, 11, 119, 242, 227, 194, 167, 149, 215, 190, 179, 176, 143, 74, 56, 221, 252, 111, 212, 224, 237, 204, 6, 0, 197, 97, 180, 119, 93, 170, 15, 222, 152, 153, 8, 168, 38, 140, 186, 14, 89, 130, 247, 23, 168, 143, 191, 156, 56, 163, 177, 216, 127, 102, 115, 15, 90, 73, 84, 30, 110, 35, 172],
                            },
                            {
                                Id: [2, 5, 29, 17],
                                Critical: false,
                                Value: "0C\x82A*.apps.aro.o6e4m4u5u4t4c4u.sbp8.westus3.aroapp-hcp.azure-test.net",
                            },
                        ],
                        ExtraExtensions: nil,
                        UnhandledCriticalExtensions: nil,
                        ExtKeyUsage: [2, 1],
                        UnknownExtKeyUsage: nil,
                        BasicConstraintsValid: true,
                        IsCA: false,
                        MaxPathLen: -1,
                        MaxPathLenZero: false,
                        SubjectKeyId: [197, 98, 86, 250, 137, 217, 107, 201, 120, 11, 250, 28, 8, 159, 50, 58, 147, 2, 127, 27, 39, 46, 176, 228, 89, 113, 119, 156, 18, 254, 186, 225, 239, 236, 201, 198, 127, 141, 38, 73, 33, 245, 173, 2, 193, 175, 61, 245, 132, 223, 97, 52, 79, 55, 120, 201, 63, 76, 209, 109, 45, 189, 34, 122],
                        AuthorityKeyId: [248, 50, 11, 119, 242, 227, 194, 167, 149, 215, 190, 179, 176, 143, 74, 56, 221, 252, 111, 212, 224, 237, 204, 6, 0, 197, 97, 180, 119, 93, 170, 15, 222, 152, 153, 8, 168, 38, 140, 186, 14, 89, 130, 247, 23, 168, 143, 191, 156, 56, 163, 177, 216, 127, 102, 115, 15, 90, 73, 84, 30, 110, 35, 172],
                        OCSPServer: nil,
                        IssuingCertificateURL: nil,
                        DNSNames: [
                            "*.apps.aro.o6e4m4u5u4t4c4u.sbp8.westus3.aroapp-hcp.azure-test.net",
                        ],
                        EmailAddresses: nil,
                        IPAddresses: nil,
                        URIs: nil,
                        PermittedDNSDomainsCritical: false,
                        PermittedDNSDomains: nil,
                        ExcludedDNSDomains: nil,
                        PermittedIPRanges: nil,
                        ExcludedIPRanges: nil,
                        PermittedEmailAddresses: nil,
                        ExcludedEmailAddresses: nil,
                        PermittedURIDomains: nil,
                        ExcludedURIDomains: nil,
                        CRLDistributionPoints: nil,
                        PolicyIdentifiers: nil,
                        Policies: nil,
                        InhibitAnyPolicy: 0,
                        InhibitAnyPolicyZero: false,
                        InhibitPolicyMapping: 0,
                        InhibitPolicyMappingZero: false,
                        RequireExplicitPolicy: 0,
                        RequireExplicitPolicyZero: false,
                        PolicyMappings: nil,
                    },
                ],
                Err: <x509.UnknownAuthorityError>{
                    Cert: 0x11cfed040008 (cyclic reference),
                    hintErr: nil,
                    hintCert: nil,
                },
            },
        },
    }
occurred
VerifySimpleWebApp failed: DNS for route host <route-host> did not resolve: DNS for <route-host> did not resolve within <duration> (last error: lookup <host> on <dns-server>: dial udp <ip>: i/o timeout): context deadline exceeded
22:15:18 UTC · failed at unknown · test Customer should be able to create an HCP cluster with custom autoscaling and update the autoscaling configuration · suite integration/parallel/all
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_autoscaling.go:203]: failed to verify cluster workloads and pods after autoscaling update
Unexpected error:
    <*errors.joinError | 0x22d4bdfbc060>: 
    VerifySimpleWebApp failed: DNS for route host agnhost-e2e-sample-app-mcq7p.apps.aro.m0y7f5l6s2i7t2m.fbfu.westus3.aroapp-hcp.azure-test.net did not resolve: DNS for agnhost-e2e-sample-app-mcq7p.apps.aro.m0y7f5l6s2i7t2m.fbfu.westus3.aroapp-hcp.azure-test.net did not resolve within 10m0s (last error: lookup agnhost-e2e-sample-app-mcq7p.apps.aro.m0y7f5l6s2i7t2m.fbfu.westus3.aroapp-hcp.azure-test.net on 172.30.0.10:53: dial udp 172.30.0.10:53: i/o timeout): context deadline exceeded
    {
        errs: [
            <*fmt.wrapError | 0x22d4bdf9c1a0>{
                msg: "VerifySimpleWebApp failed: DNS for route host agnhost-e2e-sample-app-mcq7p.apps.aro.m0y7f5l6s2i7t2m.fbfu.westus3.aroapp-hcp.azure-test.net did not resolve: DNS for agnhost-e2e-sample-app-mcq7p.apps.aro.m0y7f5l6s2i7t2m.fbfu.westus3.aroapp-hcp.azure-test.net did not resolve within 10m0s (last error: lookup agnhost-e2e-sample-app-mcq7p.apps.aro.m0y7f5l6s2i7t2m.fbfu.westus3.aroapp-hcp.azure-test.net on 172.30.0.10:53: dial udp 172.30.0.10:53: i/o timeout): context deadline exceeded",
                err: <*fmt.wrapError | 0x22d4bdee0f00>{
                    msg: "DNS for route host agnhost-e2e-sample-app-mcq7p.apps.aro.m0y7f5l6s2i7t2m.fbfu.westus3.aroapp-hcp.azure-test.net did not resolve: DNS for agnhost-e2e-sample-app-mcq7p.apps.aro.m0y7f5l6s2i7t2m.fbfu.westus3.aroapp-hcp.azure-test.net did not resolve within 10m0s (last error: lookup agnhost-e2e-sample-app-mcq7p.apps.aro.m0y7f5l6s2i7t2m.fbfu.westus3.aroapp-hcp.azure-test.net on 172.30.0.10:53: dial udp 172.30.0.10:53: i/o timeout): context deadline exceeded",
                    err: <*fmt.wrapError | 0x22d4bdf9da20>{
                        msg: "DNS for agnhost-e2e-sample-app-mcq7p.apps.aro.m0y7f5l6s2i7t2m.fbfu.westus3.aroapp-hcp.azure-test.net did not resolve within 10m0s (last error: lookup agnhost-e2e-sample-app-mcq7p.apps.aro.m0y7f5l6s2i7t2m.fbfu.westus3.aroapp-hcp.azure-test.net on 172.30.0.10:53: dial udp 172.30.0.10:53: i/o timeout): context deadline exceeded",
                        err: <context.deadlineExceededError>{},
                    },
                },
            },
        ],
    }
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_autoscaling.go:203]: failed to verify cluster workloads and pods after autoscaling update
Unexpected error:
    <*errors.joinError | 0x22d4bdfbc060>: 
    VerifySimpleWebApp failed: DNS for route host agnhost-e2e-sample-app-mcq7p.apps.aro.m0y7f5l6s2i7t2m.fbfu.westus3.aroapp-hcp.azure-test.net did not resolve: DNS for agnhost-e2e-sample-app-mcq7p.apps.aro.m0y7f5l6s2i7t2m.fbfu.westus3.aroapp-hcp.azure-test.net did not resolve within 10m0s (last error: lookup agnhost-e2e-sample-app-mcq7p.apps.aro.m0y7f5l6s2i7t2m.fbfu.westus3.aroapp-hcp.azure-test.net on 172.30.0.10:53: dial udp 172.30.0.10:53: i/o timeout): context deadline exceeded
    {
        errs: [
            <*fmt.wrapError | 0x22d4bdf9c1a0>{
                msg: "VerifySimpleWebApp failed: DNS for route host agnhost-e2e-sample-app-mcq7p.apps.aro.m0y7f5l6s2i7t2m.fbfu.westus3.aroapp-hcp.azure-test.net did not resolve: DNS for agnhost-e2e-sample-app-mcq7p.apps.aro.m0y7f5l6s2i7t2m.fbfu.westus3.aroapp-hcp.azure-test.net did not resolve within 10m0s (last error: lookup agnhost-e2e-sample-app-mcq7p.apps.aro.m0y7f5l6s2i7t2m.fbfu.westus3.aroapp-hcp.azure-test.net on 172.30.0.10:53: dial udp 172.30.0.10:53: i/o timeout): context deadline exceeded",
                err: <*fmt.wrapError | 0x22d4bdee0f00>{
                    msg: "DNS for route host agnhost-e2e-sample-app-mcq7p.apps.aro.m0y7f5l6s2i7t2m.fbfu.westus3.aroapp-hcp.azure-test.net did not resolve: DNS for agnhost-e2e-sample-app-mcq7p.apps.aro.m0y7f5l6s2i7t2m.fbfu.westus3.aroapp-hcp.azure-test.net did not resolve within 10m0s (last error: lookup agnhost-e2e-sample-app-mcq7p.apps.aro.m0y7f5l6s2i7t2m.fbfu.westus3.aroapp-hcp.azure-test.net on 172.30.0.10:53: dial udp 172.30.0.10:53: i/o timeout): context deadline exceeded",
                    err: <*fmt.wrapError | 0x22d4bdf9da20>{
                        msg: "DNS for agnhost-e2e-sample-app-mcq7p.apps.aro.m0y7f5l6s2i7t2m.fbfu.westus3.aroapp-hcp.azure-test.net did not resolve within 10m0s (last error: lookup agnhost-e2e-sample-app-mcq7p.apps.aro.m0y7f5l6s2i7t2m.fbfu.westus3.aroapp-hcp.azure-test.net on 172.30.0.10:53: dial udp 172.30.0.10:53: i/o timeout): context deadline exceeded",
                        err: <context.deadlineExceededError>{},
                    },
                },
            },
        ],
    }
occurred
DNS for route host <route-host> did not resolve: DNS for <route-host> did not resolve within <duration> (last error: lookup <host> on <dns-server>: no such host): context deadline exceeded
22:15:18 UTC · failed at unknown · test Customer should create a cluster using v20260901preview API and verify cluster health · suite integration/parallel/all
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_v20260901preview.go:121]: failed to verify simple web app runs on v20260901preview cluster "v20260901"
Unexpected error:
    <*fmt.wrapError | 0x3f57e0004820>: 
    DNS for route host agnhost-e2e-sample-app-g99mf.apps.aro.v20260901.9ggg.westus3.aroapp-hcp.azure-test.net did not resolve: DNS for agnhost-e2e-sample-app-g99mf.apps.aro.v20260901.9ggg.westus3.aroapp-hcp.azure-test.net did not resolve within 10m0s (last error: lookup agnhost-e2e-sample-app-g99mf.apps.aro.v20260901.9ggg.westus3.aroapp-hcp.azure-test.net on 172.30.0.10:53: no such host): context deadline exceeded
    {
        msg: "DNS for route host agnhost-e2e-sample-app-g99mf.apps.aro.v20260901.9ggg.westus3.aroapp-hcp.azure-test.net did not resolve: DNS for agnhost-e2e-sample-app-g99mf.apps.aro.v20260901.9ggg.westus3.aroapp-hcp.azure-test.net did not resolve within 10m0s (last error: lookup agnhost-e2e-sample-app-g99mf.apps.aro.v20260901.9ggg.westus3.aroapp-hcp.azure-test.net on 172.30.0.10:53: no such host): context deadline exceeded",
        err: <*fmt.wrapError | 0x3f57dfc6e000>{
            msg: "DNS for agnhost-e2e-sample-app-g99mf.apps.aro.v20260901.9ggg.westus3.aroapp-hcp.azure-test.net did not resolve within 10m0s (last error: lookup agnhost-e2e-sample-app-g99mf.apps.aro.v20260901.9ggg.westus3.aroapp-hcp.azure-test.net on 172.30.0.10:53: no such host): context deadline exceeded",
            err: <context.deadlineExceededError>{},
        },
    }
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_v20260901preview.go:121]: failed to verify simple web app runs on v20260901preview cluster "v20260901"
Unexpected error:
    <*fmt.wrapError | 0x3f57e0004820>: 
    DNS for route host agnhost-e2e-sample-app-g99mf.apps.aro.v20260901.9ggg.westus3.aroapp-hcp.azure-test.net did not resolve: DNS for agnhost-e2e-sample-app-g99mf.apps.aro.v20260901.9ggg.westus3.aroapp-hcp.azure-test.net did not resolve within 10m0s (last error: lookup agnhost-e2e-sample-app-g99mf.apps.aro.v20260901.9ggg.westus3.aroapp-hcp.azure-test.net on 172.30.0.10:53: no such host): context deadline exceeded
    {
        msg: "DNS for route host agnhost-e2e-sample-app-g99mf.apps.aro.v20260901.9ggg.westus3.aroapp-hcp.azure-test.net did not resolve: DNS for agnhost-e2e-sample-app-g99mf.apps.aro.v20260901.9ggg.westus3.aroapp-hcp.azure-test.net did not resolve within 10m0s (last error: lookup agnhost-e2e-sample-app-g99mf.apps.aro.v20260901.9ggg.westus3.aroapp-hcp.azure-test.net on 172.30.0.10:53: no such host): context deadline exceeded",
        err: <*fmt.wrapError | 0x3f57dfc6e000>{
            msg: "DNS for agnhost-e2e-sample-app-g99mf.apps.aro.v20260901.9ggg.westus3.aroapp-hcp.azure-test.net did not resolve within 10m0s (last error: lookup agnhost-e2e-sample-app-g99mf.apps.aro.v20260901.9ggg.westus3.aroapp-hcp.azure-test.net on 172.30.0.10:53: no such host): context deadline exceeded",
            err: <context.deadlineExceededError>{},
        },
    }
occurred
1h 25m 47s branch-ci-Azure-ARO-HCP-main-e2e-integration-e2e-parallel-all
post-goodmerged PR
No additional run metadata captured
westus3 unknown failed n/a 1
unknown (1)
tls: failed to verify certificate: x509: certificate signed by unknown authority
20:41:28 UTC · failed at unknown · test ARO-HCP should be able to perform a control plane and node pool install with OCP candidate channel for 4.23 · suite integration/parallel/all
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:194]: failed to verify simple web app runs on cluster "cluster-candidate-4-23-2rcw86"
Unexpected error:
    <*fmt.wrapError | 0x2b2e578984a0>: 
    [strict TLS verification] route was never reachable: Get "https://agnhost-e2e-sample-app-w86zx.apps.aro.a2s9b8m2e4p7p0g.0mu8.westus3.aroapp-hcp.azure-test.net": tls: failed to verify certificate: x509: certificate signed by unknown authority
    {
        msg: "[strict TLS verification] route was never reachable: Get \"https://agnhost-e2e-sample-app-w86zx.apps.aro.a2s9b8m2e4p7p0g.0mu8.westus3.aroapp-hcp.azure-test.net\": tls: failed to verify certificate: x509: certificate signed by unknown authority",
        err: <*url.Error | 0x2b2e57370780>{
            Op: "Get",
            URL: "https://agnhost-e2e-sample-app-w86zx.apps.aro.a2s9b8m2e4p7p0g.0mu8.westus3.aroapp-hcp.azure-test.net",
            Err: <*tls.CertificateVerificationError | 0x2b2e57370750>{
                UnverifiedCertificates: [
                    {
                        Raw: [48, 130, 4, 1, 48, 130, 2, 233, 160, 3, 2, 1, 2, 2, 8, 124, 87, 70, 122, 130, 6, 100, 82, 48, 13, 6, 9, 42, 134, 72, 134, 247, 13, 1, 1, 11, 5, 0, 48, 38, 49, 18, 48, 16, 6, 3, 85, 4, 11, 19, 9, 111, 112, 101, 110, 115, 104, 105, 102, 116, 49, 16, 48, 14, 6, 3, 85, 4, 3, 19, 7, 114, 111, 111, 116, 45, 99, 97, 48, 30, 23, 13, 50, 54, 49, 48, 50, 52, 50, 49, 48, 56, 48, 56, 90, 23, 13, 50, 55, 49, 48, 50, 52, 50, 49, 48, 56, 48, 56, 90, 48, 48, 49, 18, 48, 16, 6, 3, 85, 4, 10, 19, 9, 111, 112, 101, 110, 115, 104, 105, 102, 116, 49, 26, 48, 24, 6, 3, 85, 4, 3, 19, 17, 111, 112, 101, 110, 115, 104, 105, 102, 116, 45, 105, 110, 103, 114, 101, 115, 115, 48, 130, 1, 34, 48, 13, 6, 9, 42, 134, 72, 134, 247, 13, 1, 1, 1, 5, 0, 3, 130, 1, 15, 0, 48, 130, 1, 10, 2, 130, 1, 1, 0, 184, 6, 158, 64, 94, 24, 96, 40, 160, 154, 141, 196, 128, 87, 218, 59, 39, 71, 199, 234, 79, 240, 146, 24, 254, 106, 245, 45, 201, 136, 214, 64, 254, 110, 56, 158, 251, 103, 159, 125, 70, 9, 9, 192, 157, 185, 88, 45, 237, 209, 14, 238, 22, 11, 156, 42, 61, 172, 248, 175, 205, 29, 87, 44, 249, 69, 148, 194, 201, 88, 31, 36, 152, 175, 207, 120, 70, 231, 234, 239, 27, 120, 11, 188, 47, 126, 21, 187, 83, 61, 218, 253, 69, 150, 57, 242, 31, 129, 90, 78, 122, 162, 115, 193, 66, 231, 239, 171, 129, 241, 209, 248, 79, 198, 197, 15, 5, 20, 97, 114, 135, 82, 183, 39, 26, 12, 204, 167, 245, 131, 126, 229, 14, 152, 249, 66, 23, 145, 179, 201, 129, 84, 180, 47, 247, 191, 198, 219, 235, 14, 55, 141, 41, 250, 184, 135, 128, 115, 246, 220, 161, 224, 164, 119, 161, 199, 4, 69, 138, 127, 197, 71, 128, 167, 90, 212, 150, 2, 13, 143, 216, 191, 58, 228, 209, 93, 22, 58, 9, 137, 127, 203, 163, 74, 40, 217, 253, 186, 51, 153, 40, 52, 238, 73, 60, 57, 57, 108, 128, 2, 48, 134, 115, 108, 142, 65, 158, 113, 75, 25, 249, 189, 39, 130, 154, 24, 112, 209, 120, 229, 129, 223, 190, 147, 169, 232, 115, 83, 39, 47, 230, 201, 87, 216, 98, 161, 27, 95, 39, 237, 157, 102, 109, 222, 170, 149, 2, 3, 1, 0, 1, 163, 130, 1, 39, 48, 130, 1, 35, 48, 14, 6, 3, 85, 29, 15, 1, 1, 255, 4, 4, 3, 2, 5, 160, 48, 29, 6, 3, 85, 29, 37, 4, 22, 48, 20, 6, 8, 43, 6, 1, 5, 5, 7, 3, 2, 6, 8, 43, 6, 1, 5, 5, 7, 3, 1, 48, 12, 6, 3, 85, 29, 19, 1, 1, 255, 4, 2, 48, 0, 48, 73, 6, 3, 85, 29, 14, 4, 66, 4, 64, 43, 61, 184, 58, 190, 246, 117, 145, 150, 149, 52, 148, 50, 70, 171, 24, 140, 56, 164, 203, 105, 63, 65, 146, 183, 66, 114, 129, 170, 239, 140, 79, 13, 223, 192, 116, 206, 195, 234, 70, 154, 102, 164, 7, 77, 72, 8, 99, 67, 120, 61, 224, 48, 163, 84, 4, 250, 254, 27, 88, 242, 13, 99, 21, 48, 75, 6, 3, 85, 29, 35, 4, 68, 48, 66, 128, 64, 145, 115, 72, 248, 147, 3, 210, 10, 72, 90, 51, 130, 36, 255, 101, 123, 45, 151, 78, 158, 148, 254, 229, 16, 121, 180, 193, 131, 183, 127, 36, 88, 142, 33, 22, 114, 228, 61, 202, 29, 164, 17, 211, 221, 107, 244, 245, 206, 36, 16, 177, 96, 20, 74, 22, 17, 184, 132, 100, 119, 214, 222, 88, 253, 48, 76, 6, 3, 85, 29, 17, 4, 69, 48, 67, 130, 65, 42, 46, 97, 112, 112, 115, 46, 97, 114, 111, 46, 97, 50, 115, 57, 98, 56, 109, 50, 101, 52, 112, 55, 112, 48, 103, 46, 48, 109, 117, 56, 46, 119, 101, 115, 116, 117, 115, 51, 46, 97, 114, 111, 97, 112, 112, 45, 104, 99, 112, 46, 97, 122, 117, 114, 101, 45, 116, 101, 115, 116, 46, 110, 101, 116, 48, 13, 6, 9, 42, 134, 72, 134, 247, 13, 1, 1, 11, 5, 0, 3, 130, 1, 1, 0, 144, 63, 1, 9, 63, 203, 104, 158, 72, 190, 218, 173, 218, 174, 171, 175, 249, 99, 198, 163, 154, 183, 80, 70, 18, 90, 143, 16, 110, 40, 74, 15, 152, 104, 155, 119, 3, 189, 92, 182, 181, 193, 112, 193, 179, 171, 177, 195, 218, 45, 246, 105, 209, 209, 4, 147, 233, 109, 252, 165, 52, 200, 179, 47, 64, 202, 66, 155, 170, 48, 184, 165, 51, 160, 106, 229, 210, 44, 72, 109, 204, 32, 9, 38, 222, 130, 253, 176, 6, 246, 50, 104, 240, 45, 124, 77, 233, 84, 164, 109, 154, 110, 171, 77, 164, 253, 134, 225, 58, 131, 116, 48, 78, 1, 85, 150, 81, 149, 67, 171, 102, 216, 126, 229, 90, 56, 26, 33, 171, 124, 161, 209, 140, 187, 61, 34, 212, 215, 17, 134, 62, 6, 13, 208, 221, 55, 5, 187, 22, 63, 93, 164, 76, 124, 166, 239, 104, 210, 84, 78, 146, 114, 146, 113, 215, 97, 33, 13, 186, 135, 110, 182, 194, 44, 64, 215, 36, 43, 178, 21, 170, 171, 251, 180, 100, 40, 239, 189, 24, 174, 192, 195, 98, 108, 222, 96, 135, 172, 56, 114, 81, 213, 239, 84, 199, 27, 118, 56, 193, 239, 243, 191, 178, 97, 106, 201, 66, 252, 31, 190, 254, 107, 226, 184, 19, 188, 113, 44, 99, 24, 245, 52, 154, 97, 249, 178, 221, 232, 203, 63, 51, 4, 109, 167, 92, 246, 143, 188, 32, 32, 87, 98, 186, 5, 69, 17],
                        RawTBSCertificate: [48, 130, 2, 233, 160, 3, 2, 1, 2, 2, 8, 124, 87, 70, 122, 130, 6, 100, 82, 48, 13, 6, 9, 42, 134, 72, 134, 247, 13, 1, 1, 11, 5, 0, 48, 38, 49, 18, 48, 16, 6, 3, 85, 4, 11, 19, 9, 111, 112, 101, 110, 115, 104, 105, 102, 116, 49, 16, 48, 14, 6, 3, 85, 4, 3, 19, 7, 114, 111, 111, 116, 45, 99, 97, 48, 30, 23, 13, 50, 54, 49, 48, 50, 52, 50, 49, 48, 56, 48, 56, 90, 23, 13, 50, 55, 49, 48, 50, 52, 50, 49, 48, 56, 48, 56, 90, 48, 48, 49, 18, 48, 16, 6, 3, 85, 4, 10, 19, 9, 111, 112, 101, 110, 115, 104, 105, 102, 116, 49, 26, 48, 24, 6, 3, 85, 4, 3, 19, 17, 111, 112, 101, 110, 115, 104, 105, 102, 116, 45, 105, 110, 103, 114, 101, 115, 115, 48, 130, 1, 34, 48, 13, 6, 9, 42, 134, 72, 134, 247, 13, 1, 1, 1, 5, 0, 3, 130, 1, 15, 0, 48, 130, 1, 10, 2, 130, 1, 1, 0, 184, 6, 158, 64, 94, 24, 96, 40, 160, 154, 141, 196, 128, 87, 218, 59, 39, 71, 199, 234, 79, 240, 146, 24, 254, 106, 245, 45, 201, 136, 214, 64, 254, 110, 56, 158, 251, 103, 159, 125, 70, 9, 9, 192, 157, 185, 88, 45, 237, 209, 14, 238, 22, 11, 156, 42, 61, 172, 248, 175, 205, 29, 87, 44, 249, 69, 148, 194, 201, 88, 31, 36, 152, 175, 207, 120, 70, 231, 234, 239, 27, 120, 11, 188, 47, 126, 21, 187, 83, 61, 218, 253, 69, 150, 57, 242, 31, 129, 90, 78, 122, 162, 115, 193, 66, 231, 239, 171, 129, 241, 209, 248, 79, 198, 197, 15, 5, 20, 97, 114, 135, 82, 183, 39, 26, 12, 204, 167, 245, 131, 126, 229, 14, 152, 249, 66, 23, 145, 179, 201, 129, 84, 180, 47, 247, 191, 198, 219, 235, 14, 55, 141, 41, 250, 184, 135, 128, 115, 246, 220, 161, 224, 164, 119, 161, 199, 4, 69, 138, 127, 197, 71, 128, 167, 90, 212, 150, 2, 13, 143, 216, 191, 58, 228, 209, 93, 22, 58, 9, 137, 127, 203, 163, 74, 40, 217, 253, 186, 51, 153, 40, 52, 238, 73, 60, 57, 57, 108, 128, 2, 48, 134, 115, 108, 142, 65, 158, 113, 75, 25, 249, 189, 39, 130, 154, 24, 112, 209, 120, 229, 129, 223, 190, 147, 169, 232, 115, 83, 39, 47, 230, 201, 87, 216, 98, 161, 27, 95, 39, 237, 157, 102, 109, 222, 170, 149, 2, 3, 1, 0, 1, 163, 130, 1, 39, 48, 130, 1, 35, 48, 14, 6, 3, 85, 29, 15, 1, 1, 255, 4, 4, 3, 2, 5, 160, 48, 29, 6, 3, 85, 29, 37, 4, 22, 48, 20, 6, 8, 43, 6, 1, 5, 5, 7, 3, 2, 6, 8, 43, 6, 1, 5, 5, 7, 3, 1, 48, 12, 6, 3, 85, 29, 19, 1, 1, 255, 4, 2, 48, 0, 48, 73, 6, 3, 85, 29, 14, 4, 66, 4, 64, 43, 61, 184, 58, 190, 246, 117, 145, 150, 149, 52, 148, 50, 70, 171, 24, 140, 56, 164, 203, 105, 63, 65, 146, 183, 66, 114, 129, 170, 239, 140, 79, 13, 223, 192, 116, 206, 195, 234, 70, 154, 102, 164, 7, 77, 72, 8, 99, 67, 120, 61, 224, 48, 163, 84, 4, 250, 254, 27, 88, 242, 13, 99, 21, 48, 75, 6, 3, 85, 29, 35, 4, 68, 48, 66, 128, 64, 145, 115, 72, 248, 147, 3, 210, 10, 72, 90, 51, 130, 36, 255, 101, 123, 45, 151, 78, 158, 148, 254, 229, 16, 121, 180, 193, 131, 183, 127, 36, 88, 142, 33, 22, 114, 228, 61, 202, 29, 164, 17, 211, 221, 107, 244, 245, 206, 36, 16, 177, 96, 20, 74, 22, 17, 184, 132, 100, 119, 214, 222, 88, 253, 48, 76, 6, 3, 85, 29, 17, 4, 69, 48, 67, 130, 65, 42, 46, 97, 112, 112, 115, 46, 97, 114, 111, 46, 97, 50, 115, 57, 98, 56, 109, 50, 101, 52, 112, 55, 112, 48, 103, 46, 48, 109, 117, 56, 46, 119, 101, 115, 116, 117, 115, 51, 46, 97, 114, 111, 97, 112, 112, 45, 104, 99, 112, 46, 97, 122, 117, 114, 101, 45, 116, 101, 115, 116, 46, 110, 101, 116],
                        RawSubjectPublicKeyInfo: [48, 130, 1, 34, 48, 13, 6, 9, 42, 134, 72, 134, 247, 13, 1, 1, 1, 5, 0, 3, 130, 1, 15, 0, 48, 130, 1, 10, 2, 130, 1, 1, 0, 184, 6, 158, 64, 94, 24, 96, 40, 160, 154, 141, 196, 128, 87, 218, 59, 39, 71, 199, 234, 79, 240, 146, 24, 254, 106, 245, 45, 201, 136, 214, 64, 254, 110, 56, 158, 251, 103, 159, 125, 70, 9, 9, 192, 157, 185, 88, 45, 237, 209, 14, 238, 22, 11, 156, 42, 61, 172, 248, 175, 205, 29, 87, 44, 249, 69, 148, 194, 201, 88, 31, 36, 152, 175, 207, 120, 70, 231, 234, 239, 27, 120, 11, 188, 47, 126, 21, 187, 83, 61, 218, 253, 69, 150, 57, 242, 31, 129, 90, 78, 122, 162, 115, 193, 66, 231, 239, 171, 129, 241, 209, 248, 79, 198, 197, 15, 5, 20, 97, 114, 135, 82, 183, 39, 26, 12, 204, 167, 245, 131, 126, 229, 14, 152, 249, 66, 23, 145, 179, 201, 129, 84, 180, 47, 247, 191, 198, 219, 235, 14, 55, 141, 41, 250, 184, 135, 128, 115, 246, 220, 161, 224, 164, 119, 161, 199, 4, 69, 138, 127, 197, 71, 128, 167, 90, 212, 150, 2, 13, 143, 216, 191, 58, 228, 209, 93, 22, 58, 9, 137, 127, 203, 163, 74, 40, 217, 253, 186, 51, 153, 40, 52, 238, 73, 60, 57, 57, 108, 128, 2, 48, 134, 115, 108, 142, 65, 158, 113, 75, 25, 249, 189, 39, 130, 154, 24, 112, 209, 120, 229, 129, 223, 190, 147, 169, 232, 115, 83, 39, 47, 230, 201, 87, 216, 98, 161, 27, 95, 39, 237, 157, 102, 109, 222, 170, 149, 2, 3, 1, 0, 1],
                        RawSubject: [48, 48, 49, 18, 48, 16, 6, 3, 85, 4, 10, 19, 9, 111, 112, 101, 110, 115, 104, 105, 102, 116, 49, 26, 48, 24, 6, 3, 85, 4, 3, 19, 17, 111, 112, 101, 110, 115, 104, 105, 102, 116, 45, 105, 110, 103, 114, 101, 115, 115],
                        RawIssuer: [48, 38, 49, 18, 48, 16, 6, 3, 85, 4, 11, 19, 9, 111, 112, 101, 110, 115, 104, 105, 102, 116, 49, 16, 48, 14, 6, 3, 85, 4, 3, 19, 7, 114, 111, 111, 116, 45, 99, 97],
                        Signature: [144, 63, 1, 9, 63, 203, 104, 158, 72, 190, 218, 173, 218, 174, 171, 175, 249, 99, 198, 163, 154, 183, 80, 70, 18, 90, 143, 16, 110, 40, 74, 15, 152, 104, 155, 119, 3, 189, 92, 182, 181, 193, 112, 193, 179, 171, 177, 195, 218, 45, 246, 105, 209, 209, 4, 147, 233, 109, 252, 165, 52, 200, 179, 47, 64, 202, 66, 155, 170, 48, 184, 165, 51, 160, 106, 229, 210, 44, 72, 109, 204, 32, 9, 38, 222, 130, 253, 176, 6, 246, 50, 104, 240, 45, 124, 77, 233, 84, 164, 109, 154, 110, 171, 77, 164, 253, 134, 225, 58, 131, 116, 48, 78, 1, 85, 150, 81, 149, 67, 171, 102, 216, 126, 229, 90, 56, 26, 33, 171, 124, 161, 209, 140, 187, 61, 34, 212, 215, 17, 134, 62, 6, 13, 208, 221, 55, 5, 187, 22, 63, 93, 164, 76, 124, 166, 239, 104, 210, 84, 78, 146, 114, 146, 113, 215, 97, 33, 13, 186, 135, 110, 182, 194, 44, 64, 215, 36, 43, 178, 21, 170, 171, 251, 180, 100, 40, 239, 189, 24, 174, 192, 195, 98, 108, 222, 96, 135, 172, 56, 114, 81, 213, 239, 84, 199, 27, 118, 56, 193, 239, 243, 191, 178, 97, 106, 201, 66, 252, 31, 190, 254, 107, 226, 184, 19, 188, 113, 44, 99, 24, 245, 52, 154, 97, 249, 178, 221, 232, 203, 63, 51, 4, 109, 167, 92, 246, 143, 188, 32, 32, 87, 98, 186, 5, 69, 17],
                        SignatureAlgorithm: 4,
                        PublicKeyAlgorithm: 1,
                        PublicKey: <*rsa.PublicKey | 0x2b2e57843d70>{
                            N: {
                                neg: false,
                                abs: [2877128800215149205, 16629919685921545055, 13732506504615110447, 11103748926260740575, 11416989105297106818, 9223988340828442177, 2897202458616478060, 11766261893188694937, 15086238765284949963, 10809216967110507236, 9979912210168109780, 11664503868611822661, 3024932991847888604, 17852206095763650445, 1698336212534998063, 17691123283498957122, 9751057521779199143, 5748498642685419890, 4821085446021960184, 2270194980282856385, 5998191060056422898, 1979344939107882427, 11002240530182040303, 17961926253159653156, 4444200366252054316, 17136494472856312874, 5046575580329891885, 18333653388545728381, 18332734810168546880, 2830450700003086872, 11572718067588258363, 13260460152132755496],
                            },
                            E: 65537,
                        },
                        Version: 3,
                        SerialNumber: {
                            neg: false,
                            abs: [8959707475658302546],
                        },
                        Issuer: {
                            Country: nil,
                            Organization: nil,
                            OrganizationalUnit: ["openshift"],
                            Locality: nil,
                            Province: nil,
                            StreetAddress: nil,
                            PostalCode: nil,
                            SerialNumber: "",
                            CommonName: "root-ca",
                            Names: [
                                {
                                    Type: [2, 5, 4, 11],
                                    Value: <string>"openshift",
                                },
                                {
                                    Type: [2, 5, 4, 3],
                                    Value: <string>"root-ca",
                                },
                            ],
                            ExtraNames: nil,
                        },
                        Subject: {
                            Country: nil,
                            Organization: ["openshift"],
                            OrganizationalUnit: nil,
                            Locality: nil,
                            Province: nil,
                            StreetAddress: nil,
                            PostalCode: nil,
                            SerialNumber: "",
                            CommonName: "openshift-ingress",
                            Names: [
                                {
                                    Type: [2, 5, 4, 10],
                                    Value: <string>"openshift",
                                },
                                {
                                    Type: [2, 5, 4, 3],
                                    Value: <string>"openshift-ingress",
                                },
                            ],
                            ExtraNames: nil,
                        },
                        NotBefore: {wall: 0, ext: 63928472888, loc: nil},
                        NotAfter: {wall: 0, ext: 63960008888, loc: nil},
                        KeyUsage: 5,
                        Extensions: [
                            {
                                Id: [2, 5, 29, 15],
                                Critical: true,
                                Value: [3, 2, 5, 160],
                            },
                            {
                                Id: [2, 5, 29, 37],
                                Critical: false,
                                Value: [48, 20, 6, 8, 43, 6, 1, 5, 5, 7, 3, 2, 6, 8, 43, 6, 1, 5, 5, 7, 3, 1],
                            },
                            {Id: [2, 5, 29, 19], Critical: true, Value: [48, 0]},
                            {
                                Id: [2, 5, 29, 14],
                                Critical: false,
                                Value: [4, 64, 43, 61, 184, 58, 190, 246, 117, 145, 150, 149, 52, 148, 50, 70, 171, 24, 140, 56, 164, 203, 105, 63, 65, 146, 183, 66, 114, 129, 170, 239, 140, 79, 13, 223, 192, 116, 206, 195, 234, 70, 154, 102, 164, 7, 77, 72, 8, 99, 67, 120, 61, 224, 48, 163, 84, 4, 250, 254, 27, 88, 242, 13, 99, 21],
                            },
                            {
                                Id: [2, 5, 29, 35],
                                Critical: false,
                                Value: [48, 66, 128, 64, 145, 115, 72, 248, 147, 3, 210, 10, 72, 90, 51, 130, 36, 255, 101, 123, 45, 151, 78, 158, 148, 254, 229, 16, 121, 180, 193, 131, 183, 127, 36, 88, 142, 33, 22, 114, 228, 61, 202, 29, 164, 17, 211, 221, 107, 244, 245, 206, 36, 16, 177, 96, 20, 74, 22, 17, 184, 132, 100, 119, 214, 222, 88, 253],
                            },
                            {
                                Id: [2, 5, 29, 17],
                                Critical: false,
                                Value: "0C\x82A*.apps.aro.a2s9b8m2e4p7p0g.0mu8.westus3.aroapp-hcp.azure-test.net",
                            },
                        ],
                        ExtraExtensions: nil,
                        UnhandledCriticalExtensions: nil,
                        ExtKeyUsage: [2, 1],
                        UnknownExtKeyUsage: nil,
                        BasicConstraintsValid: true,
                        IsCA: false,
                        MaxPathLen: -1,
                        MaxPathLenZero: false,
                        SubjectKeyId: [43, 61, 184, 58, 190, 246, 117, 145, 150, 149, 52, 148, 50, 70, 171, 24, 140, 56, 164, 203, 105, 63, 65, 146, 183, 66, 114, 129, 170, 239, 140, 79, 13, 223, 192, 116, 206, 195, 234, 70, 154, 102, 164, 7, 77, 72, 8, 99, 67, 120, 61, 224, 48, 163, 84, 4, 250, 254, 27, 88, 242, 13, 99, 21],
                        AuthorityKeyId: [145, 115, 72, 248, 147, 3, 210, 10, 72, 90, 51, 130, 36, 255, 101, 123, 45, 151, 78, 158, 148, 254, 229, 16, 121, 180, 193, 131, 183, 127, 36, 88, 142, 33, 22, 114, 228, 61, 202, 29, 164, 17, 211, 221, 107, 244, 245, 206, 36, 16, 177, 96, 20, 74, 22, 17, 184, 132, 100, 119, 214, 222, 88, 253],
                        OCSPServer: nil,
                        IssuingCertificateURL: nil,
                        DNSNames: [
                            "*.apps.aro.a2s9b8m2e4p7p0g.0mu8.westus3.aroapp-hcp.azure-test.net",
                        ],
                        EmailAddresses: nil,
                        IPAddresses: nil,
                        URIs: nil,
                        PermittedDNSDomainsCritical: false,
                        PermittedDNSDomains: nil,
                        ExcludedDNSDomains: nil,
                        PermittedIPRanges: nil,
                        ExcludedIPRanges: nil,
                        PermittedEmailAddresses: nil,
                        ExcludedEmailAddresses: nil,
                        PermittedURIDomains: nil,
                        ExcludedURIDomains: nil,
                        CRLDistributionPoints: nil,
                        PolicyIdentifiers: nil,
                        Policies: nil,
                        InhibitAnyPolicy: 0,
                        InhibitAnyPolicyZero: false,
                        InhibitPolicyMapping: 0,
                        InhibitPolicyMappingZero: false,
                        RequireExplicitPolicy: 0,
                        RequireExplicitPolicyZero: false,
                        PolicyMappings: nil,
                    },
                ],
                Err: <x509.UnknownAuthorityError>{
                    Cert: 0x2b2e55ba2008 (cyclic reference),
                    hintErr: nil,
                    hintCert: nil,
                },
            },
        },
    }
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:194]: failed to verify simple web app runs on cluster "cluster-candidate-4-23-2rcw86"
Unexpected error:
    <*fmt.wrapError | 0x2b2e578984a0>: 
    [strict TLS verification] route was never reachable: Get "https://agnhost-e2e-sample-app-w86zx.apps.aro.a2s9b8m2e4p7p0g.0mu8.westus3.aroapp-hcp.azure-test.net": tls: failed to verify certificate: x509: certificate signed by unknown authority
    {
        msg: "[strict TLS verification] route was never reachable: Get \"https://agnhost-e2e-sample-app-w86zx.apps.aro.a2s9b8m2e4p7p0g.0mu8.westus3.aroapp-hcp.azure-test.net\": tls: failed to verify certificate: x509: certificate signed by unknown authority",
        err: <*url.Error | 0x2b2e57370780>{
            Op: "Get",
            URL: "https://agnhost-e2e-sample-app-w86zx.apps.aro.a2s9b8m2e4p7p0g.0mu8.westus3.aroapp-hcp.azure-test.net",
            Err: <*tls.CertificateVerificationError | 0x2b2e57370750>{
                UnverifiedCertificates: [
                    {
                        Raw: [48, 130, 4, 1, 48, 130, 2, 233, 160, 3, 2, 1, 2, 2, 8, 124, 87, 70, 122, 130, 6, 100, 82, 48, 13, 6, 9, 42, 134, 72, 134, 247, 13, 1, 1, 11, 5, 0, 48, 38, 49, 18, 48, 16, 6, 3, 85, 4, 11, 19, 9, 111, 112, 101, 110, 115, 104, 105, 102, 116, 49, 16, 48, 14, 6, 3, 85, 4, 3, 19, 7, 114, 111, 111, 116, 45, 99, 97, 48, 30, 23, 13, 50, 54, 49, 48, 50, 52, 50, 49, 48, 56, 48, 56, 90, 23, 13, 50, 55, 49, 48, 50, 52, 50, 49, 48, 56, 48, 56, 90, 48, 48, 49, 18, 48, 16, 6, 3, 85, 4, 10, 19, 9, 111, 112, 101, 110, 115, 104, 105, 102, 116, 49, 26, 48, 24, 6, 3, 85, 4, 3, 19, 17, 111, 112, 101, 110, 115, 104, 105, 102, 116, 45, 105, 110, 103, 114, 101, 115, 115, 48, 130, 1, 34, 48, 13, 6, 9, 42, 134, 72, 134, 247, 13, 1, 1, 1, 5, 0, 3, 130, 1, 15, 0, 48, 130, 1, 10, 2, 130, 1, 1, 0, 184, 6, 158, 64, 94, 24, 96, 40, 160, 154, 141, 196, 128, 87, 218, 59, 39, 71, 199, 234, 79, 240, 146, 24, 254, 106, 245, 45, 201, 136, 214, 64, 254, 110, 56, 158, 251, 103, 159, 125, 70, 9, 9, 192, 157, 185, 88, 45, 237, 209, 14, 238, 22, 11, 156, 42, 61, 172, 248, 175, 205, 29, 87, 44, 249, 69, 148, 194, 201, 88, 31, 36, 152, 175, 207, 120, 70, 231, 234, 239, 27, 120, 11, 188, 47, 126, 21, 187, 83, 61, 218, 253, 69, 150, 57, 242, 31, 129, 90, 78, 122, 162, 115, 193, 66, 231, 239, 171, 129, 241, 209, 248, 79, 198, 197, 15, 5, 20, 97, 114, 135, 82, 183, 39, 26, 12, 204, 167, 245, 131, 126, 229, 14, 152, 249, 66, 23, 145, 179, 201, 129, 84, 180, 47, 247, 191, 198, 219, 235, 14, 55, 141, 41, 250, 184, 135, 128, 115, 246, 220, 161, 224, 164, 119, 161, 199, 4, 69, 138, 127, 197, 71, 128, 167, 90, 212, 150, 2, 13, 143, 216, 191, 58, 228, 209, 93, 22, 58, 9, 137, 127, 203, 163, 74, 40, 217, 253, 186, 51, 153, 40, 52, 238, 73, 60, 57, 57, 108, 128, 2, 48, 134, 115, 108, 142, 65, 158, 113, 75, 25, 249, 189, 39, 130, 154, 24, 112, 209, 120, 229, 129, 223, 190, 147, 169, 232, 115, 83, 39, 47, 230, 201, 87, 216, 98, 161, 27, 95, 39, 237, 157, 102, 109, 222, 170, 149, 2, 3, 1, 0, 1, 163, 130, 1, 39, 48, 130, 1, 35, 48, 14, 6, 3, 85, 29, 15, 1, 1, 255, 4, 4, 3, 2, 5, 160, 48, 29, 6, 3, 85, 29, 37, 4, 22, 48, 20, 6, 8, 43, 6, 1, 5, 5, 7, 3, 2, 6, 8, 43, 6, 1, 5, 5, 7, 3, 1, 48, 12, 6, 3, 85, 29, 19, 1, 1, 255, 4, 2, 48, 0, 48, 73, 6, 3, 85, 29, 14, 4, 66, 4, 64, 43, 61, 184, 58, 190, 246, 117, 145, 150, 149, 52, 148, 50, 70, 171, 24, 140, 56, 164, 203, 105, 63, 65, 146, 183, 66, 114, 129, 170, 239, 140, 79, 13, 223, 192, 116, 206, 195, 234, 70, 154, 102, 164, 7, 77, 72, 8, 99, 67, 120, 61, 224, 48, 163, 84, 4, 250, 254, 27, 88, 242, 13, 99, 21, 48, 75, 6, 3, 85, 29, 35, 4, 68, 48, 66, 128, 64, 145, 115, 72, 248, 147, 3, 210, 10, 72, 90, 51, 130, 36, 255, 101, 123, 45, 151, 78, 158, 148, 254, 229, 16, 121, 180, 193, 131, 183, 127, 36, 88, 142, 33, 22, 114, 228, 61, 202, 29, 164, 17, 211, 221, 107, 244, 245, 206, 36, 16, 177, 96, 20, 74, 22, 17, 184, 132, 100, 119, 214, 222, 88, 253, 48, 76, 6, 3, 85, 29, 17, 4, 69, 48, 67, 130, 65, 42, 46, 97, 112, 112, 115, 46, 97, 114, 111, 46, 97, 50, 115, 57, 98, 56, 109, 50, 101, 52, 112, 55, 112, 48, 103, 46, 48, 109, 117, 56, 46, 119, 101, 115, 116, 117, 115, 51, 46, 97, 114, 111, 97, 112, 112, 45, 104, 99, 112, 46, 97, 122, 117, 114, 101, 45, 116, 101, 115, 116, 46, 110, 101, 116, 48, 13, 6, 9, 42, 134, 72, 134, 247, 13, 1, 1, 11, 5, 0, 3, 130, 1, 1, 0, 144, 63, 1, 9, 63, 203, 104, 158, 72, 190, 218, 173, 218, 174, 171, 175, 249, 99, 198, 163, 154, 183, 80, 70, 18, 90, 143, 16, 110, 40, 74, 15, 152, 104, 155, 119, 3, 189, 92, 182, 181, 193, 112, 193, 179, 171, 177, 195, 218, 45, 246, 105, 209, 209, 4, 147, 233, 109, 252, 165, 52, 200, 179, 47, 64, 202, 66, 155, 170, 48, 184, 165, 51, 160, 106, 229, 210, 44, 72, 109, 204, 32, 9, 38, 222, 130, 253, 176, 6, 246, 50, 104, 240, 45, 124, 77, 233, 84, 164, 109, 154, 110, 171, 77, 164, 253, 134, 225, 58, 131, 116, 48, 78, 1, 85, 150, 81, 149, 67, 171, 102, 216, 126, 229, 90, 56, 26, 33, 171, 124, 161, 209, 140, 187, 61, 34, 212, 215, 17, 134, 62, 6, 13, 208, 221, 55, 5, 187, 22, 63, 93, 164, 76, 124, 166, 239, 104, 210, 84, 78, 146, 114, 146, 113, 215, 97, 33, 13, 186, 135, 110, 182, 194, 44, 64, 215, 36, 43, 178, 21, 170, 171, 251, 180, 100, 40, 239, 189, 24, 174, 192, 195, 98, 108, 222, 96, 135, 172, 56, 114, 81, 213, 239, 84, 199, 27, 118, 56, 193, 239, 243, 191, 178, 97, 106, 201, 66, 252, 31, 190, 254, 107, 226, 184, 19, 188, 113, 44, 99, 24, 245, 52, 154, 97, 249, 178, 221, 232, 203, 63, 51, 4, 109, 167, 92, 246, 143, 188, 32, 32, 87, 98, 186, 5, 69, 17],
                        RawTBSCertificate: [48, 130, 2, 233, 160, 3, 2, 1, 2, 2, 8, 124, 87, 70, 122, 130, 6, 100, 82, 48, 13, 6, 9, 42, 134, 72, 134, 247, 13, 1, 1, 11, 5, 0, 48, 38, 49, 18, 48, 16, 6, 3, 85, 4, 11, 19, 9, 111, 112, 101, 110, 115, 104, 105, 102, 116, 49, 16, 48, 14, 6, 3, 85, 4, 3, 19, 7, 114, 111, 111, 116, 45, 99, 97, 48, 30, 23, 13, 50, 54, 49, 48, 50, 52, 50, 49, 48, 56, 48, 56, 90, 23, 13, 50, 55, 49, 48, 50, 52, 50, 49, 48, 56, 48, 56, 90, 48, 48, 49, 18, 48, 16, 6, 3, 85, 4, 10, 19, 9, 111, 112, 101, 110, 115, 104, 105, 102, 116, 49, 26, 48, 24, 6, 3, 85, 4, 3, 19, 17, 111, 112, 101, 110, 115, 104, 105, 102, 116, 45, 105, 110, 103, 114, 101, 115, 115, 48, 130, 1, 34, 48, 13, 6, 9, 42, 134, 72, 134, 247, 13, 1, 1, 1, 5, 0, 3, 130, 1, 15, 0, 48, 130, 1, 10, 2, 130, 1, 1, 0, 184, 6, 158, 64, 94, 24, 96, 40, 160, 154, 141, 196, 128, 87, 218, 59, 39, 71, 199, 234, 79, 240, 146, 24, 254, 106, 245, 45, 201, 136, 214, 64, 254, 110, 56, 158, 251, 103, 159, 125, 70, 9, 9, 192, 157, 185, 88, 45, 237, 209, 14, 238, 22, 11, 156, 42, 61, 172, 248, 175, 205, 29, 87, 44, 249, 69, 148, 194, 201, 88, 31, 36, 152, 175, 207, 120, 70, 231, 234, 239, 27, 120, 11, 188, 47, 126, 21, 187, 83, 61, 218, 253, 69, 150, 57, 242, 31, 129, 90, 78, 122, 162, 115, 193, 66, 231, 239, 171, 129, 241, 209, 248, 79, 198, 197, 15, 5, 20, 97, 114, 135, 82, 183, 39, 26, 12, 204, 167, 245, 131, 126, 229, 14, 152, 249, 66, 23, 145, 179, 201, 129, 84, 180, 47, 247, 191, 198, 219, 235, 14, 55, 141, 41, 250, 184, 135, 128, 115, 246, 220, 161, 224, 164, 119, 161, 199, 4, 69, 138, 127, 197, 71, 128, 167, 90, 212, 150, 2, 13, 143, 216, 191, 58, 228, 209, 93, 22, 58, 9, 137, 127, 203, 163, 74, 40, 217, 253, 186, 51, 153, 40, 52, 238, 73, 60, 57, 57, 108, 128, 2, 48, 134, 115, 108, 142, 65, 158, 113, 75, 25, 249, 189, 39, 130, 154, 24, 112, 209, 120, 229, 129, 223, 190, 147, 169, 232, 115, 83, 39, 47, 230, 201, 87, 216, 98, 161, 27, 95, 39, 237, 157, 102, 109, 222, 170, 149, 2, 3, 1, 0, 1, 163, 130, 1, 39, 48, 130, 1, 35, 48, 14, 6, 3, 85, 29, 15, 1, 1, 255, 4, 4, 3, 2, 5, 160, 48, 29, 6, 3, 85, 29, 37, 4, 22, 48, 20, 6, 8, 43, 6, 1, 5, 5, 7, 3, 2, 6, 8, 43, 6, 1, 5, 5, 7, 3, 1, 48, 12, 6, 3, 85, 29, 19, 1, 1, 255, 4, 2, 48, 0, 48, 73, 6, 3, 85, 29, 14, 4, 66, 4, 64, 43, 61, 184, 58, 190, 246, 117, 145, 150, 149, 52, 148, 50, 70, 171, 24, 140, 56, 164, 203, 105, 63, 65, 146, 183, 66, 114, 129, 170, 239, 140, 79, 13, 223, 192, 116, 206, 195, 234, 70, 154, 102, 164, 7, 77, 72, 8, 99, 67, 120, 61, 224, 48, 163, 84, 4, 250, 254, 27, 88, 242, 13, 99, 21, 48, 75, 6, 3, 85, 29, 35, 4, 68, 48, 66, 128, 64, 145, 115, 72, 248, 147, 3, 210, 10, 72, 90, 51, 130, 36, 255, 101, 123, 45, 151, 78, 158, 148, 254, 229, 16, 121, 180, 193, 131, 183, 127, 36, 88, 142, 33, 22, 114, 228, 61, 202, 29, 164, 17, 211, 221, 107, 244, 245, 206, 36, 16, 177, 96, 20, 74, 22, 17, 184, 132, 100, 119, 214, 222, 88, 253, 48, 76, 6, 3, 85, 29, 17, 4, 69, 48, 67, 130, 65, 42, 46, 97, 112, 112, 115, 46, 97, 114, 111, 46, 97, 50, 115, 57, 98, 56, 109, 50, 101, 52, 112, 55, 112, 48, 103, 46, 48, 109, 117, 56, 46, 119, 101, 115, 116, 117, 115, 51, 46, 97, 114, 111, 97, 112, 112, 45, 104, 99, 112, 46, 97, 122, 117, 114, 101, 45, 116, 101, 115, 116, 46, 110, 101, 116],
                        RawSubjectPublicKeyInfo: [48, 130, 1, 34, 48, 13, 6, 9, 42, 134, 72, 134, 247, 13, 1, 1, 1, 5, 0, 3, 130, 1, 15, 0, 48, 130, 1, 10, 2, 130, 1, 1, 0, 184, 6, 158, 64, 94, 24, 96, 40, 160, 154, 141, 196, 128, 87, 218, 59, 39, 71, 199, 234, 79, 240, 146, 24, 254, 106, 245, 45, 201, 136, 214, 64, 254, 110, 56, 158, 251, 103, 159, 125, 70, 9, 9, 192, 157, 185, 88, 45, 237, 209, 14, 238, 22, 11, 156, 42, 61, 172, 248, 175, 205, 29, 87, 44, 249, 69, 148, 194, 201, 88, 31, 36, 152, 175, 207, 120, 70, 231, 234, 239, 27, 120, 11, 188, 47, 126, 21, 187, 83, 61, 218, 253, 69, 150, 57, 242, 31, 129, 90, 78, 122, 162, 115, 193, 66, 231, 239, 171, 129, 241, 209, 248, 79, 198, 197, 15, 5, 20, 97, 114, 135, 82, 183, 39, 26, 12, 204, 167, 245, 131, 126, 229, 14, 152, 249, 66, 23, 145, 179, 201, 129, 84, 180, 47, 247, 191, 198, 219, 235, 14, 55, 141, 41, 250, 184, 135, 128, 115, 246, 220, 161, 224, 164, 119, 161, 199, 4, 69, 138, 127, 197, 71, 128, 167, 90, 212, 150, 2, 13, 143, 216, 191, 58, 228, 209, 93, 22, 58, 9, 137, 127, 203, 163, 74, 40, 217, 253, 186, 51, 153, 40, 52, 238, 73, 60, 57, 57, 108, 128, 2, 48, 134, 115, 108, 142, 65, 158, 113, 75, 25, 249, 189, 39, 130, 154, 24, 112, 209, 120, 229, 129, 223, 190, 147, 169, 232, 115, 83, 39, 47, 230, 201, 87, 216, 98, 161, 27, 95, 39, 237, 157, 102, 109, 222, 170, 149, 2, 3, 1, 0, 1],
                        RawSubject: [48, 48, 49, 18, 48, 16, 6, 3, 85, 4, 10, 19, 9, 111, 112, 101, 110, 115, 104, 105, 102, 116, 49, 26, 48, 24, 6, 3, 85, 4, 3, 19, 17, 111, 112, 101, 110, 115, 104, 105, 102, 116, 45, 105, 110, 103, 114, 101, 115, 115],
                        RawIssuer: [48, 38, 49, 18, 48, 16, 6, 3, 85, 4, 11, 19, 9, 111, 112, 101, 110, 115, 104, 105, 102, 116, 49, 16, 48, 14, 6, 3, 85, 4, 3, 19, 7, 114, 111, 111, 116, 45, 99, 97],
                        Signature: [144, 63, 1, 9, 63, 203, 104, 158, 72, 190, 218, 173, 218, 174, 171, 175, 249, 99, 198, 163, 154, 183, 80, 70, 18, 90, 143, 16, 110, 40, 74, 15, 152, 104, 155, 119, 3, 189, 92, 182, 181, 193, 112, 193, 179, 171, 177, 195, 218, 45, 246, 105, 209, 209, 4, 147, 233, 109, 252, 165, 52, 200, 179, 47, 64, 202, 66, 155, 170, 48, 184, 165, 51, 160, 106, 229, 210, 44, 72, 109, 204, 32, 9, 38, 222, 130, 253, 176, 6, 246, 50, 104, 240, 45, 124, 77, 233, 84, 164, 109, 154, 110, 171, 77, 164, 253, 134, 225, 58, 131, 116, 48, 78, 1, 85, 150, 81, 149, 67, 171, 102, 216, 126, 229, 90, 56, 26, 33, 171, 124, 161, 209, 140, 187, 61, 34, 212, 215, 17, 134, 62, 6, 13, 208, 221, 55, 5, 187, 22, 63, 93, 164, 76, 124, 166, 239, 104, 210, 84, 78, 146, 114, 146, 113, 215, 97, 33, 13, 186, 135, 110, 182, 194, 44, 64, 215, 36, 43, 178, 21, 170, 171, 251, 180, 100, 40, 239, 189, 24, 174, 192, 195, 98, 108, 222, 96, 135, 172, 56, 114, 81, 213, 239, 84, 199, 27, 118, 56, 193, 239, 243, 191, 178, 97, 106, 201, 66, 252, 31, 190, 254, 107, 226, 184, 19, 188, 113, 44, 99, 24, 245, 52, 154, 97, 249, 178, 221, 232, 203, 63, 51, 4, 109, 167, 92, 246, 143, 188, 32, 32, 87, 98, 186, 5, 69, 17],
                        SignatureAlgorithm: 4,
                        PublicKeyAlgorithm: 1,
                        PublicKey: <*rsa.PublicKey | 0x2b2e57843d70>{
                            N: {
                                neg: false,
                                abs: [2877128800215149205, 16629919685921545055, 13732506504615110447, 11103748926260740575, 11416989105297106818, 9223988340828442177, 2897202458616478060, 11766261893188694937, 15086238765284949963, 10809216967110507236, 9979912210168109780, 11664503868611822661, 3024932991847888604, 17852206095763650445, 1698336212534998063, 17691123283498957122, 9751057521779199143, 5748498642685419890, 4821085446021960184, 2270194980282856385, 5998191060056422898, 1979344939107882427, 11002240530182040303, 17961926253159653156, 4444200366252054316, 17136494472856312874, 5046575580329891885, 18333653388545728381, 18332734810168546880, 2830450700003086872, 11572718067588258363, 13260460152132755496],
                            },
                            E: 65537,
                        },
                        Version: 3,
                        SerialNumber: {
                            neg: false,
                            abs: [8959707475658302546],
                        },
                        Issuer: {
                            Country: nil,
                            Organization: nil,
                            OrganizationalUnit: ["openshift"],
                            Locality: nil,
                            Province: nil,
                            StreetAddress: nil,
                            PostalCode: nil,
                            SerialNumber: "",
                            CommonName: "root-ca",
                            Names: [
                                {
                                    Type: [2, 5, 4, 11],
                                    Value: <string>"openshift",
                                },
                                {
                                    Type: [2, 5, 4, 3],
                                    Value: <string>"root-ca",
                                },
                            ],
                            ExtraNames: nil,
                        },
                        Subject: {
                            Country: nil,
                            Organization: ["openshift"],
                            OrganizationalUnit: nil,
                            Locality: nil,
                            Province: nil,
                            StreetAddress: nil,
                            PostalCode: nil,
                            SerialNumber: "",
                            CommonName: "openshift-ingress",
                            Names: [
                                {
                                    Type: [2, 5, 4, 10],
                                    Value: <string>"openshift",
                                },
                                {
                                    Type: [2, 5, 4, 3],
                                    Value: <string>"openshift-ingress",
                                },
                            ],
                            ExtraNames: nil,
                        },
                        NotBefore: {wall: 0, ext: 63928472888, loc: nil},
                        NotAfter: {wall: 0, ext: 63960008888, loc: nil},
                        KeyUsage: 5,
                        Extensions: [
                            {
                                Id: [2, 5, 29, 15],
                                Critical: true,
                                Value: [3, 2, 5, 160],
                            },
                            {
                                Id: [2, 5, 29, 37],
                                Critical: false,
                                Value: [48, 20, 6, 8, 43, 6, 1, 5, 5, 7, 3, 2, 6, 8, 43, 6, 1, 5, 5, 7, 3, 1],
                            },
                            {Id: [2, 5, 29, 19], Critical: true, Value: [48, 0]},
                            {
                                Id: [2, 5, 29, 14],
                                Critical: false,
                                Value: [4, 64, 43, 61, 184, 58, 190, 246, 117, 145, 150, 149, 52, 148, 50, 70, 171, 24, 140, 56, 164, 203, 105, 63, 65, 146, 183, 66, 114, 129, 170, 239, 140, 79, 13, 223, 192, 116, 206, 195, 234, 70, 154, 102, 164, 7, 77, 72, 8, 99, 67, 120, 61, 224, 48, 163, 84, 4, 250, 254, 27, 88, 242, 13, 99, 21],
                            },
                            {
                                Id: [2, 5, 29, 35],
                                Critical: false,
                                Value: [48, 66, 128, 64, 145, 115, 72, 248, 147, 3, 210, 10, 72, 90, 51, 130, 36, 255, 101, 123, 45, 151, 78, 158, 148, 254, 229, 16, 121, 180, 193, 131, 183, 127, 36, 88, 142, 33, 22, 114, 228, 61, 202, 29, 164, 17, 211, 221, 107, 244, 245, 206, 36, 16, 177, 96, 20, 74, 22, 17, 184, 132, 100, 119, 214, 222, 88, 253],
                            },
                            {
                                Id: [2, 5, 29, 17],
                                Critical: false,
                                Value: "0C\x82A*.apps.aro.a2s9b8m2e4p7p0g.0mu8.westus3.aroapp-hcp.azure-test.net",
                            },
                        ],
                        ExtraExtensions: nil,
                        UnhandledCriticalExtensions: nil,
                        ExtKeyUsage: [2, 1],
                        UnknownExtKeyUsage: nil,
                        BasicConstraintsValid: true,
                        IsCA: false,
                        MaxPathLen: -1,
                        MaxPathLenZero: false,
                        SubjectKeyId: [43, 61, 184, 58, 190, 246, 117, 145, 150, 149, 52, 148, 50, 70, 171, 24, 140, 56, 164, 203, 105, 63, 65, 146, 183, 66, 114, 129, 170, 239, 140, 79, 13, 223, 192, 116, 206, 195, 234, 70, 154, 102, 164, 7, 77, 72, 8, 99, 67, 120, 61, 224, 48, 163, 84, 4, 250, 254, 27, 88, 242, 13, 99, 21],
                        AuthorityKeyId: [145, 115, 72, 248, 147, 3, 210, 10, 72, 90, 51, 130, 36, 255, 101, 123, 45, 151, 78, 158, 148, 254, 229, 16, 121, 180, 193, 131, 183, 127, 36, 88, 142, 33, 22, 114, 228, 61, 202, 29, 164, 17, 211, 221, 107, 244, 245, 206, 36, 16, 177, 96, 20, 74, 22, 17, 184, 132, 100, 119, 214, 222, 88, 253],
                        OCSPServer: nil,
                        IssuingCertificateURL: nil,
                        DNSNames: [
                            "*.apps.aro.a2s9b8m2e4p7p0g.0mu8.westus3.aroapp-hcp.azure-test.net",
                        ],
                        EmailAddresses: nil,
                        IPAddresses: nil,
                        URIs: nil,
                        PermittedDNSDomainsCritical: false,
                        PermittedDNSDomains: nil,
                        ExcludedDNSDomains: nil,
                        PermittedIPRanges: nil,
                        ExcludedIPRanges: nil,
                        PermittedEmailAddresses: nil,
                        ExcludedEmailAddresses: nil,
                        PermittedURIDomains: nil,
                        ExcludedURIDomains: nil,
                        CRLDistributionPoints: nil,
                        PolicyIdentifiers: nil,
                        Policies: nil,
                        InhibitAnyPolicy: 0,
                        InhibitAnyPolicyZero: false,
                        InhibitPolicyMapping: 0,
                        InhibitPolicyMappingZero: false,
                        RequireExplicitPolicy: 0,
                        RequireExplicitPolicyZero: false,
                        PolicyMappings: nil,
                    },
                ],
                Err: <x509.UnknownAuthorityError>{
                    Cert: 0x2b2e55ba2008 (cyclic reference),
                    hintErr: nil,
                    hintCert: nil,
                },
            },
        },
    }
occurred
1h 40m 47s branch-ci-Azure-ARO-HCP-main-e2e-integration-e2e-parallel-all
post-goodmerged PR
No additional run metadata captured
westus3 unknown failed n/a 4
unknown (4)
ERROR CODE: DeadlineExceeded; detail message cluster creation did not complete before the deadline; [hypershiftHostedCluster] hosted cluster control plane version not yet completed: version <version> is Partial (want Completed), started <duration> ago; hosted cluster degraded: UnavailableReplicas: [router]; provider Microsoft.RedHatOpenShift
16:40:04 UTC · failed at unknown · test Customer should be able to create a cluster and node pools with aggregated advanced features · suite integration/parallel/all
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_feature_aggregation.go:208]: failed to create HCP cluster "agg-cluster" with aggregated settings
Unexpected error:
    <*fmt.wrapError | 0x9134b719420>: 
    failed waiting for cluster="agg-cluster" in resourcegroup="feature-aggregation-c2zb7x4tvbjz" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westus3/hcpOperationStatuses/2d4481c2-5c8d-4e2e-a526-9b051ca6139f
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: DeadlineExceeded
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westus3/hcpOperationStatuses/2d4481c2-5c8d-4e2e-a526-9b051ca6139f",
      "name": "2d4481c2-5c8d-4e2e-a526-9b051ca6139f",
      "status": "Failed",
      "startTime": "2026-10-24T17:16:34.851278228Z",
      "endTime": "2026-10-24T17:45:35.221810696Z",
      "error": {
        "code": "DeadlineExceeded",
        "message": "cluster creation did not complete before the deadline; [hypershiftHostedCluster] hosted cluster control plane version not yet completed: version 4.21.39 is Partial (want Completed), started 22m57s ago; hosted cluster degraded: UnavailableReplicas: router deployment has 1 unavailable replicas"
      }
    }
    --------------------------------------------------------------------------------
    
    <Azure error internals omitted>
        Azure request IDs:
          GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westus3/hcpOperationStatuses/2d4481c2-5c8d-4e2e-a526-9b051ca6139f
            correlation ID: 4e6d7a8e-ea3e-4773-abae-818d5133da86
            client request ID: unavailable
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_feature_aggregation.go:208]: failed to create HCP cluster "agg-cluster" with aggregated settings
Unexpected error:
    <*fmt.wrapError | 0x9134b719420>: 
    failed waiting for cluster="agg-cluster" in resourcegroup="feature-aggregation-c2zb7x4tvbjz" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westus3/hcpOperationStatuses/2d4481c2-5c8d-4e2e-a526-9b051ca6139f
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: DeadlineExceeded
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westus3/hcpOperationStatuses/2d4481c2-5c8d-4e2e-a526-9b051ca6139f",
      "name": "2d4481c2-5c8d-4e2e-a526-9b051ca6139f",
      "status": "Failed",
      "startTime": "2026-10-24T17:16:34.851278228Z",
      "endTime": "2026-10-24T17:45:35.221810696Z",
      "error": {
        "code": "DeadlineExceeded",
        "message": "cluster creation did not complete before the deadline; [hypershiftHostedCluster] hosted cluster control plane version not yet completed: version 4.21.39 is Partial (want Completed), started 22m57s ago; hosted cluster degraded: UnavailableReplicas: router deployment has 1 unavailable replicas"
      }
    }
    --------------------------------------------------------------------------------
    
    <Azure error internals omitted>
        Azure request IDs:
          GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westus3/hcpOperationStatuses/2d4481c2-5c8d-4e2e-a526-9b051ca6139f
            correlation ID: 4e6d7a8e-ea3e-4773-abae-818d5133da86
            client request ID: unavailable
occurred
tls: failed to verify certificate: x509: certificate signed by unknown authority
16:40:04 UTC · failed at unknown · test ARO-HCP should be able to perform a control plane and node pool install with OCP candidate channel for 4.23 · suite integration/parallel/all
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:194]: failed to verify simple web app runs on cluster "cluster-candidate-4-23-wk4gjj"
Unexpected error:
    <*fmt.wrapError | 0x30fb7a9c6d80>: 
    [strict TLS verification] route was never reachable: Get "https://agnhost-e2e-sample-app-25lgf.apps.aro.c9p0v7j9q7j4a1q.afld.westus3.aroapp-hcp.azure-test.net": tls: failed to verify certificate: x509: certificate signed by unknown authority
    {
        msg: "[strict TLS verification] route was never reachable: Get \"https://agnhost-e2e-sample-app-25lgf.apps.aro.c9p0v7j9q7j4a1q.afld.westus3.aroapp-hcp.azure-test.net\": tls: failed to verify certificate: x509: certificate signed by unknown authority",
        err: <*url.Error | 0x30fb7cb9b320>{
            Op: "Get",
            URL: "https://agnhost-e2e-sample-app-25lgf.apps.aro.c9p0v7j9q7j4a1q.afld.westus3.aroapp-hcp.azure-test.net",
            Err: <*tls.CertificateVerificationError | 0x30fb7cb9b2f0>{
                UnverifiedCertificates: [
                    {
                        Raw: [48, 130, 4, 1, 48, 130, 2, 233, 160, 3, 2, 1, 2, 2, 8, 110, 83, 80, 98, 163, 186, 156, 29, 48, 13, 6, 9, 42, 134, 72, 134, 247, 13, 1, 1, 11, 5, 0, 48, 38, 49, 18, 48, 16, 6, 3, 85, 4, 11, 19, 9, 111, 112, 101, 110, 115, 104, 105, 102, 116, 49, 16, 48, 14, 6, 3, 85, 4, 3, 19, 7, 114, 111, 111, 116, 45, 99, 97, 48, 30, 23, 13, 50, 54, 49, 48, 50, 52, 49, 55, 50, 48, 48, 48, 90, 23, 13, 50, 55, 49, 48, 50, 52, 49, 55, 50, 48, 48, 48, 90, 48, 48, 49, 18, 48, 16, 6, 3, 85, 4, 10, 19, 9, 111, 112, 101, 110, 115, 104, 105, 102, 116, 49, 26, 48, 24, 6, 3, 85, 4, 3, 19, 17, 111, 112, 101, 110, 115, 104, 105, 102, 116, 45, 105, 110, 103, 114, 101, 115, 115, 48, 130, 1, 34, 48, 13, 6, 9, 42, 134, 72, 134, 247, 13, 1, 1, 1, 5, 0, 3, 130, 1, 15, 0, 48, 130, 1, 10, 2, 130, 1, 1, 0, 175, 180, 66, 13, 148, 254, 185, 112, 159, 180, 48, 58, 217, 14, 123, 23, 229, 28, 224, 241, 179, 210, 115, 71, 123, 94, 152, 56, 200, 179, 120, 80, 246, 68, 78, 194, 165, 17, 161, 64, 168, 40, 31, 243, 155, 207, 171, 17, 238, 179, 159, 199, 90, 32, 61, 243, 133, 233, 153, 229, 126, 69, 220, 234, 188, 36, 185, 6, 114, 77, 215, 59, 104, 34, 80, 95, 66, 69, 104, 166, 84, 17, 159, 234, 101, 109, 164, 153, 172, 156, 240, 134, 16, 200, 61, 179, 79, 164, 99, 227, 229, 46, 174, 192, 70, 194, 161, 207, 5, 78, 18, 21, 247, 101, 35, 19, 102, 159, 29, 233, 182, 86, 199, 53, 244, 175, 208, 228, 116, 49, 45, 21, 118, 178, 72, 104, 19, 188, 1, 131, 254, 127, 37, 139, 95, 60, 129, 254, 7, 18, 184, 51, 199, 217, 71, 107, 203, 113, 33, 123, 83, 246, 30, 234, 39, 214, 99, 207, 167, 116, 247, 66, 241, 15, 133, 22, 9, 238, 99, 105, 166, 167, 72, 142, 182, 229, 22, 51, 127, 84, 32, 214, 8, 185, 111, 104, 100, 152, 181, 219, 81, 90, 133, 133, 127, 114, 2, 134, 80, 248, 53, 178, 96, 3, 247, 147, 225, 186, 125, 75, 198, 237, 177, 202, 247, 71, 49, 31, 123, 47, 222, 47, 12, 75, 219, 147, 5, 27, 92, 161, 76, 204, 58, 211, 26, 120, 156, 19, 51, 136, 52, 116, 18, 166, 216, 205, 2, 3, 1, 0, 1, 163, 130, 1, 39, 48, 130, 1, 35, 48, 14, 6, 3, 85, 29, 15, 1, 1, 255, 4, 4, 3, 2, 5, 160, 48, 29, 6, 3, 85, 29, 37, 4, 22, 48, 20, 6, 8, 43, 6, 1, 5, 5, 7, 3, 2, 6, 8, 43, 6, 1, 5, 5, 7, 3, 1, 48, 12, 6, 3, 85, 29, 19, 1, 1, 255, 4, 2, 48, 0, 48, 73, 6, 3, 85, 29, 14, 4, 66, 4, 64, 184, 27, 111, 100, 185, 3, 60, 74, 232, 82, 187, 169, 146, 147, 185, 95, 74, 1, 30, 137, 225, 41, 223, 249, 128, 141, 254, 150, 25, 225, 242, 3, 235, 207, 147, 83, 107, 148, 237, 0, 226, 240, 42, 175, 47, 113, 123, 148, 128, 190, 12, 146, 119, 216, 235, 223, 47, 29, 124, 171, 8, 133, 245, 128, 48, 75, 6, 3, 85, 29, 35, 4, 68, 48, 66, 128, 64, 107, 35, 93, 140, 120, 74, 68, 242, 166, 39, 191, 159, 156, 255, 152, 139, 116, 160, 21, 220, 61, 249, 246, 136, 64, 148, 57, 252, 107, 221, 102, 126, 167, 207, 36, 223, 46, 112, 92, 229, 162, 35, 131, 186, 43, 139, 91, 12, 221, 184, 234, 124, 29, 20, 52, 38, 121, 7, 144, 253, 45, 84, 166, 41, 48, 76, 6, 3, 85, 29, 17, 4, 69, 48, 67, 130, 65, 42, 46, 97, 112, 112, 115, 46, 97, 114, 111, 46, 99, 57, 112, 48, 118, 55, 106, 57, 113, 55, 106, 52, 97, 49, 113, 46, 97, 102, 108, 100, 46, 119, 101, 115, 116, 117, 115, 51, 46, 97, 114, 111, 97, 112, 112, 45, 104, 99, 112, 46, 97, 122, 117, 114, 101, 45, 116, 101, 115, 116, 46, 110, 101, 116, 48, 13, 6, 9, 42, 134, 72, 134, 247, 13, 1, 1, 11, 5, 0, 3, 130, 1, 1, 0, 62, 37, 28, 104, 244, 85, 128, 159, 33, 132, 56, 114, 149, 155, 76, 150, 56, 190, 182, 145, 23, 80, 230, 251, 69, 191, 95, 185, 206, 12, 203, 223, 159, 176, 108, 31, 228, 30, 29, 147, 188, 224, 159, 222, 209, 80, 26, 111, 48, 5, 218, 228, 214, 243, 4, 174, 170, 173, 102, 5, 47, 112, 16, 26, 249, 76, 134, 179, 187, 39, 158, 241, 110, 27, 4, 56, 253, 35, 167, 85, 21, 75, 203, 60, 52, 83, 13, 153, 94, 185, 69, 153, 196, 55, 251, 226, 149, 80, 157, 57, 244, 94, 195, 179, 137, 174, 166, 85, 12, 161, 202, 212, 188, 14, 213, 25, 59, 74, 197, 148, 98, 52, 206, 23, 41, 197, 32, 214, 165, 60, 202, 205, 67, 25, 42, 38, 206, 141, 201, 237, 101, 166, 209, 131, 101, 208, 225, 123, 244, 235, 82, 50, 143, 145, 70, 217, 191, 27, 3, 130, 194, 10, 227, 129, 5, 165, 150, 45, 143, 154, 59, 167, 145, 196, 78, 140, 76, 209, 96, 166, 23, 54, 227, 85, 65, 210, 246, 64, 194, 155, 249, 210, 255, 119, 224, 229, 103, 89, 68, 18, 164, 119, 9, 207, 73, 245, 219, 68, 214, 3, 155, 42, 94, 212, 192, 93, 148, 231, 125, 114, 77, 118, 234, 177, 58, 169, 209, 85, 145, 75, 243, 82, 234, 209, 51, 198, 131, 119, 63, 14, 40, 242, 219, 78, 215, 35, 237, 91, 234, 2, 142, 86, 121, 121, 249, 150],
                        RawTBSCertificate: [48, 130, 2, 233, 160, 3, 2, 1, 2, 2, 8, 110, 83, 80, 98, 163, 186, 156, 29, 48, 13, 6, 9, 42, 134, 72, 134, 247, 13, 1, 1, 11, 5, 0, 48, 38, 49, 18, 48, 16, 6, 3, 85, 4, 11, 19, 9, 111, 112, 101, 110, 115, 104, 105, 102, 116, 49, 16, 48, 14, 6, 3, 85, 4, 3, 19, 7, 114, 111, 111, 116, 45, 99, 97, 48, 30, 23, 13, 50, 54, 49, 48, 50, 52, 49, 55, 50, 48, 48, 48, 90, 23, 13, 50, 55, 49, 48, 50, 52, 49, 55, 50, 48, 48, 48, 90, 48, 48, 49, 18, 48, 16, 6, 3, 85, 4, 10, 19, 9, 111, 112, 101, 110, 115, 104, 105, 102, 116, 49, 26, 48, 24, 6, 3, 85, 4, 3, 19, 17, 111, 112, 101, 110, 115, 104, 105, 102, 116, 45, 105, 110, 103, 114, 101, 115, 115, 48, 130, 1, 34, 48, 13, 6, 9, 42, 134, 72, 134, 247, 13, 1, 1, 1, 5, 0, 3, 130, 1, 15, 0, 48, 130, 1, 10, 2, 130, 1, 1, 0, 175, 180, 66, 13, 148, 254, 185, 112, 159, 180, 48, 58, 217, 14, 123, 23, 229, 28, 224, 241, 179, 210, 115, 71, 123, 94, 152, 56, 200, 179, 120, 80, 246, 68, 78, 194, 165, 17, 161, 64, 168, 40, 31, 243, 155, 207, 171, 17, 238, 179, 159, 199, 90, 32, 61, 243, 133, 233, 153, 229, 126, 69, 220, 234, 188, 36, 185, 6, 114, 77, 215, 59, 104, 34, 80, 95, 66, 69, 104, 166, 84, 17, 159, 234, 101, 109, 164, 153, 172, 156, 240, 134, 16, 200, 61, 179, 79, 164, 99, 227, 229, 46, 174, 192, 70, 194, 161, 207, 5, 78, 18, 21, 247, 101, 35, 19, 102, 159, 29, 233, 182, 86, 199, 53, 244, 175, 208, 228, 116, 49, 45, 21, 118, 178, 72, 104, 19, 188, 1, 131, 254, 127, 37, 139, 95, 60, 129, 254, 7, 18, 184, 51, 199, 217, 71, 107, 203, 113, 33, 123, 83, 246, 30, 234, 39, 214, 99, 207, 167, 116, 247, 66, 241, 15, 133, 22, 9, 238, 99, 105, 166, 167, 72, 142, 182, 229, 22, 51, 127, 84, 32, 214, 8, 185, 111, 104, 100, 152, 181, 219, 81, 90, 133, 133, 127, 114, 2, 134, 80, 248, 53, 178, 96, 3, 247, 147, 225, 186, 125, 75, 198, 237, 177, 202, 247, 71, 49, 31, 123, 47, 222, 47, 12, 75, 219, 147, 5, 27, 92, 161, 76, 204, 58, 211, 26, 120, 156, 19, 51, 136, 52, 116, 18, 166, 216, 205, 2, 3, 1, 0, 1, 163, 130, 1, 39, 48, 130, 1, 35, 48, 14, 6, 3, 85, 29, 15, 1, 1, 255, 4, 4, 3, 2, 5, 160, 48, 29, 6, 3, 85, 29, 37, 4, 22, 48, 20, 6, 8, 43, 6, 1, 5, 5, 7, 3, 2, 6, 8, 43, 6, 1, 5, 5, 7, 3, 1, 48, 12, 6, 3, 85, 29, 19, 1, 1, 255, 4, 2, 48, 0, 48, 73, 6, 3, 85, 29, 14, 4, 66, 4, 64, 184, 27, 111, 100, 185, 3, 60, 74, 232, 82, 187, 169, 146, 147, 185, 95, 74, 1, 30, 137, 225, 41, 223, 249, 128, 141, 254, 150, 25, 225, 242, 3, 235, 207, 147, 83, 107, 148, 237, 0, 226, 240, 42, 175, 47, 113, 123, 148, 128, 190, 12, 146, 119, 216, 235, 223, 47, 29, 124, 171, 8, 133, 245, 128, 48, 75, 6, 3, 85, 29, 35, 4, 68, 48, 66, 128, 64, 107, 35, 93, 140, 120, 74, 68, 242, 166, 39, 191, 159, 156, 255, 152, 139, 116, 160, 21, 220, 61, 249, 246, 136, 64, 148, 57, 252, 107, 221, 102, 126, 167, 207, 36, 223, 46, 112, 92, 229, 162, 35, 131, 186, 43, 139, 91, 12, 221, 184, 234, 124, 29, 20, 52, 38, 121, 7, 144, 253, 45, 84, 166, 41, 48, 76, 6, 3, 85, 29, 17, 4, 69, 48, 67, 130, 65, 42, 46, 97, 112, 112, 115, 46, 97, 114, 111, 46, 99, 57, 112, 48, 118, 55, 106, 57, 113, 55, 106, 52, 97, 49, 113, 46, 97, 102, 108, 100, 46, 119, 101, 115, 116, 117, 115, 51, 46, 97, 114, 111, 97, 112, 112, 45, 104, 99, 112, 46, 97, 122, 117, 114, 101, 45, 116, 101, 115, 116, 46, 110, 101, 116],
                        RawSubjectPublicKeyInfo: [48, 130, 1, 34, 48, 13, 6, 9, 42, 134, 72, 134, 247, 13, 1, 1, 1, 5, 0, 3, 130, 1, 15, 0, 48, 130, 1, 10, 2, 130, 1, 1, 0, 175, 180, 66, 13, 148, 254, 185, 112, 159, 180, 48, 58, 217, 14, 123, 23, 229, 28, 224, 241, 179, 210, 115, 71, 123, 94, 152, 56, 200, 179, 120, 80, 246, 68, 78, 194, 165, 17, 161, 64, 168, 40, 31, 243, 155, 207, 171, 17, 238, 179, 159, 199, 90, 32, 61, 243, 133, 233, 153, 229, 126, 69, 220, 234, 188, 36, 185, 6, 114, 77, 215, 59, 104, 34, 80, 95, 66, 69, 104, 166, 84, 17, 159, 234, 101, 109, 164, 153, 172, 156, 240, 134, 16, 200, 61, 179, 79, 164, 99, 227, 229, 46, 174, 192, 70, 194, 161, 207, 5, 78, 18, 21, 247, 101, 35, 19, 102, 159, 29, 233, 182, 86, 199, 53, 244, 175, 208, 228, 116, 49, 45, 21, 118, 178, 72, 104, 19, 188, 1, 131, 254, 127, 37, 139, 95, 60, 129, 254, 7, 18, 184, 51, 199, 217, 71, 107, 203, 113, 33, 123, 83, 246, 30, 234, 39, 214, 99, 207, 167, 116, 247, 66, 241, 15, 133, 22, 9, 238, 99, 105, 166, 167, 72, 142, 182, 229, 22, 51, 127, 84, 32, 214, 8, 185, 111, 104, 100, 152, 181, 219, 81, 90, 133, 133, 127, 114, 2, 134, 80, 248, 53, 178, 96, 3, 247, 147, 225, 186, 125, 75, 198, 237, 177, 202, 247, 71, 49, 31, 123, 47, 222, 47, 12, 75, 219, 147, 5, 27, 92, 161, 76, 204, 58, 211, 26, 120, 156, 19, 51, 136, 52, 116, 18, 166, 216, 205, 2, 3, 1, 0, 1],
                        RawSubject: [48, 48, 49, 18, 48, 16, 6, 3, 85, 4, 10, 19, 9, 111, 112, 101, 110, 115, 104, 105, 102, 116, 49, 26, 48, 24, 6, 3, 85, 4, 3, 19, 17, 111, 112, 101, 110, 115, 104, 105, 102, 116, 45, 105, 110, 103, 114, 101, 115, 115],
                        RawIssuer: [48, 38, 49, 18, 48, 16, 6, 3, 85, 4, 11, 19, 9, 111, 112, 101, 110, 115, 104, 105, 102, 116, 49, 16, 48, 14, 6, 3, 85, 4, 3, 19, 7, 114, 111, 111, 116, 45, 99, 97],
                        Signature: [62, 37, 28, 104, 244, 85, 128, 159, 33, 132, 56, 114, 149, 155, 76, 150, 56, 190, 182, 145, 23, 80, 230, 251, 69, 191, 95, 185, 206, 12, 203, 223, 159, 176, 108, 31, 228, 30, 29, 147, 188, 224, 159, 222, 209, 80, 26, 111, 48, 5, 218, 228, 214, 243, 4, 174, 170, 173, 102, 5, 47, 112, 16, 26, 249, 76, 134, 179, 187, 39, 158, 241, 110, 27, 4, 56, 253, 35, 167, 85, 21, 75, 203, 60, 52, 83, 13, 153, 94, 185, 69, 153, 196, 55, 251, 226, 149, 80, 157, 57, 244, 94, 195, 179, 137, 174, 166, 85, 12, 161, 202, 212, 188, 14, 213, 25, 59, 74, 197, 148, 98, 52, 206, 23, 41, 197, 32, 214, 165, 60, 202, 205, 67, 25, 42, 38, 206, 141, 201, 237, 101, 166, 209, 131, 101, 208, 225, 123, 244, 235, 82, 50, 143, 145, 70, 217, 191, 27, 3, 130, 194, 10, 227, 129, 5, 165, 150, 45, 143, 154, 59, 167, 145, 196, 78, 140, 76, 209, 96, 166, 23, 54, 227, 85, 65, 210, 246, 64, 194, 155, 249, 210, 255, 119, 224, 229, 103, 89, 68, 18, 164, 119, 9, 207, 73, 245, 219, 68, 214, 3, 155, 42, 94, 212, 192, 93, 148, 231, 125, 114, 77, 118, 234, 177, 58, 169, 209, 85, 145, 75, 243, 82, 234, 209, 51, 198, 131, 119, 63, 14, 40, 242, 219, 78, 215, 35, 237, 91, 234, 2, 142, 86, 121, 121, 249, 150],
                        SignatureAlgorithm: 4,
                        PublicKeyAlgorithm: 1,
                        PublicKey: <*rsa.PublicKey | 0x30fb7a9895b0>{
                            N: {
                                neg: false,
                                abs: [3713275565900749005, 5533862720488119315, 886043176200789153, 17818264461995138607, 16265450768615453130, 5834472357489801107, 5862144673391444614, 628656117149906395, 13178964295004987606, 715618696217184398, 12066541168516629782, 6050057140458382287, 14400619811794133371, 6862502860246136883, 1422013248764323211, 8372522752467421288, 13138907997452357860, 17826693266249031145, 5098815638656127509, 5738821655630622400, 12438080729488309683, 6057798802862875801, 7503648299216627878, 13557164215630681915, 9649412887563066602, 17200267080417426931, 12116969928591911697, 17745395029646614848, 8889710084128733264, 16509317662742049607, 11507875977146301207, 12660817078547036528],
                            },
                            E: 65537,
                        },
                        Version: 3,
                        SerialNumber: {
                            neg: false,
                            abs: [7949786151822990365],
                        },
                        Issuer: {
                            Country: nil,
                            Organization: nil,
                            OrganizationalUnit: ["openshift"],
                            Locality: nil,
                            Province: nil,
                            StreetAddress: nil,
                            PostalCode: nil,
                            SerialNumber: "",
                            CommonName: "root-ca",
                            Names: [
                                {
                                    Type: [2, 5, 4, 11],
                                    Value: <string>"openshift",
                                },
                                {
                                    Type: [2, 5, 4, 3],
                                    Value: <string>"root-ca",
                                },
                            ],
                            ExtraNames: nil,
                        },
                        Subject: {
                            Country: nil,
                            Organization: ["openshift"],
                            OrganizationalUnit: nil,
                            Locality: nil,
                            Province: nil,
                            StreetAddress: nil,
                            PostalCode: nil,
                            SerialNumber: "",
                            CommonName: "openshift-ingress",
                            Names: [
                                {
                                    Type: [2, 5, 4, 10],
                                    Value: <string>"openshift",
                                },
                                {
                                    Type: [2, 5, 4, 3],
                                    Value: <string>"openshift-ingress",
                                },
                            ],
                            ExtraNames: nil,
                        },
                        NotBefore: {wall: 0, ext: 63928459200, loc: nil},
                        NotAfter: {wall: 0, ext: 63959995200, loc: nil},
                        KeyUsage: 5,
                        Extensions: [
                            {
                                Id: [2, 5, 29, 15],
                                Critical: true,
                                Value: [3, 2, 5, 160],
                            },
                            {
                                Id: [2, 5, 29, 37],
                                Critical: false,
                                Value: [48, 20, 6, 8, 43, 6, 1, 5, 5, 7, 3, 2, 6, 8, 43, 6, 1, 5, 5, 7, 3, 1],
                            },
                            {Id: [2, 5, 29, 19], Critical: true, Value: [48, 0]},
                            {
                                Id: [2, 5, 29, 14],
                                Critical: false,
                                Value: [4, 64, 184, 27, 111, 100, 185, 3, 60, 74, 232, 82, 187, 169, 146, 147, 185, 95, 74, 1, 30, 137, 225, 41, 223, 249, 128, 141, 254, 150, 25, 225, 242, 3, 235, 207, 147, 83, 107, 148, 237, 0, 226, 240, 42, 175, 47, 113, 123, 148, 128, 190, 12, 146, 119, 216, 235, 223, 47, 29, 124, 171, 8, 133, 245, 128],
                            },
                            {
                                Id: [2, 5, 29, 35],
                                Critical: false,
                                Value: [48, 66, 128, 64, 107, 35, 93, 140, 120, 74, 68, 242, 166, 39, 191, 159, 156, 255, 152, 139, 116, 160, 21, 220, 61, 249, 246, 136, 64, 148, 57, 252, 107, 221, 102, 126, 167, 207, 36, 223, 46, 112, 92, 229, 162, 35, 131, 186, 43, 139, 91, 12, 221, 184, 234, 124, 29, 20, 52, 38, 121, 7, 144, 253, 45, 84, 166, 41],
                            },
                            {
                                Id: [2, 5, 29, 17],
                                Critical: false,
                                Value: "0C\x82A*.apps.aro.c9p0v7j9q7j4a1q.afld.westus3.aroapp-hcp.azure-test.net",
                            },
                        ],
                        ExtraExtensions: nil,
                        UnhandledCriticalExtensions: nil,
                        ExtKeyUsage: [2, 1],
                        UnknownExtKeyUsage: nil,
                        BasicConstraintsValid: true,
                        IsCA: false,
                        MaxPathLen: -1,
                        MaxPathLenZero: false,
                        SubjectKeyId: [184, 27, 111, 100, 185, 3, 60, 74, 232, 82, 187, 169, 146, 147, 185, 95, 74, 1, 30, 137, 225, 41, 223, 249, 128, 141, 254, 150, 25, 225, 242, 3, 235, 207, 147, 83, 107, 148, 237, 0, 226, 240, 42, 175, 47, 113, 123, 148, 128, 190, 12, 146, 119, 216, 235, 223, 47, 29, 124, 171, 8, 133, 245, 128],
                        AuthorityKeyId: [107, 35, 93, 140, 120, 74, 68, 242, 166, 39, 191, 159, 156, 255, 152, 139, 116, 160, 21, 220, 61, 249, 246, 136, 64, 148, 57, 252, 107, 221, 102, 126, 167, 207, 36, 223, 46, 112, 92, 229, 162, 35, 131, 186, 43, 139, 91, 12, 221, 184, 234, 124, 29, 20, 52, 38, 121, 7, 144, 253, 45, 84, 166, 41],
                        OCSPServer: nil,
                        IssuingCertificateURL: nil,
                        DNSNames: [
                            "*.apps.aro.c9p0v7j9q7j4a1q.afld.westus3.aroapp-hcp.azure-test.net",
                        ],
                        EmailAddresses: nil,
                        IPAddresses: nil,
                        URIs: nil,
                        PermittedDNSDomainsCritical: false,
                        PermittedDNSDomains: nil,
                        ExcludedDNSDomains: nil,
                        PermittedIPRanges: nil,
                        ExcludedIPRanges: nil,
                        PermittedEmailAddresses: nil,
                        ExcludedEmailAddresses: nil,
                        PermittedURIDomains: nil,
                        ExcludedURIDomains: nil,
                        CRLDistributionPoints: nil,
                        PolicyIdentifiers: nil,
                        Policies: nil,
                        InhibitAnyPolicy: 0,
                        InhibitAnyPolicyZero: false,
                        InhibitPolicyMapping: 0,
                        InhibitPolicyMappingZero: false,
                        RequireExplicitPolicy: 0,
                        RequireExplicitPolicyZero: false,
                        PolicyMappings: nil,
                    },
                ],
                Err: <x509.UnknownAuthorityError>{
                    Cert: 0x30fb7a8f2c08 (cyclic reference),
                    hintErr: nil,
                    hintCert: nil,
                },
            },
        },
    }
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:194]: failed to verify simple web app runs on cluster "cluster-candidate-4-23-wk4gjj"
Unexpected error:
    <*fmt.wrapError | 0x30fb7a9c6d80>: 
    [strict TLS verification] route was never reachable: Get "https://agnhost-e2e-sample-app-25lgf.apps.aro.c9p0v7j9q7j4a1q.afld.westus3.aroapp-hcp.azure-test.net": tls: failed to verify certificate: x509: certificate signed by unknown authority
    {
        msg: "[strict TLS verification] route was never reachable: Get \"https://agnhost-e2e-sample-app-25lgf.apps.aro.c9p0v7j9q7j4a1q.afld.westus3.aroapp-hcp.azure-test.net\": tls: failed to verify certificate: x509: certificate signed by unknown authority",
        err: <*url.Error | 0x30fb7cb9b320>{
            Op: "Get",
            URL: "https://agnhost-e2e-sample-app-25lgf.apps.aro.c9p0v7j9q7j4a1q.afld.westus3.aroapp-hcp.azure-test.net",
            Err: <*tls.CertificateVerificationError | 0x30fb7cb9b2f0>{
                UnverifiedCertificates: [
                    {
                        Raw: [48, 130, 4, 1, 48, 130, 2, 233, 160, 3, 2, 1, 2, 2, 8, 110, 83, 80, 98, 163, 186, 156, 29, 48, 13, 6, 9, 42, 134, 72, 134, 247, 13, 1, 1, 11, 5, 0, 48, 38, 49, 18, 48, 16, 6, 3, 85, 4, 11, 19, 9, 111, 112, 101, 110, 115, 104, 105, 102, 116, 49, 16, 48, 14, 6, 3, 85, 4, 3, 19, 7, 114, 111, 111, 116, 45, 99, 97, 48, 30, 23, 13, 50, 54, 49, 48, 50, 52, 49, 55, 50, 48, 48, 48, 90, 23, 13, 50, 55, 49, 48, 50, 52, 49, 55, 50, 48, 48, 48, 90, 48, 48, 49, 18, 48, 16, 6, 3, 85, 4, 10, 19, 9, 111, 112, 101, 110, 115, 104, 105, 102, 116, 49, 26, 48, 24, 6, 3, 85, 4, 3, 19, 17, 111, 112, 101, 110, 115, 104, 105, 102, 116, 45, 105, 110, 103, 114, 101, 115, 115, 48, 130, 1, 34, 48, 13, 6, 9, 42, 134, 72, 134, 247, 13, 1, 1, 1, 5, 0, 3, 130, 1, 15, 0, 48, 130, 1, 10, 2, 130, 1, 1, 0, 175, 180, 66, 13, 148, 254, 185, 112, 159, 180, 48, 58, 217, 14, 123, 23, 229, 28, 224, 241, 179, 210, 115, 71, 123, 94, 152, 56, 200, 179, 120, 80, 246, 68, 78, 194, 165, 17, 161, 64, 168, 40, 31, 243, 155, 207, 171, 17, 238, 179, 159, 199, 90, 32, 61, 243, 133, 233, 153, 229, 126, 69, 220, 234, 188, 36, 185, 6, 114, 77, 215, 59, 104, 34, 80, 95, 66, 69, 104, 166, 84, 17, 159, 234, 101, 109, 164, 153, 172, 156, 240, 134, 16, 200, 61, 179, 79, 164, 99, 227, 229, 46, 174, 192, 70, 194, 161, 207, 5, 78, 18, 21, 247, 101, 35, 19, 102, 159, 29, 233, 182, 86, 199, 53, 244, 175, 208, 228, 116, 49, 45, 21, 118, 178, 72, 104, 19, 188, 1, 131, 254, 127, 37, 139, 95, 60, 129, 254, 7, 18, 184, 51, 199, 217, 71, 107, 203, 113, 33, 123, 83, 246, 30, 234, 39, 214, 99, 207, 167, 116, 247, 66, 241, 15, 133, 22, 9, 238, 99, 105, 166, 167, 72, 142, 182, 229, 22, 51, 127, 84, 32, 214, 8, 185, 111, 104, 100, 152, 181, 219, 81, 90, 133, 133, 127, 114, 2, 134, 80, 248, 53, 178, 96, 3, 247, 147, 225, 186, 125, 75, 198, 237, 177, 202, 247, 71, 49, 31, 123, 47, 222, 47, 12, 75, 219, 147, 5, 27, 92, 161, 76, 204, 58, 211, 26, 120, 156, 19, 51, 136, 52, 116, 18, 166, 216, 205, 2, 3, 1, 0, 1, 163, 130, 1, 39, 48, 130, 1, 35, 48, 14, 6, 3, 85, 29, 15, 1, 1, 255, 4, 4, 3, 2, 5, 160, 48, 29, 6, 3, 85, 29, 37, 4, 22, 48, 20, 6, 8, 43, 6, 1, 5, 5, 7, 3, 2, 6, 8, 43, 6, 1, 5, 5, 7, 3, 1, 48, 12, 6, 3, 85, 29, 19, 1, 1, 255, 4, 2, 48, 0, 48, 73, 6, 3, 85, 29, 14, 4, 66, 4, 64, 184, 27, 111, 100, 185, 3, 60, 74, 232, 82, 187, 169, 146, 147, 185, 95, 74, 1, 30, 137, 225, 41, 223, 249, 128, 141, 254, 150, 25, 225, 242, 3, 235, 207, 147, 83, 107, 148, 237, 0, 226, 240, 42, 175, 47, 113, 123, 148, 128, 190, 12, 146, 119, 216, 235, 223, 47, 29, 124, 171, 8, 133, 245, 128, 48, 75, 6, 3, 85, 29, 35, 4, 68, 48, 66, 128, 64, 107, 35, 93, 140, 120, 74, 68, 242, 166, 39, 191, 159, 156, 255, 152, 139, 116, 160, 21, 220, 61, 249, 246, 136, 64, 148, 57, 252, 107, 221, 102, 126, 167, 207, 36, 223, 46, 112, 92, 229, 162, 35, 131, 186, 43, 139, 91, 12, 221, 184, 234, 124, 29, 20, 52, 38, 121, 7, 144, 253, 45, 84, 166, 41, 48, 76, 6, 3, 85, 29, 17, 4, 69, 48, 67, 130, 65, 42, 46, 97, 112, 112, 115, 46, 97, 114, 111, 46, 99, 57, 112, 48, 118, 55, 106, 57, 113, 55, 106, 52, 97, 49, 113, 46, 97, 102, 108, 100, 46, 119, 101, 115, 116, 117, 115, 51, 46, 97, 114, 111, 97, 112, 112, 45, 104, 99, 112, 46, 97, 122, 117, 114, 101, 45, 116, 101, 115, 116, 46, 110, 101, 116, 48, 13, 6, 9, 42, 134, 72, 134, 247, 13, 1, 1, 11, 5, 0, 3, 130, 1, 1, 0, 62, 37, 28, 104, 244, 85, 128, 159, 33, 132, 56, 114, 149, 155, 76, 150, 56, 190, 182, 145, 23, 80, 230, 251, 69, 191, 95, 185, 206, 12, 203, 223, 159, 176, 108, 31, 228, 30, 29, 147, 188, 224, 159, 222, 209, 80, 26, 111, 48, 5, 218, 228, 214, 243, 4, 174, 170, 173, 102, 5, 47, 112, 16, 26, 249, 76, 134, 179, 187, 39, 158, 241, 110, 27, 4, 56, 253, 35, 167, 85, 21, 75, 203, 60, 52, 83, 13, 153, 94, 185, 69, 153, 196, 55, 251, 226, 149, 80, 157, 57, 244, 94, 195, 179, 137, 174, 166, 85, 12, 161, 202, 212, 188, 14, 213, 25, 59, 74, 197, 148, 98, 52, 206, 23, 41, 197, 32, 214, 165, 60, 202, 205, 67, 25, 42, 38, 206, 141, 201, 237, 101, 166, 209, 131, 101, 208, 225, 123, 244, 235, 82, 50, 143, 145, 70, 217, 191, 27, 3, 130, 194, 10, 227, 129, 5, 165, 150, 45, 143, 154, 59, 167, 145, 196, 78, 140, 76, 209, 96, 166, 23, 54, 227, 85, 65, 210, 246, 64, 194, 155, 249, 210, 255, 119, 224, 229, 103, 89, 68, 18, 164, 119, 9, 207, 73, 245, 219, 68, 214, 3, 155, 42, 94, 212, 192, 93, 148, 231, 125, 114, 77, 118, 234, 177, 58, 169, 209, 85, 145, 75, 243, 82, 234, 209, 51, 198, 131, 119, 63, 14, 40, 242, 219, 78, 215, 35, 237, 91, 234, 2, 142, 86, 121, 121, 249, 150],
                        RawTBSCertificate: [48, 130, 2, 233, 160, 3, 2, 1, 2, 2, 8, 110, 83, 80, 98, 163, 186, 156, 29, 48, 13, 6, 9, 42, 134, 72, 134, 247, 13, 1, 1, 11, 5, 0, 48, 38, 49, 18, 48, 16, 6, 3, 85, 4, 11, 19, 9, 111, 112, 101, 110, 115, 104, 105, 102, 116, 49, 16, 48, 14, 6, 3, 85, 4, 3, 19, 7, 114, 111, 111, 116, 45, 99, 97, 48, 30, 23, 13, 50, 54, 49, 48, 50, 52, 49, 55, 50, 48, 48, 48, 90, 23, 13, 50, 55, 49, 48, 50, 52, 49, 55, 50, 48, 48, 48, 90, 48, 48, 49, 18, 48, 16, 6, 3, 85, 4, 10, 19, 9, 111, 112, 101, 110, 115, 104, 105, 102, 116, 49, 26, 48, 24, 6, 3, 85, 4, 3, 19, 17, 111, 112, 101, 110, 115, 104, 105, 102, 116, 45, 105, 110, 103, 114, 101, 115, 115, 48, 130, 1, 34, 48, 13, 6, 9, 42, 134, 72, 134, 247, 13, 1, 1, 1, 5, 0, 3, 130, 1, 15, 0, 48, 130, 1, 10, 2, 130, 1, 1, 0, 175, 180, 66, 13, 148, 254, 185, 112, 159, 180, 48, 58, 217, 14, 123, 23, 229, 28, 224, 241, 179, 210, 115, 71, 123, 94, 152, 56, 200, 179, 120, 80, 246, 68, 78, 194, 165, 17, 161, 64, 168, 40, 31, 243, 155, 207, 171, 17, 238, 179, 159, 199, 90, 32, 61, 243, 133, 233, 153, 229, 126, 69, 220, 234, 188, 36, 185, 6, 114, 77, 215, 59, 104, 34, 80, 95, 66, 69, 104, 166, 84, 17, 159, 234, 101, 109, 164, 153, 172, 156, 240, 134, 16, 200, 61, 179, 79, 164, 99, 227, 229, 46, 174, 192, 70, 194, 161, 207, 5, 78, 18, 21, 247, 101, 35, 19, 102, 159, 29, 233, 182, 86, 199, 53, 244, 175, 208, 228, 116, 49, 45, 21, 118, 178, 72, 104, 19, 188, 1, 131, 254, 127, 37, 139, 95, 60, 129, 254, 7, 18, 184, 51, 199, 217, 71, 107, 203, 113, 33, 123, 83, 246, 30, 234, 39, 214, 99, 207, 167, 116, 247, 66, 241, 15, 133, 22, 9, 238, 99, 105, 166, 167, 72, 142, 182, 229, 22, 51, 127, 84, 32, 214, 8, 185, 111, 104, 100, 152, 181, 219, 81, 90, 133, 133, 127, 114, 2, 134, 80, 248, 53, 178, 96, 3, 247, 147, 225, 186, 125, 75, 198, 237, 177, 202, 247, 71, 49, 31, 123, 47, 222, 47, 12, 75, 219, 147, 5, 27, 92, 161, 76, 204, 58, 211, 26, 120, 156, 19, 51, 136, 52, 116, 18, 166, 216, 205, 2, 3, 1, 0, 1, 163, 130, 1, 39, 48, 130, 1, 35, 48, 14, 6, 3, 85, 29, 15, 1, 1, 255, 4, 4, 3, 2, 5, 160, 48, 29, 6, 3, 85, 29, 37, 4, 22, 48, 20, 6, 8, 43, 6, 1, 5, 5, 7, 3, 2, 6, 8, 43, 6, 1, 5, 5, 7, 3, 1, 48, 12, 6, 3, 85, 29, 19, 1, 1, 255, 4, 2, 48, 0, 48, 73, 6, 3, 85, 29, 14, 4, 66, 4, 64, 184, 27, 111, 100, 185, 3, 60, 74, 232, 82, 187, 169, 146, 147, 185, 95, 74, 1, 30, 137, 225, 41, 223, 249, 128, 141, 254, 150, 25, 225, 242, 3, 235, 207, 147, 83, 107, 148, 237, 0, 226, 240, 42, 175, 47, 113, 123, 148, 128, 190, 12, 146, 119, 216, 235, 223, 47, 29, 124, 171, 8, 133, 245, 128, 48, 75, 6, 3, 85, 29, 35, 4, 68, 48, 66, 128, 64, 107, 35, 93, 140, 120, 74, 68, 242, 166, 39, 191, 159, 156, 255, 152, 139, 116, 160, 21, 220, 61, 249, 246, 136, 64, 148, 57, 252, 107, 221, 102, 126, 167, 207, 36, 223, 46, 112, 92, 229, 162, 35, 131, 186, 43, 139, 91, 12, 221, 184, 234, 124, 29, 20, 52, 38, 121, 7, 144, 253, 45, 84, 166, 41, 48, 76, 6, 3, 85, 29, 17, 4, 69, 48, 67, 130, 65, 42, 46, 97, 112, 112, 115, 46, 97, 114, 111, 46, 99, 57, 112, 48, 118, 55, 106, 57, 113, 55, 106, 52, 97, 49, 113, 46, 97, 102, 108, 100, 46, 119, 101, 115, 116, 117, 115, 51, 46, 97, 114, 111, 97, 112, 112, 45, 104, 99, 112, 46, 97, 122, 117, 114, 101, 45, 116, 101, 115, 116, 46, 110, 101, 116],
                        RawSubjectPublicKeyInfo: [48, 130, 1, 34, 48, 13, 6, 9, 42, 134, 72, 134, 247, 13, 1, 1, 1, 5, 0, 3, 130, 1, 15, 0, 48, 130, 1, 10, 2, 130, 1, 1, 0, 175, 180, 66, 13, 148, 254, 185, 112, 159, 180, 48, 58, 217, 14, 123, 23, 229, 28, 224, 241, 179, 210, 115, 71, 123, 94, 152, 56, 200, 179, 120, 80, 246, 68, 78, 194, 165, 17, 161, 64, 168, 40, 31, 243, 155, 207, 171, 17, 238, 179, 159, 199, 90, 32, 61, 243, 133, 233, 153, 229, 126, 69, 220, 234, 188, 36, 185, 6, 114, 77, 215, 59, 104, 34, 80, 95, 66, 69, 104, 166, 84, 17, 159, 234, 101, 109, 164, 153, 172, 156, 240, 134, 16, 200, 61, 179, 79, 164, 99, 227, 229, 46, 174, 192, 70, 194, 161, 207, 5, 78, 18, 21, 247, 101, 35, 19, 102, 159, 29, 233, 182, 86, 199, 53, 244, 175, 208, 228, 116, 49, 45, 21, 118, 178, 72, 104, 19, 188, 1, 131, 254, 127, 37, 139, 95, 60, 129, 254, 7, 18, 184, 51, 199, 217, 71, 107, 203, 113, 33, 123, 83, 246, 30, 234, 39, 214, 99, 207, 167, 116, 247, 66, 241, 15, 133, 22, 9, 238, 99, 105, 166, 167, 72, 142, 182, 229, 22, 51, 127, 84, 32, 214, 8, 185, 111, 104, 100, 152, 181, 219, 81, 90, 133, 133, 127, 114, 2, 134, 80, 248, 53, 178, 96, 3, 247, 147, 225, 186, 125, 75, 198, 237, 177, 202, 247, 71, 49, 31, 123, 47, 222, 47, 12, 75, 219, 147, 5, 27, 92, 161, 76, 204, 58, 211, 26, 120, 156, 19, 51, 136, 52, 116, 18, 166, 216, 205, 2, 3, 1, 0, 1],
                        RawSubject: [48, 48, 49, 18, 48, 16, 6, 3, 85, 4, 10, 19, 9, 111, 112, 101, 110, 115, 104, 105, 102, 116, 49, 26, 48, 24, 6, 3, 85, 4, 3, 19, 17, 111, 112, 101, 110, 115, 104, 105, 102, 116, 45, 105, 110, 103, 114, 101, 115, 115],
                        RawIssuer: [48, 38, 49, 18, 48, 16, 6, 3, 85, 4, 11, 19, 9, 111, 112, 101, 110, 115, 104, 105, 102, 116, 49, 16, 48, 14, 6, 3, 85, 4, 3, 19, 7, 114, 111, 111, 116, 45, 99, 97],
                        Signature: [62, 37, 28, 104, 244, 85, 128, 159, 33, 132, 56, 114, 149, 155, 76, 150, 56, 190, 182, 145, 23, 80, 230, 251, 69, 191, 95, 185, 206, 12, 203, 223, 159, 176, 108, 31, 228, 30, 29, 147, 188, 224, 159, 222, 209, 80, 26, 111, 48, 5, 218, 228, 214, 243, 4, 174, 170, 173, 102, 5, 47, 112, 16, 26, 249, 76, 134, 179, 187, 39, 158, 241, 110, 27, 4, 56, 253, 35, 167, 85, 21, 75, 203, 60, 52, 83, 13, 153, 94, 185, 69, 153, 196, 55, 251, 226, 149, 80, 157, 57, 244, 94, 195, 179, 137, 174, 166, 85, 12, 161, 202, 212, 188, 14, 213, 25, 59, 74, 197, 148, 98, 52, 206, 23, 41, 197, 32, 214, 165, 60, 202, 205, 67, 25, 42, 38, 206, 141, 201, 237, 101, 166, 209, 131, 101, 208, 225, 123, 244, 235, 82, 50, 143, 145, 70, 217, 191, 27, 3, 130, 194, 10, 227, 129, 5, 165, 150, 45, 143, 154, 59, 167, 145, 196, 78, 140, 76, 209, 96, 166, 23, 54, 227, 85, 65, 210, 246, 64, 194, 155, 249, 210, 255, 119, 224, 229, 103, 89, 68, 18, 164, 119, 9, 207, 73, 245, 219, 68, 214, 3, 155, 42, 94, 212, 192, 93, 148, 231, 125, 114, 77, 118, 234, 177, 58, 169, 209, 85, 145, 75, 243, 82, 234, 209, 51, 198, 131, 119, 63, 14, 40, 242, 219, 78, 215, 35, 237, 91, 234, 2, 142, 86, 121, 121, 249, 150],
                        SignatureAlgorithm: 4,
                        PublicKeyAlgorithm: 1,
                        PublicKey: <*rsa.PublicKey | 0x30fb7a9895b0>{
                            N: {
                                neg: false,
                                abs: [3713275565900749005, 5533862720488119315, 886043176200789153, 17818264461995138607, 16265450768615453130, 5834472357489801107, 5862144673391444614, 628656117149906395, 13178964295004987606, 715618696217184398, 12066541168516629782, 6050057140458382287, 14400619811794133371, 6862502860246136883, 1422013248764323211, 8372522752467421288, 13138907997452357860, 17826693266249031145, 5098815638656127509, 5738821655630622400, 12438080729488309683, 6057798802862875801, 7503648299216627878, 13557164215630681915, 9649412887563066602, 17200267080417426931, 12116969928591911697, 17745395029646614848, 8889710084128733264, 16509317662742049607, 11507875977146301207, 12660817078547036528],
                            },
                            E: 65537,
                        },
                        Version: 3,
                        SerialNumber: {
                            neg: false,
                            abs: [7949786151822990365],
                        },
                        Issuer: {
                            Country: nil,
                            Organization: nil,
                            OrganizationalUnit: ["openshift"],
                            Locality: nil,
                            Province: nil,
                            StreetAddress: nil,
                            PostalCode: nil,
                            SerialNumber: "",
                            CommonName: "root-ca",
                            Names: [
                                {
                                    Type: [2, 5, 4, 11],
                                    Value: <string>"openshift",
                                },
                                {
                                    Type: [2, 5, 4, 3],
                                    Value: <string>"root-ca",
                                },
                            ],
                            ExtraNames: nil,
                        },
                        Subject: {
                            Country: nil,
                            Organization: ["openshift"],
                            OrganizationalUnit: nil,
                            Locality: nil,
                            Province: nil,
                            StreetAddress: nil,
                            PostalCode: nil,
                            SerialNumber: "",
                            CommonName: "openshift-ingress",
                            Names: [
                                {
                                    Type: [2, 5, 4, 10],
                                    Value: <string>"openshift",
                                },
                                {
                                    Type: [2, 5, 4, 3],
                                    Value: <string>"openshift-ingress",
                                },
                            ],
                            ExtraNames: nil,
                        },
                        NotBefore: {wall: 0, ext: 63928459200, loc: nil},
                        NotAfter: {wall: 0, ext: 63959995200, loc: nil},
                        KeyUsage: 5,
                        Extensions: [
                            {
                                Id: [2, 5, 29, 15],
                                Critical: true,
                                Value: [3, 2, 5, 160],
                            },
                            {
                                Id: [2, 5, 29, 37],
                                Critical: false,
                                Value: [48, 20, 6, 8, 43, 6, 1, 5, 5, 7, 3, 2, 6, 8, 43, 6, 1, 5, 5, 7, 3, 1],
                            },
                            {Id: [2, 5, 29, 19], Critical: true, Value: [48, 0]},
                            {
                                Id: [2, 5, 29, 14],
                                Critical: false,
                                Value: [4, 64, 184, 27, 111, 100, 185, 3, 60, 74, 232, 82, 187, 169, 146, 147, 185, 95, 74, 1, 30, 137, 225, 41, 223, 249, 128, 141, 254, 150, 25, 225, 242, 3, 235, 207, 147, 83, 107, 148, 237, 0, 226, 240, 42, 175, 47, 113, 123, 148, 128, 190, 12, 146, 119, 216, 235, 223, 47, 29, 124, 171, 8, 133, 245, 128],
                            },
                            {
                                Id: [2, 5, 29, 35],
                                Critical: false,
                                Value: [48, 66, 128, 64, 107, 35, 93, 140, 120, 74, 68, 242, 166, 39, 191, 159, 156, 255, 152, 139, 116, 160, 21, 220, 61, 249, 246, 136, 64, 148, 57, 252, 107, 221, 102, 126, 167, 207, 36, 223, 46, 112, 92, 229, 162, 35, 131, 186, 43, 139, 91, 12, 221, 184, 234, 124, 29, 20, 52, 38, 121, 7, 144, 253, 45, 84, 166, 41],
                            },
                            {
                                Id: [2, 5, 29, 17],
                                Critical: false,
                                Value: "0C\x82A*.apps.aro.c9p0v7j9q7j4a1q.afld.westus3.aroapp-hcp.azure-test.net",
                            },
                        ],
                        ExtraExtensions: nil,
                        UnhandledCriticalExtensions: nil,
                        ExtKeyUsage: [2, 1],
                        UnknownExtKeyUsage: nil,
                        BasicConstraintsValid: true,
                        IsCA: false,
                        MaxPathLen: -1,
                        MaxPathLenZero: false,
                        SubjectKeyId: [184, 27, 111, 100, 185, 3, 60, 74, 232, 82, 187, 169, 146, 147, 185, 95, 74, 1, 30, 137, 225, 41, 223, 249, 128, 141, 254, 150, 25, 225, 242, 3, 235, 207, 147, 83, 107, 148, 237, 0, 226, 240, 42, 175, 47, 113, 123, 148, 128, 190, 12, 146, 119, 216, 235, 223, 47, 29, 124, 171, 8, 133, 245, 128],
                        AuthorityKeyId: [107, 35, 93, 140, 120, 74, 68, 242, 166, 39, 191, 159, 156, 255, 152, 139, 116, 160, 21, 220, 61, 249, 246, 136, 64, 148, 57, 252, 107, 221, 102, 126, 167, 207, 36, 223, 46, 112, 92, 229, 162, 35, 131, 186, 43, 139, 91, 12, 221, 184, 234, 124, 29, 20, 52, 38, 121, 7, 144, 253, 45, 84, 166, 41],
                        OCSPServer: nil,
                        IssuingCertificateURL: nil,
                        DNSNames: [
                            "*.apps.aro.c9p0v7j9q7j4a1q.afld.westus3.aroapp-hcp.azure-test.net",
                        ],
                        EmailAddresses: nil,
                        IPAddresses: nil,
                        URIs: nil,
                        PermittedDNSDomainsCritical: false,
                        PermittedDNSDomains: nil,
                        ExcludedDNSDomains: nil,
                        PermittedIPRanges: nil,
                        ExcludedIPRanges: nil,
                        PermittedEmailAddresses: nil,
                        ExcludedEmailAddresses: nil,
                        PermittedURIDomains: nil,
                        ExcludedURIDomains: nil,
                        CRLDistributionPoints: nil,
                        PolicyIdentifiers: nil,
                        Policies: nil,
                        InhibitAnyPolicy: 0,
                        InhibitAnyPolicyZero: false,
                        InhibitPolicyMapping: 0,
                        InhibitPolicyMappingZero: false,
                        RequireExplicitPolicy: 0,
                        RequireExplicitPolicyZero: false,
                        PolicyMappings: nil,
                    },
                ],
                Err: <x509.UnknownAuthorityError>{
                    Cert: 0x30fb7a8f2c08 (cyclic reference),
                    hintErr: nil,
                    hintCert: nil,
                },
            },
        },
    }
occurred
DNS for route host agnhost-e2e-sample-app-<id>
16:40:04 UTC · failed at unknown · test Customer should be able to issue the default ingress serving certificate from Let's Encrypt using cert-manager with ACME DNS-01 challenges for cluster version 4.21 · suite integration/parallel/all
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_ingress_cert_letsencrypt.go:528]: sample application must serve the exact customer-specified certificate with a valid Let's Encrypt chain
Expected success, but got an error:
    <*fmt.wrapError | 0x7f30d39c620>: 
    DNS for route host agnhost-e2e-sample-app-gt2zw.apps.aro.ingress-cert-le.msqo.westus3.aroapp-hcp.azure-test.net did not resolve: DNS for agnhost-e2e-sample-app-gt2zw.apps.aro.ingress-cert-le.msqo.westus3.aroapp-hcp.azure-test.net did not resolve within 10m0s (last error: lookup agnhost-e2e-sample-app-gt2zw.apps.aro.ingress-cert-le.msqo.westus3.aroapp-hcp.azure-test.net on 172.30.0.10:53: dial udp 172.30.0.10:53: i/o timeout): context deadline exceeded
    {
        msg: "DNS for route host agnhost-e2e-sample-app-gt2zw.apps.aro.ingress-cert-le.msqo.westus3.aroapp-hcp.azure-test.net did not resolve: DNS for agnhost-e2e-sample-app-gt2zw.apps.aro.ingress-cert-le.msqo.westus3.aroapp-hcp.azure-test.net did not resolve within 10m0s (last error: lookup agnhost-e2e-sample-app-gt2zw.apps.aro.ingress-cert-le.msqo.westus3.aroapp-hcp.azure-test.net on 172.30.0.10:53: dial udp 172.30.0.10:53: i/o timeout): context deadline exceeded",
        err: <*fmt.wrapError | 0x7f30d39c600>{
            msg: "DNS for agnhost-e2e-sample-app-gt2zw.apps.aro.ingress-cert-le.msqo.westus3.aroapp-hcp.azure-test.net did not resolve within 10m0s (last error: lookup agnhost-e2e-sample-app-gt2zw.apps.aro.ingress-cert-le.msqo.westus3.aroapp-hcp.azure-test.net on 172.30.0.10:53: dial udp 172.30.0.10:53: i/o timeout): context deadline exceeded",
            err: <context.deadlineExceededError>{},
        },
    }
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_ingress_cert_letsencrypt.go:528]: sample application must serve the exact customer-specified certificate with a valid Let's Encrypt chain
Expected success, but got an error:
    <*fmt.wrapError | 0x7f30d39c620>: 
    DNS for route host agnhost-e2e-sample-app-gt2zw.apps.aro.ingress-cert-le.msqo.westus3.aroapp-hcp.azure-test.net did not resolve: DNS for agnhost-e2e-sample-app-gt2zw.apps.aro.ingress-cert-le.msqo.westus3.aroapp-hcp.azure-test.net did not resolve within 10m0s (last error: lookup agnhost-e2e-sample-app-gt2zw.apps.aro.ingress-cert-le.msqo.westus3.aroapp-hcp.azure-test.net on 172.30.0.10:53: dial udp 172.30.0.10:53: i/o timeout): context deadline exceeded
    {
        msg: "DNS for route host agnhost-e2e-sample-app-gt2zw.apps.aro.ingress-cert-le.msqo.westus3.aroapp-hcp.azure-test.net did not resolve: DNS for agnhost-e2e-sample-app-gt2zw.apps.aro.ingress-cert-le.msqo.westus3.aroapp-hcp.azure-test.net did not resolve within 10m0s (last error: lookup agnhost-e2e-sample-app-gt2zw.apps.aro.ingress-cert-le.msqo.westus3.aroapp-hcp.azure-test.net on 172.30.0.10:53: dial udp 172.30.0.10:53: i/o timeout): context deadline exceeded",
        err: <*fmt.wrapError | 0x7f30d39c600>{
            msg: "DNS for agnhost-e2e-sample-app-gt2zw.apps.aro.ingress-cert-le.msqo.westus3.aroapp-hcp.azure-test.net did not resolve within 10m0s (last error: lookup agnhost-e2e-sample-app-gt2zw.apps.aro.ingress-cert-le.msqo.westus3.aroapp-hcp.azure-test.net on 172.30.0.10:53: dial udp 172.30.0.10:53: i/o timeout): context deadline exceeded",
            err: <context.deadlineExceededError>{},
        },
    }
DNS for route host <route-host> did not resolve: DNS for <route-host> did not resolve within <duration> (last error: lookup <host> on <dns-server>: dial udp <ip>: i/o timeout): context deadline exceeded
16:40:04 UTC · failed at unknown · test Customer should be able to retrieve cluster and nodepool status active versions · suite integration/parallel/all
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_nodepool_active_versions.go:149]: failed to verify simple web app runs on v20261001 cluster "v20261001"
Unexpected error:
    <*fmt.wrapError | 0xdad4da81e80>: 
    DNS for route host agnhost-e2e-sample-app-4gnm9.apps.aro.v20261001.bhnx.westus3.aroapp-hcp.azure-test.net did not resolve: DNS for agnhost-e2e-sample-app-4gnm9.apps.aro.v20261001.bhnx.westus3.aroapp-hcp.azure-test.net did not resolve within 10m0s (last error: lookup agnhost-e2e-sample-app-4gnm9.apps.aro.v20261001.bhnx.westus3.aroapp-hcp.azure-test.net on 172.30.0.10:53: dial udp 172.30.0.10:53: i/o timeout): context deadline exceeded
    {
        msg: "DNS for route host agnhost-e2e-sample-app-4gnm9.apps.aro.v20261001.bhnx.westus3.aroapp-hcp.azure-test.net did not resolve: DNS for agnhost-e2e-sample-app-4gnm9.apps.aro.v20261001.bhnx.westus3.aroapp-hcp.azure-test.net did not resolve within 10m0s (last error: lookup agnhost-e2e-sample-app-4gnm9.apps.aro.v20261001.bhnx.westus3.aroapp-hcp.azure-test.net on 172.30.0.10:53: dial udp 172.30.0.10:53: i/o timeout): context deadline exceeded",
        err: <*fmt.wrapError | 0xdad4d9b41a0>{
            msg: "DNS for agnhost-e2e-sample-app-4gnm9.apps.aro.v20261001.bhnx.westus3.aroapp-hcp.azure-test.net did not resolve within 10m0s (last error: lookup agnhost-e2e-sample-app-4gnm9.apps.aro.v20261001.bhnx.westus3.aroapp-hcp.azure-test.net on 172.30.0.10:53: dial udp 172.30.0.10:53: i/o timeout): context deadline exceeded",
            err: <context.deadlineExceededError>{},
        },
    }
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_nodepool_active_versions.go:149]: failed to verify simple web app runs on v20261001 cluster "v20261001"
Unexpected error:
    <*fmt.wrapError | 0xdad4da81e80>: 
    DNS for route host agnhost-e2e-sample-app-4gnm9.apps.aro.v20261001.bhnx.westus3.aroapp-hcp.azure-test.net did not resolve: DNS for agnhost-e2e-sample-app-4gnm9.apps.aro.v20261001.bhnx.westus3.aroapp-hcp.azure-test.net did not resolve within 10m0s (last error: lookup agnhost-e2e-sample-app-4gnm9.apps.aro.v20261001.bhnx.westus3.aroapp-hcp.azure-test.net on 172.30.0.10:53: dial udp 172.30.0.10:53: i/o timeout): context deadline exceeded
    {
        msg: "DNS for route host agnhost-e2e-sample-app-4gnm9.apps.aro.v20261001.bhnx.westus3.aroapp-hcp.azure-test.net did not resolve: DNS for agnhost-e2e-sample-app-4gnm9.apps.aro.v20261001.bhnx.westus3.aroapp-hcp.azure-test.net did not resolve within 10m0s (last error: lookup agnhost-e2e-sample-app-4gnm9.apps.aro.v20261001.bhnx.westus3.aroapp-hcp.azure-test.net on 172.30.0.10:53: dial udp 172.30.0.10:53: i/o timeout): context deadline exceeded",
        err: <*fmt.wrapError | 0xdad4d9b41a0>{
            msg: "DNS for agnhost-e2e-sample-app-4gnm9.apps.aro.v20261001.bhnx.westus3.aroapp-hcp.azure-test.net did not resolve within 10m0s (last error: lookup agnhost-e2e-sample-app-4gnm9.apps.aro.v20261001.bhnx.westus3.aroapp-hcp.azure-test.net on 172.30.0.10:53: dial udp 172.30.0.10:53: i/o timeout): context deadline exceeded",
            err: <context.deadlineExceededError>{},
        },
    }
occurred
1h 15m 23s branch-ci-Azure-ARO-HCP-main-e2e-integration-e2e-parallel-all
post-goodmerged PR
No additional run metadata captured
westus3 unknown failed n/a 1
unknown (1)
tls: failed to verify certificate: x509: certificate signed by unknown authority
07:33:43 UTC · failed at unknown · test ARO-HCP should be able to perform a control plane and node pool install with OCP candidate channel for 4.23 · suite integration/parallel/all
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:194]: failed to verify simple web app runs on cluster "cluster-candidate-4-23-p8z7ps"
Unexpected error:
    <*fmt.wrapError | 0x3489af4cbc00>: 
    [strict TLS verification] route was never reachable: Get "https://agnhost-e2e-sample-app-82qkn.apps.aro.s6u8l5i9p5a2r2r.x5o7.westus3.aroapp-hcp.azure-test.net": tls: failed to verify certificate: x509: certificate signed by unknown authority
    {
        msg: "[strict TLS verification] route was never reachable: Get \"https://agnhost-e2e-sample-app-82qkn.apps.aro.s6u8l5i9p5a2r2r.x5o7.westus3.aroapp-hcp.azure-test.net\": tls: failed to verify certificate: x509: certificate signed by unknown authority",
        err: <*url.Error | 0x3489b0e89ec0>{
            Op: "Get",
            URL: "https://agnhost-e2e-sample-app-82qkn.apps.aro.s6u8l5i9p5a2r2r.x5o7.westus3.aroapp-hcp.azure-test.net",
            Err: <*tls.CertificateVerificationError | 0x3489b0e89e90>{
                UnverifiedCertificates: [
                    {
                        Raw: [48, 130, 4, 1, 48, 130, 2, 233, 160, 3, 2, 1, 2, 2, 8, 43, 238, 229, 240, 21, 153, 99, 197, 48, 13, 6, 9, 42, 134, 72, 134, 247, 13, 1, 1, 11, 5, 0, 48, 38, 49, 18, 48, 16, 6, 3, 85, 4, 11, 19, 9, 111, 112, 101, 110, 115, 104, 105, 102, 116, 49, 16, 48, 14, 6, 3, 85, 4, 3, 19, 7, 114, 111, 111, 116, 45, 99, 97, 48, 30, 23, 13, 50, 54, 49, 48, 50, 52, 48, 55, 52, 54, 51, 49, 90, 23, 13, 50, 55, 49, 48, 50, 52, 48, 55, 52, 54, 51, 49, 90, 48, 48, 49, 18, 48, 16, 6, 3, 85, 4, 10, 19, 9, 111, 112, 101, 110, 115, 104, 105, 102, 116, 49, 26, 48, 24, 6, 3, 85, 4, 3, 19, 17, 111, 112, 101, 110, 115, 104, 105, 102, 116, 45, 105, 110, 103, 114, 101, 115, 115, 48, 130, 1, 34, 48, 13, 6, 9, 42, 134, 72, 134, 247, 13, 1, 1, 1, 5, 0, 3, 130, 1, 15, 0, 48, 130, 1, 10, 2, 130, 1, 1, 0, 158, 111, 39, 196, 0, 80, 49, 176, 30, 189, 213, 85, 250, 251, 181, 188, 150, 62, 113, 25, 141, 175, 149, 164, 27, 2, 38, 246, 53, 141, 137, 114, 171, 48, 154, 117, 70, 114, 145, 206, 247, 157, 219, 239, 189, 104, 15, 62, 80, 79, 181, 147, 69, 137, 179, 102, 101, 150, 46, 26, 209, 44, 232, 164, 100, 210, 189, 72, 138, 250, 119, 132, 147, 127, 145, 205, 19, 47, 94, 200, 57, 237, 183, 255, 120, 149, 7, 254, 195, 195, 130, 169, 219, 117, 252, 75, 140, 177, 52, 180, 49, 196, 156, 178, 8, 231, 5, 238, 214, 167, 179, 54, 239, 130, 78, 181, 58, 133, 156, 2, 8, 15, 159, 72, 25, 85, 198, 170, 187, 245, 62, 130, 145, 128, 67, 48, 182, 232, 194, 20, 15, 104, 33, 231, 108, 156, 56, 90, 229, 248, 221, 71, 105, 238, 237, 83, 244, 44, 112, 174, 38, 139, 77, 150, 145, 31, 165, 13, 167, 10, 136, 108, 72, 232, 249, 192, 115, 38, 117, 183, 31, 134, 156, 103, 202, 99, 106, 69, 51, 223, 14, 115, 32, 93, 24, 6, 153, 129, 175, 17, 63, 202, 109, 233, 56, 75, 87, 80, 147, 28, 70, 160, 229, 87, 181, 34, 53, 223, 172, 119, 241, 56, 131, 144, 20, 138, 28, 73, 129, 122, 162, 3, 187, 23, 218, 232, 140, 65, 52, 71, 184, 117, 40, 197, 78, 116, 38, 248, 6, 148, 15, 34, 201, 88, 176, 243, 2, 3, 1, 0, 1, 163, 130, 1, 39, 48, 130, 1, 35, 48, 14, 6, 3, 85, 29, 15, 1, 1, 255, 4, 4, 3, 2, 5, 160, 48, 29, 6, 3, 85, 29, 37, 4, 22, 48, 20, 6, 8, 43, 6, 1, 5, 5, 7, 3, 2, 6, 8, 43, 6, 1, 5, 5, 7, 3, 1, 48, 12, 6, 3, 85, 29, 19, 1, 1, 255, 4, 2, 48, 0, 48, 73, 6, 3, 85, 29, 14, 4, 66, 4, 64, 219, 47, 173, 50, 97, 112, 158, 229, 198, 244, 185, 216, 225, 191, 88, 22, 77, 0, 234, 97, 129, 231, 80, 228, 92, 178, 0, 52, 83, 87, 158, 115, 94, 72, 111, 209, 220, 55, 16, 127, 98, 248, 205, 26, 77, 249, 92, 91, 236, 8, 82, 113, 93, 251, 205, 152, 219, 254, 197, 208, 47, 119, 143, 181, 48, 75, 6, 3, 85, 29, 35, 4, 68, 48, 66, 128, 64, 79, 93, 7, 222, 176, 37, 12, 196, 224, 158, 229, 29, 81, 92, 248, 116, 80, 238, 46, 189, 156, 203, 133, 105, 107, 93, 137, 32, 23, 78, 176, 251, 113, 92, 180, 16, 114, 85, 131, 218, 105, 0, 244, 88, 176, 180, 148, 6, 119, 219, 149, 117, 27, 51, 22, 216, 24, 255, 81, 62, 12, 64, 10, 184, 48, 76, 6, 3, 85, 29, 17, 4, 69, 48, 67, 130, 65, 42, 46, 97, 112, 112, 115, 46, 97, 114, 111, 46, 115, 54, 117, 56, 108, 53, 105, 57, 112, 53, 97, 50, 114, 50, 114, 46, 120, 53, 111, 55, 46, 119, 101, 115, 116, 117, 115, 51, 46, 97, 114, 111, 97, 112, 112, 45, 104, 99, 112, 46, 97, 122, 117, 114, 101, 45, 116, 101, 115, 116, 46, 110, 101, 116, 48, 13, 6, 9, 42, 134, 72, 134, 247, 13, 1, 1, 11, 5, 0, 3, 130, 1, 1, 0, 65, 22, 134, 247, 228, 246, 36, 128, 182, 182, 244, 211, 102, 134, 109, 155, 171, 246, 80, 230, 175, 242, 200, 121, 170, 15, 231, 61, 30, 214, 43, 76, 202, 69, 76, 61, 232, 34, 150, 103, 156, 30, 87, 9, 251, 54, 13, 22, 50, 15, 186, 189, 51, 5, 9, 112, 178, 116, 29, 114, 168, 183, 30, 46, 238, 184, 247, 18, 68, 187, 209, 102, 42, 144, 123, 148, 189, 172, 148, 231, 172, 163, 24, 180, 12, 57, 57, 178, 28, 24, 47, 149, 2, 234, 44, 182, 125, 31, 5, 6, 94, 212, 163, 96, 132, 15, 25, 239, 102, 10, 14, 137, 196, 239, 124, 196, 146, 234, 2, 154, 140, 93, 166, 102, 235, 230, 253, 165, 47, 222, 167, 190, 70, 248, 172, 11, 69, 126, 194, 30, 221, 154, 97, 116, 195, 146, 31, 196, 31, 89, 67, 18, 84, 239, 174, 111, 105, 212, 184, 30, 84, 72, 237, 112, 85, 195, 211, 87, 184, 193, 65, 175, 132, 9, 196, 121, 203, 35, 118, 177, 151, 41, 106, 163, 169, 108, 86, 51, 253, 163, 7, 87, 188, 188, 151, 64, 24, 95, 174, 171, 164, 183, 237, 175, 25, 65, 122, 245, 38, 195, 178, 150, 104, 85, 228, 73, 164, 50, 181, 10, 140, 141, 232, 207, 156, 86, 106, 228, 38, 234, 189, 150, 66, 176, 12, 174, 201, 173, 149, 206, 151, 120, 120, 106, 251, 69, 123, 225, 252, 241, 122, 27, 207, 29, 170, 4],
                        RawTBSCertificate: [48, 130, 2, 233, 160, 3, 2, 1, 2, 2, 8, 43, 238, 229, 240, 21, 153, 99, 197, 48, 13, 6, 9, 42, 134, 72, 134, 247, 13, 1, 1, 11, 5, 0, 48, 38, 49, 18, 48, 16, 6, 3, 85, 4, 11, 19, 9, 111, 112, 101, 110, 115, 104, 105, 102, 116, 49, 16, 48, 14, 6, 3, 85, 4, 3, 19, 7, 114, 111, 111, 116, 45, 99, 97, 48, 30, 23, 13, 50, 54, 49, 48, 50, 52, 48, 55, 52, 54, 51, 49, 90, 23, 13, 50, 55, 49, 48, 50, 52, 48, 55, 52, 54, 51, 49, 90, 48, 48, 49, 18, 48, 16, 6, 3, 85, 4, 10, 19, 9, 111, 112, 101, 110, 115, 104, 105, 102, 116, 49, 26, 48, 24, 6, 3, 85, 4, 3, 19, 17, 111, 112, 101, 110, 115, 104, 105, 102, 116, 45, 105, 110, 103, 114, 101, 115, 115, 48, 130, 1, 34, 48, 13, 6, 9, 42, 134, 72, 134, 247, 13, 1, 1, 1, 5, 0, 3, 130, 1, 15, 0, 48, 130, 1, 10, 2, 130, 1, 1, 0, 158, 111, 39, 196, 0, 80, 49, 176, 30, 189, 213, 85, 250, 251, 181, 188, 150, 62, 113, 25, 141, 175, 149, 164, 27, 2, 38, 246, 53, 141, 137, 114, 171, 48, 154, 117, 70, 114, 145, 206, 247, 157, 219, 239, 189, 104, 15, 62, 80, 79, 181, 147, 69, 137, 179, 102, 101, 150, 46, 26, 209, 44, 232, 164, 100, 210, 189, 72, 138, 250, 119, 132, 147, 127, 145, 205, 19, 47, 94, 200, 57, 237, 183, 255, 120, 149, 7, 254, 195, 195, 130, 169, 219, 117, 252, 75, 140, 177, 52, 180, 49, 196, 156, 178, 8, 231, 5, 238, 214, 167, 179, 54, 239, 130, 78, 181, 58, 133, 156, 2, 8, 15, 159, 72, 25, 85, 198, 170, 187, 245, 62, 130, 145, 128, 67, 48, 182, 232, 194, 20, 15, 104, 33, 231, 108, 156, 56, 90, 229, 248, 221, 71, 105, 238, 237, 83, 244, 44, 112, 174, 38, 139, 77, 150, 145, 31, 165, 13, 167, 10, 136, 108, 72, 232, 249, 192, 115, 38, 117, 183, 31, 134, 156, 103, 202, 99, 106, 69, 51, 223, 14, 115, 32, 93, 24, 6, 153, 129, 175, 17, 63, 202, 109, 233, 56, 75, 87, 80, 147, 28, 70, 160, 229, 87, 181, 34, 53, 223, 172, 119, 241, 56, 131, 144, 20, 138, 28, 73, 129, 122, 162, 3, 187, 23, 218, 232, 140, 65, 52, 71, 184, 117, 40, 197, 78, 116, 38, 248, 6, 148, 15, 34, 201, 88, 176, 243, 2, 3, 1, 0, 1, 163, 130, 1, 39, 48, 130, 1, 35, 48, 14, 6, 3, 85, 29, 15, 1, 1, 255, 4, 4, 3, 2, 5, 160, 48, 29, 6, 3, 85, 29, 37, 4, 22, 48, 20, 6, 8, 43, 6, 1, 5, 5, 7, 3, 2, 6, 8, 43, 6, 1, 5, 5, 7, 3, 1, 48, 12, 6, 3, 85, 29, 19, 1, 1, 255, 4, 2, 48, 0, 48, 73, 6, 3, 85, 29, 14, 4, 66, 4, 64, 219, 47, 173, 50, 97, 112, 158, 229, 198, 244, 185, 216, 225, 191, 88, 22, 77, 0, 234, 97, 129, 231, 80, 228, 92, 178, 0, 52, 83, 87, 158, 115, 94, 72, 111, 209, 220, 55, 16, 127, 98, 248, 205, 26, 77, 249, 92, 91, 236, 8, 82, 113, 93, 251, 205, 152, 219, 254, 197, 208, 47, 119, 143, 181, 48, 75, 6, 3, 85, 29, 35, 4, 68, 48, 66, 128, 64, 79, 93, 7, 222, 176, 37, 12, 196, 224, 158, 229, 29, 81, 92, 248, 116, 80, 238, 46, 189, 156, 203, 133, 105, 107, 93, 137, 32, 23, 78, 176, 251, 113, 92, 180, 16, 114, 85, 131, 218, 105, 0, 244, 88, 176, 180, 148, 6, 119, 219, 149, 117, 27, 51, 22, 216, 24, 255, 81, 62, 12, 64, 10, 184, 48, 76, 6, 3, 85, 29, 17, 4, 69, 48, 67, 130, 65, 42, 46, 97, 112, 112, 115, 46, 97, 114, 111, 46, 115, 54, 117, 56, 108, 53, 105, 57, 112, 53, 97, 50, 114, 50, 114, 46, 120, 53, 111, 55, 46, 119, 101, 115, 116, 117, 115, 51, 46, 97, 114, 111, 97, 112, 112, 45, 104, 99, 112, 46, 97, 122, 117, 114, 101, 45, 116, 101, 115, 116, 46, 110, 101, 116],
                        RawSubjectPublicKeyInfo: [48, 130, 1, 34, 48, 13, 6, 9, 42, 134, 72, 134, 247, 13, 1, 1, 1, 5, 0, 3, 130, 1, 15, 0, 48, 130, 1, 10, 2, 130, 1, 1, 0, 158, 111, 39, 196, 0, 80, 49, 176, 30, 189, 213, 85, 250, 251, 181, 188, 150, 62, 113, 25, 141, 175, 149, 164, 27, 2, 38, 246, 53, 141, 137, 114, 171, 48, 154, 117, 70, 114, 145, 206, 247, 157, 219, 239, 189, 104, 15, 62, 80, 79, 181, 147, 69, 137, 179, 102, 101, 150, 46, 26, 209, 44, 232, 164, 100, 210, 189, 72, 138, 250, 119, 132, 147, 127, 145, 205, 19, 47, 94, 200, 57, 237, 183, 255, 120, 149, 7, 254, 195, 195, 130, 169, 219, 117, 252, 75, 140, 177, 52, 180, 49, 196, 156, 178, 8, 231, 5, 238, 214, 167, 179, 54, 239, 130, 78, 181, 58, 133, 156, 2, 8, 15, 159, 72, 25, 85, 198, 170, 187, 245, 62, 130, 145, 128, 67, 48, 182, 232, 194, 20, 15, 104, 33, 231, 108, 156, 56, 90, 229, 248, 221, 71, 105, 238, 237, 83, 244, 44, 112, 174, 38, 139, 77, 150, 145, 31, 165, 13, 167, 10, 136, 108, 72, 232, 249, 192, 115, 38, 117, 183, 31, 134, 156, 103, 202, 99, 106, 69, 51, 223, 14, 115, 32, 93, 24, 6, 153, 129, 175, 17, 63, 202, 109, 233, 56, 75, 87, 80, 147, 28, 70, 160, 229, 87, 181, 34, 53, 223, 172, 119, 241, 56, 131, 144, 20, 138, 28, 73, 129, 122, 162, 3, 187, 23, 218, 232, 140, 65, 52, 71, 184, 117, 40, 197, 78, 116, 38, 248, 6, 148, 15, 34, 201, 88, 176, 243, 2, 3, 1, 0, 1],
                        RawSubject: [48, 48, 49, 18, 48, 16, 6, 3, 85, 4, 10, 19, 9, 111, 112, 101, 110, 115, 104, 105, 102, 116, 49, 26, 48, 24, 6, 3, 85, 4, 3, 19, 17, 111, 112, 101, 110, 115, 104, 105, 102, 116, 45, 105, 110, 103, 114, 101, 115, 115],
                        RawIssuer: [48, 38, 49, 18, 48, 16, 6, 3, 85, 4, 11, 19, 9, 111, 112, 101, 110, 115, 104, 105, 102, 116, 49, 16, 48, 14, 6, 3, 85, 4, 3, 19, 7, 114, 111, 111, 116, 45, 99, 97],
                        Signature: [65, 22, 134, 247, 228, 246, 36, 128, 182, 182, 244, 211, 102, 134, 109, 155, 171, 246, 80, 230, 175, 242, 200, 121, 170, 15, 231, 61, 30, 214, 43, 76, 202, 69, 76, 61, 232, 34, 150, 103, 156, 30, 87, 9, 251, 54, 13, 22, 50, 15, 186, 189, 51, 5, 9, 112, 178, 116, 29, 114, 168, 183, 30, 46, 238, 184, 247, 18, 68, 187, 209, 102, 42, 144, 123, 148, 189, 172, 148, 231, 172, 163, 24, 180, 12, 57, 57, 178, 28, 24, 47, 149, 2, 234, 44, 182, 125, 31, 5, 6, 94, 212, 163, 96, 132, 15, 25, 239, 102, 10, 14, 137, 196, 239, 124, 196, 146, 234, 2, 154, 140, 93, 166, 102, 235, 230, 253, 165, 47, 222, 167, 190, 70, 248, 172, 11, 69, 126, 194, 30, 221, 154, 97, 116, 195, 146, 31, 196, 31, 89, 67, 18, 84, 239, 174, 111, 105, 212, 184, 30, 84, 72, 237, 112, 85, 195, 211, 87, 184, 193, 65, 175, 132, 9, 196, 121, 203, 35, 118, 177, 151, 41, 106, 163, 169, 108, 86, 51, 253, 163, 7, 87, 188, 188, 151, 64, 24, 95, 174, 171, 164, 183, 237, 175, 25, 65, 122, 245, 38, 195, 178, 150, 104, 85, 228, 73, 164, 50, 181, 10, 140, 141, 232, 207, 156, 86, 106, 228, 38, 234, 189, 150, 66, 176, 12, 174, 201, 173, 149, 206, 151, 120, 120, 106, 251, 69, 123, 225, 252, 241, 122, 27, 207, 29, 170, 4],
                        SignatureAlgorithm: 4,
                        PublicKeyAlgorithm: 1,
                        PublicKey: <*rsa.PublicKey | 0x3489b0c807b0>{
                            N: {
                                neg: false,
                                abs: [474020502862082291, 13291574703143790328, 13481484701877220423, 1480026529575117315, 3882011034964755344, 10600425279779747106, 4596607218124281680, 2332046598672068369, 14583616863811341939, 8297448792845098087, 12036582952762145216, 2777398904279246093, 7633299363274191022, 7826192215519583559, 13179997700913373671, 13543800184894407472, 580858008965596842, 17258443262315043842, 641487995285254966, 10137942184721423538, 14106262123897551947, 4174194737509894142, 10628373955593854664, 7265077268168996740, 7320088937050728612, 5787043690325848934, 17842659121433153342, 12335529207851553230, 1946160827879754098, 10826214908810466724, 2215161156995888572, 11416387303179825584],
                            },
                            E: 65537,
                        },
                        Version: 3,
                        SerialNumber: {
                            neg: false,
                            abs: [3165720407405323205],
                        },
                        Issuer: {
                            Country: nil,
                            Organization: nil,
                            OrganizationalUnit: ["openshift"],
                            Locality: nil,
                            Province: nil,
                            StreetAddress: nil,
                            PostalCode: nil,
                            SerialNumber: "",
                            CommonName: "root-ca",
                            Names: [
                                {
                                    Type: [2, 5, 4, 11],
                                    Value: <string>"openshift",
                                },
                                {
                                    Type: [2, 5, 4, 3],
                                    Value: <string>"root-ca",
                                },
                            ],
                            ExtraNames: nil,
                        },
                        Subject: {
                            Country: nil,
                            Organization: ["openshift"],
                            OrganizationalUnit: nil,
                            Locality: nil,
                            Province: nil,
                            StreetAddress: nil,
                            PostalCode: nil,
                            SerialNumber: "",
                            CommonName: "openshift-ingress",
                            Names: [
                                {
                                    Type: [2, 5, 4, 10],
                                    Value: <string>"openshift",
                                },
                                {
                                    Type: [2, 5, 4, 3],
                                    Value: <string>"openshift-ingress",
                                },
                            ],
                            ExtraNames: nil,
                        },
                        NotBefore: {wall: 0, ext: 63928424791, loc: nil},
                        NotAfter: {wall: 0, ext: 63959960791, loc: nil},
                        KeyUsage: 5,
                        Extensions: [
                            {
                                Id: [2, 5, 29, 15],
                                Critical: true,
                                Value: [3, 2, 5, 160],
                            },
                            {
                                Id: [2, 5, 29, 37],
                                Critical: false,
                                Value: [48, 20, 6, 8, 43, 6, 1, 5, 5, 7, 3, 2, 6, 8, 43, 6, 1, 5, 5, 7, 3, 1],
                            },
                            {Id: [2, 5, 29, 19], Critical: true, Value: [48, 0]},
                            {
                                Id: [2, 5, 29, 14],
                                Critical: false,
                                Value: [4, 64, 219, 47, 173, 50, 97, 112, 158, 229, 198, 244, 185, 216, 225, 191, 88, 22, 77, 0, 234, 97, 129, 231, 80, 228, 92, 178, 0, 52, 83, 87, 158, 115, 94, 72, 111, 209, 220, 55, 16, 127, 98, 248, 205, 26, 77, 249, 92, 91, 236, 8, 82, 113, 93, 251, 205, 152, 219, 254, 197, 208, 47, 119, 143, 181],
                            },
                            {
                                Id: [2, 5, 29, 35],
                                Critical: false,
                                Value: [48, 66, 128, 64, 79, 93, 7, 222, 176, 37, 12, 196, 224, 158, 229, 29, 81, 92, 248, 116, 80, 238, 46, 189, 156, 203, 133, 105, 107, 93, 137, 32, 23, 78, 176, 251, 113, 92, 180, 16, 114, 85, 131, 218, 105, 0, 244, 88, 176, 180, 148, 6, 119, 219, 149, 117, 27, 51, 22, 216, 24, 255, 81, 62, 12, 64, 10, 184],
                            },
                            {
                                Id: [2, 5, 29, 17],
                                Critical: false,
                                Value: "0C\x82A*.apps.aro.s6u8l5i9p5a2r2r.x5o7.westus3.aroapp-hcp.azure-test.net",
                            },
                        ],
                        ExtraExtensions: nil,
                        UnhandledCriticalExtensions: nil,
                        ExtKeyUsage: [2, 1],
                        UnknownExtKeyUsage: nil,
                        BasicConstraintsValid: true,
                        IsCA: false,
                        MaxPathLen: -1,
                        MaxPathLenZero: false,
                        SubjectKeyId: [219, 47, 173, 50, 97, 112, 158, 229, 198, 244, 185, 216, 225, 191, 88, 22, 77, 0, 234, 97, 129, 231, 80, 228, 92, 178, 0, 52, 83, 87, 158, 115, 94, 72, 111, 209, 220, 55, 16, 127, 98, 248, 205, 26, 77, 249, 92, 91, 236, 8, 82, 113, 93, 251, 205, 152, 219, 254, 197, 208, 47, 119, 143, 181],
                        AuthorityKeyId: [79, 93, 7, 222, 176, 37, 12, 196, 224, 158, 229, 29, 81, 92, 248, 116, 80, 238, 46, 189, 156, 203, 133, 105, 107, 93, 137, 32, 23, 78, 176, 251, 113, 92, 180, 16, 114, 85, 131, 218, 105, 0, 244, 88, 176, 180, 148, 6, 119, 219, 149, 117, 27, 51, 22, 216, 24, 255, 81, 62, 12, 64, 10, 184],
                        OCSPServer: nil,
                        IssuingCertificateURL: nil,
                        DNSNames: [
                            "*.apps.aro.s6u8l5i9p5a2r2r.x5o7.westus3.aroapp-hcp.azure-test.net",
                        ],
                        EmailAddresses: nil,
                        IPAddresses: nil,
                        URIs: nil,
                        PermittedDNSDomainsCritical: false,
                        PermittedDNSDomains: nil,
                        ExcludedDNSDomains: nil,
                        PermittedIPRanges: nil,
                        ExcludedIPRanges: nil,
                        PermittedEmailAddresses: nil,
                        ExcludedEmailAddresses: nil,
                        PermittedURIDomains: nil,
                        ExcludedURIDomains: nil,
                        CRLDistributionPoints: nil,
                        PolicyIdentifiers: nil,
                        Policies: nil,
                        InhibitAnyPolicy: 0,
                        InhibitAnyPolicyZero: false,
                        InhibitPolicyMapping: 0,
                        InhibitPolicyMappingZero: false,
                        RequireExplicitPolicy: 0,
                        RequireExplicitPolicyZero: false,
                        PolicyMappings: nil,
                    },
                ],
                Err: <x509.UnknownAuthorityError>{
                    Cert: 0x3489afc31208 (cyclic reference),
                    hintErr: nil,
                    hintCert: nil,
                },
            },
        },
    }
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:194]: failed to verify simple web app runs on cluster "cluster-candidate-4-23-p8z7ps"
Unexpected error:
    <*fmt.wrapError | 0x3489af4cbc00>: 
    [strict TLS verification] route was never reachable: Get "https://agnhost-e2e-sample-app-82qkn.apps.aro.s6u8l5i9p5a2r2r.x5o7.westus3.aroapp-hcp.azure-test.net": tls: failed to verify certificate: x509: certificate signed by unknown authority
    {
        msg: "[strict TLS verification] route was never reachable: Get \"https://agnhost-e2e-sample-app-82qkn.apps.aro.s6u8l5i9p5a2r2r.x5o7.westus3.aroapp-hcp.azure-test.net\": tls: failed to verify certificate: x509: certificate signed by unknown authority",
        err: <*url.Error | 0x3489b0e89ec0>{
            Op: "Get",
            URL: "https://agnhost-e2e-sample-app-82qkn.apps.aro.s6u8l5i9p5a2r2r.x5o7.westus3.aroapp-hcp.azure-test.net",
            Err: <*tls.CertificateVerificationError | 0x3489b0e89e90>{
                UnverifiedCertificates: [
                    {
                        Raw: [48, 130, 4, 1, 48, 130, 2, 233, 160, 3, 2, 1, 2, 2, 8, 43, 238, 229, 240, 21, 153, 99, 197, 48, 13, 6, 9, 42, 134, 72, 134, 247, 13, 1, 1, 11, 5, 0, 48, 38, 49, 18, 48, 16, 6, 3, 85, 4, 11, 19, 9, 111, 112, 101, 110, 115, 104, 105, 102, 116, 49, 16, 48, 14, 6, 3, 85, 4, 3, 19, 7, 114, 111, 111, 116, 45, 99, 97, 48, 30, 23, 13, 50, 54, 49, 48, 50, 52, 48, 55, 52, 54, 51, 49, 90, 23, 13, 50, 55, 49, 48, 50, 52, 48, 55, 52, 54, 51, 49, 90, 48, 48, 49, 18, 48, 16, 6, 3, 85, 4, 10, 19, 9, 111, 112, 101, 110, 115, 104, 105, 102, 116, 49, 26, 48, 24, 6, 3, 85, 4, 3, 19, 17, 111, 112, 101, 110, 115, 104, 105, 102, 116, 45, 105, 110, 103, 114, 101, 115, 115, 48, 130, 1, 34, 48, 13, 6, 9, 42, 134, 72, 134, 247, 13, 1, 1, 1, 5, 0, 3, 130, 1, 15, 0, 48, 130, 1, 10, 2, 130, 1, 1, 0, 158, 111, 39, 196, 0, 80, 49, 176, 30, 189, 213, 85, 250, 251, 181, 188, 150, 62, 113, 25, 141, 175, 149, 164, 27, 2, 38, 246, 53, 141, 137, 114, 171, 48, 154, 117, 70, 114, 145, 206, 247, 157, 219, 239, 189, 104, 15, 62, 80, 79, 181, 147, 69, 137, 179, 102, 101, 150, 46, 26, 209, 44, 232, 164, 100, 210, 189, 72, 138, 250, 119, 132, 147, 127, 145, 205, 19, 47, 94, 200, 57, 237, 183, 255, 120, 149, 7, 254, 195, 195, 130, 169, 219, 117, 252, 75, 140, 177, 52, 180, 49, 196, 156, 178, 8, 231, 5, 238, 214, 167, 179, 54, 239, 130, 78, 181, 58, 133, 156, 2, 8, 15, 159, 72, 25, 85, 198, 170, 187, 245, 62, 130, 145, 128, 67, 48, 182, 232, 194, 20, 15, 104, 33, 231, 108, 156, 56, 90, 229, 248, 221, 71, 105, 238, 237, 83, 244, 44, 112, 174, 38, 139, 77, 150, 145, 31, 165, 13, 167, 10, 136, 108, 72, 232, 249, 192, 115, 38, 117, 183, 31, 134, 156, 103, 202, 99, 106, 69, 51, 223, 14, 115, 32, 93, 24, 6, 153, 129, 175, 17, 63, 202, 109, 233, 56, 75, 87, 80, 147, 28, 70, 160, 229, 87, 181, 34, 53, 223, 172, 119, 241, 56, 131, 144, 20, 138, 28, 73, 129, 122, 162, 3, 187, 23, 218, 232, 140, 65, 52, 71, 184, 117, 40, 197, 78, 116, 38, 248, 6, 148, 15, 34, 201, 88, 176, 243, 2, 3, 1, 0, 1, 163, 130, 1, 39, 48, 130, 1, 35, 48, 14, 6, 3, 85, 29, 15, 1, 1, 255, 4, 4, 3, 2, 5, 160, 48, 29, 6, 3, 85, 29, 37, 4, 22, 48, 20, 6, 8, 43, 6, 1, 5, 5, 7, 3, 2, 6, 8, 43, 6, 1, 5, 5, 7, 3, 1, 48, 12, 6, 3, 85, 29, 19, 1, 1, 255, 4, 2, 48, 0, 48, 73, 6, 3, 85, 29, 14, 4, 66, 4, 64, 219, 47, 173, 50, 97, 112, 158, 229, 198, 244, 185, 216, 225, 191, 88, 22, 77, 0, 234, 97, 129, 231, 80, 228, 92, 178, 0, 52, 83, 87, 158, 115, 94, 72, 111, 209, 220, 55, 16, 127, 98, 248, 205, 26, 77, 249, 92, 91, 236, 8, 82, 113, 93, 251, 205, 152, 219, 254, 197, 208, 47, 119, 143, 181, 48, 75, 6, 3, 85, 29, 35, 4, 68, 48, 66, 128, 64, 79, 93, 7, 222, 176, 37, 12, 196, 224, 158, 229, 29, 81, 92, 248, 116, 80, 238, 46, 189, 156, 203, 133, 105, 107, 93, 137, 32, 23, 78, 176, 251, 113, 92, 180, 16, 114, 85, 131, 218, 105, 0, 244, 88, 176, 180, 148, 6, 119, 219, 149, 117, 27, 51, 22, 216, 24, 255, 81, 62, 12, 64, 10, 184, 48, 76, 6, 3, 85, 29, 17, 4, 69, 48, 67, 130, 65, 42, 46, 97, 112, 112, 115, 46, 97, 114, 111, 46, 115, 54, 117, 56, 108, 53, 105, 57, 112, 53, 97, 50, 114, 50, 114, 46, 120, 53, 111, 55, 46, 119, 101, 115, 116, 117, 115, 51, 46, 97, 114, 111, 97, 112, 112, 45, 104, 99, 112, 46, 97, 122, 117, 114, 101, 45, 116, 101, 115, 116, 46, 110, 101, 116, 48, 13, 6, 9, 42, 134, 72, 134, 247, 13, 1, 1, 11, 5, 0, 3, 130, 1, 1, 0, 65, 22, 134, 247, 228, 246, 36, 128, 182, 182, 244, 211, 102, 134, 109, 155, 171, 246, 80, 230, 175, 242, 200, 121, 170, 15, 231, 61, 30, 214, 43, 76, 202, 69, 76, 61, 232, 34, 150, 103, 156, 30, 87, 9, 251, 54, 13, 22, 50, 15, 186, 189, 51, 5, 9, 112, 178, 116, 29, 114, 168, 183, 30, 46, 238, 184, 247, 18, 68, 187, 209, 102, 42, 144, 123, 148, 189, 172, 148, 231, 172, 163, 24, 180, 12, 57, 57, 178, 28, 24, 47, 149, 2, 234, 44, 182, 125, 31, 5, 6, 94, 212, 163, 96, 132, 15, 25, 239, 102, 10, 14, 137, 196, 239, 124, 196, 146, 234, 2, 154, 140, 93, 166, 102, 235, 230, 253, 165, 47, 222, 167, 190, 70, 248, 172, 11, 69, 126, 194, 30, 221, 154, 97, 116, 195, 146, 31, 196, 31, 89, 67, 18, 84, 239, 174, 111, 105, 212, 184, 30, 84, 72, 237, 112, 85, 195, 211, 87, 184, 193, 65, 175, 132, 9, 196, 121, 203, 35, 118, 177, 151, 41, 106, 163, 169, 108, 86, 51, 253, 163, 7, 87, 188, 188, 151, 64, 24, 95, 174, 171, 164, 183, 237, 175, 25, 65, 122, 245, 38, 195, 178, 150, 104, 85, 228, 73, 164, 50, 181, 10, 140, 141, 232, 207, 156, 86, 106, 228, 38, 234, 189, 150, 66, 176, 12, 174, 201, 173, 149, 206, 151, 120, 120, 106, 251, 69, 123, 225, 252, 241, 122, 27, 207, 29, 170, 4],
                        RawTBSCertificate: [48, 130, 2, 233, 160, 3, 2, 1, 2, 2, 8, 43, 238, 229, 240, 21, 153, 99, 197, 48, 13, 6, 9, 42, 134, 72, 134, 247, 13, 1, 1, 11, 5, 0, 48, 38, 49, 18, 48, 16, 6, 3, 85, 4, 11, 19, 9, 111, 112, 101, 110, 115, 104, 105, 102, 116, 49, 16, 48, 14, 6, 3, 85, 4, 3, 19, 7, 114, 111, 111, 116, 45, 99, 97, 48, 30, 23, 13, 50, 54, 49, 48, 50, 52, 48, 55, 52, 54, 51, 49, 90, 23, 13, 50, 55, 49, 48, 50, 52, 48, 55, 52, 54, 51, 49, 90, 48, 48, 49, 18, 48, 16, 6, 3, 85, 4, 10, 19, 9, 111, 112, 101, 110, 115, 104, 105, 102, 116, 49, 26, 48, 24, 6, 3, 85, 4, 3, 19, 17, 111, 112, 101, 110, 115, 104, 105, 102, 116, 45, 105, 110, 103, 114, 101, 115, 115, 48, 130, 1, 34, 48, 13, 6, 9, 42, 134, 72, 134, 247, 13, 1, 1, 1, 5, 0, 3, 130, 1, 15, 0, 48, 130, 1, 10, 2, 130, 1, 1, 0, 158, 111, 39, 196, 0, 80, 49, 176, 30, 189, 213, 85, 250, 251, 181, 188, 150, 62, 113, 25, 141, 175, 149, 164, 27, 2, 38, 246, 53, 141, 137, 114, 171, 48, 154, 117, 70, 114, 145, 206, 247, 157, 219, 239, 189, 104, 15, 62, 80, 79, 181, 147, 69, 137, 179, 102, 101, 150, 46, 26, 209, 44, 232, 164, 100, 210, 189, 72, 138, 250, 119, 132, 147, 127, 145, 205, 19, 47, 94, 200, 57, 237, 183, 255, 120, 149, 7, 254, 195, 195, 130, 169, 219, 117, 252, 75, 140, 177, 52, 180, 49, 196, 156, 178, 8, 231, 5, 238, 214, 167, 179, 54, 239, 130, 78, 181, 58, 133, 156, 2, 8, 15, 159, 72, 25, 85, 198, 170, 187, 245, 62, 130, 145, 128, 67, 48, 182, 232, 194, 20, 15, 104, 33, 231, 108, 156, 56, 90, 229, 248, 221, 71, 105, 238, 237, 83, 244, 44, 112, 174, 38, 139, 77, 150, 145, 31, 165, 13, 167, 10, 136, 108, 72, 232, 249, 192, 115, 38, 117, 183, 31, 134, 156, 103, 202, 99, 106, 69, 51, 223, 14, 115, 32, 93, 24, 6, 153, 129, 175, 17, 63, 202, 109, 233, 56, 75, 87, 80, 147, 28, 70, 160, 229, 87, 181, 34, 53, 223, 172, 119, 241, 56, 131, 144, 20, 138, 28, 73, 129, 122, 162, 3, 187, 23, 218, 232, 140, 65, 52, 71, 184, 117, 40, 197, 78, 116, 38, 248, 6, 148, 15, 34, 201, 88, 176, 243, 2, 3, 1, 0, 1, 163, 130, 1, 39, 48, 130, 1, 35, 48, 14, 6, 3, 85, 29, 15, 1, 1, 255, 4, 4, 3, 2, 5, 160, 48, 29, 6, 3, 85, 29, 37, 4, 22, 48, 20, 6, 8, 43, 6, 1, 5, 5, 7, 3, 2, 6, 8, 43, 6, 1, 5, 5, 7, 3, 1, 48, 12, 6, 3, 85, 29, 19, 1, 1, 255, 4, 2, 48, 0, 48, 73, 6, 3, 85, 29, 14, 4, 66, 4, 64, 219, 47, 173, 50, 97, 112, 158, 229, 198, 244, 185, 216, 225, 191, 88, 22, 77, 0, 234, 97, 129, 231, 80, 228, 92, 178, 0, 52, 83, 87, 158, 115, 94, 72, 111, 209, 220, 55, 16, 127, 98, 248, 205, 26, 77, 249, 92, 91, 236, 8, 82, 113, 93, 251, 205, 152, 219, 254, 197, 208, 47, 119, 143, 181, 48, 75, 6, 3, 85, 29, 35, 4, 68, 48, 66, 128, 64, 79, 93, 7, 222, 176, 37, 12, 196, 224, 158, 229, 29, 81, 92, 248, 116, 80, 238, 46, 189, 156, 203, 133, 105, 107, 93, 137, 32, 23, 78, 176, 251, 113, 92, 180, 16, 114, 85, 131, 218, 105, 0, 244, 88, 176, 180, 148, 6, 119, 219, 149, 117, 27, 51, 22, 216, 24, 255, 81, 62, 12, 64, 10, 184, 48, 76, 6, 3, 85, 29, 17, 4, 69, 48, 67, 130, 65, 42, 46, 97, 112, 112, 115, 46, 97, 114, 111, 46, 115, 54, 117, 56, 108, 53, 105, 57, 112, 53, 97, 50, 114, 50, 114, 46, 120, 53, 111, 55, 46, 119, 101, 115, 116, 117, 115, 51, 46, 97, 114, 111, 97, 112, 112, 45, 104, 99, 112, 46, 97, 122, 117, 114, 101, 45, 116, 101, 115, 116, 46, 110, 101, 116],
                        RawSubjectPublicKeyInfo: [48, 130, 1, 34, 48, 13, 6, 9, 42, 134, 72, 134, 247, 13, 1, 1, 1, 5, 0, 3, 130, 1, 15, 0, 48, 130, 1, 10, 2, 130, 1, 1, 0, 158, 111, 39, 196, 0, 80, 49, 176, 30, 189, 213, 85, 250, 251, 181, 188, 150, 62, 113, 25, 141, 175, 149, 164, 27, 2, 38, 246, 53, 141, 137, 114, 171, 48, 154, 117, 70, 114, 145, 206, 247, 157, 219, 239, 189, 104, 15, 62, 80, 79, 181, 147, 69, 137, 179, 102, 101, 150, 46, 26, 209, 44, 232, 164, 100, 210, 189, 72, 138, 250, 119, 132, 147, 127, 145, 205, 19, 47, 94, 200, 57, 237, 183, 255, 120, 149, 7, 254, 195, 195, 130, 169, 219, 117, 252, 75, 140, 177, 52, 180, 49, 196, 156, 178, 8, 231, 5, 238, 214, 167, 179, 54, 239, 130, 78, 181, 58, 133, 156, 2, 8, 15, 159, 72, 25, 85, 198, 170, 187, 245, 62, 130, 145, 128, 67, 48, 182, 232, 194, 20, 15, 104, 33, 231, 108, 156, 56, 90, 229, 248, 221, 71, 105, 238, 237, 83, 244, 44, 112, 174, 38, 139, 77, 150, 145, 31, 165, 13, 167, 10, 136, 108, 72, 232, 249, 192, 115, 38, 117, 183, 31, 134, 156, 103, 202, 99, 106, 69, 51, 223, 14, 115, 32, 93, 24, 6, 153, 129, 175, 17, 63, 202, 109, 233, 56, 75, 87, 80, 147, 28, 70, 160, 229, 87, 181, 34, 53, 223, 172, 119, 241, 56, 131, 144, 20, 138, 28, 73, 129, 122, 162, 3, 187, 23, 218, 232, 140, 65, 52, 71, 184, 117, 40, 197, 78, 116, 38, 248, 6, 148, 15, 34, 201, 88, 176, 243, 2, 3, 1, 0, 1],
                        RawSubject: [48, 48, 49, 18, 48, 16, 6, 3, 85, 4, 10, 19, 9, 111, 112, 101, 110, 115, 104, 105, 102, 116, 49, 26, 48, 24, 6, 3, 85, 4, 3, 19, 17, 111, 112, 101, 110, 115, 104, 105, 102, 116, 45, 105, 110, 103, 114, 101, 115, 115],
                        RawIssuer: [48, 38, 49, 18, 48, 16, 6, 3, 85, 4, 11, 19, 9, 111, 112, 101, 110, 115, 104, 105, 102, 116, 49, 16, 48, 14, 6, 3, 85, 4, 3, 19, 7, 114, 111, 111, 116, 45, 99, 97],
                        Signature: [65, 22, 134, 247, 228, 246, 36, 128, 182, 182, 244, 211, 102, 134, 109, 155, 171, 246, 80, 230, 175, 242, 200, 121, 170, 15, 231, 61, 30, 214, 43, 76, 202, 69, 76, 61, 232, 34, 150, 103, 156, 30, 87, 9, 251, 54, 13, 22, 50, 15, 186, 189, 51, 5, 9, 112, 178, 116, 29, 114, 168, 183, 30, 46, 238, 184, 247, 18, 68, 187, 209, 102, 42, 144, 123, 148, 189, 172, 148, 231, 172, 163, 24, 180, 12, 57, 57, 178, 28, 24, 47, 149, 2, 234, 44, 182, 125, 31, 5, 6, 94, 212, 163, 96, 132, 15, 25, 239, 102, 10, 14, 137, 196, 239, 124, 196, 146, 234, 2, 154, 140, 93, 166, 102, 235, 230, 253, 165, 47, 222, 167, 190, 70, 248, 172, 11, 69, 126, 194, 30, 221, 154, 97, 116, 195, 146, 31, 196, 31, 89, 67, 18, 84, 239, 174, 111, 105, 212, 184, 30, 84, 72, 237, 112, 85, 195, 211, 87, 184, 193, 65, 175, 132, 9, 196, 121, 203, 35, 118, 177, 151, 41, 106, 163, 169, 108, 86, 51, 253, 163, 7, 87, 188, 188, 151, 64, 24, 95, 174, 171, 164, 183, 237, 175, 25, 65, 122, 245, 38, 195, 178, 150, 104, 85, 228, 73, 164, 50, 181, 10, 140, 141, 232, 207, 156, 86, 106, 228, 38, 234, 189, 150, 66, 176, 12, 174, 201, 173, 149, 206, 151, 120, 120, 106, 251, 69, 123, 225, 252, 241, 122, 27, 207, 29, 170, 4],
                        SignatureAlgorithm: 4,
                        PublicKeyAlgorithm: 1,
                        PublicKey: <*rsa.PublicKey | 0x3489b0c807b0>{
                            N: {
                                neg: false,
                                abs: [474020502862082291, 13291574703143790328, 13481484701877220423, 1480026529575117315, 3882011034964755344, 10600425279779747106, 4596607218124281680, 2332046598672068369, 14583616863811341939, 8297448792845098087, 12036582952762145216, 2777398904279246093, 7633299363274191022, 7826192215519583559, 13179997700913373671, 13543800184894407472, 580858008965596842, 17258443262315043842, 641487995285254966, 10137942184721423538, 14106262123897551947, 4174194737509894142, 10628373955593854664, 7265077268168996740, 7320088937050728612, 5787043690325848934, 17842659121433153342, 12335529207851553230, 1946160827879754098, 10826214908810466724, 2215161156995888572, 11416387303179825584],
                            },
                            E: 65537,
                        },
                        Version: 3,
                        SerialNumber: {
                            neg: false,
                            abs: [3165720407405323205],
                        },
                        Issuer: {
                            Country: nil,
                            Organization: nil,
                            OrganizationalUnit: ["openshift"],
                            Locality: nil,
                            Province: nil,
                            StreetAddress: nil,
                            PostalCode: nil,
                            SerialNumber: "",
                            CommonName: "root-ca",
                            Names: [
                                {
                                    Type: [2, 5, 4, 11],
                                    Value: <string>"openshift",
                                },
                                {
                                    Type: [2, 5, 4, 3],
                                    Value: <string>"root-ca",
                                },
                            ],
                            ExtraNames: nil,
                        },
                        Subject: {
                            Country: nil,
                            Organization: ["openshift"],
                            OrganizationalUnit: nil,
                            Locality: nil,
                            Province: nil,
                            StreetAddress: nil,
                            PostalCode: nil,
                            SerialNumber: "",
                            CommonName: "openshift-ingress",
                            Names: [
                                {
                                    Type: [2, 5, 4, 10],
                                    Value: <string>"openshift",
                                },
                                {
                                    Type: [2, 5, 4, 3],
                                    Value: <string>"openshift-ingress",
                                },
                            ],
                            ExtraNames: nil,
                        },
                        NotBefore: {wall: 0, ext: 63928424791, loc: nil},
                        NotAfter: {wall: 0, ext: 63959960791, loc: nil},
                        KeyUsage: 5,
                        Extensions: [
                            {
                                Id: [2, 5, 29, 15],
                                Critical: true,
                                Value: [3, 2, 5, 160],
                            },
                            {
                                Id: [2, 5, 29, 37],
                                Critical: false,
                                Value: [48, 20, 6, 8, 43, 6, 1, 5, 5, 7, 3, 2, 6, 8, 43, 6, 1, 5, 5, 7, 3, 1],
                            },
                            {Id: [2, 5, 29, 19], Critical: true, Value: [48, 0]},
                            {
                                Id: [2, 5, 29, 14],
                                Critical: false,
                                Value: [4, 64, 219, 47, 173, 50, 97, 112, 158, 229, 198, 244, 185, 216, 225, 191, 88, 22, 77, 0, 234, 97, 129, 231, 80, 228, 92, 178, 0, 52, 83, 87, 158, 115, 94, 72, 111, 209, 220, 55, 16, 127, 98, 248, 205, 26, 77, 249, 92, 91, 236, 8, 82, 113, 93, 251, 205, 152, 219, 254, 197, 208, 47, 119, 143, 181],
                            },
                            {
                                Id: [2, 5, 29, 35],
                                Critical: false,
                                Value: [48, 66, 128, 64, 79, 93, 7, 222, 176, 37, 12, 196, 224, 158, 229, 29, 81, 92, 248, 116, 80, 238, 46, 189, 156, 203, 133, 105, 107, 93, 137, 32, 23, 78, 176, 251, 113, 92, 180, 16, 114, 85, 131, 218, 105, 0, 244, 88, 176, 180, 148, 6, 119, 219, 149, 117, 27, 51, 22, 216, 24, 255, 81, 62, 12, 64, 10, 184],
                            },
                            {
                                Id: [2, 5, 29, 17],
                                Critical: false,
                                Value: "0C\x82A*.apps.aro.s6u8l5i9p5a2r2r.x5o7.westus3.aroapp-hcp.azure-test.net",
                            },
                        ],
                        ExtraExtensions: nil,
                        UnhandledCriticalExtensions: nil,
                        ExtKeyUsage: [2, 1],
                        UnknownExtKeyUsage: nil,
                        BasicConstraintsValid: true,
                        IsCA: false,
                        MaxPathLen: -1,
                        MaxPathLenZero: false,
                        SubjectKeyId: [219, 47, 173, 50, 97, 112, 158, 229, 198, 244, 185, 216, 225, 191, 88, 22, 77, 0, 234, 97, 129, 231, 80, 228, 92, 178, 0, 52, 83, 87, 158, 115, 94, 72, 111, 209, 220, 55, 16, 127, 98, 248, 205, 26, 77, 249, 92, 91, 236, 8, 82, 113, 93, 251, 205, 152, 219, 254, 197, 208, 47, 119, 143, 181],
                        AuthorityKeyId: [79, 93, 7, 222, 176, 37, 12, 196, 224, 158, 229, 29, 81, 92, 248, 116, 80, 238, 46, 189, 156, 203, 133, 105, 107, 93, 137, 32, 23, 78, 176, 251, 113, 92, 180, 16, 114, 85, 131, 218, 105, 0, 244, 88, 176, 180, 148, 6, 119, 219, 149, 117, 27, 51, 22, 216, 24, 255, 81, 62, 12, 64, 10, 184],
                        OCSPServer: nil,
                        IssuingCertificateURL: nil,
                        DNSNames: [
                            "*.apps.aro.s6u8l5i9p5a2r2r.x5o7.westus3.aroapp-hcp.azure-test.net",
                        ],
                        EmailAddresses: nil,
                        IPAddresses: nil,
                        URIs: nil,
                        PermittedDNSDomainsCritical: false,
                        PermittedDNSDomains: nil,
                        ExcludedDNSDomains: nil,
                        PermittedIPRanges: nil,
                        ExcludedIPRanges: nil,
                        PermittedEmailAddresses: nil,
                        ExcludedEmailAddresses: nil,
                        PermittedURIDomains: nil,
                        ExcludedURIDomains: nil,
                        CRLDistributionPoints: nil,
                        PolicyIdentifiers: nil,
                        Policies: nil,
                        InhibitAnyPolicy: 0,
                        InhibitAnyPolicyZero: false,
                        InhibitPolicyMapping: 0,
                        InhibitPolicyMappingZero: false,
                        RequireExplicitPolicy: 0,
                        RequireExplicitPolicyZero: false,
                        PolicyMappings: nil,
                    },
                ],
                Err: <x509.UnknownAuthorityError>{
                    Cert: 0x3489afc31208 (cyclic reference),
                    hintErr: nil,
                    hintCert: nil,
                },
            },
        },
    }
occurred
1h 20m 0s branch-ci-Azure-ARO-HCP-main-e2e-integration-e2e-parallel-all
post-goodmerged PR
No additional run metadata captured
westus3 unknown failed n/a 1
unknown (1)
tls: failed to verify certificate: x509: certificate signed by unknown authority
06:04:44 UTC · failed at unknown · test ARO-HCP should be able to perform a control plane and node pool install with OCP candidate channel for 4.23 · suite integration/parallel/all
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:194]: failed to verify simple web app runs on cluster "cluster-candidate-4-23-l4g2v7"
Unexpected error:
    <*fmt.wrapError | 0x152b7c5bb0a0>: 
    [strict TLS verification] route was never reachable: Get "https://agnhost-e2e-sample-app-vl8dn.apps.aro.h5g1x1a1f6d6x3b.gps5.westus3.aroapp-hcp.azure-test.net": tls: failed to verify certificate: x509: certificate signed by unknown authority
    {
        msg: "[strict TLS verification] route was never reachable: Get \"https://agnhost-e2e-sample-app-vl8dn.apps.aro.h5g1x1a1f6d6x3b.gps5.westus3.aroapp-hcp.azure-test.net\": tls: failed to verify certificate: x509: certificate signed by unknown authority",
        err: <*url.Error | 0x152b7da88060>{
            Op: "Get",
            URL: "https://agnhost-e2e-sample-app-vl8dn.apps.aro.h5g1x1a1f6d6x3b.gps5.westus3.aroapp-hcp.azure-test.net",
            Err: <*tls.CertificateVerificationError | 0x152b7da88030>{
                UnverifiedCertificates: [
                    {
                        Raw: [48, 130, 4, 1, 48, 130, 2, 233, 160, 3, 2, 1, 2, 2, 8, 72, 125, 244, 209, 123, 188, 206, 189, 48, 13, 6, 9, 42, 134, 72, 134, 247, 13, 1, 1, 11, 5, 0, 48, 38, 49, 18, 48, 16, 6, 3, 85, 4, 11, 19, 9, 111, 112, 101, 110, 115, 104, 105, 102, 116, 49, 16, 48, 14, 6, 3, 85, 4, 3, 19, 7, 114, 111, 111, 116, 45, 99, 97, 48, 30, 23, 13, 50, 54, 49, 48, 50, 52, 48, 54, 50, 56, 51, 54, 90, 23, 13, 50, 55, 49, 48, 50, 52, 48, 54, 50, 56, 51, 54, 90, 48, 48, 49, 18, 48, 16, 6, 3, 85, 4, 10, 19, 9, 111, 112, 101, 110, 115, 104, 105, 102, 116, 49, 26, 48, 24, 6, 3, 85, 4, 3, 19, 17, 111, 112, 101, 110, 115, 104, 105, 102, 116, 45, 105, 110, 103, 114, 101, 115, 115, 48, 130, 1, 34, 48, 13, 6, 9, 42, 134, 72, 134, 247, 13, 1, 1, 1, 5, 0, 3, 130, 1, 15, 0, 48, 130, 1, 10, 2, 130, 1, 1, 0, 230, 190, 96, 104, 10, 59, 179, 162, 174, 46, 142, 192, 200, 19, 105, 174, 207, 152, 161, 202, 176, 15, 56, 243, 170, 157, 113, 39, 213, 91, 37, 178, 7, 11, 150, 37, 109, 247, 110, 16, 213, 139, 0, 197, 124, 14, 0, 215, 252, 76, 104, 31, 165, 18, 63, 111, 231, 249, 165, 15, 146, 216, 139, 189, 27, 110, 122, 122, 47, 7, 93, 22, 38, 58, 244, 252, 34, 32, 133, 9, 218, 213, 26, 223, 83, 182, 89, 196, 95, 203, 91, 164, 98, 168, 62, 125, 198, 234, 206, 40, 244, 188, 118, 48, 46, 72, 43, 209, 41, 172, 155, 120, 222, 176, 170, 50, 117, 208, 205, 144, 80, 247, 213, 39, 212, 26, 218, 201, 101, 208, 49, 58, 238, 11, 48, 246, 212, 64, 124, 133, 100, 178, 36, 156, 116, 61, 190, 97, 187, 51, 185, 96, 93, 222, 140, 0, 114, 212, 22, 74, 240, 19, 149, 134, 52, 132, 175, 158, 99, 109, 231, 83, 168, 100, 126, 206, 64, 230, 226, 31, 8, 31, 32, 178, 246, 254, 161, 134, 237, 124, 213, 160, 101, 83, 145, 30, 84, 167, 201, 93, 240, 25, 238, 76, 169, 145, 190, 97, 234, 15, 148, 218, 216, 74, 228, 173, 107, 191, 109, 211, 218, 68, 168, 6, 146, 226, 219, 69, 169, 143, 159, 194, 142, 119, 237, 161, 31, 229, 182, 88, 35, 251, 235, 167, 126, 85, 130, 1, 82, 182, 68, 77, 144, 154, 99, 195, 2, 3, 1, 0, 1, 163, 130, 1, 39, 48, 130, 1, 35, 48, 14, 6, 3, 85, 29, 15, 1, 1, 255, 4, 4, 3, 2, 5, 160, 48, 29, 6, 3, 85, 29, 37, 4, 22, 48, 20, 6, 8, 43, 6, 1, 5, 5, 7, 3, 2, 6, 8, 43, 6, 1, 5, 5, 7, 3, 1, 48, 12, 6, 3, 85, 29, 19, 1, 1, 255, 4, 2, 48, 0, 48, 73, 6, 3, 85, 29, 14, 4, 66, 4, 64, 124, 43, 155, 38, 67, 14, 112, 156, 220, 167, 147, 27, 204, 115, 135, 241, 3, 184, 222, 93, 50, 70, 132, 146, 25, 139, 25, 95, 207, 250, 219, 139, 68, 58, 234, 212, 61, 207, 175, 41, 112, 193, 157, 177, 6, 156, 82, 245, 173, 210, 245, 105, 124, 108, 52, 248, 245, 189, 19, 182, 11, 124, 43, 147, 48, 75, 6, 3, 85, 29, 35, 4, 68, 48, 66, 128, 64, 37, 9, 140, 129, 208, 247, 167, 63, 140, 8, 100, 56, 91, 156, 184, 205, 89, 175, 125, 151, 196, 90, 193, 216, 60, 234, 131, 252, 208, 26, 206, 229, 55, 180, 1, 107, 71, 121, 228, 7, 250, 96, 76, 59, 240, 108, 125, 148, 30, 213, 19, 231, 74, 254, 82, 156, 51, 79, 97, 37, 64, 176, 221, 66, 48, 76, 6, 3, 85, 29, 17, 4, 69, 48, 67, 130, 65, 42, 46, 97, 112, 112, 115, 46, 97, 114, 111, 46, 104, 53, 103, 49, 120, 49, 97, 49, 102, 54, 100, 54, 120, 51, 98, 46, 103, 112, 115, 53, 46, 119, 101, 115, 116, 117, 115, 51, 46, 97, 114, 111, 97, 112, 112, 45, 104, 99, 112, 46, 97, 122, 117, 114, 101, 45, 116, 101, 115, 116, 46, 110, 101, 116, 48, 13, 6, 9, 42, 134, 72, 134, 247, 13, 1, 1, 11, 5, 0, 3, 130, 1, 1, 0, 93, 97, 144, 151, 119, 127, 24, 191, 35, 91, 46, 31, 22, 213, 193, 224, 185, 125, 42, 132, 155, 1, 157, 220, 86, 62, 125, 196, 10, 101, 224, 138, 230, 13, 126, 222, 36, 93, 135, 77, 176, 165, 1, 164, 174, 94, 96, 9, 215, 225, 76, 122, 138, 7, 69, 157, 164, 22, 155, 228, 2, 85, 241, 21, 174, 51, 32, 43, 65, 19, 228, 121, 80, 95, 150, 246, 168, 44, 68, 26, 192, 84, 76, 194, 80, 53, 205, 73, 96, 252, 13, 44, 33, 242, 254, 237, 5, 223, 40, 80, 20, 187, 19, 237, 36, 248, 36, 50, 76, 9, 65, 124, 13, 0, 68, 223, 227, 132, 32, 107, 173, 164, 11, 66, 80, 172, 230, 109, 226, 64, 134, 186, 209, 232, 110, 180, 186, 163, 140, 40, 78, 22, 125, 236, 179, 99, 155, 208, 217, 248, 108, 50, 230, 57, 131, 159, 87, 122, 183, 90, 172, 87, 235, 154, 28, 71, 191, 228, 246, 209, 28, 15, 19, 121, 148, 64, 99, 238, 140, 107, 150, 155, 218, 208, 46, 134, 213, 151, 67, 109, 8, 183, 4, 28, 56, 193, 144, 28, 152, 198, 182, 241, 21, 227, 31, 182, 250, 132, 24, 64, 182, 38, 59, 84, 63, 86, 167, 157, 30, 177, 142, 222, 51, 90, 139, 52, 20, 156, 95, 203, 109, 26, 217, 46, 63, 119, 60, 97, 116, 74, 82, 241, 103, 109, 27, 20, 184, 169, 56, 149, 20, 168, 6, 187, 84, 186],
                        RawTBSCertificate: [48, 130, 2, 233, 160, 3, 2, 1, 2, 2, 8, 72, 125, 244, 209, 123, 188, 206, 189, 48, 13, 6, 9, 42, 134, 72, 134, 247, 13, 1, 1, 11, 5, 0, 48, 38, 49, 18, 48, 16, 6, 3, 85, 4, 11, 19, 9, 111, 112, 101, 110, 115, 104, 105, 102, 116, 49, 16, 48, 14, 6, 3, 85, 4, 3, 19, 7, 114, 111, 111, 116, 45, 99, 97, 48, 30, 23, 13, 50, 54, 49, 48, 50, 52, 48, 54, 50, 56, 51, 54, 90, 23, 13, 50, 55, 49, 48, 50, 52, 48, 54, 50, 56, 51, 54, 90, 48, 48, 49, 18, 48, 16, 6, 3, 85, 4, 10, 19, 9, 111, 112, 101, 110, 115, 104, 105, 102, 116, 49, 26, 48, 24, 6, 3, 85, 4, 3, 19, 17, 111, 112, 101, 110, 115, 104, 105, 102, 116, 45, 105, 110, 103, 114, 101, 115, 115, 48, 130, 1, 34, 48, 13, 6, 9, 42, 134, 72, 134, 247, 13, 1, 1, 1, 5, 0, 3, 130, 1, 15, 0, 48, 130, 1, 10, 2, 130, 1, 1, 0, 230, 190, 96, 104, 10, 59, 179, 162, 174, 46, 142, 192, 200, 19, 105, 174, 207, 152, 161, 202, 176, 15, 56, 243, 170, 157, 113, 39, 213, 91, 37, 178, 7, 11, 150, 37, 109, 247, 110, 16, 213, 139, 0, 197, 124, 14, 0, 215, 252, 76, 104, 31, 165, 18, 63, 111, 231, 249, 165, 15, 146, 216, 139, 189, 27, 110, 122, 122, 47, 7, 93, 22, 38, 58, 244, 252, 34, 32, 133, 9, 218, 213, 26, 223, 83, 182, 89, 196, 95, 203, 91, 164, 98, 168, 62, 125, 198, 234, 206, 40, 244, 188, 118, 48, 46, 72, 43, 209, 41, 172, 155, 120, 222, 176, 170, 50, 117, 208, 205, 144, 80, 247, 213, 39, 212, 26, 218, 201, 101, 208, 49, 58, 238, 11, 48, 246, 212, 64, 124, 133, 100, 178, 36, 156, 116, 61, 190, 97, 187, 51, 185, 96, 93, 222, 140, 0, 114, 212, 22, 74, 240, 19, 149, 134, 52, 132, 175, 158, 99, 109, 231, 83, 168, 100, 126, 206, 64, 230, 226, 31, 8, 31, 32, 178, 246, 254, 161, 134, 237, 124, 213, 160, 101, 83, 145, 30, 84, 167, 201, 93, 240, 25, 238, 76, 169, 145, 190, 97, 234, 15, 148, 218, 216, 74, 228, 173, 107, 191, 109, 211, 218, 68, 168, 6, 146, 226, 219, 69, 169, 143, 159, 194, 142, 119, 237, 161, 31, 229, 182, 88, 35, 251, 235, 167, 126, 85, 130, 1, 82, 182, 68, 77, 144, 154, 99, 195, 2, 3, 1, 0, 1, 163, 130, 1, 39, 48, 130, 1, 35, 48, 14, 6, 3, 85, 29, 15, 1, 1, 255, 4, 4, 3, 2, 5, 160, 48, 29, 6, 3, 85, 29, 37, 4, 22, 48, 20, 6, 8, 43, 6, 1, 5, 5, 7, 3, 2, 6, 8, 43, 6, 1, 5, 5, 7, 3, 1, 48, 12, 6, 3, 85, 29, 19, 1, 1, 255, 4, 2, 48, 0, 48, 73, 6, 3, 85, 29, 14, 4, 66, 4, 64, 124, 43, 155, 38, 67, 14, 112, 156, 220, 167, 147, 27, 204, 115, 135, 241, 3, 184, 222, 93, 50, 70, 132, 146, 25, 139, 25, 95, 207, 250, 219, 139, 68, 58, 234, 212, 61, 207, 175, 41, 112, 193, 157, 177, 6, 156, 82, 245, 173, 210, 245, 105, 124, 108, 52, 248, 245, 189, 19, 182, 11, 124, 43, 147, 48, 75, 6, 3, 85, 29, 35, 4, 68, 48, 66, 128, 64, 37, 9, 140, 129, 208, 247, 167, 63, 140, 8, 100, 56, 91, 156, 184, 205, 89, 175, 125, 151, 196, 90, 193, 216, 60, 234, 131, 252, 208, 26, 206, 229, 55, 180, 1, 107, 71, 121, 228, 7, 250, 96, 76, 59, 240, 108, 125, 148, 30, 213, 19, 231, 74, 254, 82, 156, 51, 79, 97, 37, 64, 176, 221, 66, 48, 76, 6, 3, 85, 29, 17, 4, 69, 48, 67, 130, 65, 42, 46, 97, 112, 112, 115, 46, 97, 114, 111, 46, 104, 53, 103, 49, 120, 49, 97, 49, 102, 54, 100, 54, 120, 51, 98, 46, 103, 112, 115, 53, 46, 119, 101, 115, 116, 117, 115, 51, 46, 97, 114, 111, 97, 112, 112, 45, 104, 99, 112, 46, 97, 122, 117, 114, 101, 45, 116, 101, 115, 116, 46, 110, 101, 116],
                        RawSubjectPublicKeyInfo: [48, 130, 1, 34, 48, 13, 6, 9, 42, 134, 72, 134, 247, 13, 1, 1, 1, 5, 0, 3, 130, 1, 15, 0, 48, 130, 1, 10, 2, 130, 1, 1, 0, 230, 190, 96, 104, 10, 59, 179, 162, 174, 46, 142, 192, 200, 19, 105, 174, 207, 152, 161, 202, 176, 15, 56, 243, 170, 157, 113, 39, 213, 91, 37, 178, 7, 11, 150, 37, 109, 247, 110, 16, 213, 139, 0, 197, 124, 14, 0, 215, 252, 76, 104, 31, 165, 18, 63, 111, 231, 249, 165, 15, 146, 216, 139, 189, 27, 110, 122, 122, 47, 7, 93, 22, 38, 58, 244, 252, 34, 32, 133, 9, 218, 213, 26, 223, 83, 182, 89, 196, 95, 203, 91, 164, 98, 168, 62, 125, 198, 234, 206, 40, 244, 188, 118, 48, 46, 72, 43, 209, 41, 172, 155, 120, 222, 176, 170, 50, 117, 208, 205, 144, 80, 247, 213, 39, 212, 26, 218, 201, 101, 208, 49, 58, 238, 11, 48, 246, 212, 64, 124, 133, 100, 178, 36, 156, 116, 61, 190, 97, 187, 51, 185, 96, 93, 222, 140, 0, 114, 212, 22, 74, 240, 19, 149, 134, 52, 132, 175, 158, 99, 109, 231, 83, 168, 100, 126, 206, 64, 230, 226, 31, 8, 31, 32, 178, 246, 254, 161, 134, 237, 124, 213, 160, 101, 83, 145, 30, 84, 167, 201, 93, 240, 25, 238, 76, 169, 145, 190, 97, 234, 15, 148, 218, 216, 74, 228, 173, 107, 191, 109, 211, 218, 68, 168, 6, 146, 226, 219, 69, 169, 143, 159, 194, 142, 119, 237, 161, 31, 229, 182, 88, 35, 251, 235, 167, 126, 85, 130, 1, 82, 182, 68, 77, 144, 154, 99, 195, 2, 3, 1, 0, 1],
                        RawSubject: [48, 48, 49, 18, 48, 16, 6, 3, 85, 4, 10, 19, 9, 111, 112, 101, 110, 115, 104, 105, 102, 116, 49, 26, 48, 24, 6, 3, 85, 4, 3, 19, 17, 111, 112, 101, 110, 115, 104, 105, 102, 116, 45, 105, 110, 103, 114, 101, 115, 115],
                        RawIssuer: [48, 38, 49, 18, 48, 16, 6, 3, 85, 4, 11, 19, 9, 111, 112, 101, 110, 115, 104, 105, 102, 116, 49, 16, 48, 14, 6, 3, 85, 4, 3, 19, 7, 114, 111, 111, 116, 45, 99, 97],
                        Signature: [93, 97, 144, 151, 119, 127, 24, 191, 35, 91, 46, 31, 22, 213, 193, 224, 185, 125, 42, 132, 155, 1, 157, 220, 86, 62, 125, 196, 10, 101, 224, 138, 230, 13, 126, 222, 36, 93, 135, 77, 176, 165, 1, 164, 174, 94, 96, 9, 215, 225, 76, 122, 138, 7, 69, 157, 164, 22, 155, 228, 2, 85, 241, 21, 174, 51, 32, 43, 65, 19, 228, 121, 80, 95, 150, 246, 168, 44, 68, 26, 192, 84, 76, 194, 80, 53, 205, 73, 96, 252, 13, 44, 33, 242, 254, 237, 5, 223, 40, 80, 20, 187, 19, 237, 36, 248, 36, 50, 76, 9, 65, 124, 13, 0, 68, 223, 227, 132, 32, 107, 173, 164, 11, 66, 80, 172, 230, 109, 226, 64, 134, 186, 209, 232, 110, 180, 186, 163, 140, 40, 78, 22, 125, 236, 179, 99, 155, 208, 217, 248, 108, 50, 230, 57, 131, 159, 87, 122, 183, 90, 172, 87, 235, 154, 28, 71, 191, 228, 246, 209, 28, 15, 19, 121, 148, 64, 99, 238, 140, 107, 150, 155, 218, 208, 46, 134, 213, 151, 67, 109, 8, 183, 4, 28, 56, 193, 144, 28, 152, 198, 182, 241, 21, 227, 31, 182, 250, 132, 24, 64, 182, 38, 59, 84, 63, 86, 167, 157, 30, 177, 142, 222, 51, 90, 139, 52, 20, 156, 95, 203, 109, 26, 217, 46, 63, 119, 60, 97, 116, 74, 82, 241, 103, 109, 27, 20, 184, 169, 56, 149, 20, 168, 6, 187, 84, 186],
                        SignatureAlgorithm: 4,
                        PublicKeyAlgorithm: 1,
                        PublicKey: <*rsa.PublicKey | 0x152b7da139c0>{
                            N: {
                                neg: false,
                                abs: [5960026256800637891, 2592925115093451265, 10265935151895000664, 10584263166518075330, 7764045039277484038, 16865862797178365101, 17301121456550297185, 7301338980353034589, 17797840378296849824, 4676673885978960050, 7164636928709787342, 17299334997299474334, 6763997623911454282, 8376060208940824928, 15294361246912095388, 7336417922158113014, 5834366209320737481, 16046512606022782352, 3334963701415189368, 14333495439360030256, 6902711615463374461, 15768539216792148420, 2754783485831972105, 1976651951655836950, 16715572978269064125, 18180020280910561135, 15387393400031281367, 507664470512201232, 12294106973688505778, 14958884054220355827, 12551126170169731502, 16626832874262999970],
                            },
                            E: 65537,
                        },
                        Version: 3,
                        SerialNumber: {
                            neg: false,
                            abs: [5223600323380956861],
                        },
                        Issuer: {
                            Country: nil,
                            Organization: nil,
                            OrganizationalUnit: ["openshift"],
                            Locality: nil,
                            Province: nil,
                            StreetAddress: nil,
                            PostalCode: nil,
                            SerialNumber: "",
                            CommonName: "root-ca",
                            Names: [
                                {
                                    Type: [2, 5, 4, 11],
                                    Value: <string>"openshift",
                                },
                                {
                                    Type: [2, 5, 4, 3],
                                    Value: <string>"root-ca",
                                },
                            ],
                            ExtraNames: nil,
                        },
                        Subject: {
                            Country: nil,
                            Organization: ["openshift"],
                            OrganizationalUnit: nil,
                            Locality: nil,
                            Province: nil,
                            StreetAddress: nil,
                            PostalCode: nil,
                            SerialNumber: "",
                            CommonName: "openshift-ingress",
                            Names: [
                                {
                                    Type: [2, 5, 4, 10],
                                    Value: <string>"openshift",
                                },
                                {
                                    Type: [2, 5, 4, 3],
                                    Value: <string>"openshift-ingress",
                                },
                            ],
                            ExtraNames: nil,
                        },
                        NotBefore: {wall: 0, ext: 63928420116, loc: nil},
                        NotAfter: {wall: 0, ext: 63959956116, loc: nil},
                        KeyUsage: 5,
                        Extensions: [
                            {
                                Id: [2, 5, 29, 15],
                                Critical: true,
                                Value: [3, 2, 5, 160],
                            },
                            {
                                Id: [2, 5, 29, 37],
                                Critical: false,
                                Value: [48, 20, 6, 8, 43, 6, 1, 5, 5, 7, 3, 2, 6, 8, 43, 6, 1, 5, 5, 7, 3, 1],
                            },
                            {Id: [2, 5, 29, 19], Critical: true, Value: [48, 0]},
                            {
                                Id: [2, 5, 29, 14],
                                Critical: false,
                                Value: [4, 64, 124, 43, 155, 38, 67, 14, 112, 156, 220, 167, 147, 27, 204, 115, 135, 241, 3, 184, 222, 93, 50, 70, 132, 146, 25, 139, 25, 95, 207, 250, 219, 139, 68, 58, 234, 212, 61, 207, 175, 41, 112, 193, 157, 177, 6, 156, 82, 245, 173, 210, 245, 105, 124, 108, 52, 248, 245, 189, 19, 182, 11, 124, 43, 147],
                            },
                            {
                                Id: [2, 5, 29, 35],
                                Critical: false,
                                Value: [48, 66, 128, 64, 37, 9, 140, 129, 208, 247, 167, 63, 140, 8, 100, 56, 91, 156, 184, 205, 89, 175, 125, 151, 196, 90, 193, 216, 60, 234, 131, 252, 208, 26, 206, 229, 55, 180, 1, 107, 71, 121, 228, 7, 250, 96, 76, 59, 240, 108, 125, 148, 30, 213, 19, 231, 74, 254, 82, 156, 51, 79, 97, 37, 64, 176, 221, 66],
                            },
                            {
                                Id: [2, 5, 29, 17],
                                Critical: false,
                                Value: "0C\x82A*.apps.aro.h5g1x1a1f6d6x3b.gps5.westus3.aroapp-hcp.azure-test.net",
                            },
                        ],
                        ExtraExtensions: nil,
                        UnhandledCriticalExtensions: nil,
                        ExtKeyUsage: [2, 1],
                        UnknownExtKeyUsage: nil,
                        BasicConstraintsValid: true,
                        IsCA: false,
                        MaxPathLen: -1,
                        MaxPathLenZero: false,
                        SubjectKeyId: [124, 43, 155, 38, 67, 14, 112, 156, 220, 167, 147, 27, 204, 115, 135, 241, 3, 184, 222, 93, 50, 70, 132, 146, 25, 139, 25, 95, 207, 250, 219, 139, 68, 58, 234, 212, 61, 207, 175, 41, 112, 193, 157, 177, 6, 156, 82, 245, 173, 210, 245, 105, 124, 108, 52, 248, 245, 189, 19, 182, 11, 124, 43, 147],
                        AuthorityKeyId: [37, 9, 140, 129, 208, 247, 167, 63, 140, 8, 100, 56, 91, 156, 184, 205, 89, 175, 125, 151, 196, 90, 193, 216, 60, 234, 131, 252, 208, 26, 206, 229, 55, 180, 1, 107, 71, 121, 228, 7, 250, 96, 76, 59, 240, 108, 125, 148, 30, 213, 19, 231, 74, 254, 82, 156, 51, 79, 97, 37, 64, 176, 221, 66],
                        OCSPServer: nil,
                        IssuingCertificateURL: nil,
                        DNSNames: [
                            "*.apps.aro.h5g1x1a1f6d6x3b.gps5.westus3.aroapp-hcp.azure-test.net",
                        ],
                        EmailAddresses: nil,
                        IPAddresses: nil,
                        URIs: nil,
                        PermittedDNSDomainsCritical: false,
                        PermittedDNSDomains: nil,
                        ExcludedDNSDomains: nil,
                        PermittedIPRanges: nil,
                        ExcludedIPRanges: nil,
                        PermittedEmailAddresses: nil,
                        ExcludedEmailAddresses: nil,
                        PermittedURIDomains: nil,
                        ExcludedURIDomains: nil,
                        CRLDistributionPoints: nil,
                        PolicyIdentifiers: nil,
                        Policies: nil,
                        InhibitAnyPolicy: 0,
                        InhibitAnyPolicyZero: false,
                        InhibitPolicyMapping: 0,
                        InhibitPolicyMappingZero: false,
                        RequireExplicitPolicy: 0,
                        RequireExplicitPolicyZero: false,
                        PolicyMappings: nil,
                    },
                ],
                Err: <x509.UnknownAuthorityError>{
                    Cert: 0x152b7bfb8c08 (cyclic reference),
                    hintErr: nil,
                    hintCert: nil,
                },
            },
        },
    }
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:194]: failed to verify simple web app runs on cluster "cluster-candidate-4-23-l4g2v7"
Unexpected error:
    <*fmt.wrapError | 0x152b7c5bb0a0>: 
    [strict TLS verification] route was never reachable: Get "https://agnhost-e2e-sample-app-vl8dn.apps.aro.h5g1x1a1f6d6x3b.gps5.westus3.aroapp-hcp.azure-test.net": tls: failed to verify certificate: x509: certificate signed by unknown authority
    {
        msg: "[strict TLS verification] route was never reachable: Get \"https://agnhost-e2e-sample-app-vl8dn.apps.aro.h5g1x1a1f6d6x3b.gps5.westus3.aroapp-hcp.azure-test.net\": tls: failed to verify certificate: x509: certificate signed by unknown authority",
        err: <*url.Error | 0x152b7da88060>{
            Op: "Get",
            URL: "https://agnhost-e2e-sample-app-vl8dn.apps.aro.h5g1x1a1f6d6x3b.gps5.westus3.aroapp-hcp.azure-test.net",
            Err: <*tls.CertificateVerificationError | 0x152b7da88030>{
                UnverifiedCertificates: [
                    {
                        Raw: [48, 130, 4, 1, 48, 130, 2, 233, 160, 3, 2, 1, 2, 2, 8, 72, 125, 244, 209, 123, 188, 206, 189, 48, 13, 6, 9, 42, 134, 72, 134, 247, 13, 1, 1, 11, 5, 0, 48, 38, 49, 18, 48, 16, 6, 3, 85, 4, 11, 19, 9, 111, 112, 101, 110, 115, 104, 105, 102, 116, 49, 16, 48, 14, 6, 3, 85, 4, 3, 19, 7, 114, 111, 111, 116, 45, 99, 97, 48, 30, 23, 13, 50, 54, 49, 48, 50, 52, 48, 54, 50, 56, 51, 54, 90, 23, 13, 50, 55, 49, 48, 50, 52, 48, 54, 50, 56, 51, 54, 90, 48, 48, 49, 18, 48, 16, 6, 3, 85, 4, 10, 19, 9, 111, 112, 101, 110, 115, 104, 105, 102, 116, 49, 26, 48, 24, 6, 3, 85, 4, 3, 19, 17, 111, 112, 101, 110, 115, 104, 105, 102, 116, 45, 105, 110, 103, 114, 101, 115, 115, 48, 130, 1, 34, 48, 13, 6, 9, 42, 134, 72, 134, 247, 13, 1, 1, 1, 5, 0, 3, 130, 1, 15, 0, 48, 130, 1, 10, 2, 130, 1, 1, 0, 230, 190, 96, 104, 10, 59, 179, 162, 174, 46, 142, 192, 200, 19, 105, 174, 207, 152, 161, 202, 176, 15, 56, 243, 170, 157, 113, 39, 213, 91, 37, 178, 7, 11, 150, 37, 109, 247, 110, 16, 213, 139, 0, 197, 124, 14, 0, 215, 252, 76, 104, 31, 165, 18, 63, 111, 231, 249, 165, 15, 146, 216, 139, 189, 27, 110, 122, 122, 47, 7, 93, 22, 38, 58, 244, 252, 34, 32, 133, 9, 218, 213, 26, 223, 83, 182, 89, 196, 95, 203, 91, 164, 98, 168, 62, 125, 198, 234, 206, 40, 244, 188, 118, 48, 46, 72, 43, 209, 41, 172, 155, 120, 222, 176, 170, 50, 117, 208, 205, 144, 80, 247, 213, 39, 212, 26, 218, 201, 101, 208, 49, 58, 238, 11, 48, 246, 212, 64, 124, 133, 100, 178, 36, 156, 116, 61, 190, 97, 187, 51, 185, 96, 93, 222, 140, 0, 114, 212, 22, 74, 240, 19, 149, 134, 52, 132, 175, 158, 99, 109, 231, 83, 168, 100, 126, 206, 64, 230, 226, 31, 8, 31, 32, 178, 246, 254, 161, 134, 237, 124, 213, 160, 101, 83, 145, 30, 84, 167, 201, 93, 240, 25, 238, 76, 169, 145, 190, 97, 234, 15, 148, 218, 216, 74, 228, 173, 107, 191, 109, 211, 218, 68, 168, 6, 146, 226, 219, 69, 169, 143, 159, 194, 142, 119, 237, 161, 31, 229, 182, 88, 35, 251, 235, 167, 126, 85, 130, 1, 82, 182, 68, 77, 144, 154, 99, 195, 2, 3, 1, 0, 1, 163, 130, 1, 39, 48, 130, 1, 35, 48, 14, 6, 3, 85, 29, 15, 1, 1, 255, 4, 4, 3, 2, 5, 160, 48, 29, 6, 3, 85, 29, 37, 4, 22, 48, 20, 6, 8, 43, 6, 1, 5, 5, 7, 3, 2, 6, 8, 43, 6, 1, 5, 5, 7, 3, 1, 48, 12, 6, 3, 85, 29, 19, 1, 1, 255, 4, 2, 48, 0, 48, 73, 6, 3, 85, 29, 14, 4, 66, 4, 64, 124, 43, 155, 38, 67, 14, 112, 156, 220, 167, 147, 27, 204, 115, 135, 241, 3, 184, 222, 93, 50, 70, 132, 146, 25, 139, 25, 95, 207, 250, 219, 139, 68, 58, 234, 212, 61, 207, 175, 41, 112, 193, 157, 177, 6, 156, 82, 245, 173, 210, 245, 105, 124, 108, 52, 248, 245, 189, 19, 182, 11, 124, 43, 147, 48, 75, 6, 3, 85, 29, 35, 4, 68, 48, 66, 128, 64, 37, 9, 140, 129, 208, 247, 167, 63, 140, 8, 100, 56, 91, 156, 184, 205, 89, 175, 125, 151, 196, 90, 193, 216, 60, 234, 131, 252, 208, 26, 206, 229, 55, 180, 1, 107, 71, 121, 228, 7, 250, 96, 76, 59, 240, 108, 125, 148, 30, 213, 19, 231, 74, 254, 82, 156, 51, 79, 97, 37, 64, 176, 221, 66, 48, 76, 6, 3, 85, 29, 17, 4, 69, 48, 67, 130, 65, 42, 46, 97, 112, 112, 115, 46, 97, 114, 111, 46, 104, 53, 103, 49, 120, 49, 97, 49, 102, 54, 100, 54, 120, 51, 98, 46, 103, 112, 115, 53, 46, 119, 101, 115, 116, 117, 115, 51, 46, 97, 114, 111, 97, 112, 112, 45, 104, 99, 112, 46, 97, 122, 117, 114, 101, 45, 116, 101, 115, 116, 46, 110, 101, 116, 48, 13, 6, 9, 42, 134, 72, 134, 247, 13, 1, 1, 11, 5, 0, 3, 130, 1, 1, 0, 93, 97, 144, 151, 119, 127, 24, 191, 35, 91, 46, 31, 22, 213, 193, 224, 185, 125, 42, 132, 155, 1, 157, 220, 86, 62, 125, 196, 10, 101, 224, 138, 230, 13, 126, 222, 36, 93, 135, 77, 176, 165, 1, 164, 174, 94, 96, 9, 215, 225, 76, 122, 138, 7, 69, 157, 164, 22, 155, 228, 2, 85, 241, 21, 174, 51, 32, 43, 65, 19, 228, 121, 80, 95, 150, 246, 168, 44, 68, 26, 192, 84, 76, 194, 80, 53, 205, 73, 96, 252, 13, 44, 33, 242, 254, 237, 5, 223, 40, 80, 20, 187, 19, 237, 36, 248, 36, 50, 76, 9, 65, 124, 13, 0, 68, 223, 227, 132, 32, 107, 173, 164, 11, 66, 80, 172, 230, 109, 226, 64, 134, 186, 209, 232, 110, 180, 186, 163, 140, 40, 78, 22, 125, 236, 179, 99, 155, 208, 217, 248, 108, 50, 230, 57, 131, 159, 87, 122, 183, 90, 172, 87, 235, 154, 28, 71, 191, 228, 246, 209, 28, 15, 19, 121, 148, 64, 99, 238, 140, 107, 150, 155, 218, 208, 46, 134, 213, 151, 67, 109, 8, 183, 4, 28, 56, 193, 144, 28, 152, 198, 182, 241, 21, 227, 31, 182, 250, 132, 24, 64, 182, 38, 59, 84, 63, 86, 167, 157, 30, 177, 142, 222, 51, 90, 139, 52, 20, 156, 95, 203, 109, 26, 217, 46, 63, 119, 60, 97, 116, 74, 82, 241, 103, 109, 27, 20, 184, 169, 56, 149, 20, 168, 6, 187, 84, 186],
                        RawTBSCertificate: [48, 130, 2, 233, 160, 3, 2, 1, 2, 2, 8, 72, 125, 244, 209, 123, 188, 206, 189, 48, 13, 6, 9, 42, 134, 72, 134, 247, 13, 1, 1, 11, 5, 0, 48, 38, 49, 18, 48, 16, 6, 3, 85, 4, 11, 19, 9, 111, 112, 101, 110, 115, 104, 105, 102, 116, 49, 16, 48, 14, 6, 3, 85, 4, 3, 19, 7, 114, 111, 111, 116, 45, 99, 97, 48, 30, 23, 13, 50, 54, 49, 48, 50, 52, 48, 54, 50, 56, 51, 54, 90, 23, 13, 50, 55, 49, 48, 50, 52, 48, 54, 50, 56, 51, 54, 90, 48, 48, 49, 18, 48, 16, 6, 3, 85, 4, 10, 19, 9, 111, 112, 101, 110, 115, 104, 105, 102, 116, 49, 26, 48, 24, 6, 3, 85, 4, 3, 19, 17, 111, 112, 101, 110, 115, 104, 105, 102, 116, 45, 105, 110, 103, 114, 101, 115, 115, 48, 130, 1, 34, 48, 13, 6, 9, 42, 134, 72, 134, 247, 13, 1, 1, 1, 5, 0, 3, 130, 1, 15, 0, 48, 130, 1, 10, 2, 130, 1, 1, 0, 230, 190, 96, 104, 10, 59, 179, 162, 174, 46, 142, 192, 200, 19, 105, 174, 207, 152, 161, 202, 176, 15, 56, 243, 170, 157, 113, 39, 213, 91, 37, 178, 7, 11, 150, 37, 109, 247, 110, 16, 213, 139, 0, 197, 124, 14, 0, 215, 252, 76, 104, 31, 165, 18, 63, 111, 231, 249, 165, 15, 146, 216, 139, 189, 27, 110, 122, 122, 47, 7, 93, 22, 38, 58, 244, 252, 34, 32, 133, 9, 218, 213, 26, 223, 83, 182, 89, 196, 95, 203, 91, 164, 98, 168, 62, 125, 198, 234, 206, 40, 244, 188, 118, 48, 46, 72, 43, 209, 41, 172, 155, 120, 222, 176, 170, 50, 117, 208, 205, 144, 80, 247, 213, 39, 212, 26, 218, 201, 101, 208, 49, 58, 238, 11, 48, 246, 212, 64, 124, 133, 100, 178, 36, 156, 116, 61, 190, 97, 187, 51, 185, 96, 93, 222, 140, 0, 114, 212, 22, 74, 240, 19, 149, 134, 52, 132, 175, 158, 99, 109, 231, 83, 168, 100, 126, 206, 64, 230, 226, 31, 8, 31, 32, 178, 246, 254, 161, 134, 237, 124, 213, 160, 101, 83, 145, 30, 84, 167, 201, 93, 240, 25, 238, 76, 169, 145, 190, 97, 234, 15, 148, 218, 216, 74, 228, 173, 107, 191, 109, 211, 218, 68, 168, 6, 146, 226, 219, 69, 169, 143, 159, 194, 142, 119, 237, 161, 31, 229, 182, 88, 35, 251, 235, 167, 126, 85, 130, 1, 82, 182, 68, 77, 144, 154, 99, 195, 2, 3, 1, 0, 1, 163, 130, 1, 39, 48, 130, 1, 35, 48, 14, 6, 3, 85, 29, 15, 1, 1, 255, 4, 4, 3, 2, 5, 160, 48, 29, 6, 3, 85, 29, 37, 4, 22, 48, 20, 6, 8, 43, 6, 1, 5, 5, 7, 3, 2, 6, 8, 43, 6, 1, 5, 5, 7, 3, 1, 48, 12, 6, 3, 85, 29, 19, 1, 1, 255, 4, 2, 48, 0, 48, 73, 6, 3, 85, 29, 14, 4, 66, 4, 64, 124, 43, 155, 38, 67, 14, 112, 156, 220, 167, 147, 27, 204, 115, 135, 241, 3, 184, 222, 93, 50, 70, 132, 146, 25, 139, 25, 95, 207, 250, 219, 139, 68, 58, 234, 212, 61, 207, 175, 41, 112, 193, 157, 177, 6, 156, 82, 245, 173, 210, 245, 105, 124, 108, 52, 248, 245, 189, 19, 182, 11, 124, 43, 147, 48, 75, 6, 3, 85, 29, 35, 4, 68, 48, 66, 128, 64, 37, 9, 140, 129, 208, 247, 167, 63, 140, 8, 100, 56, 91, 156, 184, 205, 89, 175, 125, 151, 196, 90, 193, 216, 60, 234, 131, 252, 208, 26, 206, 229, 55, 180, 1, 107, 71, 121, 228, 7, 250, 96, 76, 59, 240, 108, 125, 148, 30, 213, 19, 231, 74, 254, 82, 156, 51, 79, 97, 37, 64, 176, 221, 66, 48, 76, 6, 3, 85, 29, 17, 4, 69, 48, 67, 130, 65, 42, 46, 97, 112, 112, 115, 46, 97, 114, 111, 46, 104, 53, 103, 49, 120, 49, 97, 49, 102, 54, 100, 54, 120, 51, 98, 46, 103, 112, 115, 53, 46, 119, 101, 115, 116, 117, 115, 51, 46, 97, 114, 111, 97, 112, 112, 45, 104, 99, 112, 46, 97, 122, 117, 114, 101, 45, 116, 101, 115, 116, 46, 110, 101, 116],
                        RawSubjectPublicKeyInfo: [48, 130, 1, 34, 48, 13, 6, 9, 42, 134, 72, 134, 247, 13, 1, 1, 1, 5, 0, 3, 130, 1, 15, 0, 48, 130, 1, 10, 2, 130, 1, 1, 0, 230, 190, 96, 104, 10, 59, 179, 162, 174, 46, 142, 192, 200, 19, 105, 174, 207, 152, 161, 202, 176, 15, 56, 243, 170, 157, 113, 39, 213, 91, 37, 178, 7, 11, 150, 37, 109, 247, 110, 16, 213, 139, 0, 197, 124, 14, 0, 215, 252, 76, 104, 31, 165, 18, 63, 111, 231, 249, 165, 15, 146, 216, 139, 189, 27, 110, 122, 122, 47, 7, 93, 22, 38, 58, 244, 252, 34, 32, 133, 9, 218, 213, 26, 223, 83, 182, 89, 196, 95, 203, 91, 164, 98, 168, 62, 125, 198, 234, 206, 40, 244, 188, 118, 48, 46, 72, 43, 209, 41, 172, 155, 120, 222, 176, 170, 50, 117, 208, 205, 144, 80, 247, 213, 39, 212, 26, 218, 201, 101, 208, 49, 58, 238, 11, 48, 246, 212, 64, 124, 133, 100, 178, 36, 156, 116, 61, 190, 97, 187, 51, 185, 96, 93, 222, 140, 0, 114, 212, 22, 74, 240, 19, 149, 134, 52, 132, 175, 158, 99, 109, 231, 83, 168, 100, 126, 206, 64, 230, 226, 31, 8, 31, 32, 178, 246, 254, 161, 134, 237, 124, 213, 160, 101, 83, 145, 30, 84, 167, 201, 93, 240, 25, 238, 76, 169, 145, 190, 97, 234, 15, 148, 218, 216, 74, 228, 173, 107, 191, 109, 211, 218, 68, 168, 6, 146, 226, 219, 69, 169, 143, 159, 194, 142, 119, 237, 161, 31, 229, 182, 88, 35, 251, 235, 167, 126, 85, 130, 1, 82, 182, 68, 77, 144, 154, 99, 195, 2, 3, 1, 0, 1],
                        RawSubject: [48, 48, 49, 18, 48, 16, 6, 3, 85, 4, 10, 19, 9, 111, 112, 101, 110, 115, 104, 105, 102, 116, 49, 26, 48, 24, 6, 3, 85, 4, 3, 19, 17, 111, 112, 101, 110, 115, 104, 105, 102, 116, 45, 105, 110, 103, 114, 101, 115, 115],
                        RawIssuer: [48, 38, 49, 18, 48, 16, 6, 3, 85, 4, 11, 19, 9, 111, 112, 101, 110, 115, 104, 105, 102, 116, 49, 16, 48, 14, 6, 3, 85, 4, 3, 19, 7, 114, 111, 111, 116, 45, 99, 97],
                        Signature: [93, 97, 144, 151, 119, 127, 24, 191, 35, 91, 46, 31, 22, 213, 193, 224, 185, 125, 42, 132, 155, 1, 157, 220, 86, 62, 125, 196, 10, 101, 224, 138, 230, 13, 126, 222, 36, 93, 135, 77, 176, 165, 1, 164, 174, 94, 96, 9, 215, 225, 76, 122, 138, 7, 69, 157, 164, 22, 155, 228, 2, 85, 241, 21, 174, 51, 32, 43, 65, 19, 228, 121, 80, 95, 150, 246, 168, 44, 68, 26, 192, 84, 76, 194, 80, 53, 205, 73, 96, 252, 13, 44, 33, 242, 254, 237, 5, 223, 40, 80, 20, 187, 19, 237, 36, 248, 36, 50, 76, 9, 65, 124, 13, 0, 68, 223, 227, 132, 32, 107, 173, 164, 11, 66, 80, 172, 230, 109, 226, 64, 134, 186, 209, 232, 110, 180, 186, 163, 140, 40, 78, 22, 125, 236, 179, 99, 155, 208, 217, 248, 108, 50, 230, 57, 131, 159, 87, 122, 183, 90, 172, 87, 235, 154, 28, 71, 191, 228, 246, 209, 28, 15, 19, 121, 148, 64, 99, 238, 140, 107, 150, 155, 218, 208, 46, 134, 213, 151, 67, 109, 8, 183, 4, 28, 56, 193, 144, 28, 152, 198, 182, 241, 21, 227, 31, 182, 250, 132, 24, 64, 182, 38, 59, 84, 63, 86, 167, 157, 30, 177, 142, 222, 51, 90, 139, 52, 20, 156, 95, 203, 109, 26, 217, 46, 63, 119, 60, 97, 116, 74, 82, 241, 103, 109, 27, 20, 184, 169, 56, 149, 20, 168, 6, 187, 84, 186],
                        SignatureAlgorithm: 4,
                        PublicKeyAlgorithm: 1,
                        PublicKey: <*rsa.PublicKey | 0x152b7da139c0>{
                            N: {
                                neg: false,
                                abs: [5960026256800637891, 2592925115093451265, 10265935151895000664, 10584263166518075330, 7764045039277484038, 16865862797178365101, 17301121456550297185, 7301338980353034589, 17797840378296849824, 4676673885978960050, 7164636928709787342, 17299334997299474334, 6763997623911454282, 8376060208940824928, 15294361246912095388, 7336417922158113014, 5834366209320737481, 16046512606022782352, 3334963701415189368, 14333495439360030256, 6902711615463374461, 15768539216792148420, 2754783485831972105, 1976651951655836950, 16715572978269064125, 18180020280910561135, 15387393400031281367, 507664470512201232, 12294106973688505778, 14958884054220355827, 12551126170169731502, 16626832874262999970],
                            },
                            E: 65537,
                        },
                        Version: 3,
                        SerialNumber: {
                            neg: false,
                            abs: [5223600323380956861],
                        },
                        Issuer: {
                            Country: nil,
                            Organization: nil,
                            OrganizationalUnit: ["openshift"],
                            Locality: nil,
                            Province: nil,
                            StreetAddress: nil,
                            PostalCode: nil,
                            SerialNumber: "",
                            CommonName: "root-ca",
                            Names: [
                                {
                                    Type: [2, 5, 4, 11],
                                    Value: <string>"openshift",
                                },
                                {
                                    Type: [2, 5, 4, 3],
                                    Value: <string>"root-ca",
                                },
                            ],
                            ExtraNames: nil,
                        },
                        Subject: {
                            Country: nil,
                            Organization: ["openshift"],
                            OrganizationalUnit: nil,
                            Locality: nil,
                            Province: nil,
                            StreetAddress: nil,
                            PostalCode: nil,
                            SerialNumber: "",
                            CommonName: "openshift-ingress",
                            Names: [
                                {
                                    Type: [2, 5, 4, 10],
                                    Value: <string>"openshift",
                                },
                                {
                                    Type: [2, 5, 4, 3],
                                    Value: <string>"openshift-ingress",
                                },
                            ],
                            ExtraNames: nil,
                        },
                        NotBefore: {wall: 0, ext: 63928420116, loc: nil},
                        NotAfter: {wall: 0, ext: 63959956116, loc: nil},
                        KeyUsage: 5,
                        Extensions: [
                            {
                                Id: [2, 5, 29, 15],
                                Critical: true,
                                Value: [3, 2, 5, 160],
                            },
                            {
                                Id: [2, 5, 29, 37],
                                Critical: false,
                                Value: [48, 20, 6, 8, 43, 6, 1, 5, 5, 7, 3, 2, 6, 8, 43, 6, 1, 5, 5, 7, 3, 1],
                            },
                            {Id: [2, 5, 29, 19], Critical: true, Value: [48, 0]},
                            {
                                Id: [2, 5, 29, 14],
                                Critical: false,
                                Value: [4, 64, 124, 43, 155, 38, 67, 14, 112, 156, 220, 167, 147, 27, 204, 115, 135, 241, 3, 184, 222, 93, 50, 70, 132, 146, 25, 139, 25, 95, 207, 250, 219, 139, 68, 58, 234, 212, 61, 207, 175, 41, 112, 193, 157, 177, 6, 156, 82, 245, 173, 210, 245, 105, 124, 108, 52, 248, 245, 189, 19, 182, 11, 124, 43, 147],
                            },
                            {
                                Id: [2, 5, 29, 35],
                                Critical: false,
                                Value: [48, 66, 128, 64, 37, 9, 140, 129, 208, 247, 167, 63, 140, 8, 100, 56, 91, 156, 184, 205, 89, 175, 125, 151, 196, 90, 193, 216, 60, 234, 131, 252, 208, 26, 206, 229, 55, 180, 1, 107, 71, 121, 228, 7, 250, 96, 76, 59, 240, 108, 125, 148, 30, 213, 19, 231, 74, 254, 82, 156, 51, 79, 97, 37, 64, 176, 221, 66],
                            },
                            {
                                Id: [2, 5, 29, 17],
                                Critical: false,
                                Value: "0C\x82A*.apps.aro.h5g1x1a1f6d6x3b.gps5.westus3.aroapp-hcp.azure-test.net",
                            },
                        ],
                        ExtraExtensions: nil,
                        UnhandledCriticalExtensions: nil,
                        ExtKeyUsage: [2, 1],
                        UnknownExtKeyUsage: nil,
                        BasicConstraintsValid: true,
                        IsCA: false,
                        MaxPathLen: -1,
                        MaxPathLenZero: false,
                        SubjectKeyId: [124, 43, 155, 38, 67, 14, 112, 156, 220, 167, 147, 27, 204, 115, 135, 241, 3, 184, 222, 93, 50, 70, 132, 146, 25, 139, 25, 95, 207, 250, 219, 139, 68, 58, 234, 212, 61, 207, 175, 41, 112, 193, 157, 177, 6, 156, 82, 245, 173, 210, 245, 105, 124, 108, 52, 248, 245, 189, 19, 182, 11, 124, 43, 147],
                        AuthorityKeyId: [37, 9, 140, 129, 208, 247, 167, 63, 140, 8, 100, 56, 91, 156, 184, 205, 89, 175, 125, 151, 196, 90, 193, 216, 60, 234, 131, 252, 208, 26, 206, 229, 55, 180, 1, 107, 71, 121, 228, 7, 250, 96, 76, 59, 240, 108, 125, 148, 30, 213, 19, 231, 74, 254, 82, 156, 51, 79, 97, 37, 64, 176, 221, 66],
                        OCSPServer: nil,
                        IssuingCertificateURL: nil,
                        DNSNames: [
                            "*.apps.aro.h5g1x1a1f6d6x3b.gps5.westus3.aroapp-hcp.azure-test.net",
                        ],
                        EmailAddresses: nil,
                        IPAddresses: nil,
                        URIs: nil,
                        PermittedDNSDomainsCritical: false,
                        PermittedDNSDomains: nil,
                        ExcludedDNSDomains: nil,
                        PermittedIPRanges: nil,
                        ExcludedIPRanges: nil,
                        PermittedEmailAddresses: nil,
                        ExcludedEmailAddresses: nil,
                        PermittedURIDomains: nil,
                        ExcludedURIDomains: nil,
                        CRLDistributionPoints: nil,
                        PolicyIdentifiers: nil,
                        Policies: nil,
                        InhibitAnyPolicy: 0,
                        InhibitAnyPolicyZero: false,
                        InhibitPolicyMapping: 0,
                        InhibitPolicyMappingZero: false,
                        RequireExplicitPolicy: 0,
                        RequireExplicitPolicyZero: false,
                        PolicyMappings: nil,
                    },
                ],
                Err: <x509.UnknownAuthorityError>{
                    Cert: 0x152b7bfb8c08 (cyclic reference),
                    hintErr: nil,
                    hintCert: nil,
                },
            },
        },
    }
occurred
1h 30m 46s branch-ci-Azure-ARO-HCP-main-e2e-integration-e2e-parallel-all
post-goodmerged PR
No additional run metadata captured
westus3 unknown failed n/a 3
unknown (3)
tls: failed to verify certificate: x509: certificate signed by unknown authority
03:02:26 UTC · failed at unknown · test ARO-HCP should be able to perform a control plane and node pool install with OCP candidate channel for 4.23 · suite integration/parallel/all
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:194]: failed to verify simple web app runs on cluster "cluster-candidate-4-23-xhptzz"
Unexpected error:
    <*fmt.wrapError | 0x1d9b54fe6000>: 
    [strict TLS verification] route was never reachable: Get "https://agnhost-e2e-sample-app-777tx.apps.aro.h1v1j5a4q4j8y9s.1da3.westus3.aroapp-hcp.azure-test.net": tls: failed to verify certificate: x509: certificate signed by unknown authority
    {
        msg: "[strict TLS verification] route was never reachable: Get \"https://agnhost-e2e-sample-app-777tx.apps.aro.h1v1j5a4q4j8y9s.1da3.westus3.aroapp-hcp.azure-test.net\": tls: failed to verify certificate: x509: certificate signed by unknown authority",
        err: <*url.Error | 0x1d9b54f6e0c0>{
            Op: "Get",
            URL: "https://agnhost-e2e-sample-app-777tx.apps.aro.h1v1j5a4q4j8y9s.1da3.westus3.aroapp-hcp.azure-test.net",
            Err: <*tls.CertificateVerificationError | 0x1d9b54f6e060>{
                UnverifiedCertificates: [
                    {
                        Raw: [48, 130, 4, 0, 48, 130, 2, 232, 160, 3, 2, 1, 2, 2, 7, 126, 132, 197, 9, 68, 253, 60, 48, 13, 6, 9, 42, 134, 72, 134, 247, 13, 1, 1, 11, 5, 0, 48, 38, 49, 18, 48, 16, 6, 3, 85, 4, 11, 19, 9, 111, 112, 101, 110, 115, 104, 105, 102, 116, 49, 16, 48, 14, 6, 3, 85, 4, 3, 19, 7, 114, 111, 111, 116, 45, 99, 97, 48, 30, 23, 13, 50, 54, 49, 48, 50, 52, 48, 51, 51, 50, 49, 48, 90, 23, 13, 50, 55, 49, 48, 50, 52, 48, 51, 51, 50, 49, 48, 90, 48, 48, 49, 18, 48, 16, 6, 3, 85, 4, 10, 19, 9, 111, 112, 101, 110, 115, 104, 105, 102, 116, 49, 26, 48, 24, 6, 3, 85, 4, 3, 19, 17, 111, 112, 101, 110, 115, 104, 105, 102, 116, 45, 105, 110, 103, 114, 101, 115, 115, 48, 130, 1, 34, 48, 13, 6, 9, 42, 134, 72, 134, 247, 13, 1, 1, 1, 5, 0, 3, 130, 1, 15, 0, 48, 130, 1, 10, 2, 130, 1, 1, 0, 207, 128, 218, 212, 26, 132, 208, 243, 92, 156, 156, 214, 53, 193, 75, 191, 178, 88, 241, 38, 163, 215, 203, 145, 224, 112, 103, 58, 9, 153, 245, 82, 220, 111, 140, 139, 161, 99, 162, 9, 10, 163, 63, 57, 151, 129, 243, 25, 133, 62, 43, 200, 35, 141, 232, 109, 192, 10, 138, 13, 120, 44, 157, 202, 25, 139, 129, 58, 216, 111, 145, 24, 55, 24, 23, 228, 156, 172, 101, 183, 207, 223, 73, 100, 53, 41, 125, 73, 239, 89, 201, 165, 74, 243, 215, 177, 223, 112, 58, 228, 208, 240, 151, 37, 153, 119, 237, 179, 20, 85, 131, 190, 36, 125, 23, 184, 200, 218, 93, 234, 90, 97, 224, 192, 77, 135, 1, 202, 159, 217, 203, 241, 161, 52, 213, 56, 96, 249, 140, 209, 180, 80, 49, 158, 231, 47, 244, 35, 41, 7, 251, 235, 173, 156, 233, 71, 118, 32, 117, 101, 8, 235, 7, 107, 174, 208, 128, 140, 68, 177, 228, 226, 57, 173, 158, 135, 91, 200, 85, 37, 244, 126, 98, 219, 71, 183, 169, 127, 163, 164, 9, 211, 219, 34, 103, 109, 123, 209, 155, 183, 40, 28, 208, 40, 61, 63, 249, 102, 235, 233, 213, 18, 214, 16, 243, 135, 116, 93, 127, 34, 199, 14, 197, 51, 130, 225, 91, 183, 135, 179, 150, 125, 195, 25, 0, 219, 23, 147, 130, 215, 229, 224, 255, 230, 241, 126, 125, 10, 148, 17, 135, 148, 31, 6, 198, 159, 2, 3, 1, 0, 1, 163, 130, 1, 39, 48, 130, 1, 35, 48, 14, 6, 3, 85, 29, 15, 1, 1, 255, 4, 4, 3, 2, 5, 160, 48, 29, 6, 3, 85, 29, 37, 4, 22, 48, 20, 6, 8, 43, 6, 1, 5, 5, 7, 3, 2, 6, 8, 43, 6, 1, 5, 5, 7, 3, 1, 48, 12, 6, 3, 85, 29, 19, 1, 1, 255, 4, 2, 48, 0, 48, 73, 6, 3, 85, 29, 14, 4, 66, 4, 64, 151, 50, 248, 37, 209, 121, 110, 180, 165, 224, 104, 231, 132, 191, 4, 141, 230, 108, 133, 21, 249, 143, 45, 39, 159, 230, 5, 9, 221, 165, 194, 92, 82, 9, 127, 116, 96, 225, 58, 3, 240, 92, 33, 54, 115, 209, 236, 44, 113, 1, 122, 254, 132, 18, 35, 85, 13, 63, 55, 68, 245, 55, 31, 222, 48, 75, 6, 3, 85, 29, 35, 4, 68, 48, 66, 128, 64, 212, 242, 117, 237, 251, 129, 17, 40, 107, 113, 46, 32, 113, 79, 15, 14, 14, 116, 200, 32, 28, 245, 96, 213, 104, 10, 125, 78, 189, 101, 7, 34, 56, 108, 123, 92, 186, 132, 218, 166, 174, 58, 176, 69, 173, 49, 183, 61, 85, 207, 199, 230, 10, 251, 80, 73, 48, 131, 248, 174, 77, 22, 112, 231, 48, 76, 6, 3, 85, 29, 17, 4, 69, 48, 67, 130, 65, 42, 46, 97, 112, 112, 115, 46, 97, 114, 111, 46, 104, 49, 118, 49, 106, 53, 97, 52, 113, 52, 106, 56, 121, 57, 115, 46, 49, 100, 97, 51, 46, 119, 101, 115, 116, 117, 115, 51, 46, 97, 114, 111, 97, 112, 112, 45, 104, 99, 112, 46, 97, 122, 117, 114, 101, 45, 116, 101, 115, 116, 46, 110, 101, 116, 48, 13, 6, 9, 42, 134, 72, 134, 247, 13, 1, 1, 11, 5, 0, 3, 130, 1, 1, 0, 210, 85, 40, 236, 76, 26, 142, 116, 32, 133, 55, 168, 156, 128, 149, 43, 102, 234, 125, 146, 217, 61, 79, 196, 218, 194, 125, 227, 148, 218, 229, 112, 87, 55, 112, 103, 71, 167, 86, 76, 220, 157, 113, 183, 190, 66, 91, 201, 228, 199, 238, 11, 23, 126, 102, 42, 155, 62, 252, 11, 203, 150, 192, 104, 209, 47, 32, 85, 151, 78, 147, 151, 116, 174, 88, 152, 176, 253, 138, 226, 146, 247, 41, 217, 62, 240, 230, 149, 20, 224, 76, 122, 217, 220, 253, 71, 173, 51, 183, 40, 243, 149, 197, 210, 226, 65, 7, 41, 157, 67, 88, 94, 28, 42, 85, 24, 21, 163, 28, 209, 29, 2, 30, 128, 219, 8, 23, 74, 70, 195, 5, 144, 132, 227, 11, 15, 109, 67, 145, 224, 175, 168, 48, 134, 34, 243, 54, 95, 181, 212, 143, 249, 177, 226, 180, 209, 177, 104, 226, 213, 12, 143, 86, 186, 83, 219, 200, 29, 244, 229, 243, 186, 95, 226, 10, 156, 126, 111, 18, 111, 7, 214, 237, 104, 47, 96, 76, 215, 237, 244, 132, 35, 12, 57, 53, 11, 153, 175, 14, 155, 31, 121, 229, 46, 117, 124, 182, 14, 54, 12, 37, 80, 196, 185, 229, 150, 209, 168, 129, 127, 202, 188, 211, 37, 142, 99, 117, 229, 10, 240, 1, 91, 137, 21, 47, 232, 208, 202, 88, 175, 19, 12, 17, 165, 34, 137, 139, 207, 49, 154, 144, 91, 223, 210, 26, 45],
                        RawTBSCertificate: [48, 130, 2, 232, 160, 3, 2, 1, 2, 2, 7, 126, 132, 197, 9, 68, 253, 60, 48, 13, 6, 9, 42, 134, 72, 134, 247, 13, 1, 1, 11, 5, 0, 48, 38, 49, 18, 48, 16, 6, 3, 85, 4, 11, 19, 9, 111, 112, 101, 110, 115, 104, 105, 102, 116, 49, 16, 48, 14, 6, 3, 85, 4, 3, 19, 7, 114, 111, 111, 116, 45, 99, 97, 48, 30, 23, 13, 50, 54, 49, 48, 50, 52, 48, 51, 51, 50, 49, 48, 90, 23, 13, 50, 55, 49, 48, 50, 52, 48, 51, 51, 50, 49, 48, 90, 48, 48, 49, 18, 48, 16, 6, 3, 85, 4, 10, 19, 9, 111, 112, 101, 110, 115, 104, 105, 102, 116, 49, 26, 48, 24, 6, 3, 85, 4, 3, 19, 17, 111, 112, 101, 110, 115, 104, 105, 102, 116, 45, 105, 110, 103, 114, 101, 115, 115, 48, 130, 1, 34, 48, 13, 6, 9, 42, 134, 72, 134, 247, 13, 1, 1, 1, 5, 0, 3, 130, 1, 15, 0, 48, 130, 1, 10, 2, 130, 1, 1, 0, 207, 128, 218, 212, 26, 132, 208, 243, 92, 156, 156, 214, 53, 193, 75, 191, 178, 88, 241, 38, 163, 215, 203, 145, 224, 112, 103, 58, 9, 153, 245, 82, 220, 111, 140, 139, 161, 99, 162, 9, 10, 163, 63, 57, 151, 129, 243, 25, 133, 62, 43, 200, 35, 141, 232, 109, 192, 10, 138, 13, 120, 44, 157, 202, 25, 139, 129, 58, 216, 111, 145, 24, 55, 24, 23, 228, 156, 172, 101, 183, 207, 223, 73, 100, 53, 41, 125, 73, 239, 89, 201, 165, 74, 243, 215, 177, 223, 112, 58, 228, 208, 240, 151, 37, 153, 119, 237, 179, 20, 85, 131, 190, 36, 125, 23, 184, 200, 218, 93, 234, 90, 97, 224, 192, 77, 135, 1, 202, 159, 217, 203, 241, 161, 52, 213, 56, 96, 249, 140, 209, 180, 80, 49, 158, 231, 47, 244, 35, 41, 7, 251, 235, 173, 156, 233, 71, 118, 32, 117, 101, 8, 235, 7, 107, 174, 208, 128, 140, 68, 177, 228, 226, 57, 173, 158, 135, 91, 200, 85, 37, 244, 126, 98, 219, 71, 183, 169, 127, 163, 164, 9, 211, 219, 34, 103, 109, 123, 209, 155, 183, 40, 28, 208, 40, 61, 63, 249, 102, 235, 233, 213, 18, 214, 16, 243, 135, 116, 93, 127, 34, 199, 14, 197, 51, 130, 225, 91, 183, 135, 179, 150, 125, 195, 25, 0, 219, 23, 147, 130, 215, 229, 224, 255, 230, 241, 126, 125, 10, 148, 17, 135, 148, 31, 6, 198, 159, 2, 3, 1, 0, 1, 163, 130, 1, 39, 48, 130, 1, 35, 48, 14, 6, 3, 85, 29, 15, 1, 1, 255, 4, 4, 3, 2, 5, 160, 48, 29, 6, 3, 85, 29, 37, 4, 22, 48, 20, 6, 8, 43, 6, 1, 5, 5, 7, 3, 2, 6, 8, 43, 6, 1, 5, 5, 7, 3, 1, 48, 12, 6, 3, 85, 29, 19, 1, 1, 255, 4, 2, 48, 0, 48, 73, 6, 3, 85, 29, 14, 4, 66, 4, 64, 151, 50, 248, 37, 209, 121, 110, 180, 165, 224, 104, 231, 132, 191, 4, 141, 230, 108, 133, 21, 249, 143, 45, 39, 159, 230, 5, 9, 221, 165, 194, 92, 82, 9, 127, 116, 96, 225, 58, 3, 240, 92, 33, 54, 115, 209, 236, 44, 113, 1, 122, 254, 132, 18, 35, 85, 13, 63, 55, 68, 245, 55, 31, 222, 48, 75, 6, 3, 85, 29, 35, 4, 68, 48, 66, 128, 64, 212, 242, 117, 237, 251, 129, 17, 40, 107, 113, 46, 32, 113, 79, 15, 14, 14, 116, 200, 32, 28, 245, 96, 213, 104, 10, 125, 78, 189, 101, 7, 34, 56, 108, 123, 92, 186, 132, 218, 166, 174, 58, 176, 69, 173, 49, 183, 61, 85, 207, 199, 230, 10, 251, 80, 73, 48, 131, 248, 174, 77, 22, 112, 231, 48, 76, 6, 3, 85, 29, 17, 4, 69, 48, 67, 130, 65, 42, 46, 97, 112, 112, 115, 46, 97, 114, 111, 46, 104, 49, 118, 49, 106, 53, 97, 52, 113, 52, 106, 56, 121, 57, 115, 46, 49, 100, 97, 51, 46, 119, 101, 115, 116, 117, 115, 51, 46, 97, 114, 111, 97, 112, 112, 45, 104, 99, 112, 46, 97, 122, 117, 114, 101, 45, 116, 101, 115, 116, 46, 110, 101, 116],
                        RawSubjectPublicKeyInfo: [48, 130, 1, 34, 48, 13, 6, 9, 42, 134, 72, 134, 247, 13, 1, 1, 1, 5, 0, 3, 130, 1, 15, 0, 48, 130, 1, 10, 2, 130, 1, 1, 0, 207, 128, 218, 212, 26, 132, 208, 243, 92, 156, 156, 214, 53, 193, 75, 191, 178, 88, 241, 38, 163, 215, 203, 145, 224, 112, 103, 58, 9, 153, 245, 82, 220, 111, 140, 139, 161, 99, 162, 9, 10, 163, 63, 57, 151, 129, 243, 25, 133, 62, 43, 200, 35, 141, 232, 109, 192, 10, 138, 13, 120, 44, 157, 202, 25, 139, 129, 58, 216, 111, 145, 24, 55, 24, 23, 228, 156, 172, 101, 183, 207, 223, 73, 100, 53, 41, 125, 73, 239, 89, 201, 165, 74, 243, 215, 177, 223, 112, 58, 228, 208, 240, 151, 37, 153, 119, 237, 179, 20, 85, 131, 190, 36, 125, 23, 184, 200, 218, 93, 234, 90, 97, 224, 192, 77, 135, 1, 202, 159, 217, 203, 241, 161, 52, 213, 56, 96, 249, 140, 209, 180, 80, 49, 158, 231, 47, 244, 35, 41, 7, 251, 235, 173, 156, 233, 71, 118, 32, 117, 101, 8, 235, 7, 107, 174, 208, 128, 140, 68, 177, 228, 226, 57, 173, 158, 135, 91, 200, 85, 37, 244, 126, 98, 219, 71, 183, 169, 127, 163, 164, 9, 211, 219, 34, 103, 109, 123, 209, 155, 183, 40, 28, 208, 40, 61, 63, 249, 102, 235, 233, 213, 18, 214, 16, 243, 135, 116, 93, 127, 34, 199, 14, 197, 51, 130, 225, 91, 183, 135, 179, 150, 125, 195, 25, 0, 219, 23, 147, 130, 215, 229, 224, 255, 230, 241, 126, 125, 10, 148, 17, 135, 148, 31, 6, 198, 159, 2, 3, 1, 0, 1],
                        RawSubject: [48, 48, 49, 18, 48, 16, 6, 3, 85, 4, 10, 19, 9, 111, 112, 101, 110, 115, 104, 105, 102, 116, 49, 26, 48, 24, 6, 3, 85, 4, 3, 19, 17, 111, 112, 101, 110, 115, 104, 105, 102, 116, 45, 105, 110, 103, 114, 101, 115, 115],
                        RawIssuer: [48, 38, 49, 18, 48, 16, 6, 3, 85, 4, 11, 19, 9, 111, 112, 101, 110, 115, 104, 105, 102, 116, 49, 16, 48, 14, 6, 3, 85, 4, 3, 19, 7, 114, 111, 111, 116, 45, 99, 97],
                        Signature: [210, 85, 40, 236, 76, 26, 142, 116, 32, 133, 55, 168, 156, 128, 149, 43, 102, 234, 125, 146, 217, 61, 79, 196, 218, 194, 125, 227, 148, 218, 229, 112, 87, 55, 112, 103, 71, 167, 86, 76, 220, 157, 113, 183, 190, 66, 91, 201, 228, 199, 238, 11, 23, 126, 102, 42, 155, 62, 252, 11, 203, 150, 192, 104, 209, 47, 32, 85, 151, 78, 147, 151, 116, 174, 88, 152, 176, 253, 138, 226, 146, 247, 41, 217, 62, 240, 230, 149, 20, 224, 76, 122, 217, 220, 253, 71, 173, 51, 183, 40, 243, 149, 197, 210, 226, 65, 7, 41, 157, 67, 88, 94, 28, 42, 85, 24, 21, 163, 28, 209, 29, 2, 30, 128, 219, 8, 23, 74, 70, 195, 5, 144, 132, 227, 11, 15, 109, 67, 145, 224, 175, 168, 48, 134, 34, 243, 54, 95, 181, 212, 143, 249, 177, 226, 180, 209, 177, 104, 226, 213, 12, 143, 86, 186, 83, 219, 200, 29, 244, 229, 243, 186, 95, 226, 10, 156, 126, 111, 18, 111, 7, 214, 237, 104, 47, 96, 76, 215, 237, 244, 132, 35, 12, 57, 53, 11, 153, 175, 14, 155, 31, 121, 229, 46, 117, 124, 182, 14, 54, 12, 37, 80, 196, 185, 229, 150, 209, 168, 129, 127, 202, 188, 211, 37, 142, 99, 117, 229, 10, 240, 1, 91, 137, 21, 47, 232, 208, 202, 88, 175, 19, 12, 17, 165, 34, 137, 139, 207, 49, 154, 144, 91, 223, 210, 26, 45],
                        SignatureAlgorithm: 4,
                        PublicKeyAlgorithm: 1,
                        PublicKey: <*rsa.PublicKey | 0x1d9b56b98650>{
                            N: {
                                neg: false,
                                abs: [10669458062462863007, 16564520796827843850, 14058268652807094999, 9430919938504365693, 8384997868578981171, 16999352545364210567, 2890413932109887846, 15790296963440810935, 5167785463102310867, 6613629674297647835, 4949989125738307207, 642615530906222732, 12510130358062183781, 16658801978516175851, 6987771129143243166, 11518461760628905272, 6512733652693483978, 2629283839865413098, 11058568763427619774, 16100433422281905957, 17247037959756306353, 14978771580395027785, 3969949342175618487, 1840706962450256152, 13838024595504668106, 9601159594190432365, 766525878170678041, 15884068942095032841, 16172539760854299986, 12851286684961590161, 6673381191695027135, 14952191367382880499],
                            },
                            E: 65537,
                        },
                        Version: 3,
                        SerialNumber: {
                            neg: false,
                            abs: [35611828864482620],
                        },
                        Issuer: {
                            Country: nil,
                            Organization: nil,
                            OrganizationalUnit: ["openshift"],
                            Locality: nil,
                            Province: nil,
                            StreetAddress: nil,
                            PostalCode: nil,
                            SerialNumber: "",
                            CommonName: "root-ca",
                            Names: [
                                {
                                    Type: [2, 5, 4, 11],
                                    Value: <string>"openshift",
                                },
                                {
                                    Type: [2, 5, 4, 3],
                                    Value: <string>"root-ca",
                                },
                            ],
                            ExtraNames: nil,
                        },
                        Subject: {
                            Country: nil,
                            Organization: ["openshift"],
                            OrganizationalUnit: nil,
                            Locality: nil,
                            Province: nil,
                            StreetAddress: nil,
                            PostalCode: nil,
                            SerialNumber: "",
                            CommonName: "openshift-ingress",
                            Names: [
                                {
                                    Type: [2, 5, 4, 10],
                                    Value: <string>"openshift",
                                },
                                {
                                    Type: [2, 5, 4, 3],
                                    Value: <string>"openshift-ingress",
                                },
                            ],
                            ExtraNames: nil,
                        },
                        NotBefore: {wall: 0, ext: 63928409530, loc: nil},
                        NotAfter: {wall: 0, ext: 63959945530, loc: nil},
                        KeyUsage: 5,
                        Extensions: [
                            {
                                Id: [2, 5, 29, 15],
                                Critical: true,
                                Value: [3, 2, 5, 160],
                            },
                            {
                                Id: [2, 5, 29, 37],
                                Critical: false,
                                Value: [48, 20, 6, 8, 43, 6, 1, 5, 5, 7, 3, 2, 6, 8, 43, 6, 1, 5, 5, 7, 3, 1],
                            },
                            {Id: [2, 5, 29, 19], Critical: true, Value: [48, 0]},
                            {
                                Id: [2, 5, 29, 14],
                                Critical: false,
                                Value: [4, 64, 151, 50, 248, 37, 209, 121, 110, 180, 165, 224, 104, 231, 132, 191, 4, 141, 230, 108, 133, 21, 249, 143, 45, 39, 159, 230, 5, 9, 221, 165, 194, 92, 82, 9, 127, 116, 96, 225, 58, 3, 240, 92, 33, 54, 115, 209, 236, 44, 113, 1, 122, 254, 132, 18, 35, 85, 13, 63, 55, 68, 245, 55, 31, 222],
                            },
                            {
                                Id: [2, 5, 29, 35],
                                Critical: false,
                                Value: [48, 66, 128, 64, 212, 242, 117, 237, 251, 129, 17, 40, 107, 113, 46, 32, 113, 79, 15, 14, 14, 116, 200, 32, 28, 245, 96, 213, 104, 10, 125, 78, 189, 101, 7, 34, 56, 108, 123, 92, 186, 132, 218, 166, 174, 58, 176, 69, 173, 49, 183, 61, 85, 207, 199, 230, 10, 251, 80, 73, 48, 131, 248, 174, 77, 22, 112, 231],
                            },
                            {
                                Id: [2, 5, 29, 17],
                                Critical: false,
                                Value: "0C\x82A*.apps.aro.h1v1j5a4q4j8y9s.1da3.westus3.aroapp-hcp.azure-test.net",
                            },
                        ],
                        ExtraExtensions: nil,
                        UnhandledCriticalExtensions: nil,
                        ExtKeyUsage: [2, 1],
                        UnknownExtKeyUsage: nil,
                        BasicConstraintsValid: true,
                        IsCA: false,
                        MaxPathLen: -1,
                        MaxPathLenZero: false,
                        SubjectKeyId: [151, 50, 248, 37, 209, 121, 110, 180, 165, 224, 104, 231, 132, 191, 4, 141, 230, 108, 133, 21, 249, 143, 45, 39, 159, 230, 5, 9, 221, 165, 194, 92, 82, 9, 127, 116, 96, 225, 58, 3, 240, 92, 33, 54, 115, 209, 236, 44, 113, 1, 122, 254, 132, 18, 35, 85, 13, 63, 55, 68, 245, 55, 31, 222],
                        AuthorityKeyId: [212, 242, 117, 237, 251, 129, 17, 40, 107, 113, 46, 32, 113, 79, 15, 14, 14, 116, 200, 32, 28, 245, 96, 213, 104, 10, 125, 78, 189, 101, 7, 34, 56, 108, 123, 92, 186, 132, 218, 166, 174, 58, 176, 69, 173, 49, 183, 61, 85, 207, 199, 230, 10, 251, 80, 73, 48, 131, 248, 174, 77, 22, 112, 231],
                        OCSPServer: nil,
                        IssuingCertificateURL: nil,
                        DNSNames: [
                            "*.apps.aro.h1v1j5a4q4j8y9s.1da3.westus3.aroapp-hcp.azure-test.net",
                        ],
                        EmailAddresses: nil,
                        IPAddresses: nil,
                        URIs: nil,
                        PermittedDNSDomainsCritical: false,
                        PermittedDNSDomains: nil,
                        ExcludedDNSDomains: nil,
                        PermittedIPRanges: nil,
                        ExcludedIPRanges: nil,
                        PermittedEmailAddresses: nil,
                        ExcludedEmailAddresses: nil,
                        PermittedURIDomains: nil,
                        ExcludedURIDomains: nil,
                        CRLDistributionPoints: nil,
                        PolicyIdentifiers: nil,
                        Policies: nil,
                        InhibitAnyPolicy: 0,
                        InhibitAnyPolicyZero: false,
                        InhibitPolicyMapping: 0,
                        InhibitPolicyMappingZero: false,
                        RequireExplicitPolicy: 0,
                        RequireExplicitPolicyZero: false,
                        PolicyMappings: nil,
                    },
                ],
                Err: <x509.UnknownAuthorityError>{
                    Cert: 0x1d9b544f2c08 (cyclic reference),
                    hintErr: nil,
                    hintCert: nil,
                },
            },
        },
    }
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:194]: failed to verify simple web app runs on cluster "cluster-candidate-4-23-xhptzz"
Unexpected error:
    <*fmt.wrapError | 0x1d9b54fe6000>: 
    [strict TLS verification] route was never reachable: Get "https://agnhost-e2e-sample-app-777tx.apps.aro.h1v1j5a4q4j8y9s.1da3.westus3.aroapp-hcp.azure-test.net": tls: failed to verify certificate: x509: certificate signed by unknown authority
    {
        msg: "[strict TLS verification] route was never reachable: Get \"https://agnhost-e2e-sample-app-777tx.apps.aro.h1v1j5a4q4j8y9s.1da3.westus3.aroapp-hcp.azure-test.net\": tls: failed to verify certificate: x509: certificate signed by unknown authority",
        err: <*url.Error | 0x1d9b54f6e0c0>{
            Op: "Get",
            URL: "https://agnhost-e2e-sample-app-777tx.apps.aro.h1v1j5a4q4j8y9s.1da3.westus3.aroapp-hcp.azure-test.net",
            Err: <*tls.CertificateVerificationError | 0x1d9b54f6e060>{
                UnverifiedCertificates: [
                    {
                        Raw: [48, 130, 4, 0, 48, 130, 2, 232, 160, 3, 2, 1, 2, 2, 7, 126, 132, 197, 9, 68, 253, 60, 48, 13, 6, 9, 42, 134, 72, 134, 247, 13, 1, 1, 11, 5, 0, 48, 38, 49, 18, 48, 16, 6, 3, 85, 4, 11, 19, 9, 111, 112, 101, 110, 115, 104, 105, 102, 116, 49, 16, 48, 14, 6, 3, 85, 4, 3, 19, 7, 114, 111, 111, 116, 45, 99, 97, 48, 30, 23, 13, 50, 54, 49, 48, 50, 52, 48, 51, 51, 50, 49, 48, 90, 23, 13, 50, 55, 49, 48, 50, 52, 48, 51, 51, 50, 49, 48, 90, 48, 48, 49, 18, 48, 16, 6, 3, 85, 4, 10, 19, 9, 111, 112, 101, 110, 115, 104, 105, 102, 116, 49, 26, 48, 24, 6, 3, 85, 4, 3, 19, 17, 111, 112, 101, 110, 115, 104, 105, 102, 116, 45, 105, 110, 103, 114, 101, 115, 115, 48, 130, 1, 34, 48, 13, 6, 9, 42, 134, 72, 134, 247, 13, 1, 1, 1, 5, 0, 3, 130, 1, 15, 0, 48, 130, 1, 10, 2, 130, 1, 1, 0, 207, 128, 218, 212, 26, 132, 208, 243, 92, 156, 156, 214, 53, 193, 75, 191, 178, 88, 241, 38, 163, 215, 203, 145, 224, 112, 103, 58, 9, 153, 245, 82, 220, 111, 140, 139, 161, 99, 162, 9, 10, 163, 63, 57, 151, 129, 243, 25, 133, 62, 43, 200, 35, 141, 232, 109, 192, 10, 138, 13, 120, 44, 157, 202, 25, 139, 129, 58, 216, 111, 145, 24, 55, 24, 23, 228, 156, 172, 101, 183, 207, 223, 73, 100, 53, 41, 125, 73, 239, 89, 201, 165, 74, 243, 215, 177, 223, 112, 58, 228, 208, 240, 151, 37, 153, 119, 237, 179, 20, 85, 131, 190, 36, 125, 23, 184, 200, 218, 93, 234, 90, 97, 224, 192, 77, 135, 1, 202, 159, 217, 203, 241, 161, 52, 213, 56, 96, 249, 140, 209, 180, 80, 49, 158, 231, 47, 244, 35, 41, 7, 251, 235, 173, 156, 233, 71, 118, 32, 117, 101, 8, 235, 7, 107, 174, 208, 128, 140, 68, 177, 228, 226, 57, 173, 158, 135, 91, 200, 85, 37, 244, 126, 98, 219, 71, 183, 169, 127, 163, 164, 9, 211, 219, 34, 103, 109, 123, 209, 155, 183, 40, 28, 208, 40, 61, 63, 249, 102, 235, 233, 213, 18, 214, 16, 243, 135, 116, 93, 127, 34, 199, 14, 197, 51, 130, 225, 91, 183, 135, 179, 150, 125, 195, 25, 0, 219, 23, 147, 130, 215, 229, 224, 255, 230, 241, 126, 125, 10, 148, 17, 135, 148, 31, 6, 198, 159, 2, 3, 1, 0, 1, 163, 130, 1, 39, 48, 130, 1, 35, 48, 14, 6, 3, 85, 29, 15, 1, 1, 255, 4, 4, 3, 2, 5, 160, 48, 29, 6, 3, 85, 29, 37, 4, 22, 48, 20, 6, 8, 43, 6, 1, 5, 5, 7, 3, 2, 6, 8, 43, 6, 1, 5, 5, 7, 3, 1, 48, 12, 6, 3, 85, 29, 19, 1, 1, 255, 4, 2, 48, 0, 48, 73, 6, 3, 85, 29, 14, 4, 66, 4, 64, 151, 50, 248, 37, 209, 121, 110, 180, 165, 224, 104, 231, 132, 191, 4, 141, 230, 108, 133, 21, 249, 143, 45, 39, 159, 230, 5, 9, 221, 165, 194, 92, 82, 9, 127, 116, 96, 225, 58, 3, 240, 92, 33, 54, 115, 209, 236, 44, 113, 1, 122, 254, 132, 18, 35, 85, 13, 63, 55, 68, 245, 55, 31, 222, 48, 75, 6, 3, 85, 29, 35, 4, 68, 48, 66, 128, 64, 212, 242, 117, 237, 251, 129, 17, 40, 107, 113, 46, 32, 113, 79, 15, 14, 14, 116, 200, 32, 28, 245, 96, 213, 104, 10, 125, 78, 189, 101, 7, 34, 56, 108, 123, 92, 186, 132, 218, 166, 174, 58, 176, 69, 173, 49, 183, 61, 85, 207, 199, 230, 10, 251, 80, 73, 48, 131, 248, 174, 77, 22, 112, 231, 48, 76, 6, 3, 85, 29, 17, 4, 69, 48, 67, 130, 65, 42, 46, 97, 112, 112, 115, 46, 97, 114, 111, 46, 104, 49, 118, 49, 106, 53, 97, 52, 113, 52, 106, 56, 121, 57, 115, 46, 49, 100, 97, 51, 46, 119, 101, 115, 116, 117, 115, 51, 46, 97, 114, 111, 97, 112, 112, 45, 104, 99, 112, 46, 97, 122, 117, 114, 101, 45, 116, 101, 115, 116, 46, 110, 101, 116, 48, 13, 6, 9, 42, 134, 72, 134, 247, 13, 1, 1, 11, 5, 0, 3, 130, 1, 1, 0, 210, 85, 40, 236, 76, 26, 142, 116, 32, 133, 55, 168, 156, 128, 149, 43, 102, 234, 125, 146, 217, 61, 79, 196, 218, 194, 125, 227, 148, 218, 229, 112, 87, 55, 112, 103, 71, 167, 86, 76, 220, 157, 113, 183, 190, 66, 91, 201, 228, 199, 238, 11, 23, 126, 102, 42, 155, 62, 252, 11, 203, 150, 192, 104, 209, 47, 32, 85, 151, 78, 147, 151, 116, 174, 88, 152, 176, 253, 138, 226, 146, 247, 41, 217, 62, 240, 230, 149, 20, 224, 76, 122, 217, 220, 253, 71, 173, 51, 183, 40, 243, 149, 197, 210, 226, 65, 7, 41, 157, 67, 88, 94, 28, 42, 85, 24, 21, 163, 28, 209, 29, 2, 30, 128, 219, 8, 23, 74, 70, 195, 5, 144, 132, 227, 11, 15, 109, 67, 145, 224, 175, 168, 48, 134, 34, 243, 54, 95, 181, 212, 143, 249, 177, 226, 180, 209, 177, 104, 226, 213, 12, 143, 86, 186, 83, 219, 200, 29, 244, 229, 243, 186, 95, 226, 10, 156, 126, 111, 18, 111, 7, 214, 237, 104, 47, 96, 76, 215, 237, 244, 132, 35, 12, 57, 53, 11, 153, 175, 14, 155, 31, 121, 229, 46, 117, 124, 182, 14, 54, 12, 37, 80, 196, 185, 229, 150, 209, 168, 129, 127, 202, 188, 211, 37, 142, 99, 117, 229, 10, 240, 1, 91, 137, 21, 47, 232, 208, 202, 88, 175, 19, 12, 17, 165, 34, 137, 139, 207, 49, 154, 144, 91, 223, 210, 26, 45],
                        RawTBSCertificate: [48, 130, 2, 232, 160, 3, 2, 1, 2, 2, 7, 126, 132, 197, 9, 68, 253, 60, 48, 13, 6, 9, 42, 134, 72, 134, 247, 13, 1, 1, 11, 5, 0, 48, 38, 49, 18, 48, 16, 6, 3, 85, 4, 11, 19, 9, 111, 112, 101, 110, 115, 104, 105, 102, 116, 49, 16, 48, 14, 6, 3, 85, 4, 3, 19, 7, 114, 111, 111, 116, 45, 99, 97, 48, 30, 23, 13, 50, 54, 49, 48, 50, 52, 48, 51, 51, 50, 49, 48, 90, 23, 13, 50, 55, 49, 48, 50, 52, 48, 51, 51, 50, 49, 48, 90, 48, 48, 49, 18, 48, 16, 6, 3, 85, 4, 10, 19, 9, 111, 112, 101, 110, 115, 104, 105, 102, 116, 49, 26, 48, 24, 6, 3, 85, 4, 3, 19, 17, 111, 112, 101, 110, 115, 104, 105, 102, 116, 45, 105, 110, 103, 114, 101, 115, 115, 48, 130, 1, 34, 48, 13, 6, 9, 42, 134, 72, 134, 247, 13, 1, 1, 1, 5, 0, 3, 130, 1, 15, 0, 48, 130, 1, 10, 2, 130, 1, 1, 0, 207, 128, 218, 212, 26, 132, 208, 243, 92, 156, 156, 214, 53, 193, 75, 191, 178, 88, 241, 38, 163, 215, 203, 145, 224, 112, 103, 58, 9, 153, 245, 82, 220, 111, 140, 139, 161, 99, 162, 9, 10, 163, 63, 57, 151, 129, 243, 25, 133, 62, 43, 200, 35, 141, 232, 109, 192, 10, 138, 13, 120, 44, 157, 202, 25, 139, 129, 58, 216, 111, 145, 24, 55, 24, 23, 228, 156, 172, 101, 183, 207, 223, 73, 100, 53, 41, 125, 73, 239, 89, 201, 165, 74, 243, 215, 177, 223, 112, 58, 228, 208, 240, 151, 37, 153, 119, 237, 179, 20, 85, 131, 190, 36, 125, 23, 184, 200, 218, 93, 234, 90, 97, 224, 192, 77, 135, 1, 202, 159, 217, 203, 241, 161, 52, 213, 56, 96, 249, 140, 209, 180, 80, 49, 158, 231, 47, 244, 35, 41, 7, 251, 235, 173, 156, 233, 71, 118, 32, 117, 101, 8, 235, 7, 107, 174, 208, 128, 140, 68, 177, 228, 226, 57, 173, 158, 135, 91, 200, 85, 37, 244, 126, 98, 219, 71, 183, 169, 127, 163, 164, 9, 211, 219, 34, 103, 109, 123, 209, 155, 183, 40, 28, 208, 40, 61, 63, 249, 102, 235, 233, 213, 18, 214, 16, 243, 135, 116, 93, 127, 34, 199, 14, 197, 51, 130, 225, 91, 183, 135, 179, 150, 125, 195, 25, 0, 219, 23, 147, 130, 215, 229, 224, 255, 230, 241, 126, 125, 10, 148, 17, 135, 148, 31, 6, 198, 159, 2, 3, 1, 0, 1, 163, 130, 1, 39, 48, 130, 1, 35, 48, 14, 6, 3, 85, 29, 15, 1, 1, 255, 4, 4, 3, 2, 5, 160, 48, 29, 6, 3, 85, 29, 37, 4, 22, 48, 20, 6, 8, 43, 6, 1, 5, 5, 7, 3, 2, 6, 8, 43, 6, 1, 5, 5, 7, 3, 1, 48, 12, 6, 3, 85, 29, 19, 1, 1, 255, 4, 2, 48, 0, 48, 73, 6, 3, 85, 29, 14, 4, 66, 4, 64, 151, 50, 248, 37, 209, 121, 110, 180, 165, 224, 104, 231, 132, 191, 4, 141, 230, 108, 133, 21, 249, 143, 45, 39, 159, 230, 5, 9, 221, 165, 194, 92, 82, 9, 127, 116, 96, 225, 58, 3, 240, 92, 33, 54, 115, 209, 236, 44, 113, 1, 122, 254, 132, 18, 35, 85, 13, 63, 55, 68, 245, 55, 31, 222, 48, 75, 6, 3, 85, 29, 35, 4, 68, 48, 66, 128, 64, 212, 242, 117, 237, 251, 129, 17, 40, 107, 113, 46, 32, 113, 79, 15, 14, 14, 116, 200, 32, 28, 245, 96, 213, 104, 10, 125, 78, 189, 101, 7, 34, 56, 108, 123, 92, 186, 132, 218, 166, 174, 58, 176, 69, 173, 49, 183, 61, 85, 207, 199, 230, 10, 251, 80, 73, 48, 131, 248, 174, 77, 22, 112, 231, 48, 76, 6, 3, 85, 29, 17, 4, 69, 48, 67, 130, 65, 42, 46, 97, 112, 112, 115, 46, 97, 114, 111, 46, 104, 49, 118, 49, 106, 53, 97, 52, 113, 52, 106, 56, 121, 57, 115, 46, 49, 100, 97, 51, 46, 119, 101, 115, 116, 117, 115, 51, 46, 97, 114, 111, 97, 112, 112, 45, 104, 99, 112, 46, 97, 122, 117, 114, 101, 45, 116, 101, 115, 116, 46, 110, 101, 116],
                        RawSubjectPublicKeyInfo: [48, 130, 1, 34, 48, 13, 6, 9, 42, 134, 72, 134, 247, 13, 1, 1, 1, 5, 0, 3, 130, 1, 15, 0, 48, 130, 1, 10, 2, 130, 1, 1, 0, 207, 128, 218, 212, 26, 132, 208, 243, 92, 156, 156, 214, 53, 193, 75, 191, 178, 88, 241, 38, 163, 215, 203, 145, 224, 112, 103, 58, 9, 153, 245, 82, 220, 111, 140, 139, 161, 99, 162, 9, 10, 163, 63, 57, 151, 129, 243, 25, 133, 62, 43, 200, 35, 141, 232, 109, 192, 10, 138, 13, 120, 44, 157, 202, 25, 139, 129, 58, 216, 111, 145, 24, 55, 24, 23, 228, 156, 172, 101, 183, 207, 223, 73, 100, 53, 41, 125, 73, 239, 89, 201, 165, 74, 243, 215, 177, 223, 112, 58, 228, 208, 240, 151, 37, 153, 119, 237, 179, 20, 85, 131, 190, 36, 125, 23, 184, 200, 218, 93, 234, 90, 97, 224, 192, 77, 135, 1, 202, 159, 217, 203, 241, 161, 52, 213, 56, 96, 249, 140, 209, 180, 80, 49, 158, 231, 47, 244, 35, 41, 7, 251, 235, 173, 156, 233, 71, 118, 32, 117, 101, 8, 235, 7, 107, 174, 208, 128, 140, 68, 177, 228, 226, 57, 173, 158, 135, 91, 200, 85, 37, 244, 126, 98, 219, 71, 183, 169, 127, 163, 164, 9, 211, 219, 34, 103, 109, 123, 209, 155, 183, 40, 28, 208, 40, 61, 63, 249, 102, 235, 233, 213, 18, 214, 16, 243, 135, 116, 93, 127, 34, 199, 14, 197, 51, 130, 225, 91, 183, 135, 179, 150, 125, 195, 25, 0, 219, 23, 147, 130, 215, 229, 224, 255, 230, 241, 126, 125, 10, 148, 17, 135, 148, 31, 6, 198, 159, 2, 3, 1, 0, 1],
                        RawSubject: [48, 48, 49, 18, 48, 16, 6, 3, 85, 4, 10, 19, 9, 111, 112, 101, 110, 115, 104, 105, 102, 116, 49, 26, 48, 24, 6, 3, 85, 4, 3, 19, 17, 111, 112, 101, 110, 115, 104, 105, 102, 116, 45, 105, 110, 103, 114, 101, 115, 115],
                        RawIssuer: [48, 38, 49, 18, 48, 16, 6, 3, 85, 4, 11, 19, 9, 111, 112, 101, 110, 115, 104, 105, 102, 116, 49, 16, 48, 14, 6, 3, 85, 4, 3, 19, 7, 114, 111, 111, 116, 45, 99, 97],
                        Signature: [210, 85, 40, 236, 76, 26, 142, 116, 32, 133, 55, 168, 156, 128, 149, 43, 102, 234, 125, 146, 217, 61, 79, 196, 218, 194, 125, 227, 148, 218, 229, 112, 87, 55, 112, 103, 71, 167, 86, 76, 220, 157, 113, 183, 190, 66, 91, 201, 228, 199, 238, 11, 23, 126, 102, 42, 155, 62, 252, 11, 203, 150, 192, 104, 209, 47, 32, 85, 151, 78, 147, 151, 116, 174, 88, 152, 176, 253, 138, 226, 146, 247, 41, 217, 62, 240, 230, 149, 20, 224, 76, 122, 217, 220, 253, 71, 173, 51, 183, 40, 243, 149, 197, 210, 226, 65, 7, 41, 157, 67, 88, 94, 28, 42, 85, 24, 21, 163, 28, 209, 29, 2, 30, 128, 219, 8, 23, 74, 70, 195, 5, 144, 132, 227, 11, 15, 109, 67, 145, 224, 175, 168, 48, 134, 34, 243, 54, 95, 181, 212, 143, 249, 177, 226, 180, 209, 177, 104, 226, 213, 12, 143, 86, 186, 83, 219, 200, 29, 244, 229, 243, 186, 95, 226, 10, 156, 126, 111, 18, 111, 7, 214, 237, 104, 47, 96, 76, 215, 237, 244, 132, 35, 12, 57, 53, 11, 153, 175, 14, 155, 31, 121, 229, 46, 117, 124, 182, 14, 54, 12, 37, 80, 196, 185, 229, 150, 209, 168, 129, 127, 202, 188, 211, 37, 142, 99, 117, 229, 10, 240, 1, 91, 137, 21, 47, 232, 208, 202, 88, 175, 19, 12, 17, 165, 34, 137, 139, 207, 49, 154, 144, 91, 223, 210, 26, 45],
                        SignatureAlgorithm: 4,
                        PublicKeyAlgorithm: 1,
                        PublicKey: <*rsa.PublicKey | 0x1d9b56b98650>{
                            N: {
                                neg: false,
                                abs: [10669458062462863007, 16564520796827843850, 14058268652807094999, 9430919938504365693, 8384997868578981171, 16999352545364210567, 2890413932109887846, 15790296963440810935, 5167785463102310867, 6613629674297647835, 4949989125738307207, 642615530906222732, 12510130358062183781, 16658801978516175851, 6987771129143243166, 11518461760628905272, 6512733652693483978, 2629283839865413098, 11058568763427619774, 16100433422281905957, 17247037959756306353, 14978771580395027785, 3969949342175618487, 1840706962450256152, 13838024595504668106, 9601159594190432365, 766525878170678041, 15884068942095032841, 16172539760854299986, 12851286684961590161, 6673381191695027135, 14952191367382880499],
                            },
                            E: 65537,
                        },
                        Version: 3,
                        SerialNumber: {
                            neg: false,
                            abs: [35611828864482620],
                        },
                        Issuer: {
                            Country: nil,
                            Organization: nil,
                            OrganizationalUnit: ["openshift"],
                            Locality: nil,
                            Province: nil,
                            StreetAddress: nil,
                            PostalCode: nil,
                            SerialNumber: "",
                            CommonName: "root-ca",
                            Names: [
                                {
                                    Type: [2, 5, 4, 11],
                                    Value: <string>"openshift",
                                },
                                {
                                    Type: [2, 5, 4, 3],
                                    Value: <string>"root-ca",
                                },
                            ],
                            ExtraNames: nil,
                        },
                        Subject: {
                            Country: nil,
                            Organization: ["openshift"],
                            OrganizationalUnit: nil,
                            Locality: nil,
                            Province: nil,
                            StreetAddress: nil,
                            PostalCode: nil,
                            SerialNumber: "",
                            CommonName: "openshift-ingress",
                            Names: [
                                {
                                    Type: [2, 5, 4, 10],
                                    Value: <string>"openshift",
                                },
                                {
                                    Type: [2, 5, 4, 3],
                                    Value: <string>"openshift-ingress",
                                },
                            ],
                            ExtraNames: nil,
                        },
                        NotBefore: {wall: 0, ext: 63928409530, loc: nil},
                        NotAfter: {wall: 0, ext: 63959945530, loc: nil},
                        KeyUsage: 5,
                        Extensions: [
                            {
                                Id: [2, 5, 29, 15],
                                Critical: true,
                                Value: [3, 2, 5, 160],
                            },
                            {
                                Id: [2, 5, 29, 37],
                                Critical: false,
                                Value: [48, 20, 6, 8, 43, 6, 1, 5, 5, 7, 3, 2, 6, 8, 43, 6, 1, 5, 5, 7, 3, 1],
                            },
                            {Id: [2, 5, 29, 19], Critical: true, Value: [48, 0]},
                            {
                                Id: [2, 5, 29, 14],
                                Critical: false,
                                Value: [4, 64, 151, 50, 248, 37, 209, 121, 110, 180, 165, 224, 104, 231, 132, 191, 4, 141, 230, 108, 133, 21, 249, 143, 45, 39, 159, 230, 5, 9, 221, 165, 194, 92, 82, 9, 127, 116, 96, 225, 58, 3, 240, 92, 33, 54, 115, 209, 236, 44, 113, 1, 122, 254, 132, 18, 35, 85, 13, 63, 55, 68, 245, 55, 31, 222],
                            },
                            {
                                Id: [2, 5, 29, 35],
                                Critical: false,
                                Value: [48, 66, 128, 64, 212, 242, 117, 237, 251, 129, 17, 40, 107, 113, 46, 32, 113, 79, 15, 14, 14, 116, 200, 32, 28, 245, 96, 213, 104, 10, 125, 78, 189, 101, 7, 34, 56, 108, 123, 92, 186, 132, 218, 166, 174, 58, 176, 69, 173, 49, 183, 61, 85, 207, 199, 230, 10, 251, 80, 73, 48, 131, 248, 174, 77, 22, 112, 231],
                            },
                            {
                                Id: [2, 5, 29, 17],
                                Critical: false,
                                Value: "0C\x82A*.apps.aro.h1v1j5a4q4j8y9s.1da3.westus3.aroapp-hcp.azure-test.net",
                            },
                        ],
                        ExtraExtensions: nil,
                        UnhandledCriticalExtensions: nil,
                        ExtKeyUsage: [2, 1],
                        UnknownExtKeyUsage: nil,
                        BasicConstraintsValid: true,
                        IsCA: false,
                        MaxPathLen: -1,
                        MaxPathLenZero: false,
                        SubjectKeyId: [151, 50, 248, 37, 209, 121, 110, 180, 165, 224, 104, 231, 132, 191, 4, 141, 230, 108, 133, 21, 249, 143, 45, 39, 159, 230, 5, 9, 221, 165, 194, 92, 82, 9, 127, 116, 96, 225, 58, 3, 240, 92, 33, 54, 115, 209, 236, 44, 113, 1, 122, 254, 132, 18, 35, 85, 13, 63, 55, 68, 245, 55, 31, 222],
                        AuthorityKeyId: [212, 242, 117, 237, 251, 129, 17, 40, 107, 113, 46, 32, 113, 79, 15, 14, 14, 116, 200, 32, 28, 245, 96, 213, 104, 10, 125, 78, 189, 101, 7, 34, 56, 108, 123, 92, 186, 132, 218, 166, 174, 58, 176, 69, 173, 49, 183, 61, 85, 207, 199, 230, 10, 251, 80, 73, 48, 131, 248, 174, 77, 22, 112, 231],
                        OCSPServer: nil,
                        IssuingCertificateURL: nil,
                        DNSNames: [
                            "*.apps.aro.h1v1j5a4q4j8y9s.1da3.westus3.aroapp-hcp.azure-test.net",
                        ],
                        EmailAddresses: nil,
                        IPAddresses: nil,
                        URIs: nil,
                        PermittedDNSDomainsCritical: false,
                        PermittedDNSDomains: nil,
                        ExcludedDNSDomains: nil,
                        PermittedIPRanges: nil,
                        ExcludedIPRanges: nil,
                        PermittedEmailAddresses: nil,
                        ExcludedEmailAddresses: nil,
                        PermittedURIDomains: nil,
                        ExcludedURIDomains: nil,
                        CRLDistributionPoints: nil,
                        PolicyIdentifiers: nil,
                        Policies: nil,
                        InhibitAnyPolicy: 0,
                        InhibitAnyPolicyZero: false,
                        InhibitPolicyMapping: 0,
                        InhibitPolicyMappingZero: false,
                        RequireExplicitPolicy: 0,
                        RequireExplicitPolicyZero: false,
                        PolicyMappings: nil,
                    },
                ],
                Err: <x509.UnknownAuthorityError>{
                    Cert: 0x1d9b544f2c08 (cyclic reference),
                    hintErr: nil,
                    hintCert: nil,
                },
            },
        },
    }
occurred
Internal error occurred: failed calling webhook "webhook
03:02:26 UTC · failed at unknown · test Customer should be able to issue the default ingress serving certificate from Let's Encrypt using cert-manager with ACME DNS-01 challenges for cluster version 5.1 · suite integration/parallel/all
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_ingress_cert_letsencrypt.go:372]: Timed out after 120.003s.
ClusterIssuer CRD should be installed and the resource creatable
Expected success, but got an error:
    <*errors.StatusError | 0x11d151799220>: 
    Internal error occurred: failed calling webhook "webhook.cert-manager.io": failed to call webhook: Post "https://cert-manager-webhook.cert-manager.svc:443/validate?timeout=30s": no endpoints available for service "cert-manager-webhook"
    {
        ErrStatus: {
            TypeMeta: {Kind: "Status", APIVersion: "v1"},
            ListMeta: {
                SelfLink: "",
                ResourceVersion: "",
                Continue: "",
                RemainingItemCount: nil,
                ShardInfo: nil,
            },
            Status: "Failure",
            Message: "Internal error occurred: failed calling webhook \"webhook.cert-manager.io\": failed to call webhook: Post \"https://cert-manager-webhook.cert-manager.svc:443/validate?timeout=30s\": no endpoints available for service \"cert-manager-webhook\"",
            Reason: "InternalError",
            Details: {
                Name: "",
                Group: "",
                Kind: "",
                UID: "",
                Causes: [
                    {
                        Type: "",
                        Message: "failed calling webhook \"webhook.cert-manager.io\": failed to call webhook: Post \"https://cert-manager-webhook.cert-manager.svc:443/validate?timeout=30s\": no endpoints available for service \"cert-manager-webhook\"",
                        Field: "",
                    },
                ],
                RetryAfterSeconds: 0,
            },
            Code: 500,
        },
    }
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_ingress_cert_letsencrypt.go:372]: Timed out after 120.003s.
ClusterIssuer CRD should be installed and the resource creatable
Expected success, but got an error:
    <*errors.StatusError | 0x11d151799220>: 
    Internal error occurred: failed calling webhook "webhook.cert-manager.io": failed to call webhook: Post "https://cert-manager-webhook.cert-manager.svc:443/validate?timeout=30s": no endpoints available for service "cert-manager-webhook"
    {
        ErrStatus: {
            TypeMeta: {Kind: "Status", APIVersion: "v1"},
            ListMeta: {
                SelfLink: "",
                ResourceVersion: "",
                Continue: "",
                RemainingItemCount: nil,
                ShardInfo: nil,
            },
            Status: "Failure",
            Message: "Internal error occurred: failed calling webhook \"webhook.cert-manager.io\": failed to call webhook: Post \"https://cert-manager-webhook.cert-manager.svc:443/validate?timeout=30s\": no endpoints available for service \"cert-manager-webhook\"",
            Reason: "InternalError",
            Details: {
                Name: "",
                Group: "",
                Kind: "",
                UID: "",
                Causes: [
                    {
                        Type: "",
                        Message: "failed calling webhook \"webhook.cert-manager.io\": failed to call webhook: Post \"https://cert-manager-webhook.cert-manager.svc:443/validate?timeout=30s\": no endpoints available for service \"cert-manager-webhook\"",
                        Field: "",
                    },
                ],
                RetryAfterSeconds: 0,
            },
            Code: 500,
        },
    }
ERROR CODE: DeadlineExceeded; detail code SubscriptionRequestsThrottled; detail message Number of 'read' requests for subscription 'XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX' actor '<uuid>' exceeded. Ple... message truncated; provider Microsoft.RedHatOpenShift
03:02:26 UTC · failed at unknown · test Customer should upgrade control plane minor version when pods cannot be created from 5.0 minor to 5.1 minor · suite integration/parallel/all
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade_blocked_pod_creation.go:201]: failed to create node pool "np-deny-pods" for cluster "cp-ystream-deny-pods-5-1-fjj5cf"
Unexpected error:
    <*fmt.wrapError | 0x3b484fe8ea00>: 
    failed to create NodePool np-deny-pods: failed waiting for nodepool="np-deny-pods" for cluster "cp-ystream-deny-pods-5-1-fjj5cf" in resourcegroup="rg-cp-ystream-deny-pods-5-1-j5b76vjznvvh" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westus3/hcpOperationStatuses/cf1ae438-bf01-4f86-8ab5-784122b7cb5a
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: DeadlineExceeded
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westus3/hcpOperationStatuses/cf1ae438-bf01-4f86-8ab5-784122b7cb5a",
      "name": "cf1ae438-bf01-4f86-8ab5-784122b7cb5a",
      "status": "Failed",
      "startTime": "2026-10-24T03:36:53.315230765Z",
      "endTime": "2026-10-24T03:56:02.496643822Z",
      "error": {
        "code": "DeadlineExceeded",
        "message": "node pool creation did not complete before the deadline; [hypershiftNodePool] node pool condition AllMachinesReady is False: 1 of 2 machines are not ready\nMachine z4z5b9p8d5g2j9j-np-deny-pods-8gxvs-vs65j: Failed: interfaces failed to create or update. err: failed to get existing resource rg-cp-ystream-deny-pods-5-1-j5b76vjznvvh-cp-ystream-den-2ba76655/z4z5b9p8d5g2j9j-np-deny-pods-8gxvs-vs65j-nic (service: interfaces): GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-cp-ystream-deny-pods-5-1-j5b76vjznvvh-cp-ystream-den-2ba76655/providers/Microsoft.Network/networkInterfaces/z4z5b9p8d5g2j9j-np-deny-pods-8gxvs-vs65j-nic\n--------------------------------------------------------------------------------\nRESPONSE 429: 429 Too Many Requests\nERROR CODE: SubscriptionRequestsThrottled\n--------------------------------------------------------------------------------\n{\n  \"error\": {\n    \"code\": \"SubscriptionRequestsThrottled\",\n    \"message\": \"Number of 'read' requests for subscription 'XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX' actor '6096f642-63a0-4dd6-958d-715a87f33535' exceeded. Ple... message truncated\n"
      }
    }
    --------------------------------------------------------------------------------
    
    <Azure error internals omitted>
        Azure request IDs:
          GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westus3/hcpOperationStatuses/cf1ae438-bf01-4f86-8ab5-784122b7cb5a
            correlation ID: 3b16cd17-d78f-4b89-a83b-76ef2e36cf89
            client request ID: unavailable
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade_blocked_pod_creation.go:201]: failed to create node pool "np-deny-pods" for cluster "cp-ystream-deny-pods-5-1-fjj5cf"
Unexpected error:
    <*fmt.wrapError | 0x3b484fe8ea00>: 
    failed to create NodePool np-deny-pods: failed waiting for nodepool="np-deny-pods" for cluster "cp-ystream-deny-pods-5-1-fjj5cf" in resourcegroup="rg-cp-ystream-deny-pods-5-1-j5b76vjznvvh" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westus3/hcpOperationStatuses/cf1ae438-bf01-4f86-8ab5-784122b7cb5a
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: DeadlineExceeded
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westus3/hcpOperationStatuses/cf1ae438-bf01-4f86-8ab5-784122b7cb5a",
      "name": "cf1ae438-bf01-4f86-8ab5-784122b7cb5a",
      "status": "Failed",
      "startTime": "2026-10-24T03:36:53.315230765Z",
      "endTime": "2026-10-24T03:56:02.496643822Z",
      "error": {
        "code": "DeadlineExceeded",
        "message": "node pool creation did not complete before the deadline; [hypershiftNodePool] node pool condition AllMachinesReady is False: 1 of 2 machines are not ready\nMachine z4z5b9p8d5g2j9j-np-deny-pods-8gxvs-vs65j: Failed: interfaces failed to create or update. err: failed to get existing resource rg-cp-ystream-deny-pods-5-1-j5b76vjznvvh-cp-ystream-den-2ba76655/z4z5b9p8d5g2j9j-np-deny-pods-8gxvs-vs65j-nic (service: interfaces): GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-cp-ystream-deny-pods-5-1-j5b76vjznvvh-cp-ystream-den-2ba76655/providers/Microsoft.Network/networkInterfaces/z4z5b9p8d5g2j9j-np-deny-pods-8gxvs-vs65j-nic\n--------------------------------------------------------------------------------\nRESPONSE 429: 429 Too Many Requests\nERROR CODE: SubscriptionRequestsThrottled\n--------------------------------------------------------------------------------\n{\n  \"error\": {\n    \"code\": \"SubscriptionRequestsThrottled\",\n    \"message\": \"Number of 'read' requests for subscription 'XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX' actor '6096f642-63a0-4dd6-958d-715a87f33535' exceeded. Ple... message truncated\n"
      }
    }
    --------------------------------------------------------------------------------
    
    <Azure error internals omitted>
        Azure request IDs:
          GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westus3/hcpOperationStatuses/cf1ae438-bf01-4f86-8ab5-784122b7cb5a
            correlation ID: 3b16cd17-d78f-4b89-a83b-76ef2e36cf89
            client request ID: unavailable
occurred
1h 11m 12s branch-ci-Azure-ARO-HCP-main-e2e-integration-e2e-parallel-all
post-goodmerged PR
No additional run metadata captured
westus3 unknown failed n/a 1
unknown (1)
tls: failed to verify certificate: x509: certificate signed by unknown authority
00:44:35 UTC · failed at unknown · test ARO-HCP should be able to perform a control plane and node pool install with OCP candidate channel for 4.23 · suite integration/parallel/all
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:194]: failed to verify simple web app runs on cluster "cluster-candidate-4-23-8z7tws"
Unexpected error:
    <*fmt.wrapError | 0x1f1968099be0>: 
    [strict TLS verification] route was never reachable: Get "https://agnhost-e2e-sample-app-69t4s.apps.aro.j2a0a0o5x9b4d9j.ekt0.westus3.aroapp-hcp.azure-test.net": tls: failed to verify certificate: x509: certificate signed by unknown authority
    {
        msg: "[strict TLS verification] route was never reachable: Get \"https://agnhost-e2e-sample-app-69t4s.apps.aro.j2a0a0o5x9b4d9j.ekt0.westus3.aroapp-hcp.azure-test.net\": tls: failed to verify certificate: x509: certificate signed by unknown authority",
        err: <*url.Error | 0x1f1969237a70>{
            Op: "Get",
            URL: "https://agnhost-e2e-sample-app-69t4s.apps.aro.j2a0a0o5x9b4d9j.ekt0.westus3.aroapp-hcp.azure-test.net",
            Err: <*tls.CertificateVerificationError | 0x1f1969237a40>{
                UnverifiedCertificates: [
                    {
                        Raw: [48, 130, 4, 1, 48, 130, 2, 233, 160, 3, 2, 1, 2, 2, 8, 10, 79, 113, 121, 215, 189, 122, 224, 48, 13, 6, 9, 42, 134, 72, 134, 247, 13, 1, 1, 11, 5, 0, 48, 38, 49, 18, 48, 16, 6, 3, 85, 4, 11, 19, 9, 111, 112, 101, 110, 115, 104, 105, 102, 116, 49, 16, 48, 14, 6, 3, 85, 4, 3, 19, 7, 114, 111, 111, 116, 45, 99, 97, 48, 30, 23, 13, 50, 54, 49, 48, 50, 52, 48, 48, 53, 56, 51, 56, 90, 23, 13, 50, 55, 49, 48, 50, 52, 48, 48, 53, 56, 51, 56, 90, 48, 48, 49, 18, 48, 16, 6, 3, 85, 4, 10, 19, 9, 111, 112, 101, 110, 115, 104, 105, 102, 116, 49, 26, 48, 24, 6, 3, 85, 4, 3, 19, 17, 111, 112, 101, 110, 115, 104, 105, 102, 116, 45, 105, 110, 103, 114, 101, 115, 115, 48, 130, 1, 34, 48, 13, 6, 9, 42, 134, 72, 134, 247, 13, 1, 1, 1, 5, 0, 3, 130, 1, 15, 0, 48, 130, 1, 10, 2, 130, 1, 1, 0, 218, 203, 20, 74, 150, 131, 221, 244, 8, 174, 50, 200, 232, 206, 148, 177, 108, 150, 50, 246, 233, 250, 129, 223, 15, 120, 59, 11, 252, 58, 210, 100, 231, 255, 150, 147, 148, 67, 32, 69, 22, 138, 183, 209, 53, 75, 133, 88, 171, 155, 169, 197, 58, 104, 98, 205, 78, 168, 252, 61, 77, 33, 3, 158, 73, 106, 0, 150, 138, 251, 213, 201, 174, 186, 121, 229, 217, 13, 171, 213, 245, 168, 65, 246, 119, 20, 133, 253, 84, 108, 76, 119, 79, 195, 105, 41, 203, 48, 19, 92, 231, 104, 41, 4, 184, 148, 219, 49, 24, 185, 151, 204, 119, 138, 53, 247, 15, 240, 114, 107, 27, 213, 51, 169, 23, 119, 3, 230, 79, 250, 151, 65, 49, 209, 106, 166, 141, 234, 208, 18, 103, 236, 170, 251, 238, 72, 124, 91, 185, 120, 55, 26, 9, 53, 146, 92, 25, 245, 231, 186, 230, 129, 100, 67, 140, 59, 238, 118, 94, 61, 175, 178, 214, 102, 92, 160, 170, 19, 135, 220, 10, 212, 65, 97, 208, 83, 65, 118, 223, 1, 93, 237, 111, 160, 62, 207, 19, 252, 226, 197, 252, 178, 250, 241, 72, 6, 177, 128, 243, 131, 222, 45, 189, 243, 154, 157, 136, 97, 61, 47, 225, 130, 209, 141, 112, 199, 164, 88, 230, 8, 110, 183, 140, 97, 6, 185, 160, 19, 213, 167, 204, 209, 202, 43, 174, 218, 24, 87, 48, 153, 172, 43, 174, 93, 149, 199, 2, 3, 1, 0, 1, 163, 130, 1, 39, 48, 130, 1, 35, 48, 14, 6, 3, 85, 29, 15, 1, 1, 255, 4, 4, 3, 2, 5, 160, 48, 29, 6, 3, 85, 29, 37, 4, 22, 48, 20, 6, 8, 43, 6, 1, 5, 5, 7, 3, 2, 6, 8, 43, 6, 1, 5, 5, 7, 3, 1, 48, 12, 6, 3, 85, 29, 19, 1, 1, 255, 4, 2, 48, 0, 48, 73, 6, 3, 85, 29, 14, 4, 66, 4, 64, 127, 30, 31, 238, 204, 235, 1, 150, 165, 144, 185, 136, 115, 66, 108, 209, 78, 42, 144, 134, 111, 163, 128, 74, 49, 77, 97, 28, 162, 23, 60, 100, 101, 176, 65, 80, 151, 9, 113, 72, 146, 114, 248, 244, 169, 35, 242, 65, 107, 32, 38, 41, 233, 208, 241, 25, 112, 13, 147, 191, 5, 123, 206, 141, 48, 75, 6, 3, 85, 29, 35, 4, 68, 48, 66, 128, 64, 132, 61, 126, 105, 41, 254, 134, 24, 131, 36, 49, 149, 83, 114, 116, 28, 120, 16, 97, 177, 135, 61, 140, 45, 121, 166, 237, 114, 186, 178, 49, 116, 218, 147, 129, 114, 109, 194, 246, 71, 71, 33, 84, 110, 141, 168, 203, 104, 51, 172, 142, 101, 144, 79, 17, 96, 100, 15, 46, 233, 30, 31, 133, 226, 48, 76, 6, 3, 85, 29, 17, 4, 69, 48, 67, 130, 65, 42, 46, 97, 112, 112, 115, 46, 97, 114, 111, 46, 106, 50, 97, 48, 97, 48, 111, 53, 120, 57, 98, 52, 100, 57, 106, 46, 101, 107, 116, 48, 46, 119, 101, 115, 116, 117, 115, 51, 46, 97, 114, 111, 97, 112, 112, 45, 104, 99, 112, 46, 97, 122, 117, 114, 101, 45, 116, 101, 115, 116, 46, 110, 101, 116, 48, 13, 6, 9, 42, 134, 72, 134, 247, 13, 1, 1, 11, 5, 0, 3, 130, 1, 1, 0, 42, 248, 99, 170, 29, 107, 171, 6, 50, 87, 32, 241, 55, 155, 199, 148, 146, 75, 235, 67, 120, 126, 43, 42, 133, 46, 162, 196, 58, 182, 130, 51, 175, 136, 146, 44, 12, 89, 65, 106, 45, 253, 250, 126, 78, 149, 119, 123, 85, 1, 168, 243, 62, 125, 153, 2, 6, 150, 129, 76, 125, 189, 71, 70, 13, 190, 217, 203, 113, 103, 72, 246, 94, 27, 37, 230, 172, 133, 8, 164, 252, 92, 4, 85, 140, 251, 221, 211, 209, 184, 253, 214, 225, 127, 173, 157, 35, 135, 120, 217, 126, 29, 190, 1, 221, 175, 113, 125, 252, 172, 50, 19, 40, 254, 218, 71, 190, 7, 127, 214, 134, 209, 36, 40, 29, 177, 3, 184, 114, 60, 108, 251, 70, 45, 4, 39, 92, 240, 253, 36, 243, 127, 197, 110, 104, 252, 79, 250, 51, 16, 215, 160, 212, 5, 242, 17, 127, 134, 162, 147, 111, 2, 86, 169, 40, 70, 129, 48, 144, 180, 6, 160, 252, 214, 228, 159, 168, 38, 241, 201, 88, 73, 98, 239, 101, 233, 12, 148, 146, 247, 198, 117, 223, 195, 78, 78, 224, 55, 97, 147, 99, 8, 163, 151, 6, 221, 74, 191, 71, 245, 135, 66, 56, 66, 142, 235, 174, 61, 91, 85, 64, 219, 60, 203, 15, 200, 241, 80, 218, 121, 48, 79, 149, 34, 148, 54, 207, 137, 248, 48, 135, 107, 97, 224, 244, 103, 170, 195, 215, 192, 212, 17, 178, 179, 169, 78],
                        RawTBSCertificate: [48, 130, 2, 233, 160, 3, 2, 1, 2, 2, 8, 10, 79, 113, 121, 215, 189, 122, 224, 48, 13, 6, 9, 42, 134, 72, 134, 247, 13, 1, 1, 11, 5, 0, 48, 38, 49, 18, 48, 16, 6, 3, 85, 4, 11, 19, 9, 111, 112, 101, 110, 115, 104, 105, 102, 116, 49, 16, 48, 14, 6, 3, 85, 4, 3, 19, 7, 114, 111, 111, 116, 45, 99, 97, 48, 30, 23, 13, 50, 54, 49, 48, 50, 52, 48, 48, 53, 56, 51, 56, 90, 23, 13, 50, 55, 49, 48, 50, 52, 48, 48, 53, 56, 51, 56, 90, 48, 48, 49, 18, 48, 16, 6, 3, 85, 4, 10, 19, 9, 111, 112, 101, 110, 115, 104, 105, 102, 116, 49, 26, 48, 24, 6, 3, 85, 4, 3, 19, 17, 111, 112, 101, 110, 115, 104, 105, 102, 116, 45, 105, 110, 103, 114, 101, 115, 115, 48, 130, 1, 34, 48, 13, 6, 9, 42, 134, 72, 134, 247, 13, 1, 1, 1, 5, 0, 3, 130, 1, 15, 0, 48, 130, 1, 10, 2, 130, 1, 1, 0, 218, 203, 20, 74, 150, 131, 221, 244, 8, 174, 50, 200, 232, 206, 148, 177, 108, 150, 50, 246, 233, 250, 129, 223, 15, 120, 59, 11, 252, 58, 210, 100, 231, 255, 150, 147, 148, 67, 32, 69, 22, 138, 183, 209, 53, 75, 133, 88, 171, 155, 169, 197, 58, 104, 98, 205, 78, 168, 252, 61, 77, 33, 3, 158, 73, 106, 0, 150, 138, 251, 213, 201, 174, 186, 121, 229, 217, 13, 171, 213, 245, 168, 65, 246, 119, 20, 133, 253, 84, 108, 76, 119, 79, 195, 105, 41, 203, 48, 19, 92, 231, 104, 41, 4, 184, 148, 219, 49, 24, 185, 151, 204, 119, 138, 53, 247, 15, 240, 114, 107, 27, 213, 51, 169, 23, 119, 3, 230, 79, 250, 151, 65, 49, 209, 106, 166, 141, 234, 208, 18, 103, 236, 170, 251, 238, 72, 124, 91, 185, 120, 55, 26, 9, 53, 146, 92, 25, 245, 231, 186, 230, 129, 100, 67, 140, 59, 238, 118, 94, 61, 175, 178, 214, 102, 92, 160, 170, 19, 135, 220, 10, 212, 65, 97, 208, 83, 65, 118, 223, 1, 93, 237, 111, 160, 62, 207, 19, 252, 226, 197, 252, 178, 250, 241, 72, 6, 177, 128, 243, 131, 222, 45, 189, 243, 154, 157, 136, 97, 61, 47, 225, 130, 209, 141, 112, 199, 164, 88, 230, 8, 110, 183, 140, 97, 6, 185, 160, 19, 213, 167, 204, 209, 202, 43, 174, 218, 24, 87, 48, 153, 172, 43, 174, 93, 149, 199, 2, 3, 1, 0, 1, 163, 130, 1, 39, 48, 130, 1, 35, 48, 14, 6, 3, 85, 29, 15, 1, 1, 255, 4, 4, 3, 2, 5, 160, 48, 29, 6, 3, 85, 29, 37, 4, 22, 48, 20, 6, 8, 43, 6, 1, 5, 5, 7, 3, 2, 6, 8, 43, 6, 1, 5, 5, 7, 3, 1, 48, 12, 6, 3, 85, 29, 19, 1, 1, 255, 4, 2, 48, 0, 48, 73, 6, 3, 85, 29, 14, 4, 66, 4, 64, 127, 30, 31, 238, 204, 235, 1, 150, 165, 144, 185, 136, 115, 66, 108, 209, 78, 42, 144, 134, 111, 163, 128, 74, 49, 77, 97, 28, 162, 23, 60, 100, 101, 176, 65, 80, 151, 9, 113, 72, 146, 114, 248, 244, 169, 35, 242, 65, 107, 32, 38, 41, 233, 208, 241, 25, 112, 13, 147, 191, 5, 123, 206, 141, 48, 75, 6, 3, 85, 29, 35, 4, 68, 48, 66, 128, 64, 132, 61, 126, 105, 41, 254, 134, 24, 131, 36, 49, 149, 83, 114, 116, 28, 120, 16, 97, 177, 135, 61, 140, 45, 121, 166, 237, 114, 186, 178, 49, 116, 218, 147, 129, 114, 109, 194, 246, 71, 71, 33, 84, 110, 141, 168, 203, 104, 51, 172, 142, 101, 144, 79, 17, 96, 100, 15, 46, 233, 30, 31, 133, 226, 48, 76, 6, 3, 85, 29, 17, 4, 69, 48, 67, 130, 65, 42, 46, 97, 112, 112, 115, 46, 97, 114, 111, 46, 106, 50, 97, 48, 97, 48, 111, 53, 120, 57, 98, 52, 100, 57, 106, 46, 101, 107, 116, 48, 46, 119, 101, 115, 116, 117, 115, 51, 46, 97, 114, 111, 97, 112, 112, 45, 104, 99, 112, 46, 97, 122, 117, 114, 101, 45, 116, 101, 115, 116, 46, 110, 101, 116],
                        RawSubjectPublicKeyInfo: [48, 130, 1, 34, 48, 13, 6, 9, 42, 134, 72, 134, 247, 13, 1, 1, 1, 5, 0, 3, 130, 1, 15, 0, 48, 130, 1, 10, 2, 130, 1, 1, 0, 218, 203, 20, 74, 150, 131, 221, 244, 8, 174, 50, 200, 232, 206, 148, 177, 108, 150, 50, 246, 233, 250, 129, 223, 15, 120, 59, 11, 252, 58, 210, 100, 231, 255, 150, 147, 148, 67, 32, 69, 22, 138, 183, 209, 53, 75, 133, 88, 171, 155, 169, 197, 58, 104, 98, 205, 78, 168, 252, 61, 77, 33, 3, 158, 73, 106, 0, 150, 138, 251, 213, 201, 174, 186, 121, 229, 217, 13, 171, 213, 245, 168, 65, 246, 119, 20, 133, 253, 84, 108, 76, 119, 79, 195, 105, 41, 203, 48, 19, 92, 231, 104, 41, 4, 184, 148, 219, 49, 24, 185, 151, 204, 119, 138, 53, 247, 15, 240, 114, 107, 27, 213, 51, 169, 23, 119, 3, 230, 79, 250, 151, 65, 49, 209, 106, 166, 141, 234, 208, 18, 103, 236, 170, 251, 238, 72, 124, 91, 185, 120, 55, 26, 9, 53, 146, 92, 25, 245, 231, 186, 230, 129, 100, 67, 140, 59, 238, 118, 94, 61, 175, 178, 214, 102, 92, 160, 170, 19, 135, 220, 10, 212, 65, 97, 208, 83, 65, 118, 223, 1, 93, 237, 111, 160, 62, 207, 19, 252, 226, 197, 252, 178, 250, 241, 72, 6, 177, 128, 243, 131, 222, 45, 189, 243, 154, 157, 136, 97, 61, 47, 225, 130, 209, 141, 112, 199, 164, 88, 230, 8, 110, 183, 140, 97, 6, 185, 160, 19, 213, 167, 204, 209, 202, 43, 174, 218, 24, 87, 48, 153, 172, 43, 174, 93, 149, 199, 2, 3, 1, 0, 1],
                        RawSubject: [48, 48, 49, 18, 48, 16, 6, 3, 85, 4, 10, 19, 9, 111, 112, 101, 110, 115, 104, 105, 102, 116, 49, 26, 48, 24, 6, 3, 85, 4, 3, 19, 17, 111, 112, 101, 110, 115, 104, 105, 102, 116, 45, 105, 110, 103, 114, 101, 115, 115],
                        RawIssuer: [48, 38, 49, 18, 48, 16, 6, 3, 85, 4, 11, 19, 9, 111, 112, 101, 110, 115, 104, 105, 102, 116, 49, 16, 48, 14, 6, 3, 85, 4, 3, 19, 7, 114, 111, 111, 116, 45, 99, 97],
                        Signature: [42, 248, 99, 170, 29, 107, 171, 6, 50, 87, 32, 241, 55, 155, 199, 148, 146, 75, 235, 67, 120, 126, 43, 42, 133, 46, 162, 196, 58, 182, 130, 51, 175, 136, 146, 44, 12, 89, 65, 106, 45, 253, 250, 126, 78, 149, 119, 123, 85, 1, 168, 243, 62, 125, 153, 2, 6, 150, 129, 76, 125, 189, 71, 70, 13, 190, 217, 203, 113, 103, 72, 246, 94, 27, 37, 230, 172, 133, 8, 164, 252, 92, 4, 85, 140, 251, 221, 211, 209, 184, 253, 214, 225, 127, 173, 157, 35, 135, 120, 217, 126, 29, 190, 1, 221, 175, 113, 125, 252, 172, 50, 19, 40, 254, 218, 71, 190, 7, 127, 214, 134, 209, 36, 40, 29, 177, 3, 184, 114, 60, 108, 251, 70, 45, 4, 39, 92, 240, 253, 36, 243, 127, 197, 110, 104, 252, 79, 250, 51, 16, 215, 160, 212, 5, 242, 17, 127, 134, 162, 147, 111, 2, 86, 169, 40, 70, 129, 48, 144, 180, 6, 160, 252, 214, 228, 159, 168, 38, 241, 201, 88, 73, 98, 239, 101, 233, 12, 148, 146, 247, 198, 117, 223, 195, 78, 78, 224, 55, 97, 147, 99, 8, 163, 151, 6, 221, 74, 191, 71, 245, 135, 66, 56, 66, 142, 235, 174, 61, 91, 85, 64, 219, 60, 203, 15, 200, 241, 80, 218, 121, 48, 79, 149, 34, 148, 54, 207, 137, 248, 48, 135, 107, 97, 224, 244, 103, 170, 195, 215, 192, 212, 17, 178, 179, 169, 78],
                        SignatureAlgorithm: 4,
                        PublicKeyAlgorithm: 1,
                        PublicKey: <*rsa.PublicKey | 0x1f1969380610>{
                            N: {
                                neg: false,
                                abs: [3502019488866211271, 14758799742835759191, 10115373632375215527, 8126644754336738999, 9827203137755337101, 17547112861207337629, 18208892157614797184, 8043497993598198469, 15011413961759350253, 12255288390149488993, 6790776995780648096, 16609667141997424246, 663597444377208762, 17170110312745547546, 10226214681369684731, 5763084979438447270, 2005566010399589350, 8613756572540105323, 13300496603446613964, 14641223678321764612, 6083321271993723177, 17701470862195328509, 12590509736363666389, 5290041358876923337, 5668057471262983070, 12365663866429399757, 1624312724790674776, 16717245902414356549, 1114705830436590180, 7824497438671667679, 625493236731778225, 15765717231125913076],
                            },
                            E: 65537,
                        },
                        Version: 3,
                        SerialNumber: {
                            neg: false,
                            abs: [742937231663921888],
                        },
                        Issuer: {
                            Country: nil,
                            Organization: nil,
                            OrganizationalUnit: ["openshift"],
                            Locality: nil,
                            Province: nil,
                            StreetAddress: nil,
                            PostalCode: nil,
                            SerialNumber: "",
                            CommonName: "root-ca",
                            Names: [
                                {
                                    Type: [2, 5, 4, 11],
                                    Value: <string>"openshift",
                                },
                                {
                                    Type: [2, 5, 4, 3],
                                    Value: <string>"root-ca",
                                },
                            ],
                            ExtraNames: nil,
                        },
                        Subject: {
                            Country: nil,
                            Organization: ["openshift"],
                            OrganizationalUnit: nil,
                            Locality: nil,
                            Province: nil,
                            StreetAddress: nil,
                            PostalCode: nil,
                            SerialNumber: "",
                            CommonName: "openshift-ingress",
                            Names: [
                                {
                                    Type: [2, 5, 4, 10],
                                    Value: <string>"openshift",
                                },
                                {
                                    Type: [2, 5, 4, 3],
                                    Value: <string>"openshift-ingress",
                                },
                            ],
                            ExtraNames: nil,
                        },
                        NotBefore: {wall: 0, ext: 63928400318, loc: nil},
                        NotAfter: {wall: 0, ext: 63959936318, loc: nil},
                        KeyUsage: 5,
                        Extensions: [
                            {
                                Id: [2, 5, 29, 15],
                                Critical: true,
                                Value: [3, 2, 5, 160],
                            },
                            {
                                Id: [2, 5, 29, 37],
                                Critical: false,
                                Value: [48, 20, 6, 8, 43, 6, 1, 5, 5, 7, 3, 2, 6, 8, 43, 6, 1, 5, 5, 7, 3, 1],
                            },
                            {Id: [2, 5, 29, 19], Critical: true, Value: [48, 0]},
                            {
                                Id: [2, 5, 29, 14],
                                Critical: false,
                                Value: [4, 64, 127, 30, 31, 238, 204, 235, 1, 150, 165, 144, 185, 136, 115, 66, 108, 209, 78, 42, 144, 134, 111, 163, 128, 74, 49, 77, 97, 28, 162, 23, 60, 100, 101, 176, 65, 80, 151, 9, 113, 72, 146, 114, 248, 244, 169, 35, 242, 65, 107, 32, 38, 41, 233, 208, 241, 25, 112, 13, 147, 191, 5, 123, 206, 141],
                            },
                            {
                                Id: [2, 5, 29, 35],
                                Critical: false,
                                Value: [48, 66, 128, 64, 132, 61, 126, 105, 41, 254, 134, 24, 131, 36, 49, 149, 83, 114, 116, 28, 120, 16, 97, 177, 135, 61, 140, 45, 121, 166, 237, 114, 186, 178, 49, 116, 218, 147, 129, 114, 109, 194, 246, 71, 71, 33, 84, 110, 141, 168, 203, 104, 51, 172, 142, 101, 144, 79, 17, 96, 100, 15, 46, 233, 30, 31, 133, 226],
                            },
                            {
                                Id: [2, 5, 29, 17],
                                Critical: false,
                                Value: "0C\x82A*.apps.aro.j2a0a0o5x9b4d9j.ekt0.westus3.aroapp-hcp.azure-test.net",
                            },
                        ],
                        ExtraExtensions: nil,
                        UnhandledCriticalExtensions: nil,
                        ExtKeyUsage: [2, 1],
                        UnknownExtKeyUsage: nil,
                        BasicConstraintsValid: true,
                        IsCA: false,
                        MaxPathLen: -1,
                        MaxPathLenZero: false,
                        SubjectKeyId: [127, 30, 31, 238, 204, 235, 1, 150, 165, 144, 185, 136, 115, 66, 108, 209, 78, 42, 144, 134, 111, 163, 128, 74, 49, 77, 97, 28, 162, 23, 60, 100, 101, 176, 65, 80, 151, 9, 113, 72, 146, 114, 248, 244, 169, 35, 242, 65, 107, 32, 38, 41, 233, 208, 241, 25, 112, 13, 147, 191, 5, 123, 206, 141],
                        AuthorityKeyId: [132, 61, 126, 105, 41, 254, 134, 24, 131, 36, 49, 149, 83, 114, 116, 28, 120, 16, 97, 177, 135, 61, 140, 45, 121, 166, 237, 114, 186, 178, 49, 116, 218, 147, 129, 114, 109, 194, 246, 71, 71, 33, 84, 110, 141, 168, 203, 104, 51, 172, 142, 101, 144, 79, 17, 96, 100, 15, 46, 233, 30, 31, 133, 226],
                        OCSPServer: nil,
                        IssuingCertificateURL: nil,
                        DNSNames: [
                            "*.apps.aro.j2a0a0o5x9b4d9j.ekt0.westus3.aroapp-hcp.azure-test.net",
                        ],
                        EmailAddresses: nil,
                        IPAddresses: nil,
                        URIs: nil,
                        PermittedDNSDomainsCritical: false,
                        PermittedDNSDomains: nil,
                        ExcludedDNSDomains: nil,
                        PermittedIPRanges: nil,
                        ExcludedIPRanges: nil,
                        PermittedEmailAddresses: nil,
                        ExcludedEmailAddresses: nil,
                        PermittedURIDomains: nil,
                        ExcludedURIDomains: nil,
                        CRLDistributionPoints: nil,
                        PolicyIdentifiers: nil,
                        Policies: nil,
                        InhibitAnyPolicy: 0,
                        InhibitAnyPolicyZero: false,
                        InhibitPolicyMapping: 0,
                        InhibitPolicyMappingZero: false,
                        RequireExplicitPolicy: 0,
                        RequireExplicitPolicyZero: false,
                        PolicyMappings: nil,
                    },
                ],
                Err: <x509.UnknownAuthorityError>{
                    Cert: 0x1f196788e008 (cyclic reference),
                    hintErr: nil,
                    hintCert: nil,
                },
            },
        },
    }
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:194]: failed to verify simple web app runs on cluster "cluster-candidate-4-23-8z7tws"
Unexpected error:
    <*fmt.wrapError | 0x1f1968099be0>: 
    [strict TLS verification] route was never reachable: Get "https://agnhost-e2e-sample-app-69t4s.apps.aro.j2a0a0o5x9b4d9j.ekt0.westus3.aroapp-hcp.azure-test.net": tls: failed to verify certificate: x509: certificate signed by unknown authority
    {
        msg: "[strict TLS verification] route was never reachable: Get \"https://agnhost-e2e-sample-app-69t4s.apps.aro.j2a0a0o5x9b4d9j.ekt0.westus3.aroapp-hcp.azure-test.net\": tls: failed to verify certificate: x509: certificate signed by unknown authority",
        err: <*url.Error | 0x1f1969237a70>{
            Op: "Get",
            URL: "https://agnhost-e2e-sample-app-69t4s.apps.aro.j2a0a0o5x9b4d9j.ekt0.westus3.aroapp-hcp.azure-test.net",
            Err: <*tls.CertificateVerificationError | 0x1f1969237a40>{
                UnverifiedCertificates: [
                    {
                        Raw: [48, 130, 4, 1, 48, 130, 2, 233, 160, 3, 2, 1, 2, 2, 8, 10, 79, 113, 121, 215, 189, 122, 224, 48, 13, 6, 9, 42, 134, 72, 134, 247, 13, 1, 1, 11, 5, 0, 48, 38, 49, 18, 48, 16, 6, 3, 85, 4, 11, 19, 9, 111, 112, 101, 110, 115, 104, 105, 102, 116, 49, 16, 48, 14, 6, 3, 85, 4, 3, 19, 7, 114, 111, 111, 116, 45, 99, 97, 48, 30, 23, 13, 50, 54, 49, 48, 50, 52, 48, 48, 53, 56, 51, 56, 90, 23, 13, 50, 55, 49, 48, 50, 52, 48, 48, 53, 56, 51, 56, 90, 48, 48, 49, 18, 48, 16, 6, 3, 85, 4, 10, 19, 9, 111, 112, 101, 110, 115, 104, 105, 102, 116, 49, 26, 48, 24, 6, 3, 85, 4, 3, 19, 17, 111, 112, 101, 110, 115, 104, 105, 102, 116, 45, 105, 110, 103, 114, 101, 115, 115, 48, 130, 1, 34, 48, 13, 6, 9, 42, 134, 72, 134, 247, 13, 1, 1, 1, 5, 0, 3, 130, 1, 15, 0, 48, 130, 1, 10, 2, 130, 1, 1, 0, 218, 203, 20, 74, 150, 131, 221, 244, 8, 174, 50, 200, 232, 206, 148, 177, 108, 150, 50, 246, 233, 250, 129, 223, 15, 120, 59, 11, 252, 58, 210, 100, 231, 255, 150, 147, 148, 67, 32, 69, 22, 138, 183, 209, 53, 75, 133, 88, 171, 155, 169, 197, 58, 104, 98, 205, 78, 168, 252, 61, 77, 33, 3, 158, 73, 106, 0, 150, 138, 251, 213, 201, 174, 186, 121, 229, 217, 13, 171, 213, 245, 168, 65, 246, 119, 20, 133, 253, 84, 108, 76, 119, 79, 195, 105, 41, 203, 48, 19, 92, 231, 104, 41, 4, 184, 148, 219, 49, 24, 185, 151, 204, 119, 138, 53, 247, 15, 240, 114, 107, 27, 213, 51, 169, 23, 119, 3, 230, 79, 250, 151, 65, 49, 209, 106, 166, 141, 234, 208, 18, 103, 236, 170, 251, 238, 72, 124, 91, 185, 120, 55, 26, 9, 53, 146, 92, 25, 245, 231, 186, 230, 129, 100, 67, 140, 59, 238, 118, 94, 61, 175, 178, 214, 102, 92, 160, 170, 19, 135, 220, 10, 212, 65, 97, 208, 83, 65, 118, 223, 1, 93, 237, 111, 160, 62, 207, 19, 252, 226, 197, 252, 178, 250, 241, 72, 6, 177, 128, 243, 131, 222, 45, 189, 243, 154, 157, 136, 97, 61, 47, 225, 130, 209, 141, 112, 199, 164, 88, 230, 8, 110, 183, 140, 97, 6, 185, 160, 19, 213, 167, 204, 209, 202, 43, 174, 218, 24, 87, 48, 153, 172, 43, 174, 93, 149, 199, 2, 3, 1, 0, 1, 163, 130, 1, 39, 48, 130, 1, 35, 48, 14, 6, 3, 85, 29, 15, 1, 1, 255, 4, 4, 3, 2, 5, 160, 48, 29, 6, 3, 85, 29, 37, 4, 22, 48, 20, 6, 8, 43, 6, 1, 5, 5, 7, 3, 2, 6, 8, 43, 6, 1, 5, 5, 7, 3, 1, 48, 12, 6, 3, 85, 29, 19, 1, 1, 255, 4, 2, 48, 0, 48, 73, 6, 3, 85, 29, 14, 4, 66, 4, 64, 127, 30, 31, 238, 204, 235, 1, 150, 165, 144, 185, 136, 115, 66, 108, 209, 78, 42, 144, 134, 111, 163, 128, 74, 49, 77, 97, 28, 162, 23, 60, 100, 101, 176, 65, 80, 151, 9, 113, 72, 146, 114, 248, 244, 169, 35, 242, 65, 107, 32, 38, 41, 233, 208, 241, 25, 112, 13, 147, 191, 5, 123, 206, 141, 48, 75, 6, 3, 85, 29, 35, 4, 68, 48, 66, 128, 64, 132, 61, 126, 105, 41, 254, 134, 24, 131, 36, 49, 149, 83, 114, 116, 28, 120, 16, 97, 177, 135, 61, 140, 45, 121, 166, 237, 114, 186, 178, 49, 116, 218, 147, 129, 114, 109, 194, 246, 71, 71, 33, 84, 110, 141, 168, 203, 104, 51, 172, 142, 101, 144, 79, 17, 96, 100, 15, 46, 233, 30, 31, 133, 226, 48, 76, 6, 3, 85, 29, 17, 4, 69, 48, 67, 130, 65, 42, 46, 97, 112, 112, 115, 46, 97, 114, 111, 46, 106, 50, 97, 48, 97, 48, 111, 53, 120, 57, 98, 52, 100, 57, 106, 46, 101, 107, 116, 48, 46, 119, 101, 115, 116, 117, 115, 51, 46, 97, 114, 111, 97, 112, 112, 45, 104, 99, 112, 46, 97, 122, 117, 114, 101, 45, 116, 101, 115, 116, 46, 110, 101, 116, 48, 13, 6, 9, 42, 134, 72, 134, 247, 13, 1, 1, 11, 5, 0, 3, 130, 1, 1, 0, 42, 248, 99, 170, 29, 107, 171, 6, 50, 87, 32, 241, 55, 155, 199, 148, 146, 75, 235, 67, 120, 126, 43, 42, 133, 46, 162, 196, 58, 182, 130, 51, 175, 136, 146, 44, 12, 89, 65, 106, 45, 253, 250, 126, 78, 149, 119, 123, 85, 1, 168, 243, 62, 125, 153, 2, 6, 150, 129, 76, 125, 189, 71, 70, 13, 190, 217, 203, 113, 103, 72, 246, 94, 27, 37, 230, 172, 133, 8, 164, 252, 92, 4, 85, 140, 251, 221, 211, 209, 184, 253, 214, 225, 127, 173, 157, 35, 135, 120, 217, 126, 29, 190, 1, 221, 175, 113, 125, 252, 172, 50, 19, 40, 254, 218, 71, 190, 7, 127, 214, 134, 209, 36, 40, 29, 177, 3, 184, 114, 60, 108, 251, 70, 45, 4, 39, 92, 240, 253, 36, 243, 127, 197, 110, 104, 252, 79, 250, 51, 16, 215, 160, 212, 5, 242, 17, 127, 134, 162, 147, 111, 2, 86, 169, 40, 70, 129, 48, 144, 180, 6, 160, 252, 214, 228, 159, 168, 38, 241, 201, 88, 73, 98, 239, 101, 233, 12, 148, 146, 247, 198, 117, 223, 195, 78, 78, 224, 55, 97, 147, 99, 8, 163, 151, 6, 221, 74, 191, 71, 245, 135, 66, 56, 66, 142, 235, 174, 61, 91, 85, 64, 219, 60, 203, 15, 200, 241, 80, 218, 121, 48, 79, 149, 34, 148, 54, 207, 137, 248, 48, 135, 107, 97, 224, 244, 103, 170, 195, 215, 192, 212, 17, 178, 179, 169, 78],
                        RawTBSCertificate: [48, 130, 2, 233, 160, 3, 2, 1, 2, 2, 8, 10, 79, 113, 121, 215, 189, 122, 224, 48, 13, 6, 9, 42, 134, 72, 134, 247, 13, 1, 1, 11, 5, 0, 48, 38, 49, 18, 48, 16, 6, 3, 85, 4, 11, 19, 9, 111, 112, 101, 110, 115, 104, 105, 102, 116, 49, 16, 48, 14, 6, 3, 85, 4, 3, 19, 7, 114, 111, 111, 116, 45, 99, 97, 48, 30, 23, 13, 50, 54, 49, 48, 50, 52, 48, 48, 53, 56, 51, 56, 90, 23, 13, 50, 55, 49, 48, 50, 52, 48, 48, 53, 56, 51, 56, 90, 48, 48, 49, 18, 48, 16, 6, 3, 85, 4, 10, 19, 9, 111, 112, 101, 110, 115, 104, 105, 102, 116, 49, 26, 48, 24, 6, 3, 85, 4, 3, 19, 17, 111, 112, 101, 110, 115, 104, 105, 102, 116, 45, 105, 110, 103, 114, 101, 115, 115, 48, 130, 1, 34, 48, 13, 6, 9, 42, 134, 72, 134, 247, 13, 1, 1, 1, 5, 0, 3, 130, 1, 15, 0, 48, 130, 1, 10, 2, 130, 1, 1, 0, 218, 203, 20, 74, 150, 131, 221, 244, 8, 174, 50, 200, 232, 206, 148, 177, 108, 150, 50, 246, 233, 250, 129, 223, 15, 120, 59, 11, 252, 58, 210, 100, 231, 255, 150, 147, 148, 67, 32, 69, 22, 138, 183, 209, 53, 75, 133, 88, 171, 155, 169, 197, 58, 104, 98, 205, 78, 168, 252, 61, 77, 33, 3, 158, 73, 106, 0, 150, 138, 251, 213, 201, 174, 186, 121, 229, 217, 13, 171, 213, 245, 168, 65, 246, 119, 20, 133, 253, 84, 108, 76, 119, 79, 195, 105, 41, 203, 48, 19, 92, 231, 104, 41, 4, 184, 148, 219, 49, 24, 185, 151, 204, 119, 138, 53, 247, 15, 240, 114, 107, 27, 213, 51, 169, 23, 119, 3, 230, 79, 250, 151, 65, 49, 209, 106, 166, 141, 234, 208, 18, 103, 236, 170, 251, 238, 72, 124, 91, 185, 120, 55, 26, 9, 53, 146, 92, 25, 245, 231, 186, 230, 129, 100, 67, 140, 59, 238, 118, 94, 61, 175, 178, 214, 102, 92, 160, 170, 19, 135, 220, 10, 212, 65, 97, 208, 83, 65, 118, 223, 1, 93, 237, 111, 160, 62, 207, 19, 252, 226, 197, 252, 178, 250, 241, 72, 6, 177, 128, 243, 131, 222, 45, 189, 243, 154, 157, 136, 97, 61, 47, 225, 130, 209, 141, 112, 199, 164, 88, 230, 8, 110, 183, 140, 97, 6, 185, 160, 19, 213, 167, 204, 209, 202, 43, 174, 218, 24, 87, 48, 153, 172, 43, 174, 93, 149, 199, 2, 3, 1, 0, 1, 163, 130, 1, 39, 48, 130, 1, 35, 48, 14, 6, 3, 85, 29, 15, 1, 1, 255, 4, 4, 3, 2, 5, 160, 48, 29, 6, 3, 85, 29, 37, 4, 22, 48, 20, 6, 8, 43, 6, 1, 5, 5, 7, 3, 2, 6, 8, 43, 6, 1, 5, 5, 7, 3, 1, 48, 12, 6, 3, 85, 29, 19, 1, 1, 255, 4, 2, 48, 0, 48, 73, 6, 3, 85, 29, 14, 4, 66, 4, 64, 127, 30, 31, 238, 204, 235, 1, 150, 165, 144, 185, 136, 115, 66, 108, 209, 78, 42, 144, 134, 111, 163, 128, 74, 49, 77, 97, 28, 162, 23, 60, 100, 101, 176, 65, 80, 151, 9, 113, 72, 146, 114, 248, 244, 169, 35, 242, 65, 107, 32, 38, 41, 233, 208, 241, 25, 112, 13, 147, 191, 5, 123, 206, 141, 48, 75, 6, 3, 85, 29, 35, 4, 68, 48, 66, 128, 64, 132, 61, 126, 105, 41, 254, 134, 24, 131, 36, 49, 149, 83, 114, 116, 28, 120, 16, 97, 177, 135, 61, 140, 45, 121, 166, 237, 114, 186, 178, 49, 116, 218, 147, 129, 114, 109, 194, 246, 71, 71, 33, 84, 110, 141, 168, 203, 104, 51, 172, 142, 101, 144, 79, 17, 96, 100, 15, 46, 233, 30, 31, 133, 226, 48, 76, 6, 3, 85, 29, 17, 4, 69, 48, 67, 130, 65, 42, 46, 97, 112, 112, 115, 46, 97, 114, 111, 46, 106, 50, 97, 48, 97, 48, 111, 53, 120, 57, 98, 52, 100, 57, 106, 46, 101, 107, 116, 48, 46, 119, 101, 115, 116, 117, 115, 51, 46, 97, 114, 111, 97, 112, 112, 45, 104, 99, 112, 46, 97, 122, 117, 114, 101, 45, 116, 101, 115, 116, 46, 110, 101, 116],
                        RawSubjectPublicKeyInfo: [48, 130, 1, 34, 48, 13, 6, 9, 42, 134, 72, 134, 247, 13, 1, 1, 1, 5, 0, 3, 130, 1, 15, 0, 48, 130, 1, 10, 2, 130, 1, 1, 0, 218, 203, 20, 74, 150, 131, 221, 244, 8, 174, 50, 200, 232, 206, 148, 177, 108, 150, 50, 246, 233, 250, 129, 223, 15, 120, 59, 11, 252, 58, 210, 100, 231, 255, 150, 147, 148, 67, 32, 69, 22, 138, 183, 209, 53, 75, 133, 88, 171, 155, 169, 197, 58, 104, 98, 205, 78, 168, 252, 61, 77, 33, 3, 158, 73, 106, 0, 150, 138, 251, 213, 201, 174, 186, 121, 229, 217, 13, 171, 213, 245, 168, 65, 246, 119, 20, 133, 253, 84, 108, 76, 119, 79, 195, 105, 41, 203, 48, 19, 92, 231, 104, 41, 4, 184, 148, 219, 49, 24, 185, 151, 204, 119, 138, 53, 247, 15, 240, 114, 107, 27, 213, 51, 169, 23, 119, 3, 230, 79, 250, 151, 65, 49, 209, 106, 166, 141, 234, 208, 18, 103, 236, 170, 251, 238, 72, 124, 91, 185, 120, 55, 26, 9, 53, 146, 92, 25, 245, 231, 186, 230, 129, 100, 67, 140, 59, 238, 118, 94, 61, 175, 178, 214, 102, 92, 160, 170, 19, 135, 220, 10, 212, 65, 97, 208, 83, 65, 118, 223, 1, 93, 237, 111, 160, 62, 207, 19, 252, 226, 197, 252, 178, 250, 241, 72, 6, 177, 128, 243, 131, 222, 45, 189, 243, 154, 157, 136, 97, 61, 47, 225, 130, 209, 141, 112, 199, 164, 88, 230, 8, 110, 183, 140, 97, 6, 185, 160, 19, 213, 167, 204, 209, 202, 43, 174, 218, 24, 87, 48, 153, 172, 43, 174, 93, 149, 199, 2, 3, 1, 0, 1],
                        RawSubject: [48, 48, 49, 18, 48, 16, 6, 3, 85, 4, 10, 19, 9, 111, 112, 101, 110, 115, 104, 105, 102, 116, 49, 26, 48, 24, 6, 3, 85, 4, 3, 19, 17, 111, 112, 101, 110, 115, 104, 105, 102, 116, 45, 105, 110, 103, 114, 101, 115, 115],
                        RawIssuer: [48, 38, 49, 18, 48, 16, 6, 3, 85, 4, 11, 19, 9, 111, 112, 101, 110, 115, 104, 105, 102, 116, 49, 16, 48, 14, 6, 3, 85, 4, 3, 19, 7, 114, 111, 111, 116, 45, 99, 97],
                        Signature: [42, 248, 99, 170, 29, 107, 171, 6, 50, 87, 32, 241, 55, 155, 199, 148, 146, 75, 235, 67, 120, 126, 43, 42, 133, 46, 162, 196, 58, 182, 130, 51, 175, 136, 146, 44, 12, 89, 65, 106, 45, 253, 250, 126, 78, 149, 119, 123, 85, 1, 168, 243, 62, 125, 153, 2, 6, 150, 129, 76, 125, 189, 71, 70, 13, 190, 217, 203, 113, 103, 72, 246, 94, 27, 37, 230, 172, 133, 8, 164, 252, 92, 4, 85, 140, 251, 221, 211, 209, 184, 253, 214, 225, 127, 173, 157, 35, 135, 120, 217, 126, 29, 190, 1, 221, 175, 113, 125, 252, 172, 50, 19, 40, 254, 218, 71, 190, 7, 127, 214, 134, 209, 36, 40, 29, 177, 3, 184, 114, 60, 108, 251, 70, 45, 4, 39, 92, 240, 253, 36, 243, 127, 197, 110, 104, 252, 79, 250, 51, 16, 215, 160, 212, 5, 242, 17, 127, 134, 162, 147, 111, 2, 86, 169, 40, 70, 129, 48, 144, 180, 6, 160, 252, 214, 228, 159, 168, 38, 241, 201, 88, 73, 98, 239, 101, 233, 12, 148, 146, 247, 198, 117, 223, 195, 78, 78, 224, 55, 97, 147, 99, 8, 163, 151, 6, 221, 74, 191, 71, 245, 135, 66, 56, 66, 142, 235, 174, 61, 91, 85, 64, 219, 60, 203, 15, 200, 241, 80, 218, 121, 48, 79, 149, 34, 148, 54, 207, 137, 248, 48, 135, 107, 97, 224, 244, 103, 170, 195, 215, 192, 212, 17, 178, 179, 169, 78],
                        SignatureAlgorithm: 4,
                        PublicKeyAlgorithm: 1,
                        PublicKey: <*rsa.PublicKey | 0x1f1969380610>{
                            N: {
                                neg: false,
                                abs: [3502019488866211271, 14758799742835759191, 10115373632375215527, 8126644754336738999, 9827203137755337101, 17547112861207337629, 18208892157614797184, 8043497993598198469, 15011413961759350253, 12255288390149488993, 6790776995780648096, 16609667141997424246, 663597444377208762, 17170110312745547546, 10226214681369684731, 5763084979438447270, 2005566010399589350, 8613756572540105323, 13300496603446613964, 14641223678321764612, 6083321271993723177, 17701470862195328509, 12590509736363666389, 5290041358876923337, 5668057471262983070, 12365663866429399757, 1624312724790674776, 16717245902414356549, 1114705830436590180, 7824497438671667679, 625493236731778225, 15765717231125913076],
                            },
                            E: 65537,
                        },
                        Version: 3,
                        SerialNumber: {
                            neg: false,
                            abs: [742937231663921888],
                        },
                        Issuer: {
                            Country: nil,
                            Organization: nil,
                            OrganizationalUnit: ["openshift"],
                            Locality: nil,
                            Province: nil,
                            StreetAddress: nil,
                            PostalCode: nil,
                            SerialNumber: "",
                            CommonName: "root-ca",
                            Names: [
                                {
                                    Type: [2, 5, 4, 11],
                                    Value: <string>"openshift",
                                },
                                {
                                    Type: [2, 5, 4, 3],
                                    Value: <string>"root-ca",
                                },
                            ],
                            ExtraNames: nil,
                        },
                        Subject: {
                            Country: nil,
                            Organization: ["openshift"],
                            OrganizationalUnit: nil,
                            Locality: nil,
                            Province: nil,
                            StreetAddress: nil,
                            PostalCode: nil,
                            SerialNumber: "",
                            CommonName: "openshift-ingress",
                            Names: [
                                {
                                    Type: [2, 5, 4, 10],
                                    Value: <string>"openshift",
                                },
                                {
                                    Type: [2, 5, 4, 3],
                                    Value: <string>"openshift-ingress",
                                },
                            ],
                            ExtraNames: nil,
                        },
                        NotBefore: {wall: 0, ext: 63928400318, loc: nil},
                        NotAfter: {wall: 0, ext: 63959936318, loc: nil},
                        KeyUsage: 5,
                        Extensions: [
                            {
                                Id: [2, 5, 29, 15],
                                Critical: true,
                                Value: [3, 2, 5, 160],
                            },
                            {
                                Id: [2, 5, 29, 37],
                                Critical: false,
                                Value: [48, 20, 6, 8, 43, 6, 1, 5, 5, 7, 3, 2, 6, 8, 43, 6, 1, 5, 5, 7, 3, 1],
                            },
                            {Id: [2, 5, 29, 19], Critical: true, Value: [48, 0]},
                            {
                                Id: [2, 5, 29, 14],
                                Critical: false,
                                Value: [4, 64, 127, 30, 31, 238, 204, 235, 1, 150, 165, 144, 185, 136, 115, 66, 108, 209, 78, 42, 144, 134, 111, 163, 128, 74, 49, 77, 97, 28, 162, 23, 60, 100, 101, 176, 65, 80, 151, 9, 113, 72, 146, 114, 248, 244, 169, 35, 242, 65, 107, 32, 38, 41, 233, 208, 241, 25, 112, 13, 147, 191, 5, 123, 206, 141],
                            },
                            {
                                Id: [2, 5, 29, 35],
                                Critical: false,
                                Value: [48, 66, 128, 64, 132, 61, 126, 105, 41, 254, 134, 24, 131, 36, 49, 149, 83, 114, 116, 28, 120, 16, 97, 177, 135, 61, 140, 45, 121, 166, 237, 114, 186, 178, 49, 116, 218, 147, 129, 114, 109, 194, 246, 71, 71, 33, 84, 110, 141, 168, 203, 104, 51, 172, 142, 101, 144, 79, 17, 96, 100, 15, 46, 233, 30, 31, 133, 226],
                            },
                            {
                                Id: [2, 5, 29, 17],
                                Critical: false,
                                Value: "0C\x82A*.apps.aro.j2a0a0o5x9b4d9j.ekt0.westus3.aroapp-hcp.azure-test.net",
                            },
                        ],
                        ExtraExtensions: nil,
                        UnhandledCriticalExtensions: nil,
                        ExtKeyUsage: [2, 1],
                        UnknownExtKeyUsage: nil,
                        BasicConstraintsValid: true,
                        IsCA: false,
                        MaxPathLen: -1,
                        MaxPathLenZero: false,
                        SubjectKeyId: [127, 30, 31, 238, 204, 235, 1, 150, 165, 144, 185, 136, 115, 66, 108, 209, 78, 42, 144, 134, 111, 163, 128, 74, 49, 77, 97, 28, 162, 23, 60, 100, 101, 176, 65, 80, 151, 9, 113, 72, 146, 114, 248, 244, 169, 35, 242, 65, 107, 32, 38, 41, 233, 208, 241, 25, 112, 13, 147, 191, 5, 123, 206, 141],
                        AuthorityKeyId: [132, 61, 126, 105, 41, 254, 134, 24, 131, 36, 49, 149, 83, 114, 116, 28, 120, 16, 97, 177, 135, 61, 140, 45, 121, 166, 237, 114, 186, 178, 49, 116, 218, 147, 129, 114, 109, 194, 246, 71, 71, 33, 84, 110, 141, 168, 203, 104, 51, 172, 142, 101, 144, 79, 17, 96, 100, 15, 46, 233, 30, 31, 133, 226],
                        OCSPServer: nil,
                        IssuingCertificateURL: nil,
                        DNSNames: [
                            "*.apps.aro.j2a0a0o5x9b4d9j.ekt0.westus3.aroapp-hcp.azure-test.net",
                        ],
                        EmailAddresses: nil,
                        IPAddresses: nil,
                        URIs: nil,
                        PermittedDNSDomainsCritical: false,
                        PermittedDNSDomains: nil,
                        ExcludedDNSDomains: nil,
                        PermittedIPRanges: nil,
                        ExcludedIPRanges: nil,
                        PermittedEmailAddresses: nil,
                        ExcludedEmailAddresses: nil,
                        PermittedURIDomains: nil,
                        ExcludedURIDomains: nil,
                        CRLDistributionPoints: nil,
                        PolicyIdentifiers: nil,
                        Policies: nil,
                        InhibitAnyPolicy: 0,
                        InhibitAnyPolicyZero: false,
                        InhibitPolicyMapping: 0,
                        InhibitPolicyMappingZero: false,
                        RequireExplicitPolicy: 0,
                        RequireExplicitPolicyZero: false,
                        PolicyMappings: nil,
                    },
                ],
                Err: <x509.UnknownAuthorityError>{
                    Cert: 0x1f196788e008 (cyclic reference),
                    hintErr: nil,
                    hintCert: nil,
                },
            },
        },
    }
occurred