CIHealth Run Log

↗ Open Failure patterns for this day

Environment: PROD

Time (UTC)	Job	Failed at	Result	PR	Failed tests	Details
02:00:04	periodic-ci-Azure-ARO-HCP-main-periodic-prod-e2e-parallel post-goodmerged PR No additional run metadata captured	e2e	failed	n/a	4	e2e (4) tls: failed to verify certificate: x509: certificate is valid for <hcp-api-host>, <hcp-api-host>, not <hcp-api-host> 02:00:04 UTC · failed at e2e · test Service Provider should upgrade the control plane z-stream automatically on behalf of the customer for 4.21 · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:153]: Unexpected error: <errors.joinError \| 0xc001286438>: VerifyBasicAccess failed: failed to list services: Get "https://api.r8w7u2u4m1a6h8y.dxvx.uksouth.aroapp-hcp.io:443/api/v1/namespaces/default/services": tls: failed to verify certificate: x509: certificate is valid for api.r8w7u2u4m1a6h8y.f07n.uksouth.aroapp-hcp.io, reserved.aroapp-hcp.io, not api.r8w7u2u4m1a6h8y.dxvx.uksouth.aroapp-hcp.io VerifyAllAPIServicesAvailable failed: failed to list all APIServices: Get "https://api.r8w7u2u4m1a6h8y.dxvx.uksouth.aroapp-hcp.io:443/apis/apiregistration.k8s.io/v1/apiservices": tls: failed to verify certificate: x509: certificate is valid for api.r8w7u2u4m1a6h8y.f07n.uksouth.aroapp-hcp.io, reserved.aroapp-hcp.io, not api.r8w7u2u4m1a6h8y.dxvx.uksouth.aroapp-hcp.io ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:153]: Unexpected error: <errors.joinError \| 0xc001286438>: VerifyBasicAccess failed: failed to list services: Get "https://api.r8w7u2u4m1a6h8y.dxvx.uksouth.aroapp-hcp.io:443/api/v1/namespaces/default/services": tls: failed to verify certificate: x509: certificate is valid for api.r8w7u2u4m1a6h8y.f07n.uksouth.aroapp-hcp.io, reserved.aroapp-hcp.io, not api.r8w7u2u4m1a6h8y.dxvx.uksouth.aroapp-hcp.io VerifyAllAPIServicesAvailable failed: failed to list all APIServices: Get "https://api.r8w7u2u4m1a6h8y.dxvx.uksouth.aroapp-hcp.io:443/apis/apiregistration.k8s.io/v1/apiservices": tls: failed to verify certificate: x509: certificate is valid for api.r8w7u2u4m1a6h8y.f07n.uksouth.aroapp-hcp.io, reserved.aroapp-hcp.io, not api.r8w7u2u4m1a6h8y.dxvx.uksouth.aroapp-hcp.io ... occurred Interrupted by User 02:00:04 UTC · failed at e2e · test Customer should be able to create several HCP clusters in their customer resource group, but not in the same managed resource group · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/clusters_sharing_resgroup.go:35]: Interrupted by User fail [github.com/Azure/ARO-HCP/test/e2e/clusters_sharing_resgroup.go:35]: Interrupted by User tls: failed to verify certificate: x509: certificate is valid for <hcp-api-host>, <hcp-api-host>, not <hcp-api-host> 02:00:04 UTC · failed at e2e · test Customer should be able to create an HCP cluster with back-level version 4.19 · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_version_backlevel.go:160]: Unexpected error: <errors.joinError \| 0xc001880d20>: VerifyBasicAccess failed: failed to list services: Get "https://api.o9k4j6p7a1e1h0s.60d9.uksouth.aroapp-hcp.io:443/api/v1/namespaces/default/services": tls: failed to verify certificate: x509: certificate is valid for api.o9k4j6p7a1e1h0s.a1gh.uksouth.aroapp-hcp.io, reserved.aroapp-hcp.io, not api.o9k4j6p7a1e1h0s.60d9.uksouth.aroapp-hcp.io VerifyAllAPIServicesAvailable failed: failed to list all APIServices: Get "https://api.o9k4j6p7a1e1h0s.60d9.uksouth.aroapp-hcp.io:443/apis/apiregistration.k8s.io/v1/apiservices": tls: failed to verify certificate: x509: certificate is valid for api.o9k4j6p7a1e1h0s.a1gh.uksouth.aroapp-hcp.io, reserved.aroapp-hcp.io, not api.o9k4j6p7a1e1h0s.60d9.uksouth.aroapp-hcp.io ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_version_backlevel.go:160]: Unexpected error: <errors.joinError \| 0xc001880d20>: VerifyBasicAccess failed: failed to list services: Get "https://api.o9k4j6p7a1e1h0s.60d9.uksouth.aroapp-hcp.io:443/api/v1/namespaces/default/services": tls: failed to verify certificate: x509: certificate is valid for api.o9k4j6p7a1e1h0s.a1gh.uksouth.aroapp-hcp.io, reserved.aroapp-hcp.io, not api.o9k4j6p7a1e1h0s.60d9.uksouth.aroapp-hcp.io VerifyAllAPIServicesAvailable failed: failed to list all APIServices: Get "https://api.o9k4j6p7a1e1h0s.60d9.uksouth.aroapp-hcp.io:443/apis/apiregistration.k8s.io/v1/apiservices": tls: failed to verify certificate: x509: certificate is valid for api.o9k4j6p7a1e1h0s.a1gh.uksouth.aroapp-hcp.io, reserved.aroapp-hcp.io, not api.o9k4j6p7a1e1h0s.60d9.uksouth.aroapp-hcp.io ... occurred tls: failed to verify certificate: x509: certificate is valid for <hcp-api-host>, <hcp-api-host>, not <hcp-api-host> 02:00:04 UTC · failed at e2e · test Update HCPOpenShiftCluster Negative creates a cluster and fails to update its name with a PATCH request · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_update.go:98]: Unexpected error: <errors.joinError \| 0xc00107ef30>: VerifyBasicAccess failed: failed to list services: Get "https://api.u0b6k0h2n5h7u1f.ccn5.uksouth.aroapp-hcp.io:443/api/v1/namespaces/default/services": tls: failed to verify certificate: x509: certificate is valid for api.u0b6k0h2n5h7u1f.vrwb.uksouth.aroapp-hcp.io, reserved.aroapp-hcp.io, not api.u0b6k0h2n5h7u1f.ccn5.uksouth.aroapp-hcp.io VerifyAllAPIServicesAvailable failed: failed to list all APIServices: Get "https://api.u0b6k0h2n5h7u1f.ccn5.uksouth.aroapp-hcp.io:443/apis/apiregistration.k8s.io/v1/apiservices": tls: failed to verify certificate: x509: certificate is valid for api.u0b6k0h2n5h7u1f.vrwb.uksouth.aroapp-hcp.io, reserved.aroapp-hcp.io, not api.u0b6k0h2n5h7u1f.ccn5.uksouth.aroapp-hcp.io ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_update.go:98]: Unexpected error: <errors.joinError \| 0xc00107ef30>: VerifyBasicAccess failed: failed to list services: Get "https://api.u0b6k0h2n5h7u1f.ccn5.uksouth.aroapp-hcp.io:443/api/v1/namespaces/default/services": tls: failed to verify certificate: x509: certificate is valid for api.u0b6k0h2n5h7u1f.vrwb.uksouth.aroapp-hcp.io, reserved.aroapp-hcp.io, not api.u0b6k0h2n5h7u1f.ccn5.uksouth.aroapp-hcp.io VerifyAllAPIServicesAvailable failed: failed to list all APIServices: Get "https://api.u0b6k0h2n5h7u1f.ccn5.uksouth.aroapp-hcp.io:443/apis/apiregistration.k8s.io/v1/apiservices": tls: failed to verify certificate: x509: certificate is valid for api.u0b6k0h2n5h7u1f.vrwb.uksouth.aroapp-hcp.io, reserved.aroapp-hcp.io, not api.u0b6k0h2n5h7u1f.ccn5.uksouth.aroapp-hcp.io ... occurred

Time (UTC)

Job

Failed at

Result

Failed tests

Details

02:00:04

periodic-ci-Azure-ARO-HCP-main-periodic-prod-e2e-parallel

post-goodmerged PR

No additional run metadata captured

e2e

failed

n/a

e2e (4)

tls: failed to verify certificate: x509: certificate is valid for <hcp-api-host>, <hcp-api-host>, not <hcp-api-host>

02:00:04 UTC · failed at e2e · test Service Provider should upgrade the control plane z-stream automatically on behalf of the customer for 4.21 · suite prod/parallel

Show raw failure

fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:153]: Unexpected error:
    <*errors.joinError | 0xc001286438>: 
    VerifyBasicAccess failed: failed to list services: Get "https://api.r8w7u2u4m1a6h8y.dxvx.uksouth.aroapp-hcp.io:443/api/v1/namespaces/default/services": tls: failed to verify certificate: x509: certificate is valid for api.r8w7u2u4m1a6h8y.f07n.uksouth.aroapp-hcp.io, reserved.aroapp-hcp.io, not api.r8w7u2u4m1a6h8y.dxvx.uksouth.aroapp-hcp.io
    VerifyAllAPIServicesAvailable failed: failed to list all APIServices: Get "https://api.r8w7u2u4m1a6h8y.dxvx.uksouth.aroapp-hcp.io:443/apis/apiregistration.k8s.io/v1/apiservices": tls: failed to verify certificate: x509: certificate is valid for api.r8w7u2u4m1a6h8y.f07n.uksouth.aroapp-hcp.io, reserved.aroapp-hcp.io, not api.r8w7u2u4m1a6h8y.dxvx.uksouth.aroapp-hcp.io
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:153]: Unexpected error:
    <*errors.joinError | 0xc001286438>: 
    VerifyBasicAccess failed: failed to list services: Get "https://api.r8w7u2u4m1a6h8y.dxvx.uksouth.aroapp-hcp.io:443/api/v1/namespaces/default/services": tls: failed to verify certificate: x509: certificate is valid for api.r8w7u2u4m1a6h8y.f07n.uksouth.aroapp-hcp.io, reserved.aroapp-hcp.io, not api.r8w7u2u4m1a6h8y.dxvx.uksouth.aroapp-hcp.io
    VerifyAllAPIServicesAvailable failed: failed to list all APIServices: Get "https://api.r8w7u2u4m1a6h8y.dxvx.uksouth.aroapp-hcp.io:443/apis/apiregistration.k8s.io/v1/apiservices": tls: failed to verify certificate: x509: certificate is valid for api.r8w7u2u4m1a6h8y.f07n.uksouth.aroapp-hcp.io, reserved.aroapp-hcp.io, not api.r8w7u2u4m1a6h8y.dxvx.uksouth.aroapp-hcp.io
    ...
occurred

Interrupted by User

02:00:04 UTC · failed at e2e · test Customer should be able to create several HCP clusters in their customer resource group, but not in the same managed resource group · suite prod/parallel

Show raw failure

fail [github.com/Azure/ARO-HCP/test/e2e/clusters_sharing_resgroup.go:35]: Interrupted by User
fail [github.com/Azure/ARO-HCP/test/e2e/clusters_sharing_resgroup.go:35]: Interrupted by User

tls: failed to verify certificate: x509: certificate is valid for <hcp-api-host>, <hcp-api-host>, not <hcp-api-host>

02:00:04 UTC · failed at e2e · test Customer should be able to create an HCP cluster with back-level version 4.19 · suite prod/parallel

Show raw failure

fail [github.com/Azure/ARO-HCP/test/e2e/cluster_version_backlevel.go:160]: Unexpected error:
    <*errors.joinError | 0xc001880d20>: 
    VerifyBasicAccess failed: failed to list services: Get "https://api.o9k4j6p7a1e1h0s.60d9.uksouth.aroapp-hcp.io:443/api/v1/namespaces/default/services": tls: failed to verify certificate: x509: certificate is valid for api.o9k4j6p7a1e1h0s.a1gh.uksouth.aroapp-hcp.io, reserved.aroapp-hcp.io, not api.o9k4j6p7a1e1h0s.60d9.uksouth.aroapp-hcp.io
    VerifyAllAPIServicesAvailable failed: failed to list all APIServices: Get "https://api.o9k4j6p7a1e1h0s.60d9.uksouth.aroapp-hcp.io:443/apis/apiregistration.k8s.io/v1/apiservices": tls: failed to verify certificate: x509: certificate is valid for api.o9k4j6p7a1e1h0s.a1gh.uksouth.aroapp-hcp.io, reserved.aroapp-hcp.io, not api.o9k4j6p7a1e1h0s.60d9.uksouth.aroapp-hcp.io
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_version_backlevel.go:160]: Unexpected error:
    <*errors.joinError | 0xc001880d20>: 
    VerifyBasicAccess failed: failed to list services: Get "https://api.o9k4j6p7a1e1h0s.60d9.uksouth.aroapp-hcp.io:443/api/v1/namespaces/default/services": tls: failed to verify certificate: x509: certificate is valid for api.o9k4j6p7a1e1h0s.a1gh.uksouth.aroapp-hcp.io, reserved.aroapp-hcp.io, not api.o9k4j6p7a1e1h0s.60d9.uksouth.aroapp-hcp.io
    VerifyAllAPIServicesAvailable failed: failed to list all APIServices: Get "https://api.o9k4j6p7a1e1h0s.60d9.uksouth.aroapp-hcp.io:443/apis/apiregistration.k8s.io/v1/apiservices": tls: failed to verify certificate: x509: certificate is valid for api.o9k4j6p7a1e1h0s.a1gh.uksouth.aroapp-hcp.io, reserved.aroapp-hcp.io, not api.o9k4j6p7a1e1h0s.60d9.uksouth.aroapp-hcp.io
    ...
occurred

tls: failed to verify certificate: x509: certificate is valid for <hcp-api-host>, <hcp-api-host>, not <hcp-api-host>

02:00:04 UTC · failed at e2e · test Update HCPOpenShiftCluster Negative creates a cluster and fails to update its name with a PATCH request · suite prod/parallel

Show raw failure

fail [github.com/Azure/ARO-HCP/test/e2e/cluster_update.go:98]: Unexpected error:
    <*errors.joinError | 0xc00107ef30>: 
    VerifyBasicAccess failed: failed to list services: Get "https://api.u0b6k0h2n5h7u1f.ccn5.uksouth.aroapp-hcp.io:443/api/v1/namespaces/default/services": tls: failed to verify certificate: x509: certificate is valid for api.u0b6k0h2n5h7u1f.vrwb.uksouth.aroapp-hcp.io, reserved.aroapp-hcp.io, not api.u0b6k0h2n5h7u1f.ccn5.uksouth.aroapp-hcp.io
    VerifyAllAPIServicesAvailable failed: failed to list all APIServices: Get "https://api.u0b6k0h2n5h7u1f.ccn5.uksouth.aroapp-hcp.io:443/apis/apiregistration.k8s.io/v1/apiservices": tls: failed to verify certificate: x509: certificate is valid for api.u0b6k0h2n5h7u1f.vrwb.uksouth.aroapp-hcp.io, reserved.aroapp-hcp.io, not api.u0b6k0h2n5h7u1f.ccn5.uksouth.aroapp-hcp.io
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_update.go:98]: Unexpected error:
    <*errors.joinError | 0xc00107ef30>: 
    VerifyBasicAccess failed: failed to list services: Get "https://api.u0b6k0h2n5h7u1f.ccn5.uksouth.aroapp-hcp.io:443/api/v1/namespaces/default/services": tls: failed to verify certificate: x509: certificate is valid for api.u0b6k0h2n5h7u1f.vrwb.uksouth.aroapp-hcp.io, reserved.aroapp-hcp.io, not api.u0b6k0h2n5h7u1f.ccn5.uksouth.aroapp-hcp.io
    VerifyAllAPIServicesAvailable failed: failed to list all APIServices: Get "https://api.u0b6k0h2n5h7u1f.ccn5.uksouth.aroapp-hcp.io:443/apis/apiregistration.k8s.io/v1/apiservices": tls: failed to verify certificate: x509: certificate is valid for api.u0b6k0h2n5h7u1f.vrwb.uksouth.aroapp-hcp.io, reserved.aroapp-hcp.io, not api.u0b6k0h2n5h7u1f.ccn5.uksouth.aroapp-hcp.io
    ...
occurred