CIHealth Run Log

Environment: DEV

Time (UTC)	Runtime	Job	Region	Failed at	Result	PR	Failed tests	Details
23:56:26	n/a	pull-ci-Azure-ARO-HCP-main-e2e-parallel merged PR head 9d6e88a · merge ac3354a	n/a	n/a	passed	#4234 (merged)	0	n/a
23:52:41	n/a	pull-ci-Azure-ARO-HCP-main-e2e-parallel merged PR head 906f507 · merge 61fd736	n/a	e2e	failed	⚠#4409 (merged)	1	e2e (1) Yet, Eventually failed because the matcher was not satisfied: 23:52:41 UTC · failed at e2e · test Customer should be able to use workload identity via the cluster OIDC issuer URL · suite rp-api-compat-all/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/oidc_issuer_workload_identity.go:310]: Told to stop trying after 30.313s. Pod failed; see logs for details At one point, however, the function did return successfully. Yet, Eventually failed because the matcher was not satisfied: Expected <v1.PodPhase>: Pending to equal <v1.PodPhase>: Succeeded fail [github.com/Azure/ARO-HCP/test/e2e/oidc_issuer_workload_identity.go:310]: Told to stop trying after 30.313s. Pod failed; see logs for details At one point, however, the function did return successfully. Yet, Eventually failed because the matcher was not satisfied: Expected <v1.PodPhase>: Pending to equal <v1.PodPhase>: Succeeded
23:34:09	n/a	pull-ci-Azure-ARO-HCP-main-e2e-parallel head d0078db	n/a	n/a	passed	#4393 (open)	0	n/a
23:31:33	n/a	pull-ci-Azure-ARO-HCP-main-e2e-parallel head 147684b	n/a	unknown	failed	#4419 (closed)	1	non-artifact-backed failure (no junit artifacts)
23:29:44	n/a	pull-ci-Azure-ARO-HCP-main-e2e-parallel post-goodmerged PR head 5b9e8e9 · merge 5b9e8e9	n/a	n/a	passed	#4416 (merged)	0	n/a
22:52:55	n/a	pull-ci-Azure-ARO-HCP-main-e2e-parallel head 701828d	n/a	unknown	failed	#4418 (closed)	1	non-artifact-backed failure (no junit artifacts)
22:50:23	n/a	pull-ci-Azure-ARO-HCP-main-e2e-parallel batch head e0b528c · merge e0b528c	n/a	n/a	passed	#4371 (merged)	0	n/a
22:05:37	n/a	pull-ci-Azure-ARO-HCP-main-e2e-parallel head ec88e65	n/a	unknown	failed	#4418 (closed)	1	non-artifact-backed failure (no junit artifacts)
21:36:16	n/a	pull-ci-Azure-ARO-HCP-main-e2e-parallel post-goodmerged PR head 1892906 · merge 1892906	n/a	n/a	passed	#3820 (merged)	0	n/a
21:35:05	n/a	pull-ci-Azure-ARO-HCP-main-e2e-parallel head d0078db	n/a	e2e	failed	#4393 (open)	1	e2e (1) ERROR CODE: InternalServerError; detail code SubscriptionRequestsThrottled; detail message Number of 'read' requests for subscription 'XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX' actor '<uuid>' exceeded. Please try again after '<seconds>' seconds after additional tokens are available.; provider Microsoft.ManagedIdentity 21:35:05 UTC · failed at e2e · test Customer should be able to list HCP clusters without node pools at both subscription and resource group levels · suite rp-api-compat-all/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_list_no_infra.go:86]: Unexpected error: <fmt.wrapError \| 0xc00161a480>: failed to create HCP cluster list-test-cluster-9kr7h4: failed waiting for cluster="list-test-cluster-9kr7h4" in resourcegroup="cluster-listing-gf288m" to finish creating: GET https://rp.j9688192.hcpsvc.osadev.cloud/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westus3/hcpOperationStatuses/9e740b0f-be2a-4872-80dc-2be64627366a -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westus3/hcpOperationStatuses/9e740b0f-be2a-4872-80dc-2be64627366a", "name": "9e740b0f-be2a-4872-80dc-2be64627366a", "status": "Failed", "startTime": "2026-03-11T22:38:49.336574312Z", "endTime": "2026-03-11T22:39:46.867353527Z", "error": { "code": "InternalServerError", "message": "failed to get managed identity '/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-dev-121/providers/Microsoft.ManagedIdentity/userAssignedIdentities/kms': GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-dev-121/providers/Microsoft.ManagedIdentity/userAssignedIdentities/kms\n--------------------------------------------------------------------------------\nRESPONSE 429: 429 Too Many Requests\nERROR CODE: SubscriptionRequestsThrottled\n--------------------------------------------------------------------------------\n{\n \"error\": {\n \"code\": \"SubscriptionRequestsThrottled\",\n \"message\": \"Number of 'read' requests for subscription 'XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX' actor 'd6b62dfa-87f5-49b3-bbcb-4a687c4faa96' exceeded. Please try again after '10' seconds after additional tokens are available. Refer to https://aka.ms/arm-throttling for additional information.\"\n }\n}\n--------------------------------------------------------------------------------\n" } } -------------------------------------------------------------------------------- { msg: "failed to create HCP cluster list-test-cluster-9kr7h4: failed waiting for cluster=\"list-test-cluster-9kr7h4\" in resourcegroup=\"cluster-listing-gf288m\" to finish creating: GET https://rp.j9688192.hcpsvc.osadev.cloud/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westus3/hcpOperationStatuses/9e740b0f-be2a-4872-80dc-2be64627366a\n--------------------------------------------------------------------------------\nRESPONSE 200: 200 OK\nERROR CODE: InternalServerError\n--------------------------------------------------------------------------------\n{\n \"id\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westus3/hcpOperationStatuses/9e740b0f-be2a-4872-80dc-2be64627366a\",\n \"name\": \"9e740b0f-be2a-4872-80dc-2be64627366a\",\n \"status\": \"Failed\",\n \"startTime\": \"2026-03-11T22:38:49.336574312Z\",\n \"endTime\": \"2026-03-11T22:39:46.867353527Z\",\n \"error\": {\n \"code\": \"InternalServerError\",\n \"message\": \"failed to get managed identity '/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-dev-121/providers/Microsoft.ManagedIdentity/userAssignedIdentities/kms': GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-dev-121/providers/Microsoft.ManagedIdentity/userAssignedIdentities/kms\\n--------------------------------------------------------------------------------\\nRESPONSE 429: 429 Too Many Requests\\nERROR CODE: SubscriptionRequestsThrottled\\n--------------------------------------------------------------------------------\\n{\\n \\\"error\\\": {\\n \\\"code\\\": \\\"SubscriptionRequestsThrottled\\\",\\n \\\"message\\\": \\\"Number of 'read' requests for subscription 'XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX' actor 'd6b62dfa-87f5-49b3-bbcb-4a687c4faa96' exceeded. Please try again after '10' seconds after additional tokens are available. Refer to https://aka.ms/arm-throttling for additional information.\\\"\\n }\\n}\\n--------------------------------------------------------------------------------\\n\"\n }\n}\n--------------------------------------------------------------------------------\n", err: <fmt.wrapError \| 0xc00161a460>{ msg: "failed waiting for cluster=\"list-test-cluster-9kr7h4\" in resourcegroup=\"cluster-listing-gf288m\" to finish creating: GET https://rp.j9688192.hcpsvc.osadev.cloud/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westus3/hcpOperationStatuses/9e740b0f-be2a-4872-80dc-2be64627366a\n--------------------------------------------------------------------------------\nRESPONSE 200: 200 OK\nERROR CODE: InternalServerError\n--------------------------------------------------------------------------------\n{\n \"id\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westus3/hcpOperationStatuses/9e740b0f-be2a-4872-80dc-2be64627366a\",\n \"name\": \"9e740b0f-be2a-4872-80dc-2be64627366a\",\n \"status\": \"Failed\",\n \"startTime\": \"2026-03-11T22:38:49.336574312Z\",\n \"endTime\": \"2026-03-11T22:39:46.867353527Z\",\n \"error\": {\n \"code\": \"InternalServerError\",\n \"message\": \"failed to get managed identity '/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-dev-121/providers/Microsoft.ManagedIdentity/userAssignedIdentities/kms': GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-dev-121/providers/Microsoft.ManagedIdentity/userAssignedIdentities/kms\\n--------------------------------------------------------------------------------\\nRESPONSE 429: 429 Too Many Requests\\nERROR CODE: SubscriptionRequestsThrottled\\n--------------------------------------------------------------------------------\\... Gomega truncated this representation as it exceeds 'format.MaxLength'. Consider having the object provide a custom 'GomegaStringer' representation or adjust the parameters in Gomega's 'format' package. Learn more here: https://onsi.github.io/gomega/#adjusting-output occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_list_no_infra.go:86]: Unexpected error: <fmt.wrapError \| 0xc00161a480>: failed to create HCP cluster list-test-cluster-9kr7h4: failed waiting for cluster="list-test-cluster-9kr7h4" in resourcegroup="cluster-listing-gf288m" to finish creating: GET https://rp.j9688192.hcpsvc.osadev.cloud/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westus3/hcpOperationStatuses/9e740b0f-be2a-4872-80dc-2be64627366a -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westus3/hcpOperationStatuses/9e740b0f-be2a-4872-80dc-2be64627366a", "name": "9e740b0f-be2a-4872-80dc-2be64627366a", "status": "Failed", "startTime": "2026-03-11T22:38:49.336574312Z", "endTime": "2026-03-11T22:39:46.867353527Z", "error": { "code": "InternalServerError", "message": "failed to get managed identity '/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-dev-121/providers/Microsoft.ManagedIdentity/userAssignedIdentities/kms': GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-dev-121/providers/Microsoft.ManagedIdentity/userAssignedIdentities/kms\n--------------------------------------------------------------------------------\nRESPONSE 429: 429 Too Many Requests\nERROR CODE: SubscriptionRequestsThrottled\n--------------------------------------------------------------------------------\n{\n \"error\": {\n \"code\": \"SubscriptionRequestsThrottled\",\n \"message\": \"Number of 'read' requests for subscription 'XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX' actor 'd6b62dfa-87f5-49b3-bbcb-4a687c4faa96' exceeded. Please try again after '10' seconds after additional tokens are available. Refer to https://aka.ms/arm-throttling for additional information.\"\n }\n}\n--------------------------------------------------------------------------------\n" } } -------------------------------------------------------------------------------- { msg: "failed to create HCP cluster list-test-cluster-9kr7h4: failed waiting for cluster=\"list-test-cluster-9kr7h4\" in resourcegroup=\"cluster-listing-gf288m\" to finish creating: GET https://rp.j9688192.hcpsvc.osadev.cloud/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westus3/hcpOperationStatuses/9e740b0f-be2a-4872-80dc-2be64627366a\n--------------------------------------------------------------------------------\nRESPONSE 200: 200 OK\nERROR CODE: InternalServerError\n--------------------------------------------------------------------------------\n{\n \"id\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westus3/hcpOperationStatuses/9e740b0f-be2a-4872-80dc-2be64627366a\",\n \"name\": \"9e740b0f-be2a-4872-80dc-2be64627366a\",\n \"status\": \"Failed\",\n \"startTime\": \"2026-03-11T22:38:49.336574312Z\",\n \"endTime\": \"2026-03-11T22:39:46.867353527Z\",\n \"error\": {\n \"code\": \"InternalServerError\",\n \"message\": \"failed to get managed identity '/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-dev-121/providers/Microsoft.ManagedIdentity/userAssignedIdentities/kms': GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-dev-121/providers/Microsoft.ManagedIdentity/userAssignedIdentities/kms\\n--------------------------------------------------------------------------------\\nRESPONSE 429: 429 Too Many Requests\\nERROR CODE: SubscriptionRequestsThrottled\\n--------------------------------------------------------------------------------\\n{\\n \\\"error\\\": {\\n \\\"code\\\": \\\"SubscriptionRequestsThrottled\\\",\\n \\\"message\\\": \\\"Number of 'read' requests for subscription 'XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX' actor 'd6b62dfa-87f5-49b3-bbcb-4a687c4faa96' exceeded. Please try again after '10' seconds after additional tokens are available. Refer to https://aka.ms/arm-throttling for additional information.\\\"\\n }\\n}\\n--------------------------------------------------------------------------------\\n\"\n }\n}\n--------------------------------------------------------------------------------\n", err: <fmt.wrapError \| 0xc00161a460>{ msg: "failed waiting for cluster=\"list-test-cluster-9kr7h4\" in resourcegroup=\"cluster-listing-gf288m\" to finish creating: GET https://rp.j9688192.hcpsvc.osadev.cloud/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westus3/hcpOperationStatuses/9e740b0f-be2a-4872-80dc-2be64627366a\n--------------------------------------------------------------------------------\nRESPONSE 200: 200 OK\nERROR CODE: InternalServerError\n--------------------------------------------------------------------------------\n{\n \"id\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westus3/hcpOperationStatuses/9e740b0f-be2a-4872-80dc-2be64627366a\",\n \"name\": \"9e740b0f-be2a-4872-80dc-2be64627366a\",\n \"status\": \"Failed\",\n \"startTime\": \"2026-03-11T22:38:49.336574312Z\",\n \"endTime\": \"2026-03-11T22:39:46.867353527Z\",\n \"error\": {\n \"code\": \"InternalServerError\",\n \"message\": \"failed to get managed identity '/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-dev-121/providers/Microsoft.ManagedIdentity/userAssignedIdentities/kms': GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-dev-121/providers/Microsoft.ManagedIdentity/userAssignedIdentities/kms\\n--------------------------------------------------------------------------------\\nRESPONSE 429: 429 Too Many Requests\\nERROR CODE: SubscriptionRequestsThrottled\\n--------------------------------------------------------------------------------\\... Gomega truncated this representation as it exceeds 'format.MaxLength'. Consider having the object provide a custom 'GomegaStringer' representation or adjust the parameters in Gomega's 'format' package. Learn more here: https://onsi.github.io/gomega/#adjusting-output occurred
21:33:25	n/a	pull-ci-Azure-ARO-HCP-main-e2e-parallel batch head 1892906 · merge 1892906	n/a	n/a	passed	#3820 (merged)	0	n/a
21:33:09	n/a	pull-ci-Azure-ARO-HCP-main-e2e-parallel merged PR head 698a2b9 · merge 0b33790	n/a	n/a	passed	#4376 (merged)	0	n/a
21:12:22	n/a	pull-ci-Azure-ARO-HCP-main-e2e-parallel post-goodmerged PR head 5b9e8e9 · merge 5b9e8e9	n/a	e2e	failed	#4416 (merged)	3	e2e (3) timeout during CreateHCPClusterAndWait; context deadline exceeded 21:12:22 UTC · failed at e2e · test Engineering should be able to retrieve kusto logs for a cluster and services · suite rp-api-compat-all/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/kusto_logs_present.go:87]: Unexpected error: <fmt.wrapErrors \| 0xc0012251a0>: failed to create HCP cluster engineering-hcp-cluster, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster engineering-hcp-cluster in resource group kusto-logs-dthjd6, error: failed waiting for cluster="engineering-hcp-cluster" in resourcegroup="kusto-logs-dthjd6" to finish creating, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster engineering-hcp-cluster in resource group kusto-logs-dthjd6, error: context deadline exceeded { msg: "failed to create HCP cluster engineering-hcp-cluster, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster engineering-hcp-cluster in resource group kusto-logs-dthjd6, error: failed waiting for cluster=\"engineering-hcp-cluster\" in resourcegroup=\"kusto-logs-dthjd6\" to finish creating, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster engineering-hcp-cluster in resource group kusto-logs-dthjd6, error: context deadline exceeded", errs: [ <errors.errorString \| 0xc0011d7710>{ s: "timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster engineering-hcp-cluster in resource group kusto-logs-dthjd6", }, <fmt.wrapErrors \| 0xc001225110>{ msg: "failed waiting for cluster=\"engineering-hcp-cluster\" in resourcegroup=\"kusto-logs-dthjd6\" to finish creating, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster engineering-hcp-cluster in resource group kusto-logs-dthjd6, error: context deadline exceeded", errs: [ <errors.errorString \| 0xc0011d7710>{ s: "timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster engineering-hcp-cluster in resource group kusto-logs-dthjd6", }, <context.deadlineExceededError>{}, ], }, ], } occurred fail [github.com/Azure/ARO-HCP/test/e2e/kusto_logs_present.go:87]: Unexpected error: <fmt.wrapErrors \| 0xc0012251a0>: failed to create HCP cluster engineering-hcp-cluster, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster engineering-hcp-cluster in resource group kusto-logs-dthjd6, error: failed waiting for cluster="engineering-hcp-cluster" in resourcegroup="kusto-logs-dthjd6" to finish creating, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster engineering-hcp-cluster in resource group kusto-logs-dthjd6, error: context deadline exceeded { msg: "failed to create HCP cluster engineering-hcp-cluster, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster engineering-hcp-cluster in resource group kusto-logs-dthjd6, error: failed waiting for cluster=\"engineering-hcp-cluster\" in resourcegroup=\"kusto-logs-dthjd6\" to finish creating, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster engineering-hcp-cluster in resource group kusto-logs-dthjd6, error: context deadline exceeded", errs: [ <errors.errorString \| 0xc0011d7710>{ s: "timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster engineering-hcp-cluster in resource group kusto-logs-dthjd6", }, <fmt.wrapErrors \| 0xc001225110>{ msg: "failed waiting for cluster=\"engineering-hcp-cluster\" in resourcegroup=\"kusto-logs-dthjd6\" to finish creating, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster engineering-hcp-cluster in resource group kusto-logs-dthjd6, error: context deadline exceeded", errs: [ <errors.errorString \| 0xc0011d7710>{ s: "timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster engineering-hcp-cluster in resource group kusto-logs-dthjd6", }, <context.deadlineExceededError>{}, ], }, ], } occurred timeout during CreateHCPClusterAndWait; context deadline exceeded 21:12:22 UTC · failed at e2e · test Customer should respect cluster-wide node limits with nodepool autoscaling · suite rp-api-compat-all/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_autoscaling.go:166]: Unexpected error: <fmt.wrapErrors \| 0xc0009ad230>: failed to create HCP cluster node-limit-cluster, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster node-limit-cluster in resource group node-limit-test-gwnwk6, error: failed waiting for cluster="node-limit-cluster" in resourcegroup="node-limit-test-gwnwk6" to finish creating, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster node-limit-cluster in resource group node-limit-test-gwnwk6, error: context deadline exceeded { msg: "failed to create HCP cluster node-limit-cluster, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster node-limit-cluster in resource group node-limit-test-gwnwk6, error: failed waiting for cluster=\"node-limit-cluster\" in resourcegroup=\"node-limit-test-gwnwk6\" to finish creating, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster node-limit-cluster in resource group node-limit-test-gwnwk6, error: context deadline exceeded", errs: [ <errors.errorString \| 0xc0014ff980>{ s: "timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster node-limit-cluster in resource group node-limit-test-gwnwk6", }, <fmt.wrapErrors \| 0xc0009ad1a0>{ msg: "failed waiting for cluster=\"node-limit-cluster\" in resourcegroup=\"node-limit-test-gwnwk6\" to finish creating, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster node-limit-cluster in resource group node-limit-test-gwnwk6, error: context deadline exceeded", errs: [ <errors.errorString \| 0xc0014ff980>{ s: "timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster node-limit-cluster in resource group node-limit-test-gwnwk6", }, <context.deadlineExceededError>{}, ], }, ], } occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_autoscaling.go:166]: Unexpected error: <fmt.wrapErrors \| 0xc0009ad230>: failed to create HCP cluster node-limit-cluster, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster node-limit-cluster in resource group node-limit-test-gwnwk6, error: failed waiting for cluster="node-limit-cluster" in resourcegroup="node-limit-test-gwnwk6" to finish creating, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster node-limit-cluster in resource group node-limit-test-gwnwk6, error: context deadline exceeded { msg: "failed to create HCP cluster node-limit-cluster, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster node-limit-cluster in resource group node-limit-test-gwnwk6, error: failed waiting for cluster=\"node-limit-cluster\" in resourcegroup=\"node-limit-test-gwnwk6\" to finish creating, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster node-limit-cluster in resource group node-limit-test-gwnwk6, error: context deadline exceeded", errs: [ <errors.errorString \| 0xc0014ff980>{ s: "timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster node-limit-cluster in resource group node-limit-test-gwnwk6", }, <fmt.wrapErrors \| 0xc0009ad1a0>{ msg: "failed waiting for cluster=\"node-limit-cluster\" in resourcegroup=\"node-limit-test-gwnwk6\" to finish creating, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster node-limit-cluster in resource group node-limit-test-gwnwk6, error: context deadline exceeded", errs: [ <errors.errorString \| 0xc0014ff980>{ s: "timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster node-limit-cluster in resource group node-limit-test-gwnwk6", }, <context.deadlineExceededError>{}, ], }, ], } occurred timeout during CreateHCPClusterAndWait; context deadline exceeded 21:12:22 UTC · failed at e2e · test Customer should be able to create an HCP cluster with Image Registry not present · suite rp-api-compat-all/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/image_registry_cluster_create.go:89]: Unexpected error: <fmt.wrapErrors \| 0xc0014e6ed0>: failed to create HCP cluster disabled-image-registry-hcp-cluster, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster disabled-image-registry-hcp-cluster in resource group disabled-image-registry-z24fch, error: failed waiting for cluster="disabled-image-registry-hcp-cluster" in resourcegroup="disabled-image-registry-z24fch" to finish creating, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster disabled-image-registry-hcp-cluster in resource group disabled-image-registry-z24fch, error: context deadline exceeded { msg: "failed to create HCP cluster disabled-image-registry-hcp-cluster, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster disabled-image-registry-hcp-cluster in resource group disabled-image-registry-z24fch, error: failed waiting for cluster=\"disabled-image-registry-hcp-cluster\" in resourcegroup=\"disabled-image-registry-z24fch\" to finish creating, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster disabled-image-registry-hcp-cluster in resource group disabled-image-registry-z24fch, error: context deadline exceeded", errs: [ <errors.errorString \| 0xc001879410>{ s: "timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster disabled-image-registry-hcp-cluster in resource group disabled-image-registry-z24fch", }, <fmt.wrapErrors \| 0xc0014e6ea0>{ msg: "failed waiting for cluster=\"disabled-image-registry-hcp-cluster\" in resourcegroup=\"disabled-image-registry-z24fch\" to finish creating, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster disabled-image-registry-hcp-cluster in resource group disabled-image-registry-z24fch, error: context deadline exceeded", errs: [ <errors.errorString \| 0xc001879410>{ s: "timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster disabled-image-registry-hcp-cluster in resource group disabled-image-registry-z24fch", }, <context.deadlineExceededError>{}, ], }, ], } occurred fail [github.com/Azure/ARO-HCP/test/e2e/image_registry_cluster_create.go:89]: Unexpected error: <fmt.wrapErrors \| 0xc0014e6ed0>: failed to create HCP cluster disabled-image-registry-hcp-cluster, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster disabled-image-registry-hcp-cluster in resource group disabled-image-registry-z24fch, error: failed waiting for cluster="disabled-image-registry-hcp-cluster" in resourcegroup="disabled-image-registry-z24fch" to finish creating, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster disabled-image-registry-hcp-cluster in resource group disabled-image-registry-z24fch, error: context deadline exceeded { msg: "failed to create HCP cluster disabled-image-registry-hcp-cluster, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster disabled-image-registry-hcp-cluster in resource group disabled-image-registry-z24fch, error: failed waiting for cluster=\"disabled-image-registry-hcp-cluster\" in resourcegroup=\"disabled-image-registry-z24fch\" to finish creating, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster disabled-image-registry-hcp-cluster in resource group disabled-image-registry-z24fch, error: context deadline exceeded", errs: [ <errors.errorString \| 0xc001879410>{ s: "timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster disabled-image-registry-hcp-cluster in resource group disabled-image-registry-z24fch", }, <fmt.wrapErrors \| 0xc0014e6ea0>{ msg: "failed waiting for cluster=\"disabled-image-registry-hcp-cluster\" in resourcegroup=\"disabled-image-registry-z24fch\" to finish creating, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster disabled-image-registry-hcp-cluster in resource group disabled-image-registry-z24fch, error: context deadline exceeded", errs: [ <errors.errorString \| 0xc001879410>{ s: "timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster disabled-image-registry-hcp-cluster in resource group disabled-image-registry-z24fch", }, <context.deadlineExceededError>{}, ], }, ], } occurred
20:38:00	n/a	pull-ci-Azure-ARO-HCP-main-e2e-parallel merged PR head 19c1684 · merge f91e23c	n/a	n/a	passed	#4410 (merged)	0	n/a
19:41:45	n/a	pull-ci-Azure-ARO-HCP-main-e2e-parallel head 1e50ea6	n/a	n/a	passed	#4415 (closed)	0	n/a
18:56:09	n/a	pull-ci-Azure-ARO-HCP-main-e2e-parallel post-goodmerged PR head 4655674 · merge 4655674	n/a	n/a	passed	#4405 (merged)	0	n/a
18:32:46	n/a	pull-ci-Azure-ARO-HCP-main-e2e-parallel merged PR head 19c1684 · merge f91e23c	n/a	e2e	failed	#4410 (merged)	1	e2e (1) ERROR CODE: InternalServerError; detail code AuthorizationPermissionMismatch; provider Microsoft.RedHatOpenShift 18:32:46 UTC · failed at e2e · test Customer should be able to create a cluster with default autoscaling and a nodepool with autoscaling enabled · suite rp-api-compat-all/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_autoscaling.go:83]: Unexpected error: <fmt.wrapError \| 0xc00047e220>: failed to create HCP cluster np-autoscale-cluster: failed waiting for cluster="np-autoscale-cluster" in resourcegroup="np-autoscaling-hmt75w" to finish creating: GET https://rp.j2141696.hcpsvc.osadev.cloud/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westus3/hcpOperationStatuses/deec8042-2914-4e4c-a390-a1755b5f9b9d -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westus3/hcpOperationStatuses/deec8042-2914-4e4c-a390-a1755b5f9b9d", "name": "deec8042-2914-4e4c-a390-a1755b5f9b9d", "status": "Failed", "startTime": "2026-03-11T19:52:56.363794552Z", "endTime": "2026-03-11T19:54:41.481115713Z", "error": { "code": "InternalServerError", "message": "GET https://arohcpprowoidcj2141696.blob.core.windows.net/$web\n--------------------------------------------------------------------------------\nRESPONSE 403: 403 This request is not authorized to perform this operation using this permission.\nERROR CODE: AuthorizationPermissionMismatch\n--------------------------------------------------------------------------------\n\u003c?xml version=\"1.0\" encoding=\"utf-8\"?\u003e\u003cError\u003e\u003cCode\u003eAuthorizationPermissionMismatch\u003c/Code\u003e\u003cMessage\u003eThis request is not authorized to perform this operation using this permission.\nRequestId:67c1957e-301e-0013-7190-b1fbf4000000\nTime:2026-03-11T19:54:20.9610355Z\u003c/Message\u003e\u003c/Error\u003e\n--------------------------------------------------------------------------------\n" } } -------------------------------------------------------------------------------- { msg: "failed to create HCP cluster np-autoscale-cluster: failed waiting for cluster=\"np-autoscale-cluster\" in resourcegroup=\"np-autoscaling-hmt75w\" to finish creating: GET https://rp.j2141696.hcpsvc.osadev.cloud/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westus3/hcpOperationStatuses/deec8042-2914-4e4c-a390-a1755b5f9b9d\n--------------------------------------------------------------------------------\nRESPONSE 200: 200 OK\nERROR CODE: InternalServerError\n--------------------------------------------------------------------------------\n{\n \"id\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westus3/hcpOperationStatuses/deec8042-2914-4e4c-a390-a1755b5f9b9d\",\n \"name\": \"deec8042-2914-4e4c-a390-a1755b5f9b9d\",\n \"status\": \"Failed\",\n \"startTime\": \"2026-03-11T19:52:56.363794552Z\",\n \"endTime\": \"2026-03-11T19:54:41.481115713Z\",\n \"error\": {\n \"code\": \"InternalServerError\",\n \"message\": \"GET https://arohcpprowoidcj2141696.blob.core.windows.net/$web\\n--------------------------------------------------------------------------------\\nRESPONSE 403: 403 This request is not authorized to perform this operation using this permission.\\nERROR CODE: AuthorizationPermissionMismatch\\n--------------------------------------------------------------------------------\\n\ufeff\\u003c?xml version=\\\"1.0\\\" encoding=\\\"utf-8\\\"?\\u003e\\u003cError\\u003e\\u003cCode\\u003eAuthorizationPermissionMismatch\\u003c/Code\\u003e\\u003cMessage\\u003eThis request is not authorized to perform this operation using this permission.\\nRequestId:67c1957e-301e-0013-7190-b1fbf4000000\\nTime:2026-03-11T19:54:20.9610355Z\\u003c/Message\\u003e\\u003c/Error\\u003e\\n--------------------------------------------------------------------------------\\n\"\n }\n}\n--------------------------------------------------------------------------------\n", err: <fmt.wrapError \| 0xc00047e200>{ msg: "failed waiting for cluster=\"np-autoscale-cluster\" in resourcegroup=\"np-autoscaling-hmt75w\" to finish creating: GET https://rp.j2141696.hcpsvc.osadev.cloud/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westus3/hcpOperationStatuses/deec8042-2914-4e4c-a390-a1755b5f9b9d\n--------------------------------------------------------------------------------\nRESPONSE 200: 200 OK\nERROR CODE: InternalServerError\n--------------------------------------------------------------------------------\n{\n \"id\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westus3/hcpOperationStatuses/deec8042-2914-4e4c-a390-a1755b5f9b9d\",\n \"name\": \"deec8042-2914-4e4c-a390-a1755b5f9b9d\",\n \"status\": \"Failed\",\n \"startTime\": \"2026-03-11T19:52:56.363794552Z\",\n \"endTime\": \"2026-03-11T19:54:41.481115713Z\",\n \"error\": {\n \"code\": \"InternalServerError\",\n \"message\": \"GET https://arohcpprowoidcj2141696.blob.core.windows.net/$web\\n--------------------------------------------------------------------------------\\nRESPONSE 403: 403 This request is not authorized to perform this operation using this permission.\\nERROR CODE: AuthorizationPermissionMismatch\\n--------------------------------------------------------------------------------\\n\ufeff\\u003c?xml version=\\\"1.0\\\" encoding=\\\"utf-8\\\"?\\u003e\\u003cError\\u003e\\u003cCode\\u003eAuthorizationPermissionMismatch\\u003c/Code\\u003e\\u003cMessage\\u003eThis request is not authorized to perform this operation using this permission.\\nRequestId:67c1957e-301e-0013-7190-b1fbf4000000\\nTime:2026-03-11T19:54:20.9610355Z\\u003c/Message\\u003e\\u003c/Error\\u003e\\n--------------------------------------------------------------------------------\\n\"\n }\n}\n--------------------------------------------------------------------------------\n", ... Gomega truncated this representation as it exceeds 'format.MaxLength'. Consider having the object provide a custom 'GomegaStringer' representation or adjust the parameters in Gomega's 'format' package. Learn more here: https://onsi.github.io/gomega/#adjusting-output occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_autoscaling.go:83]: Unexpected error: <fmt.wrapError \| 0xc00047e220>: failed to create HCP cluster np-autoscale-cluster: failed waiting for cluster="np-autoscale-cluster" in resourcegroup="np-autoscaling-hmt75w" to finish creating: GET https://rp.j2141696.hcpsvc.osadev.cloud/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westus3/hcpOperationStatuses/deec8042-2914-4e4c-a390-a1755b5f9b9d -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westus3/hcpOperationStatuses/deec8042-2914-4e4c-a390-a1755b5f9b9d", "name": "deec8042-2914-4e4c-a390-a1755b5f9b9d", "status": "Failed", "startTime": "2026-03-11T19:52:56.363794552Z", "endTime": "2026-03-11T19:54:41.481115713Z", "error": { "code": "InternalServerError", "message": "GET https://arohcpprowoidcj2141696.blob.core.windows.net/$web\n--------------------------------------------------------------------------------\nRESPONSE 403: 403 This request is not authorized to perform this operation using this permission.\nERROR CODE: AuthorizationPermissionMismatch\n--------------------------------------------------------------------------------\n\u003c?xml version=\"1.0\" encoding=\"utf-8\"?\u003e\u003cError\u003e\u003cCode\u003eAuthorizationPermissionMismatch\u003c/Code\u003e\u003cMessage\u003eThis request is not authorized to perform this operation using this permission.\nRequestId:67c1957e-301e-0013-7190-b1fbf4000000\nTime:2026-03-11T19:54:20.9610355Z\u003c/Message\u003e\u003c/Error\u003e\n--------------------------------------------------------------------------------\n" } } -------------------------------------------------------------------------------- { msg: "failed to create HCP cluster np-autoscale-cluster: failed waiting for cluster=\"np-autoscale-cluster\" in resourcegroup=\"np-autoscaling-hmt75w\" to finish creating: GET https://rp.j2141696.hcpsvc.osadev.cloud/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westus3/hcpOperationStatuses/deec8042-2914-4e4c-a390-a1755b5f9b9d\n--------------------------------------------------------------------------------\nRESPONSE 200: 200 OK\nERROR CODE: InternalServerError\n--------------------------------------------------------------------------------\n{\n \"id\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westus3/hcpOperationStatuses/deec8042-2914-4e4c-a390-a1755b5f9b9d\",\n \"name\": \"deec8042-2914-4e4c-a390-a1755b5f9b9d\",\n \"status\": \"Failed\",\n \"startTime\": \"2026-03-11T19:52:56.363794552Z\",\n \"endTime\": \"2026-03-11T19:54:41.481115713Z\",\n \"error\": {\n \"code\": \"InternalServerError\",\n \"message\": \"GET https://arohcpprowoidcj2141696.blob.core.windows.net/$web\\n--------------------------------------------------------------------------------\\nRESPONSE 403: 403 This request is not authorized to perform this operation using this permission.\\nERROR CODE: AuthorizationPermissionMismatch\\n--------------------------------------------------------------------------------\\n\ufeff\\u003c?xml version=\\\"1.0\\\" encoding=\\\"utf-8\\\"?\\u003e\\u003cError\\u003e\\u003cCode\\u003eAuthorizationPermissionMismatch\\u003c/Code\\u003e\\u003cMessage\\u003eThis request is not authorized to perform this operation using this permission.\\nRequestId:67c1957e-301e-0013-7190-b1fbf4000000\\nTime:2026-03-11T19:54:20.9610355Z\\u003c/Message\\u003e\\u003c/Error\\u003e\\n--------------------------------------------------------------------------------\\n\"\n }\n}\n--------------------------------------------------------------------------------\n", err: <fmt.wrapError \| 0xc00047e200>{ msg: "failed waiting for cluster=\"np-autoscale-cluster\" in resourcegroup=\"np-autoscaling-hmt75w\" to finish creating: GET https://rp.j2141696.hcpsvc.osadev.cloud/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westus3/hcpOperationStatuses/deec8042-2914-4e4c-a390-a1755b5f9b9d\n--------------------------------------------------------------------------------\nRESPONSE 200: 200 OK\nERROR CODE: InternalServerError\n--------------------------------------------------------------------------------\n{\n \"id\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westus3/hcpOperationStatuses/deec8042-2914-4e4c-a390-a1755b5f9b9d\",\n \"name\": \"deec8042-2914-4e4c-a390-a1755b5f9b9d\",\n \"status\": \"Failed\",\n \"startTime\": \"2026-03-11T19:52:56.363794552Z\",\n \"endTime\": \"2026-03-11T19:54:41.481115713Z\",\n \"error\": {\n \"code\": \"InternalServerError\",\n \"message\": \"GET https://arohcpprowoidcj2141696.blob.core.windows.net/$web\\n--------------------------------------------------------------------------------\\nRESPONSE 403: 403 This request is not authorized to perform this operation using this permission.\\nERROR CODE: AuthorizationPermissionMismatch\\n--------------------------------------------------------------------------------\\n\ufeff\\u003c?xml version=\\\"1.0\\\" encoding=\\\"utf-8\\\"?\\u003e\\u003cError\\u003e\\u003cCode\\u003eAuthorizationPermissionMismatch\\u003c/Code\\u003e\\u003cMessage\\u003eThis request is not authorized to perform this operation using this permission.\\nRequestId:67c1957e-301e-0013-7190-b1fbf4000000\\nTime:2026-03-11T19:54:20.9610355Z\\u003c/Message\\u003e\\u003c/Error\\u003e\\n--------------------------------------------------------------------------------\\n\"\n }\n}\n--------------------------------------------------------------------------------\n", ... Gomega truncated this representation as it exceeds 'format.MaxLength'. Consider having the object provide a custom 'GomegaStringer' representation or adjust the parameters in Gomega's 'format' package. Learn more here: https://onsi.github.io/gomega/#adjusting-output occurred
17:55:23	n/a	pull-ci-Azure-ARO-HCP-main-e2e-parallel head c008cfe	n/a	e2e	failed	⚠#4281 (open)	10	e2e (10) whoami failed: Post "<url> 17:55:23 UTC · failed at e2e · test SRE should be able to log into a cluster via a breakglass session · suite rp-api-compat-all/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/admin_api.go:276]: Timed out after 30.000s. Expected success, but got an error: <fmt.wrapError \| 0xc000f24c80>: whoami failed: Post "https://sessiongate.j3408384.hcpsvc.osadev.cloud/sessiongate/breakglass-tnn5j/kas/apis/authentication.k8s.io/v1/selfsubjectreviews": tls: failed to verify certificate: x509: certificate signed by unknown authority { msg: "whoami failed: Post \"https://sessiongate.j3408384.hcpsvc.osadev.cloud/sessiongate/breakglass-tnn5j/kas/apis/authentication.k8s.io/v1/selfsubjectreviews\": tls: failed to verify certificate: x509: certificate signed by unknown authority", err: <url.Error \| 0xc000f295c0>{ Op: "Post", URL: "https://sessiongate.j3408384.hcpsvc.osadev.cloud/sessiongate/breakglass-tnn5j/kas/apis/authentication.k8s.io/v1/selfsubjectreviews", Err: <tls.CertificateVerificationError \| 0xc000f29590>{ UnverifiedCertificates: [ { Raw: [48, 130, 3, 163, 48, 130, 2, 139, 160, 3, 2, 1, 2, 2, 16, 60, 220, 3, 66, 206, 206, 67, 161, 182, 60, 238, 232, 174, 60, 119, 220, 48, 13, 6, 9, 42, 134, 72, 134, 247, 13, 1, 1, 11, 5, 0, 48, 51, 49, 49, 48, 47, 6, 3, 85, 4, 3, 19, 40, 115, 101, 115, 115, 105, 111, 110, 103, 97, 116, 101, 46, 106, 51, 52, 48, 56, 51, 56, 52, 46, 104, 99, 112, 115, 118, 99, 46, 111, 115, 97, 100, 101, 118, 46, 99, 108, 111, 117, 100, 48, 30, 23, 13, 50, 54, 48, 51, 49, 49, 49, 56, 49, 54, 49, 48, 90, 23, 13, 50, 55, 48, 51, 49, 49, 49, 56, 50, 54, 49, 48, 90, 48, 51, 49, 49, 48, 47, 6, 3, 85, 4, 3, 19, 40, 115, 101, 115, 115, 105, 111, 110, 103, 97, 116, 101, 46, 106, 51, 52, 48, 56, 51, 56, 52, 46, 104, 99, 112, 115, 118, 99, 46, 111, 115, 97, 100, 101, 118, 46, 99, 108, 111, 117, 100, 48, 130, 1, 34, 48, 13, 6, 9, 42, 134, 72, 134, 247, 13, 1, 1, 1, 5, 0, 3, 130, 1, 15, 0, 48, 130, 1, 10, 2, 130, 1, 1, 0, 170, 244, 209, 29, 122, 17, 25, 157, 128, 66, 253, 235, 120, 129, 195, 140, 97, 76, 238, 221, 255, 130, 99, 250, 128, 209, 64, 240, 3, 188, 202, 18, 192, 221, 168, 98, 13, 36, 43, 46, 137, 97, 131, 165, 112, 158, 38, 73, 241, 222, 193, 234, 177, 73, 221, 209, 205, 190, 108, 197, 100, 52, 241, 85, 141, 44, 209, 99, 255, 2, 172, 252, 83, 170, 77, 240, 68, 121, 171, 76, 42, 9, 93, 138, 200, 169, 196, 124, 218, 23, 187, 66, 6, 195, 93, 198, 27, 61, 51, 163, 17, 246, 131, 158, 240, 223, 123, 175, 172, 255, 67, 62, 243, 29, 55, 146, 171, 18, 17, 38, 253, 141, 95, 35, 49, 236, 7, 229, 99, 49, 228, 144, 62, 143, 95, 5, 197, 168, 73, 217, 41, 76, 222, 76, 211, 179, 23, 154, 215, 33, 176, 216, 116, 13, 154, 182, 62, 224, 57, 105, 241, 1, 229, 143, 130, 187, 62, 190, 32, 103, 20, 206, 42, 212, 248, 114, 183, 148, 36, 125, 15, 156, 56, 229, 92, 41, 70, 13, 50, 202, 151, 163, 22, 50, 43, 18, 145, 53, 114, 176, 84, 64, 253, 56, 222, 192, 113, 239, 241, 39, 205, 207, 152, 225, 13, 132, 208, 118, 69, 64, 26, 152, 128, 138, 120, 56, 175, 190, 96, 233, 90, 138, 25, 69, 226, 129, 77, 255, 249, 127, 193, 160, 206, 158, 134, 71, 233, 65, 213, 246, 232, 225, 164, 99, 87, 217, 2, 3, 1, 0, 1, 163, 129, 178, 48, 129, 175, 48, 14, 6, 3, 85, 29, 15, 1, 1, 255, 4, 4, 3, 2, 5, 160, 48, 9, 6, 3, 85, 29, 19, 4, 2, 48, 0, 48, 29, 6, 3, 85, 29, 37, 4, 22, 48, 20, 6, 8, 43, 6, 1, 5, 5, 7, 3, 1, 6, 8, 43, 6, 1, 5, 5, 7, 3, 2, 48, 51, 6, 3, 85, 29, 17, 4, 44, 48, 42, 130, 40, 115, 101, 115, 115, 105, 111, 110, 103, 97, 116, 101, 46, 106, 51, 52, 48, 56, 51, 56, 52, 46, 104, 99, 112, 115, 118, 99, 46, 111, 115, 97, 100, 101, 118, 46, 99, 108, 111, 117, 100, 48, 31, 6, 3, 85, 29, 35, 4, 24, 48, 22, 128, 20, 116, 7, 40, 185, 8, 167, 119, 212, 64, 244, 210, 236, 71, 125, 139, 221, 246, 3, 1, 144, 48, 29, 6, 3, 85, 29, 14, 4, 22, 4, 20, 116, 7, 40, 185, 8, 167, 119, 212, 64, 244, 210, 236, 71, 125, 139, 221, 246, 3, 1, 144, 48, 13, 6, 9, 42, 134, 72, 134, 247, 13, 1, 1, 11, 5, 0, 3, 130, 1, 1, 0, 22, 254, 58, 158, 51, 93, 65, 139, 104, 175, 133, 162, 252, 123, 74, 219, 158, 162, 79, 62, 252, 12, 165, 218, 0, 175, 171, 133, 226, 96, 93, 76, 128, 114, 93, 143, 155, 38, 25, 189, 184, 249, 7, 176, 9, 48, 242, 209, 163, 119, 34, 221, 129, 34, 1, 115, 123, 225, 70, 108, 250, 149, 17, 130, 139, 203, 159, 65, 27, 103, 148, 131, 68, 82, 4, 47, 39, 49, 206, 64, 48, 15, 7, 127, 22, 81, 35, 85, 194, 134, 97, 64, 71, 170, 122, 207, 134, 73, 61, 67, 120, 41, 121, 187, 122,... Gomega truncated this representation as it exceeds 'format.MaxLength'. Consider having the object provide a custom 'GomegaStringer' representation or adjust the parameters in Gomega's 'format' package. Learn more here: https://onsi.github.io/gomega/#adjusting-output fail [github.com/Azure/ARO-HCP/test/e2e/admin_api.go:276]: Timed out after 30.000s. Expected success, but got an error: <fmt.wrapError \| 0xc000f24c80>: whoami failed: Post "https://sessiongate.j3408384.hcpsvc.osadev.cloud/sessiongate/breakglass-tnn5j/kas/apis/authentication.k8s.io/v1/selfsubjectreviews": tls: failed to verify certificate: x509: certificate signed by unknown authority { msg: "whoami failed: Post \"https://sessiongate.j3408384.hcpsvc.osadev.cloud/sessiongate/breakglass-tnn5j/kas/apis/authentication.k8s.io/v1/selfsubjectreviews\": tls: failed to verify certificate: x509: certificate signed by unknown authority", err: <url.Error \| 0xc000f295c0>{ Op: "Post", URL: "https://sessiongate.j3408384.hcpsvc.osadev.cloud/sessiongate/breakglass-tnn5j/kas/apis/authentication.k8s.io/v1/selfsubjectreviews", Err: <tls.CertificateVerificationError \| 0xc000f29590>{ UnverifiedCertificates: [ { Raw: [48, 130, 3, 163, 48, 130, 2, 139, 160, 3, 2, 1, 2, 2, 16, 60, 220, 3, 66, 206, 206, 67, 161, 182, 60, 238, 232, 174, 60, 119, 220, 48, 13, 6, 9, 42, 134, 72, 134, 247, 13, 1, 1, 11, 5, 0, 48, 51, 49, 49, 48, 47, 6, 3, 85, 4, 3, 19, 40, 115, 101, 115, 115, 105, 111, 110, 103, 97, 116, 101, 46, 106, 51, 52, 48, 56, 51, 56, 52, 46, 104, 99, 112, 115, 118, 99, 46, 111, 115, 97, 100, 101, 118, 46, 99, 108, 111, 117, 100, 48, 30, 23, 13, 50, 54, 48, 51, 49, 49, 49, 56, 49, 54, 49, 48, 90, 23, 13, 50, 55, 48, 51, 49, 49, 49, 56, 50, 54, 49, 48, 90, 48, 51, 49, 49, 48, 47, 6, 3, 85, 4, 3, 19, 40, 115, 101, 115, 115, 105, 111, 110, 103, 97, 116, 101, 46, 106, 51, 52, 48, 56, 51, 56, 52, 46, 104, 99, 112, 115, 118, 99, 46, 111, 115, 97, 100, 101, 118, 46, 99, 108, 111, 117, 100, 48, 130, 1, 34, 48, 13, 6, 9, 42, 134, 72, 134, 247, 13, 1, 1, 1, 5, 0, 3, 130, 1, 15, 0, 48, 130, 1, 10, 2, 130, 1, 1, 0, 170, 244, 209, 29, 122, 17, 25, 157, 128, 66, 253, 235, 120, 129, 195, 140, 97, 76, 238, 221, 255, 130, 99, 250, 128, 209, 64, 240, 3, 188, 202, 18, 192, 221, 168, 98, 13, 36, 43, 46, 137, 97, 131, 165, 112, 158, 38, 73, 241, 222, 193, 234, 177, 73, 221, 209, 205, 190, 108, 197, 100, 52, 241, 85, 141, 44, 209, 99, 255, 2, 172, 252, 83, 170, 77, 240, 68, 121, 171, 76, 42, 9, 93, 138, 200, 169, 196, 124, 218, 23, 187, 66, 6, 195, 93, 198, 27, 61, 51, 163, 17, 246, 131, 158, 240, 223, 123, 175, 172, 255, 67, 62, 243, 29, 55, 146, 171, 18, 17, 38, 253, 141, 95, 35, 49, 236, 7, 229, 99, 49, 228, 144, 62, 143, 95, 5, 197, 168, 73, 217, 41, 76, 222, 76, 211, 179, 23, 154, 215, 33, 176, 216, 116, 13, 154, 182, 62, 224, 57, 105, 241, 1, 229, 143, 130, 187, 62, 190, 32, 103, 20, 206, 42, 212, 248, 114, 183, 148, 36, 125, 15, 156, 56, 229, 92, 41, 70, 13, 50, 202, 151, 163, 22, 50, 43, 18, 145, 53, 114, 176, 84, 64, 253, 56, 222, 192, 113, 239, 241, 39, 205, 207, 152, 225, 13, 132, 208, 118, 69, 64, 26, 152, 128, 138, 120, 56, 175, 190, 96, 233, 90, 138, 25, 69, 226, 129, 77, 255, 249, 127, 193, 160, 206, 158, 134, 71, 233, 65, 213, 246, 232, 225, 164, 99, 87, 217, 2, 3, 1, 0, 1, 163, 129, 178, 48, 129, 175, 48, 14, 6, 3, 85, 29, 15, 1, 1, 255, 4, 4, 3, 2, 5, 160, 48, 9, 6, 3, 85, 29, 19, 4, 2, 48, 0, 48, 29, 6, 3, 85, 29, 37, 4, 22, 48, 20, 6, 8, 43, 6, 1, 5, 5, 7, 3, 1, 6, 8, 43, 6, 1, 5, 5, 7, 3, 2, 48, 51, 6, 3, 85, 29, 17, 4, 44, 48, 42, 130, 40, 115, 101, 115, 115, 105, 111, 110, 103, 97, 116, 101, 46, 106, 51, 52, 48, 56, 51, 56, 52, 46, 104, 99, 112, 115, 118, 99, 46, 111, 115, 97, 100, 101, 118, 46, 99, 108, 111, 117, 100, 48, 31, 6, 3, 85, 29, 35, 4, 24, 48, 22, 128, 20, 116, 7, 40, 185, 8, 167, 119, 212, 64, 244, 210, 236, 71, 125, 139, 221, 246, 3, 1, 144, 48, 29, 6, 3, 85, 29, 14, 4, 22, 4, 20, 116, 7, 40, 185, 8, 167, 119, 212, 64, 244, 210, 236, 71, 125, 139, 221, 246, 3, 1, 144, 48, 13, 6, 9, 42, 134, 72, 134, 247, 13, 1, 1, 11, 5, 0, 3, 130, 1, 1, 0, 22, 254, 58, 158, 51, 93, 65, 139, 104, 175, 133, 162, 252, 123, 74, 219, 158, 162, 79, 62, 252, 12, 165, 218, 0, 175, 171, 133, 226, 96, 93, 76, 128, 114, 93, 143, 155, 38, 25, 189, 184, 249, 7, 176, 9, 48, 242, 209, 163, 119, 34, 221, 129, 34, 1, 115, 123, 225, 70, 108, 250, 149, 17, 130, 139, 203, 159, 65, 27, 103, 148, 131, 68, 82, 4, 47, 39, 49, 206, 64, 48, 15, 7, 127, 22, 81, 35, 85, 194, 134, 97, 64, 71, 170, 122, 207, 134, 73, 61, 67, 120, 41, 121, 187, 122,... Gomega truncated this representation as it exceeds 'format.MaxLength'. Consider having the object provide a custom 'GomegaStringer' representation or adjust the parameters in Gomega's 'format' package. Learn more here: https://onsi.github.io/gomega/#adjusting-output tls: failed to verify certificate: x509: certificate signed by unknown authority 17:55:23 UTC · failed at e2e · test Update HCPOpenShiftCluster Positive creates a cluster and updates tags with a PATCH request · suite rp-api-compat-all/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_update.go:174]: Unexpected error: <errors.joinError \| 0xc000f463a8>: VerifyAllAPIServicesAvailable failed: failed to list all APIServices: Get "https://api.a9u9y9u2h3o8l6k.8pac.j3408384.hcp.osadev.cloud:443/apis/apiregistration.k8s.io/v1/apiservices": tls: failed to verify certificate: x509: certificate signed by unknown authority VerifyBasicAccess failed: failed to list services: Get "https://api.a9u9y9u2h3o8l6k.8pac.j3408384.hcp.osadev.cloud:443/api/v1/namespaces/default/services": tls: failed to verify certificate: x509: certificate signed by unknown authority { errs: [ <fmt.wrapError \| 0xc001082220>{ msg: "VerifyAllAPIServicesAvailable failed: failed to list all APIServices: Get \"https://api.a9u9y9u2h3o8l6k.8pac.j3408384.hcp.osadev.cloud:443/apis/apiregistration.k8s.io/v1/apiservices\": tls: failed to verify certificate: x509: certificate signed by unknown authority", err: <fmt.wrapError \| 0xc001082200>{ msg: "failed to list all APIServices: Get \"https://api.a9u9y9u2h3o8l6k.8pac.j3408384.hcp.osadev.cloud:443/apis/apiregistration.k8s.io/v1/apiservices\": tls: failed to verify certificate: x509: certificate signed by unknown authority", err: <url.Error \| 0xc001080660>{ Op: "Get", URL: "https://api.a9u9y9u2h3o8l6k.8pac.j3408384.hcp.osadev.cloud:443/apis/apiregistration.k8s.io/v1/apiservices", Err: <tls.CertificateVerificationError \| 0xc001080630>{ UnverifiedCertificates: [ { Raw: [48, 130, 3, 143, 48, 130, 2, 119, 160, 3, 2, 1, 2, 2, 16, 17, 97, 189, 215, 12, 9, 69, 140, 146, 242, 21, 222, 229, 48, 69, 241, 48, 13, 6, 9, 42, 134, 72, 134, 247, 13, 1, 1, 11, 5, 0, 48, 36, 49, 34, 48, 32, 6, 3, 85, 4, 3, 19, 25, 114, 101, 115, 101, 114, 118, 101, 100, 46, 104, 99, 112, 46, 111, 115, 97, 100, 101, 118, 46, 99, 108, 111, 117, 100, 48, 30, 23, 13, 50, 54, 48, 51, 49, 49, 49, 56, 52, 51, 48, 53, 90, 23, 13, 50, 54, 48, 57, 49, 49, 49, 56, 53, 51, 48, 53, 90, 48, 36, 49, 34, 48, 32, 6, 3, 85, 4, 3, 19, 25, 114, 101, 115, 101, 114, 118, 101, 100, 46, 104, 99, 112, 46, 111, 115, 97, 100, 101, 118, 46, 99, 108, 111, 117, 100, 48, 130, 1, 34, 48, 13, 6, 9, 42, 134, 72, 134, 247, 13, 1, 1, 1, 5, 0, 3, 130, 1, 15, 0, 48, 130, 1, 10, 2, 130, 1, 1, 0, 197, 66, 136, 134, 170, 50, 11, 185, 128, 94, 78, 252, 110, 146, 62, 200, 235, 253, 115, 212, 60, 238, 70, 223, 119, 167, 62, 146, 218, 1, 201, 77, 97, 230, 202, 163, 114, 91, 170, 201, 184, 80, 172, 106, 114, 139, 156, 50, 52, 135, 21, 218, 52, 217, 180, 220, 135, 226, 216, 99, 59, 99, 118, 38, 50, 197, 146, 121, 206, 176, 232, 74, 191, 211, 140, 42, 88, 88, 255, 171, 11, 243, 189, 233, 57, 217, 156, 193, 213, 115, 197, 127, 84, 172, 178, 207, 249, 184, 190, 162, 205, 112, 239, 220, 43, 142, 56, 101, 6, 14, 219, 41, 150, 232, 109, 24, 67, 73, 170, 217, 80, 252, 54, 161, 28, 169, 65, 57, 40, 198, 210, 140, 170, 189, 250, 116, 31, 71, 4, 15, 133, 145, 74, 95, 94, 174, 16, 206, 152, 230, 73, 191, 76, 129, 91, 152, 176, 58, 59, 191, 49, 28, 246, 32, 209, 251, 74, 25, 101, 212, 222, 236, 236, 83, 186, 63, 168, 253, 194, 155, 18, 141, 156, 208, 153, 141, 169, 189, 170, 230, 243, 200, 65, 63, 156, 38, 62, 90, 170, 21, 133, 113, 73, 240, 85, 193, 56, 166, 135, 137, 151, 33, 154, 174, 4, 54, 140, 236, 116, 205, 173, 218, 144, 217, 172, 111, 53, 181, 141, 92, 218, 198, 64, 164, 162, 26, 242, 196, 19, 76, 51, 1, 37, 81, 49, 24, 98, 161, 177, 70, 172, 255, 155, 73, 147, 221, 2, 3, 1, 0, 1, 163, 129, 188, 48, 129, 185, 48, 14, 6, 3, 85, 29, 15, 1, 1, 255, 4, 4, 3, 2, 5, 160, 48, 9, 6, 3, 85, 29, 19, 4, 2, 48, 0, 48, 29, 6, 3, 85, 29, 37, 4, 22, 48, 20, 6, 8, 43, 6, 1, 5, 5, 7, 3, 1, 6, 8, 43, 6, 1, 5, 5, 7, 3, 2, 48, 61, 6, 3, 85, 29, 17, 4, 54, 48, 52, 130, 50, 97, 112, 105, 46, 97, 57, 117, 57, 121, 57, 117, 50, 104, 51, 111, 56, 108, 54, 107, 46, 56, 112, 97, 99, 46, 106, 51, 52, 48, 56, 51, 56, 52, 46, 104, 99, 112, 46, 111, 115, 97, 100, 101, 118, 46, 99, 108, 111, 117, 100, 48, 31, 6, 3, 85, 29, 35, 4, 24, 48, 22, 128, 20, 200, 215, 22, 129, 41, 106, 120, 252, 229, 17, 248, 83, 17, 227, 193, 236, 95, 77, 104, 216, 48, 29, 6, 3, 85, 29, 14, 4, 22, 4, 20, 200, 215, 22, 129, 41, 106, 120, 252, 229, 17, 248, 83, 17, 227, 193, 236, 95, 77, 104, 216, 48, 13, 6, 9, 42, 134, 72, 134, 247, 13, 1, 1, 11, 5, 0, 3, 130, 1, 1, 0, 172, 207, 15, 242, 173, 48, 23, 37, 26, 64, 119, 22, 168, 120, 196, 179, 56, 20, ... Gomega truncated this representation as it exceeds 'format.MaxLength'. Consider having the object provide a custom 'GomegaStringer' representation or adjust the parameters in Gomega's 'format' package. Learn more here: https://onsi.github.io/gomega/#adjusting-output occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_update.go:174]: Unexpected error: <errors.joinError \| 0xc000f463a8>: VerifyAllAPIServicesAvailable failed: failed to list all APIServices: Get "https://api.a9u9y9u2h3o8l6k.8pac.j3408384.hcp.osadev.cloud:443/apis/apiregistration.k8s.io/v1/apiservices": tls: failed to verify certificate: x509: certificate signed by unknown authority VerifyBasicAccess failed: failed to list services: Get "https://api.a9u9y9u2h3o8l6k.8pac.j3408384.hcp.osadev.cloud:443/api/v1/namespaces/default/services": tls: failed to verify certificate: x509: certificate signed by unknown authority { errs: [ <fmt.wrapError \| 0xc001082220>{ msg: "VerifyAllAPIServicesAvailable failed: failed to list all APIServices: Get \"https://api.a9u9y9u2h3o8l6k.8pac.j3408384.hcp.osadev.cloud:443/apis/apiregistration.k8s.io/v1/apiservices\": tls: failed to verify certificate: x509: certificate signed by unknown authority", err: <fmt.wrapError \| 0xc001082200>{ msg: "failed to list all APIServices: Get \"https://api.a9u9y9u2h3o8l6k.8pac.j3408384.hcp.osadev.cloud:443/apis/apiregistration.k8s.io/v1/apiservices\": tls: failed to verify certificate: x509: certificate signed by unknown authority", err: <url.Error \| 0xc001080660>{ Op: "Get", URL: "https://api.a9u9y9u2h3o8l6k.8pac.j3408384.hcp.osadev.cloud:443/apis/apiregistration.k8s.io/v1/apiservices", Err: <tls.CertificateVerificationError \| 0xc001080630>{ UnverifiedCertificates: [ { Raw: [48, 130, 3, 143, 48, 130, 2, 119, 160, 3, 2, 1, 2, 2, 16, 17, 97, 189, 215, 12, 9, 69, 140, 146, 242, 21, 222, 229, 48, 69, 241, 48, 13, 6, 9, 42, 134, 72, 134, 247, 13, 1, 1, 11, 5, 0, 48, 36, 49, 34, 48, 32, 6, 3, 85, 4, 3, 19, 25, 114, 101, 115, 101, 114, 118, 101, 100, 46, 104, 99, 112, 46, 111, 115, 97, 100, 101, 118, 46, 99, 108, 111, 117, 100, 48, 30, 23, 13, 50, 54, 48, 51, 49, 49, 49, 56, 52, 51, 48, 53, 90, 23, 13, 50, 54, 48, 57, 49, 49, 49, 56, 53, 51, 48, 53, 90, 48, 36, 49, 34, 48, 32, 6, 3, 85, 4, 3, 19, 25, 114, 101, 115, 101, 114, 118, 101, 100, 46, 104, 99, 112, 46, 111, 115, 97, 100, 101, 118, 46, 99, 108, 111, 117, 100, 48, 130, 1, 34, 48, 13, 6, 9, 42, 134, 72, 134, 247, 13, 1, 1, 1, 5, 0, 3, 130, 1, 15, 0, 48, 130, 1, 10, 2, 130, 1, 1, 0, 197, 66, 136, 134, 170, 50, 11, 185, 128, 94, 78, 252, 110, 146, 62, 200, 235, 253, 115, 212, 60, 238, 70, 223, 119, 167, 62, 146, 218, 1, 201, 77, 97, 230, 202, 163, 114, 91, 170, 201, 184, 80, 172, 106, 114, 139, 156, 50, 52, 135, 21, 218, 52, 217, 180, 220, 135, 226, 216, 99, 59, 99, 118, 38, 50, 197, 146, 121, 206, 176, 232, 74, 191, 211, 140, 42, 88, 88, 255, 171, 11, 243, 189, 233, 57, 217, 156, 193, 213, 115, 197, 127, 84, 172, 178, 207, 249, 184, 190, 162, 205, 112, 239, 220, 43, 142, 56, 101, 6, 14, 219, 41, 150, 232, 109, 24, 67, 73, 170, 217, 80, 252, 54, 161, 28, 169, 65, 57, 40, 198, 210, 140, 170, 189, 250, 116, 31, 71, 4, 15, 133, 145, 74, 95, 94, 174, 16, 206, 152, 230, 73, 191, 76, 129, 91, 152, 176, 58, 59, 191, 49, 28, 246, 32, 209, 251, 74, 25, 101, 212, 222, 236, 236, 83, 186, 63, 168, 253, 194, 155, 18, 141, 156, 208, 153, 141, 169, 189, 170, 230, 243, 200, 65, 63, 156, 38, 62, 90, 170, 21, 133, 113, 73, 240, 85, 193, 56, 166, 135, 137, 151, 33, 154, 174, 4, 54, 140, 236, 116, 205, 173, 218, 144, 217, 172, 111, 53, 181, 141, 92, 218, 198, 64, 164, 162, 26, 242, 196, 19, 76, 51, 1, 37, 81, 49, 24, 98, 161, 177, 70, 172, 255, 155, 73, 147, 221, 2, 3, 1, 0, 1, 163, 129, 188, 48, 129, 185, 48, 14, 6, 3, 85, 29, 15, 1, 1, 255, 4, 4, 3, 2, 5, 160, 48, 9, 6, 3, 85, 29, 19, 4, 2, 48, 0, 48, 29, 6, 3, 85, 29, 37, 4, 22, 48, 20, 6, 8, 43, 6, 1, 5, 5, 7, 3, 1, 6, 8, 43, 6, 1, 5, 5, 7, 3, 2, 48, 61, 6, 3, 85, 29, 17, 4, 54, 48, 52, 130, 50, 97, 112, 105, 46, 97, 57, 117, 57, 121, 57, 117, 50, 104, 51, 111, 56, 108, 54, 107, 46, 56, 112, 97, 99, 46, 106, 51, 52, 48, 56, 51, 56, 52, 46, 104, 99, 112, 46, 111, 115, 97, 100, 101, 118, 46, 99, 108, 111, 117, 100, 48, 31, 6, 3, 85, 29, 35, 4, 24, 48, 22, 128, 20, 200, 215, 22, 129, 41, 106, 120, 252, 229, 17, 248, 83, 17, 227, 193, 236, 95, 77, 104, 216, 48, 29, 6, 3, 85, 29, 14, 4, 22, 4, 20, 200, 215, 22, 129, 41, 106, 120, 252, 229, 17, 248, 83, 17, 227, 193, 236, 95, 77, 104, 216, 48, 13, 6, 9, 42, 134, 72, 134, 247, 13, 1, 1, 11, 5, 0, 3, 130, 1, 1, 0, 172, 207, 15, 242, 173, 48, 23, 37, 26, 64, 119, 22, 168, 120, 196, 179, 56, 20, ... Gomega truncated this representation as it exceeds 'format.MaxLength'. Consider having the object provide a custom 'GomegaStringer' representation or adjust the parameters in Gomega's 'format' package. Learn more here: https://onsi.github.io/gomega/#adjusting-output occurred cluster must have CertificateAuthorityData 17:55:23 UTC · failed at e2e · test Customer should be able to test admin credentials before cluster ready, then full admin credential lifecycle · suite rp-api-compat-all/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/admin_credential_lifecycle.go:216]: cluster must have CertificateAuthorityData Expected <[]uint8 \| len:0, cap:0>: nil not to be empty fail [github.com/Azure/ARO-HCP/test/e2e/admin_credential_lifecycle.go:216]: cluster must have CertificateAuthorityData Expected <[]uint8 \| len:0, cap:0>: nil not to be empty can't list nodes in the cluster: Get "<url> 17:55:23 UTC · failed at e2e · test Customer should be able to update nodepool replicas and autoscaling · suite rp-api-compat-all/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_update_nodes.go:134]: Expected success, but got an error: <fmt.wrapError \| 0xc0014d6b60>: can't list nodes in the cluster: Get "https://api.p0w3z1y5o0y0j1t.8vh4.j3408384.hcp.osadev.cloud:443/api/v1/nodes": tls: failed to verify certificate: x509: certificate signed by unknown authority { msg: "can't list nodes in the cluster: Get \"https://api.p0w3z1y5o0y0j1t.8vh4.j3408384.hcp.osadev.cloud:443/api/v1/nodes\": tls: failed to verify certificate: x509: certificate signed by unknown authority", err: <url.Error \| 0xc0009f9e60>{ Op: "Get", URL: "https://api.p0w3z1y5o0y0j1t.8vh4.j3408384.hcp.osadev.cloud:443/api/v1/nodes", Err: <tls.CertificateVerificationError \| 0xc0009f9ad0>{ UnverifiedCertificates: [ { Raw: [48, 130, 3, 143, 48, 130, 2, 119, 160, 3, 2, 1, 2, 2, 16, 60, 31, 24, 214, 46, 206, 75, 119, 175, 165, 76, 146, 43, 61, 225, 78, 48, 13, 6, 9, 42, 134, 72, 134, 247, 13, 1, 1, 11, 5, 0, 48, 36, 49, 34, 48, 32, 6, 3, 85, 4, 3, 19, 25, 114, 101, 115, 101, 114, 118, 101, 100, 46, 104, 99, 112, 46, 111, 115, 97, 100, 101, 118, 46, 99, 108, 111, 117, 100, 48, 30, 23, 13, 50, 54, 48, 51, 49, 49, 49, 56, 52, 51, 48, 53, 90, 23, 13, 50, 54, 48, 57, 49, 49, 49, 56, 53, 51, 48, 53, 90, 48, 36, 49, 34, 48, 32, 6, 3, 85, 4, 3, 19, 25, 114, 101, 115, 101, 114, 118, 101, 100, 46, 104, 99, 112, 46, 111, 115, 97, 100, 101, 118, 46, 99, 108, 111, 117, 100, 48, 130, 1, 34, 48, 13, 6, 9, 42, 134, 72, 134, 247, 13, 1, 1, 1, 5, 0, 3, 130, 1, 15, 0, 48, 130, 1, 10, 2, 130, 1, 1, 0, 217, 186, 89, 123, 218, 201, 53, 94, 92, 131, 150, 81, 117, 108, 249, 22, 43, 189, 216, 173, 6, 149, 66, 79, 145, 254, 150, 183, 235, 188, 251, 117, 55, 118, 144, 80, 106, 234, 99, 38, 63, 157, 25, 57, 12, 251, 65, 99, 136, 251, 155, 122, 25, 40, 107, 27, 222, 133, 147, 88, 226, 100, 8, 80, 161, 66, 171, 40, 95, 12, 109, 27, 198, 175, 59, 149, 90, 70, 55, 148, 246, 134, 249, 161, 192, 27, 177, 157, 15, 240, 239, 235, 126, 228, 173, 37, 188, 38, 187, 21, 42, 118, 204, 225, 140, 126, 243, 123, 25, 106, 11, 130, 10, 135, 148, 214, 243, 186, 245, 100, 21, 174, 61, 162, 78, 60, 0, 175, 56, 148, 44, 5, 46, 26, 99, 186, 254, 122, 225, 105, 218, 94, 201, 141, 246, 228, 23, 209, 40, 223, 138, 158, 110, 207, 16, 209, 109, 131, 210, 59, 140, 182, 120, 76, 53, 121, 243, 255, 154, 175, 113, 244, 211, 133, 208, 201, 244, 237, 248, 47, 38, 126, 212, 164, 206, 181, 219, 174, 202, 118, 32, 158, 107, 138, 124, 61, 73, 222, 114, 57, 124, 7, 179, 90, 175, 60, 134, 15, 185, 187, 159, 154, 108, 197, 82, 124, 166, 194, 83, 155, 171, 118, 21, 223, 57, 72, 128, 86, 95, 213, 77, 137, 180, 208, 44, 63, 190, 118, 220, 50, 163, 78, 227, 84, 210, 138, 118, 250, 40, 85, 221, 107, 76, 83, 7, 133, 2, 3, 1, 0, 1, 163, 129, 188, 48, 129, 185, 48, 14, 6, 3, 85, 29, 15, 1, 1, 255, 4, 4, 3, 2, 5, 160, 48, 9, 6, 3, 85, 29, 19, 4, 2, 48, 0, 48, 29, 6, 3, 85, 29, 37, 4, 22, 48, 20, 6, 8, 43, 6, 1, 5, 5, 7, 3, 1, 6, 8, 43, 6, 1, 5, 5, 7, 3, 2, 48, 61, 6, 3, 85, 29, 17, 4, 54, 48, 52, 130, 50, 97, 112, 105, 46, 112, 48, 119, 51, 122, 49, 121, 53, 111, 48, 121, 48, 106, 49, 116, 46, 56, 118, 104, 52, 46, 106, 51, 52, 48, 56, 51, 56, 52, 46, 104, 99, 112, 46, 111, 115, 97, 100, 101, 118, 46, 99, 108, 111, 117, 100, 48, 31, 6, 3, 85, 29, 35, 4, 24, 48, 22, 128, 20, 243, 201, 157, 138, 182, 5, 223, 220, 105, 6, 0, 62, 113, 55, 162, 155, 46, 141, 44, 4, 48, 29, 6, 3, 85, 29, 14, 4, 22, 4, 20, 243, 201, 157, 138, 182, 5, 223, 220, 105, 6, 0, 62, 113, 55, 162, 155, 46, 141, 44, 4, 48, 13, 6, 9, 42, 134, 72, 134, 247, 13, 1, 1, 11, 5, 0, 3, 130, 1, 1, 0, 127, 28, 24, 222, 156, 171, 68, 186, 210, 96, 86, 56, 131, 210, 244, 133, 8, 45, 198, 193, 47, 223, 125, 201, 217, 60, 14, 16, 45, 96, 107, 183, 184, 65, 76, 10, 15, 69, 96, 189, 14, 208, 35, 62, 1, 178, 121, 70, 98, 121, 27, 104, 148, 10, 54, 2, 165, 45, 82, 248, 13, 161, 166, 92, 197, 213, 1, 220, 32, 238, 205, 195, 236, 139, 98, 125, 200, 230, 76, 250, 154, 100, 65, 42, 154, 87, 26, 7, 193, 65, 111, 219, 175, 144, 136, 255, 139, 5, 166, 74, 100, 176, 242, 151, 182, 62, 86, 18, 29, 56, 98, 188, 59, 146, 96, 132, 8, 173, 242, 13, 216, 82, 156, 8, 61, 145, 109, 52, 96, 187, 224, 27, 35, 239, 227, 44, 20, 189, 37, 91, 184, 83, 245, 45, 94, 1... Gomega truncated this representation as it exceeds 'format.MaxLength'. Consider having the object provide a custom 'GomegaStringer' representation or adjust the parameters in Gomega's 'format' package. Learn more here: https://onsi.github.io/gomega/#adjusting-output fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_update_nodes.go:134]: Expected success, but got an error: <fmt.wrapError \| 0xc0014d6b60>: can't list nodes in the cluster: Get "https://api.p0w3z1y5o0y0j1t.8vh4.j3408384.hcp.osadev.cloud:443/api/v1/nodes": tls: failed to verify certificate: x509: certificate signed by unknown authority { msg: "can't list nodes in the cluster: Get \"https://api.p0w3z1y5o0y0j1t.8vh4.j3408384.hcp.osadev.cloud:443/api/v1/nodes\": tls: failed to verify certificate: x509: certificate signed by unknown authority", err: <url.Error \| 0xc0009f9e60>{ Op: "Get", URL: "https://api.p0w3z1y5o0y0j1t.8vh4.j3408384.hcp.osadev.cloud:443/api/v1/nodes", Err: <tls.CertificateVerificationError \| 0xc0009f9ad0>{ UnverifiedCertificates: [ { Raw: [48, 130, 3, 143, 48, 130, 2, 119, 160, 3, 2, 1, 2, 2, 16, 60, 31, 24, 214, 46, 206, 75, 119, 175, 165, 76, 146, 43, 61, 225, 78, 48, 13, 6, 9, 42, 134, 72, 134, 247, 13, 1, 1, 11, 5, 0, 48, 36, 49, 34, 48, 32, 6, 3, 85, 4, 3, 19, 25, 114, 101, 115, 101, 114, 118, 101, 100, 46, 104, 99, 112, 46, 111, 115, 97, 100, 101, 118, 46, 99, 108, 111, 117, 100, 48, 30, 23, 13, 50, 54, 48, 51, 49, 49, 49, 56, 52, 51, 48, 53, 90, 23, 13, 50, 54, 48, 57, 49, 49, 49, 56, 53, 51, 48, 53, 90, 48, 36, 49, 34, 48, 32, 6, 3, 85, 4, 3, 19, 25, 114, 101, 115, 101, 114, 118, 101, 100, 46, 104, 99, 112, 46, 111, 115, 97, 100, 101, 118, 46, 99, 108, 111, 117, 100, 48, 130, 1, 34, 48, 13, 6, 9, 42, 134, 72, 134, 247, 13, 1, 1, 1, 5, 0, 3, 130, 1, 15, 0, 48, 130, 1, 10, 2, 130, 1, 1, 0, 217, 186, 89, 123, 218, 201, 53, 94, 92, 131, 150, 81, 117, 108, 249, 22, 43, 189, 216, 173, 6, 149, 66, 79, 145, 254, 150, 183, 235, 188, 251, 117, 55, 118, 144, 80, 106, 234, 99, 38, 63, 157, 25, 57, 12, 251, 65, 99, 136, 251, 155, 122, 25, 40, 107, 27, 222, 133, 147, 88, 226, 100, 8, 80, 161, 66, 171, 40, 95, 12, 109, 27, 198, 175, 59, 149, 90, 70, 55, 148, 246, 134, 249, 161, 192, 27, 177, 157, 15, 240, 239, 235, 126, 228, 173, 37, 188, 38, 187, 21, 42, 118, 204, 225, 140, 126, 243, 123, 25, 106, 11, 130, 10, 135, 148, 214, 243, 186, 245, 100, 21, 174, 61, 162, 78, 60, 0, 175, 56, 148, 44, 5, 46, 26, 99, 186, 254, 122, 225, 105, 218, 94, 201, 141, 246, 228, 23, 209, 40, 223, 138, 158, 110, 207, 16, 209, 109, 131, 210, 59, 140, 182, 120, 76, 53, 121, 243, 255, 154, 175, 113, 244, 211, 133, 208, 201, 244, 237, 248, 47, 38, 126, 212, 164, 206, 181, 219, 174, 202, 118, 32, 158, 107, 138, 124, 61, 73, 222, 114, 57, 124, 7, 179, 90, 175, 60, 134, 15, 185, 187, 159, 154, 108, 197, 82, 124, 166, 194, 83, 155, 171, 118, 21, 223, 57, 72, 128, 86, 95, 213, 77, 137, 180, 208, 44, 63, 190, 118, 220, 50, 163, 78, 227, 84, 210, 138, 118, 250, 40, 85, 221, 107, 76, 83, 7, 133, 2, 3, 1, 0, 1, 163, 129, 188, 48, 129, 185, 48, 14, 6, 3, 85, 29, 15, 1, 1, 255, 4, 4, 3, 2, 5, 160, 48, 9, 6, 3, 85, 29, 19, 4, 2, 48, 0, 48, 29, 6, 3, 85, 29, 37, 4, 22, 48, 20, 6, 8, 43, 6, 1, 5, 5, 7, 3, 1, 6, 8, 43, 6, 1, 5, 5, 7, 3, 2, 48, 61, 6, 3, 85, 29, 17, 4, 54, 48, 52, 130, 50, 97, 112, 105, 46, 112, 48, 119, 51, 122, 49, 121, 53, 111, 48, 121, 48, 106, 49, 116, 46, 56, 118, 104, 52, 46, 106, 51, 52, 48, 56, 51, 56, 52, 46, 104, 99, 112, 46, 111, 115, 97, 100, 101, 118, 46, 99, 108, 111, 117, 100, 48, 31, 6, 3, 85, 29, 35, 4, 24, 48, 22, 128, 20, 243, 201, 157, 138, 182, 5, 223, 220, 105, 6, 0, 62, 113, 55, 162, 155, 46, 141, 44, 4, 48, 29, 6, 3, 85, 29, 14, 4, 22, 4, 20, 243, 201, 157, 138, 182, 5, 223, 220, 105, 6, 0, 62, 113, 55, 162, 155, 46, 141, 44, 4, 48, 13, 6, 9, 42, 134, 72, 134, 247, 13, 1, 1, 11, 5, 0, 3, 130, 1, 1, 0, 127, 28, 24, 222, 156, 171, 68, 186, 210, 96, 86, 56, 131, 210, 244, 133, 8, 45, 198, 193, 47, 223, 125, 201, 217, 60, 14, 16, 45, 96, 107, 183, 184, 65, 76, 10, 15, 69, 96, 189, 14, 208, 35, 62, 1, 178, 121, 70, 98, 121, 27, 104, 148, 10, 54, 2, 165, 45, 82, 248, 13, 161, 166, 92, 197, 213, 1, 220, 32, 238, 205, 195, 236, 139, 98, 125, 200, 230, 76, 250, 154, 100, 65, 42, 154, 87, 26, 7, 193, 65, 111, 219, 175, 144, 136, 255, 139, 5, 166, 74, 100, 176, 242, 151, 182, 62, 86, 18, 29, 56, 98, 188, 59, 146, 96, 132, 8, 173, 242, 13, 216, 82, 156, 8, 61, 145, 109, 52, 96, 187, 224, 27, 35, 239, 227, 44, 20, 189, 37, 91, 184, 83, 245, 45, 94, 1... Gomega truncated this representation as it exceeds 'format.MaxLength'. Consider having the object provide a custom 'GomegaStringer' representation or adjust the parameters in Gomega's 'format' package. Learn more here: https://onsi.github.io/gomega/#adjusting-output tls: failed to verify certificate: x509: certificate signed by unknown authority 17:55:23 UTC · failed at e2e · test Customer should be able to create an HCP cluster with back-level version 4.19 · suite rp-api-compat-all/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_version_backlevel.go:160]: Unexpected error: <errors.joinError \| 0xc000e26d98>: VerifyBasicAccess failed: failed to list services: Get "https://api.t3j0b8c9z2o2c6w.lfrh.j3408384.hcp.osadev.cloud:443/api/v1/namespaces/default/services": tls: failed to verify certificate: x509: certificate signed by unknown authority VerifyAllAPIServicesAvailable failed: failed to list all APIServices: Get "https://api.t3j0b8c9z2o2c6w.lfrh.j3408384.hcp.osadev.cloud:443/apis/apiregistration.k8s.io/v1/apiservices": tls: failed to verify certificate: x509: certificate signed by unknown authority { errs: [ <fmt.wrapError \| 0xc00076f940>{ msg: "VerifyBasicAccess failed: failed to list services: Get \"https://api.t3j0b8c9z2o2c6w.lfrh.j3408384.hcp.osadev.cloud:443/api/v1/namespaces/default/services\": tls: failed to verify certificate: x509: certificate signed by unknown authority", err: <fmt.wrapError \| 0xc00076f8c0>{ msg: "failed to list services: Get \"https://api.t3j0b8c9z2o2c6w.lfrh.j3408384.hcp.osadev.cloud:443/api/v1/namespaces/default/services\": tls: failed to verify certificate: x509: certificate signed by unknown authority", err: <url.Error \| 0xc00140ce10>{ Op: "Get", URL: "https://api.t3j0b8c9z2o2c6w.lfrh.j3408384.hcp.osadev.cloud:443/api/v1/namespaces/default/services", Err: <tls.CertificateVerificationError \| 0xc00140cde0>{ UnverifiedCertificates: [ { Raw: [48, 130, 3, 143, 48, 130, 2, 119, 160, 3, 2, 1, 2, 2, 16, 40, 172, 242, 74, 36, 249, 68, 133, 130, 96, 49, 251, 137, 66, 85, 80, 48, 13, 6, 9, 42, 134, 72, 134, 247, 13, 1, 1, 11, 5, 0, 48, 36, 49, 34, 48, 32, 6, 3, 85, 4, 3, 19, 25, 114, 101, 115, 101, 114, 118, 101, 100, 46, 104, 99, 112, 46, 111, 115, 97, 100, 101, 118, 46, 99, 108, 111, 117, 100, 48, 30, 23, 13, 50, 54, 48, 51, 49, 49, 49, 56, 52, 51, 50, 53, 90, 23, 13, 50, 54, 48, 57, 49, 49, 49, 56, 53, 51, 50, 53, 90, 48, 36, 49, 34, 48, 32, 6, 3, 85, 4, 3, 19, 25, 114, 101, 115, 101, 114, 118, 101, 100, 46, 104, 99, 112, 46, 111, 115, 97, 100, 101, 118, 46, 99, 108, 111, 117, 100, 48, 130, 1, 34, 48, 13, 6, 9, 42, 134, 72, 134, 247, 13, 1, 1, 1, 5, 0, 3, 130, 1, 15, 0, 48, 130, 1, 10, 2, 130, 1, 1, 0, 187, 64, 157, 58, 32, 94, 142, 72, 104, 247, 105, 189, 188, 133, 177, 253, 177, 177, 129, 81, 18, 196, 88, 11, 83, 31, 112, 118, 103, 87, 179, 89, 116, 151, 242, 18, 200, 144, 97, 176, 165, 239, 18, 142, 42, 194, 11, 10, 223, 132, 254, 146, 101, 177, 192, 23, 58, 194, 80, 174, 202, 217, 108, 79, 195, 112, 157, 127, 157, 155, 12, 10, 27, 190, 172, 126, 48, 133, 141, 101, 30, 124, 147, 112, 34, 250, 186, 192, 127, 223, 179, 71, 192, 39, 191, 193, 10, 18, 218, 29, 125, 55, 173, 40, 132, 16, 255, 200, 169, 199, 222, 204, 190, 75, 121, 130, 193, 26, 242, 227, 104, 241, 235, 144, 241, 25, 186, 17, 253, 129, 234, 4, 107, 90, 80, 139, 88, 87, 207, 96, 50, 100, 113, 121, 171, 20, 71, 180, 211, 97, 254, 104, 171, 7, 210, 251, 2, 229, 144, 165, 141, 150, 211, 143, 11, 90, 108, 70, 140, 84, 112, 146, 229, 194, 44, 229, 28, 220, 85, 32, 53, 36, 132, 253, 68, 189, 158, 88, 203, 154, 44, 220, 46, 113, 192, 158, 151, 47, 201, 161, 186, 105, 146, 31, 53, 56, 184, 149, 124, 13, 94, 229, 98, 183, 86, 208, 91, 28, 221, 28, 174, 150, 97, 174, 112, 222, 86, 150, 178, 137, 182, 85, 80, 32, 205, 185, 131, 77, 44, 61, 79, 175, 97, 244, 122, 147, 120, 191, 161, 142, 114, 255, 72, 157, 186, 117, 2, 3, 1, 0, 1, 163, 129, 188, 48, 129, 185, 48, 14, 6, 3, 85, 29, 15, 1, 1, 255, 4, 4, 3, 2, 5, 160, 48, 9, 6, 3, 85, 29, 19, 4, 2, 48, 0, 48, 29, 6, 3, 85, 29, 37, 4, 22, 48, 20, 6, 8, 43, 6, 1, 5, 5, 7, 3, 1, 6, 8, 43, 6, 1, 5, 5, 7, 3, 2, 48, 61, 6, 3, 85, 29, 17, 4, 54, 48, 52, 130, 50, 97, 112, 105, 46, 116, 51, 106, 48, 98, 56, 99, 57, 122, 50, 111, 50, 99, 54, 119, 46, 108, 102, 114, 104, 46, 106, 51, 52, 48, 56, 51, 56, 52, 46, 104, 99, 112, 46, 111, 115, 97, 100, 101, 118, 46, 99, 108, 111, 117, 100, 48, 31, 6, 3, 85, 29, 35, 4, 24, 48, 22, 128, 20, 97, 49, 248, 226, 248, 166, 141, 165, 42, 239, 167, 238, 41, 153, 21, 2, 119, 146, 172, 155, 48, 29, 6, 3, 85, 29, 14, 4, 22, 4, 20, 97, 49, 248, 226, 248, 166, 141, 165, 42, 239, 167, 238, 41, 153, 21, 2, 119, 146, 172, 155, 48, 13, 6, 9, 42, 134, 72, 134, 247, 13, 1, 1, 11, 5, 0, 3, 130, 1, 1, 0, 107, 114, 185, 188, 231, 204, 151, 246, 90, 222, 161, 231, 44, 227, 250, 111, 231, 177, 158, 211, 158, 4, 200, 24, 79, 207, 2... Gomega truncated this representation as it exceeds 'format.MaxLength'. Consider having the object provide a custom 'GomegaStringer' representation or adjust the parameters in Gomega's 'format' package. Learn more here: https://onsi.github.io/gomega/#adjusting-output occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_version_backlevel.go:160]: Unexpected error: <errors.joinError \| 0xc000e26d98>: VerifyBasicAccess failed: failed to list services: Get "https://api.t3j0b8c9z2o2c6w.lfrh.j3408384.hcp.osadev.cloud:443/api/v1/namespaces/default/services": tls: failed to verify certificate: x509: certificate signed by unknown authority VerifyAllAPIServicesAvailable failed: failed to list all APIServices: Get "https://api.t3j0b8c9z2o2c6w.lfrh.j3408384.hcp.osadev.cloud:443/apis/apiregistration.k8s.io/v1/apiservices": tls: failed to verify certificate: x509: certificate signed by unknown authority { errs: [ <fmt.wrapError \| 0xc00076f940>{ msg: "VerifyBasicAccess failed: failed to list services: Get \"https://api.t3j0b8c9z2o2c6w.lfrh.j3408384.hcp.osadev.cloud:443/api/v1/namespaces/default/services\": tls: failed to verify certificate: x509: certificate signed by unknown authority", err: <fmt.wrapError \| 0xc00076f8c0>{ msg: "failed to list services: Get \"https://api.t3j0b8c9z2o2c6w.lfrh.j3408384.hcp.osadev.cloud:443/api/v1/namespaces/default/services\": tls: failed to verify certificate: x509: certificate signed by unknown authority", err: <url.Error \| 0xc00140ce10>{ Op: "Get", URL: "https://api.t3j0b8c9z2o2c6w.lfrh.j3408384.hcp.osadev.cloud:443/api/v1/namespaces/default/services", Err: <tls.CertificateVerificationError \| 0xc00140cde0>{ UnverifiedCertificates: [ { Raw: [48, 130, 3, 143, 48, 130, 2, 119, 160, 3, 2, 1, 2, 2, 16, 40, 172, 242, 74, 36, 249, 68, 133, 130, 96, 49, 251, 137, 66, 85, 80, 48, 13, 6, 9, 42, 134, 72, 134, 247, 13, 1, 1, 11, 5, 0, 48, 36, 49, 34, 48, 32, 6, 3, 85, 4, 3, 19, 25, 114, 101, 115, 101, 114, 118, 101, 100, 46, 104, 99, 112, 46, 111, 115, 97, 100, 101, 118, 46, 99, 108, 111, 117, 100, 48, 30, 23, 13, 50, 54, 48, 51, 49, 49, 49, 56, 52, 51, 50, 53, 90, 23, 13, 50, 54, 48, 57, 49, 49, 49, 56, 53, 51, 50, 53, 90, 48, 36, 49, 34, 48, 32, 6, 3, 85, 4, 3, 19, 25, 114, 101, 115, 101, 114, 118, 101, 100, 46, 104, 99, 112, 46, 111, 115, 97, 100, 101, 118, 46, 99, 108, 111, 117, 100, 48, 130, 1, 34, 48, 13, 6, 9, 42, 134, 72, 134, 247, 13, 1, 1, 1, 5, 0, 3, 130, 1, 15, 0, 48, 130, 1, 10, 2, 130, 1, 1, 0, 187, 64, 157, 58, 32, 94, 142, 72, 104, 247, 105, 189, 188, 133, 177, 253, 177, 177, 129, 81, 18, 196, 88, 11, 83, 31, 112, 118, 103, 87, 179, 89, 116, 151, 242, 18, 200, 144, 97, 176, 165, 239, 18, 142, 42, 194, 11, 10, 223, 132, 254, 146, 101, 177, 192, 23, 58, 194, 80, 174, 202, 217, 108, 79, 195, 112, 157, 127, 157, 155, 12, 10, 27, 190, 172, 126, 48, 133, 141, 101, 30, 124, 147, 112, 34, 250, 186, 192, 127, 223, 179, 71, 192, 39, 191, 193, 10, 18, 218, 29, 125, 55, 173, 40, 132, 16, 255, 200, 169, 199, 222, 204, 190, 75, 121, 130, 193, 26, 242, 227, 104, 241, 235, 144, 241, 25, 186, 17, 253, 129, 234, 4, 107, 90, 80, 139, 88, 87, 207, 96, 50, 100, 113, 121, 171, 20, 71, 180, 211, 97, 254, 104, 171, 7, 210, 251, 2, 229, 144, 165, 141, 150, 211, 143, 11, 90, 108, 70, 140, 84, 112, 146, 229, 194, 44, 229, 28, 220, 85, 32, 53, 36, 132, 253, 68, 189, 158, 88, 203, 154, 44, 220, 46, 113, 192, 158, 151, 47, 201, 161, 186, 105, 146, 31, 53, 56, 184, 149, 124, 13, 94, 229, 98, 183, 86, 208, 91, 28, 221, 28, 174, 150, 97, 174, 112, 222, 86, 150, 178, 137, 182, 85, 80, 32, 205, 185, 131, 77, 44, 61, 79, 175, 97, 244, 122, 147, 120, 191, 161, 142, 114, 255, 72, 157, 186, 117, 2, 3, 1, 0, 1, 163, 129, 188, 48, 129, 185, 48, 14, 6, 3, 85, 29, 15, 1, 1, 255, 4, 4, 3, 2, 5, 160, 48, 9, 6, 3, 85, 29, 19, 4, 2, 48, 0, 48, 29, 6, 3, 85, 29, 37, 4, 22, 48, 20, 6, 8, 43, 6, 1, 5, 5, 7, 3, 1, 6, 8, 43, 6, 1, 5, 5, 7, 3, 2, 48, 61, 6, 3, 85, 29, 17, 4, 54, 48, 52, 130, 50, 97, 112, 105, 46, 116, 51, 106, 48, 98, 56, 99, 57, 122, 50, 111, 50, 99, 54, 119, 46, 108, 102, 114, 104, 46, 106, 51, 52, 48, 56, 51, 56, 52, 46, 104, 99, 112, 46, 111, 115, 97, 100, 101, 118, 46, 99, 108, 111, 117, 100, 48, 31, 6, 3, 85, 29, 35, 4, 24, 48, 22, 128, 20, 97, 49, 248, 226, 248, 166, 141, 165, 42, 239, 167, 238, 41, 153, 21, 2, 119, 146, 172, 155, 48, 29, 6, 3, 85, 29, 14, 4, 22, 4, 20, 97, 49, 248, 226, 248, 166, 141, 165, 42, 239, 167, 238, 41, 153, 21, 2, 119, 146, 172, 155, 48, 13, 6, 9, 42, 134, 72, 134, 247, 13, 1, 1, 11, 5, 0, 3, 130, 1, 1, 0, 107, 114, 185, 188, 231, 204, 151, 246, 90, 222, 161, 231, 44, 227, 250, 111, 231, 177, 158, 211, 158, 4, 200, 24, 79, 207, 2... Gomega truncated this representation as it exceeds 'format.MaxLength'. Consider having the object provide a custom 'GomegaStringer' representation or adjust the parameters in Gomega's 'format' package. Learn more here: https://onsi.github.io/gomega/#adjusting-output occurred tls: failed to verify certificate: x509: certificate signed by unknown authority 17:55:23 UTC · failed at e2e · test Customer should be able to create an HCP cluster with Image Registry not present · suite rp-api-compat-all/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/image_registry_cluster_create.go:108]: Unexpected error: <errors.joinError \| 0xc001488b58>: VerifyAllAPIServicesAvailable failed: failed to list all APIServices: Get "https://api.t9o0k0i4z9t0h5k.hbwf.j3408384.hcp.osadev.cloud:443/apis/apiregistration.k8s.io/v1/apiservices": tls: failed to verify certificate: x509: certificate signed by unknown authority VerifyBasicAccess failed: failed to list services: Get "https://api.t9o0k0i4z9t0h5k.hbwf.j3408384.hcp.osadev.cloud:443/api/v1/namespaces/default/services": tls: failed to verify certificate: x509: certificate signed by unknown authority VerifyImageRegistryDisabled failed: wrong type of error: url.Error, Get "https://api.t9o0k0i4z9t0h5k.hbwf.j3408384.hcp.osadev.cloud:443/api/v1/namespaces/openshift-image-registry/services/image-registry": tls: failed to verify certificate: x509: certificate signed by unknown authority { errs: [ <fmt.wrapError \| 0xc0016153c0>{ msg: "VerifyAllAPIServicesAvailable failed: failed to list all APIServices: Get \"https://api.t9o0k0i4z9t0h5k.hbwf.j3408384.hcp.osadev.cloud:443/apis/apiregistration.k8s.io/v1/apiservices\": tls: failed to verify certificate: x509: certificate signed by unknown authority", err: <fmt.wrapError \| 0xc0016153a0>{ msg: "failed to list all APIServices: Get \"https://api.t9o0k0i4z9t0h5k.hbwf.j3408384.hcp.osadev.cloud:443/apis/apiregistration.k8s.io/v1/apiservices\": tls: failed to verify certificate: x509: certificate signed by unknown authority", err: <url.Error \| 0xc001496780>{ Op: "Get", URL: "https://api.t9o0k0i4z9t0h5k.hbwf.j3408384.hcp.osadev.cloud:443/apis/apiregistration.k8s.io/v1/apiservices", Err: <tls.CertificateVerificationError \| 0xc001496750>{ UnverifiedCertificates: [ { Raw: [48, 130, 3, 143, 48, 130, 2, 119, 160, 3, 2, 1, 2, 2, 16, 96, 69, 88, 85, 100, 71, 68, 198, 148, 141, 21, 75, 38, 119, 140, 156, 48, 13, 6, 9, 42, 134, 72, 134, 247, 13, 1, 1, 11, 5, 0, 48, 36, 49, 34, 48, 32, 6, 3, 85, 4, 3, 19, 25, 114, 101, 115, 101, 114, 118, 101, 100, 46, 104, 99, 112, 46, 111, 115, 97, 100, 101, 118, 46, 99, 108, 111, 117, 100, 48, 30, 23, 13, 50, 54, 48, 51, 49, 49, 49, 56, 52, 51, 49, 53, 90, 23, 13, 50, 54, 48, 57, 49, 49, 49, 56, 53, 51, 49, 53, 90, 48, 36, 49, 34, 48, 32, 6, 3, 85, 4, 3, 19, 25, 114, 101, 115, 101, 114, 118, 101, 100, 46, 104, 99, 112, 46, 111, 115, 97, 100, 101, 118, 46, 99, 108, 111, 117, 100, 48, 130, 1, 34, 48, 13, 6, 9, 42, 134, 72, 134, 247, 13, 1, 1, 1, 5, 0, 3, 130, 1, 15, 0, 48, 130, 1, 10, 2, 130, 1, 1, 0, 227, 77, 160, 89, 157, 206, 16, 255, 0, 34, 47, 156, 199, 165, 38, 0, 73, 140, 145, 4, 26, 159, 169, 27, 95, 42, 89, 231, 88, 127, 238, 81, 204, 53, 141, 62, 159, 153, 235, 15, 250, 143, 10, 223, 103, 61, 93, 203, 153, 99, 182, 70, 251, 215, 243, 25, 15, 183, 235, 86, 83, 243, 234, 49, 122, 26, 217, 202, 219, 90, 123, 85, 181, 31, 123, 14, 109, 240, 55, 89, 143, 249, 170, 228, 245, 184, 18, 3, 44, 188, 55, 3, 224, 193, 88, 201, 176, 250, 127, 105, 137, 143, 22, 145, 231, 247, 65, 141, 121, 166, 12, 151, 213, 71, 209, 101, 12, 204, 100, 62, 68, 217, 65, 117, 234, 139, 103, 0, 253, 191, 6, 185, 230, 55, 55, 238, 238, 33, 211, 81, 87, 183, 203, 101, 223, 188, 209, 27, 47, 14, 215, 101, 180, 70, 147, 114, 103, 137, 172, 152, 197, 185, 84, 17, 189, 73, 62, 51, 242, 11, 147, 182, 230, 26, 247, 248, 176, 84, 248, 24, 69, 136, 183, 12, 34, 190, 8, 252, 86, 92, 68, 50, 60, 244, 128, 196, 21, 45, 28, 39, 56, 43, 89, 98, 127, 185, 136, 1, 181, 7, 180, 134, 253, 83, 58, 201, 124, 151, 199, 172, 34, 183, 230, 38, 136, 64, 134, 32, 50, 241, 2, 32, 243, 120, 209, 240, 255, 59, 203, 232, 40, 68, 56, 12, 171, 8, 68, 180, 95, 175, 72, 68, 203, 182, 186, 221, 2, 3, 1, 0, 1, 163, 129, 188, 48, 129, 185, 48, 14, 6, 3, 85, 29, 15, 1, 1, 255, 4, 4, 3, 2, 5, 160, 48, 9, 6, 3, 85, 29, 19, 4, 2, 48, 0, 48, 29, 6, 3, 85, 29, 37, 4, 22, 48, 20, 6, 8, 43, 6, 1, 5, 5, 7, 3, 1, 6, 8, 43, 6, 1, 5, 5, 7, 3, 2, 48, 61, 6, 3, 85, 29, 17, 4, 54, 48, 52, 130, 50, 97, 112, 105, 46, 116, 57, 111, 48, 107, 48, 105, 52, 122, 57, 116, 48, 104, 53, 107, 46, 104, 98, 119, 102, 46, 106, 51, 52, 48, 56, 51, 56, 52, 46, 104, 99, 112, 46, 111, 115, 97, 100, 101, 118, 46, 99, 108, 111, 117, 100, 48, 31, 6, 3, 85, 29, 35, 4, 24, 48, 22, 128, 20, 78, 126, 137, 46, 12, 95, 43, 216, 93, 234, 77, 250, 203, 95, 254, 67, 50, 75, 134, 92, 48, 29, 6, 3, 85, 29, 14, 4, 22, 4, 20, 78, 126, 137, 46, 12, 95, 43, 216, 93, 234, 77, 250, 203, 95, 254, 67, 50, 75, 134, 92, 48, 13, 6, 9, 42, 134, 72, 134, 247, 13, 1, 1, 11, 5, 0, 3, 130, 1, 1, 0, 104, 68, 254, 156, 30, 123, 98, 156, 149, 49, 106, 239, 181, 234, 61, 247, 123, 53, 127, 76, 70, 139, 217, 167, ... Gomega truncated this representation as it exceeds 'format.MaxLength'. Consider having the object provide a custom 'GomegaStringer' representation or adjust the parameters in Gomega's 'format' package. Learn more here: https://onsi.github.io/gomega/#adjusting-output occurred fail [github.com/Azure/ARO-HCP/test/e2e/image_registry_cluster_create.go:108]: Unexpected error: <errors.joinError \| 0xc001488b58>: VerifyAllAPIServicesAvailable failed: failed to list all APIServices: Get "https://api.t9o0k0i4z9t0h5k.hbwf.j3408384.hcp.osadev.cloud:443/apis/apiregistration.k8s.io/v1/apiservices": tls: failed to verify certificate: x509: certificate signed by unknown authority VerifyBasicAccess failed: failed to list services: Get "https://api.t9o0k0i4z9t0h5k.hbwf.j3408384.hcp.osadev.cloud:443/api/v1/namespaces/default/services": tls: failed to verify certificate: x509: certificate signed by unknown authority VerifyImageRegistryDisabled failed: wrong type of error: url.Error, Get "https://api.t9o0k0i4z9t0h5k.hbwf.j3408384.hcp.osadev.cloud:443/api/v1/namespaces/openshift-image-registry/services/image-registry": tls: failed to verify certificate: x509: certificate signed by unknown authority { errs: [ <fmt.wrapError \| 0xc0016153c0>{ msg: "VerifyAllAPIServicesAvailable failed: failed to list all APIServices: Get \"https://api.t9o0k0i4z9t0h5k.hbwf.j3408384.hcp.osadev.cloud:443/apis/apiregistration.k8s.io/v1/apiservices\": tls: failed to verify certificate: x509: certificate signed by unknown authority", err: <fmt.wrapError \| 0xc0016153a0>{ msg: "failed to list all APIServices: Get \"https://api.t9o0k0i4z9t0h5k.hbwf.j3408384.hcp.osadev.cloud:443/apis/apiregistration.k8s.io/v1/apiservices\": tls: failed to verify certificate: x509: certificate signed by unknown authority", err: <url.Error \| 0xc001496780>{ Op: "Get", URL: "https://api.t9o0k0i4z9t0h5k.hbwf.j3408384.hcp.osadev.cloud:443/apis/apiregistration.k8s.io/v1/apiservices", Err: <tls.CertificateVerificationError \| 0xc001496750>{ UnverifiedCertificates: [ { Raw: [48, 130, 3, 143, 48, 130, 2, 119, 160, 3, 2, 1, 2, 2, 16, 96, 69, 88, 85, 100, 71, 68, 198, 148, 141, 21, 75, 38, 119, 140, 156, 48, 13, 6, 9, 42, 134, 72, 134, 247, 13, 1, 1, 11, 5, 0, 48, 36, 49, 34, 48, 32, 6, 3, 85, 4, 3, 19, 25, 114, 101, 115, 101, 114, 118, 101, 100, 46, 104, 99, 112, 46, 111, 115, 97, 100, 101, 118, 46, 99, 108, 111, 117, 100, 48, 30, 23, 13, 50, 54, 48, 51, 49, 49, 49, 56, 52, 51, 49, 53, 90, 23, 13, 50, 54, 48, 57, 49, 49, 49, 56, 53, 51, 49, 53, 90, 48, 36, 49, 34, 48, 32, 6, 3, 85, 4, 3, 19, 25, 114, 101, 115, 101, 114, 118, 101, 100, 46, 104, 99, 112, 46, 111, 115, 97, 100, 101, 118, 46, 99, 108, 111, 117, 100, 48, 130, 1, 34, 48, 13, 6, 9, 42, 134, 72, 134, 247, 13, 1, 1, 1, 5, 0, 3, 130, 1, 15, 0, 48, 130, 1, 10, 2, 130, 1, 1, 0, 227, 77, 160, 89, 157, 206, 16, 255, 0, 34, 47, 156, 199, 165, 38, 0, 73, 140, 145, 4, 26, 159, 169, 27, 95, 42, 89, 231, 88, 127, 238, 81, 204, 53, 141, 62, 159, 153, 235, 15, 250, 143, 10, 223, 103, 61, 93, 203, 153, 99, 182, 70, 251, 215, 243, 25, 15, 183, 235, 86, 83, 243, 234, 49, 122, 26, 217, 202, 219, 90, 123, 85, 181, 31, 123, 14, 109, 240, 55, 89, 143, 249, 170, 228, 245, 184, 18, 3, 44, 188, 55, 3, 224, 193, 88, 201, 176, 250, 127, 105, 137, 143, 22, 145, 231, 247, 65, 141, 121, 166, 12, 151, 213, 71, 209, 101, 12, 204, 100, 62, 68, 217, 65, 117, 234, 139, 103, 0, 253, 191, 6, 185, 230, 55, 55, 238, 238, 33, 211, 81, 87, 183, 203, 101, 223, 188, 209, 27, 47, 14, 215, 101, 180, 70, 147, 114, 103, 137, 172, 152, 197, 185, 84, 17, 189, 73, 62, 51, 242, 11, 147, 182, 230, 26, 247, 248, 176, 84, 248, 24, 69, 136, 183, 12, 34, 190, 8, 252, 86, 92, 68, 50, 60, 244, 128, 196, 21, 45, 28, 39, 56, 43, 89, 98, 127, 185, 136, 1, 181, 7, 180, 134, 253, 83, 58, 201, 124, 151, 199, 172, 34, 183, 230, 38, 136, 64, 134, 32, 50, 241, 2, 32, 243, 120, 209, 240, 255, 59, 203, 232, 40, 68, 56, 12, 171, 8, 68, 180, 95, 175, 72, 68, 203, 182, 186, 221, 2, 3, 1, 0, 1, 163, 129, 188, 48, 129, 185, 48, 14, 6, 3, 85, 29, 15, 1, 1, 255, 4, 4, 3, 2, 5, 160, 48, 9, 6, 3, 85, 29, 19, 4, 2, 48, 0, 48, 29, 6, 3, 85, 29, 37, 4, 22, 48, 20, 6, 8, 43, 6, 1, 5, 5, 7, 3, 1, 6, 8, 43, 6, 1, 5, 5, 7, 3, 2, 48, 61, 6, 3, 85, 29, 17, 4, 54, 48, 52, 130, 50, 97, 112, 105, 46, 116, 57, 111, 48, 107, 48, 105, 52, 122, 57, 116, 48, 104, 53, 107, 46, 104, 98, 119, 102, 46, 106, 51, 52, 48, 56, 51, 56, 52, 46, 104, 99, 112, 46, 111, 115, 97, 100, 101, 118, 46, 99, 108, 111, 117, 100, 48, 31, 6, 3, 85, 29, 35, 4, 24, 48, 22, 128, 20, 78, 126, 137, 46, 12, 95, 43, 216, 93, 234, 77, 250, 203, 95, 254, 67, 50, 75, 134, 92, 48, 29, 6, 3, 85, 29, 14, 4, 22, 4, 20, 78, 126, 137, 46, 12, 95, 43, 216, 93, 234, 77, 250, 203, 95, 254, 67, 50, 75, 134, 92, 48, 13, 6, 9, 42, 134, 72, 134, 247, 13, 1, 1, 11, 5, 0, 3, 130, 1, 1, 0, 104, 68, 254, 156, 30, 123, 98, 156, 149, 49, 106, 239, 181, 234, 61, 247, 123, 53, 127, 76, 70, 139, 217, 167, ... Gomega truncated this representation as it exceeds 'format.MaxLength'. Consider having the object provide a custom 'GomegaStringer' representation or adjust the parameters in Gomega's 'format' package. Learn more here: https://onsi.github.io/gomega/#adjusting-output occurred tls: failed to verify certificate: x509: certificate signed by unknown authority 17:55:23 UTC · failed at e2e · test Customer should be able to create an HCP cluster and custom node pool osDisk size · suite rp-api-compat-all/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_nodepool_osdisk.go:108]: Unexpected error: <errors.joinError \| 0xc000c8cfc0>: VerifyBasicAccess failed: failed to list services: Get "https://api.v2b6j7t3i0i0k0u.cf87.j3408384.hcp.osadev.cloud:443/api/v1/namespaces/default/services": tls: failed to verify certificate: x509: certificate signed by unknown authority VerifyAllAPIServicesAvailable failed: failed to list all APIServices: Get "https://api.v2b6j7t3i0i0k0u.cf87.j3408384.hcp.osadev.cloud:443/apis/apiregistration.k8s.io/v1/apiservices": tls: failed to verify certificate: x509: certificate signed by unknown authority { errs: [ <fmt.wrapError \| 0xc0004dbce0>{ msg: "VerifyBasicAccess failed: failed to list services: Get \"https://api.v2b6j7t3i0i0k0u.cf87.j3408384.hcp.osadev.cloud:443/api/v1/namespaces/default/services\": tls: failed to verify certificate: x509: certificate signed by unknown authority", err: <fmt.wrapError \| 0xc0004dbcc0>{ msg: "failed to list services: Get \"https://api.v2b6j7t3i0i0k0u.cf87.j3408384.hcp.osadev.cloud:443/api/v1/namespaces/default/services\": tls: failed to verify certificate: x509: certificate signed by unknown authority", err: <url.Error \| 0xc00156a8d0>{ Op: "Get", URL: "https://api.v2b6j7t3i0i0k0u.cf87.j3408384.hcp.osadev.cloud:443/api/v1/namespaces/default/services", Err: <tls.CertificateVerificationError \| 0xc00156a8a0>{ UnverifiedCertificates: [ { Raw: [48, 130, 3, 143, 48, 130, 2, 119, 160, 3, 2, 1, 2, 2, 16, 67, 5, 166, 237, 225, 226, 68, 194, 163, 198, 86, 165, 116, 45, 192, 109, 48, 13, 6, 9, 42, 134, 72, 134, 247, 13, 1, 1, 11, 5, 0, 48, 36, 49, 34, 48, 32, 6, 3, 85, 4, 3, 19, 25, 114, 101, 115, 101, 114, 118, 101, 100, 46, 104, 99, 112, 46, 111, 115, 97, 100, 101, 118, 46, 99, 108, 111, 117, 100, 48, 30, 23, 13, 50, 54, 48, 51, 49, 49, 49, 57, 48, 48, 52, 54, 90, 23, 13, 50, 54, 48, 57, 49, 49, 49, 57, 49, 48, 52, 54, 90, 48, 36, 49, 34, 48, 32, 6, 3, 85, 4, 3, 19, 25, 114, 101, 115, 101, 114, 118, 101, 100, 46, 104, 99, 112, 46, 111, 115, 97, 100, 101, 118, 46, 99, 108, 111, 117, 100, 48, 130, 1, 34, 48, 13, 6, 9, 42, 134, 72, 134, 247, 13, 1, 1, 1, 5, 0, 3, 130, 1, 15, 0, 48, 130, 1, 10, 2, 130, 1, 1, 0, 176, 205, 240, 241, 16, 53, 250, 250, 229, 234, 163, 179, 1, 180, 44, 212, 205, 69, 46, 248, 6, 98, 81, 118, 98, 45, 151, 230, 28, 235, 224, 209, 35, 57, 12, 205, 218, 35, 104, 117, 163, 167, 26, 143, 191, 115, 209, 206, 2, 216, 144, 137, 80, 177, 109, 24, 75, 102, 150, 13, 76, 195, 33, 69, 222, 250, 17, 214, 233, 242, 69, 166, 233, 99, 19, 31, 35, 69, 174, 247, 119, 15, 18, 10, 38, 133, 214, 203, 236, 224, 254, 7, 228, 235, 119, 106, 19, 157, 73, 191, 225, 15, 122, 90, 78, 115, 63, 144, 40, 208, 59, 93, 226, 22, 210, 155, 117, 182, 142, 177, 46, 125, 209, 120, 106, 135, 215, 60, 7, 137, 82, 63, 147, 88, 89, 22, 43, 32, 22, 2, 98, 4, 186, 142, 2, 237, 156, 81, 41, 150, 241, 6, 92, 176, 123, 203, 96, 122, 20, 46, 23, 22, 216, 138, 36, 128, 154, 174, 113, 223, 131, 210, 152, 215, 249, 244, 33, 108, 126, 177, 80, 128, 72, 214, 228, 3, 211, 18, 212, 179, 112, 140, 158, 6, 73, 9, 86, 108, 181, 27, 237, 237, 167, 172, 163, 202, 182, 39, 23, 53, 66, 209, 247, 101, 45, 162, 15, 154, 145, 184, 57, 61, 183, 16, 105, 214, 116, 230, 234, 224, 255, 211, 41, 85, 125, 216, 252, 191, 124, 59, 138, 205, 117, 130, 226, 102, 150, 174, 80, 56, 91, 167, 43, 124, 62, 237, 2, 3, 1, 0, 1, 163, 129, 188, 48, 129, 185, 48, 14, 6, 3, 85, 29, 15, 1, 1, 255, 4, 4, 3, 2, 5, 160, 48, 9, 6, 3, 85, 29, 19, 4, 2, 48, 0, 48, 29, 6, 3, 85, 29, 37, 4, 22, 48, 20, 6, 8, 43, 6, 1, 5, 5, 7, 3, 1, 6, 8, 43, 6, 1, 5, 5, 7, 3, 2, 48, 61, 6, 3, 85, 29, 17, 4, 54, 48, 52, 130, 50, 97, 112, 105, 46, 118, 50, 98, 54, 106, 55, 116, 51, 105, 48, 105, 48, 107, 48, 117, 46, 99, 102, 56, 55, 46, 106, 51, 52, 48, 56, 51, 56, 52, 46, 104, 99, 112, 46, 111, 115, 97, 100, 101, 118, 46, 99, 108, 111, 117, 100, 48, 31, 6, 3, 85, 29, 35, 4, 24, 48, 22, 128, 20, 114, 177, 140, 206, 207, 94, 9, 168, 135, 3, 170, 114, 82, 7, 7, 31, 200, 109, 95, 210, 48, 29, 6, 3, 85, 29, 14, 4, 22, 4, 20, 114, 177, 140, 206, 207, 94, 9, 168, 135, 3, 170, 114, 82, 7, 7, 31, 200, 109, 95, 210, 48, 13, 6, 9, 42, 134, 72, 134, 247, 13, 1, 1, 11, 5, 0, 3, 130, 1, 1, 0, 58, 177, 141, 169, 86, 18, 70, 238, 141, 114, 67, 204, 31, 234, 2, 117, 215, 222, 236, 170, 28, 137, 213, 56, 191, 53, 161, 230, 189, 67, 113, 68, 69, 9... Gomega truncated this representation as it exceeds 'format.MaxLength'. Consider having the object provide a custom 'GomegaStringer' representation or adjust the parameters in Gomega's 'format' package. Learn more here: https://onsi.github.io/gomega/#adjusting-output occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_nodepool_osdisk.go:108]: Unexpected error: <errors.joinError \| 0xc000c8cfc0>: VerifyBasicAccess failed: failed to list services: Get "https://api.v2b6j7t3i0i0k0u.cf87.j3408384.hcp.osadev.cloud:443/api/v1/namespaces/default/services": tls: failed to verify certificate: x509: certificate signed by unknown authority VerifyAllAPIServicesAvailable failed: failed to list all APIServices: Get "https://api.v2b6j7t3i0i0k0u.cf87.j3408384.hcp.osadev.cloud:443/apis/apiregistration.k8s.io/v1/apiservices": tls: failed to verify certificate: x509: certificate signed by unknown authority { errs: [ <fmt.wrapError \| 0xc0004dbce0>{ msg: "VerifyBasicAccess failed: failed to list services: Get \"https://api.v2b6j7t3i0i0k0u.cf87.j3408384.hcp.osadev.cloud:443/api/v1/namespaces/default/services\": tls: failed to verify certificate: x509: certificate signed by unknown authority", err: <fmt.wrapError \| 0xc0004dbcc0>{ msg: "failed to list services: Get \"https://api.v2b6j7t3i0i0k0u.cf87.j3408384.hcp.osadev.cloud:443/api/v1/namespaces/default/services\": tls: failed to verify certificate: x509: certificate signed by unknown authority", err: <url.Error \| 0xc00156a8d0>{ Op: "Get", URL: "https://api.v2b6j7t3i0i0k0u.cf87.j3408384.hcp.osadev.cloud:443/api/v1/namespaces/default/services", Err: <tls.CertificateVerificationError \| 0xc00156a8a0>{ UnverifiedCertificates: [ { Raw: [48, 130, 3, 143, 48, 130, 2, 119, 160, 3, 2, 1, 2, 2, 16, 67, 5, 166, 237, 225, 226, 68, 194, 163, 198, 86, 165, 116, 45, 192, 109, 48, 13, 6, 9, 42, 134, 72, 134, 247, 13, 1, 1, 11, 5, 0, 48, 36, 49, 34, 48, 32, 6, 3, 85, 4, 3, 19, 25, 114, 101, 115, 101, 114, 118, 101, 100, 46, 104, 99, 112, 46, 111, 115, 97, 100, 101, 118, 46, 99, 108, 111, 117, 100, 48, 30, 23, 13, 50, 54, 48, 51, 49, 49, 49, 57, 48, 48, 52, 54, 90, 23, 13, 50, 54, 48, 57, 49, 49, 49, 57, 49, 48, 52, 54, 90, 48, 36, 49, 34, 48, 32, 6, 3, 85, 4, 3, 19, 25, 114, 101, 115, 101, 114, 118, 101, 100, 46, 104, 99, 112, 46, 111, 115, 97, 100, 101, 118, 46, 99, 108, 111, 117, 100, 48, 130, 1, 34, 48, 13, 6, 9, 42, 134, 72, 134, 247, 13, 1, 1, 1, 5, 0, 3, 130, 1, 15, 0, 48, 130, 1, 10, 2, 130, 1, 1, 0, 176, 205, 240, 241, 16, 53, 250, 250, 229, 234, 163, 179, 1, 180, 44, 212, 205, 69, 46, 248, 6, 98, 81, 118, 98, 45, 151, 230, 28, 235, 224, 209, 35, 57, 12, 205, 218, 35, 104, 117, 163, 167, 26, 143, 191, 115, 209, 206, 2, 216, 144, 137, 80, 177, 109, 24, 75, 102, 150, 13, 76, 195, 33, 69, 222, 250, 17, 214, 233, 242, 69, 166, 233, 99, 19, 31, 35, 69, 174, 247, 119, 15, 18, 10, 38, 133, 214, 203, 236, 224, 254, 7, 228, 235, 119, 106, 19, 157, 73, 191, 225, 15, 122, 90, 78, 115, 63, 144, 40, 208, 59, 93, 226, 22, 210, 155, 117, 182, 142, 177, 46, 125, 209, 120, 106, 135, 215, 60, 7, 137, 82, 63, 147, 88, 89, 22, 43, 32, 22, 2, 98, 4, 186, 142, 2, 237, 156, 81, 41, 150, 241, 6, 92, 176, 123, 203, 96, 122, 20, 46, 23, 22, 216, 138, 36, 128, 154, 174, 113, 223, 131, 210, 152, 215, 249, 244, 33, 108, 126, 177, 80, 128, 72, 214, 228, 3, 211, 18, 212, 179, 112, 140, 158, 6, 73, 9, 86, 108, 181, 27, 237, 237, 167, 172, 163, 202, 182, 39, 23, 53, 66, 209, 247, 101, 45, 162, 15, 154, 145, 184, 57, 61, 183, 16, 105, 214, 116, 230, 234, 224, 255, 211, 41, 85, 125, 216, 252, 191, 124, 59, 138, 205, 117, 130, 226, 102, 150, 174, 80, 56, 91, 167, 43, 124, 62, 237, 2, 3, 1, 0, 1, 163, 129, 188, 48, 129, 185, 48, 14, 6, 3, 85, 29, 15, 1, 1, 255, 4, 4, 3, 2, 5, 160, 48, 9, 6, 3, 85, 29, 19, 4, 2, 48, 0, 48, 29, 6, 3, 85, 29, 37, 4, 22, 48, 20, 6, 8, 43, 6, 1, 5, 5, 7, 3, 1, 6, 8, 43, 6, 1, 5, 5, 7, 3, 2, 48, 61, 6, 3, 85, 29, 17, 4, 54, 48, 52, 130, 50, 97, 112, 105, 46, 118, 50, 98, 54, 106, 55, 116, 51, 105, 48, 105, 48, 107, 48, 117, 46, 99, 102, 56, 55, 46, 106, 51, 52, 48, 56, 51, 56, 52, 46, 104, 99, 112, 46, 111, 115, 97, 100, 101, 118, 46, 99, 108, 111, 117, 100, 48, 31, 6, 3, 85, 29, 35, 4, 24, 48, 22, 128, 20, 114, 177, 140, 206, 207, 94, 9, 168, 135, 3, 170, 114, 82, 7, 7, 31, 200, 109, 95, 210, 48, 29, 6, 3, 85, 29, 14, 4, 22, 4, 20, 114, 177, 140, 206, 207, 94, 9, 168, 135, 3, 170, 114, 82, 7, 7, 31, 200, 109, 95, 210, 48, 13, 6, 9, 42, 134, 72, 134, 247, 13, 1, 1, 11, 5, 0, 3, 130, 1, 1, 0, 58, 177, 141, 169, 86, 18, 70, 238, 141, 114, 67, 204, 31, 234, 2, 117, 215, 222, 236, 170, 28, 137, 213, 56, 191, 53, 161, 230, 189, 67, 113, 68, 69, 9... Gomega truncated this representation as it exceeds 'format.MaxLength'. Consider having the object provide a custom 'GomegaStringer' representation or adjust the parameters in Gomega's 'format' package. Learn more here: https://onsi.github.io/gomega/#adjusting-output occurred tls: failed to verify certificate: x509: certificate signed by unknown authority 17:55:23 UTC · failed at e2e · test Customer should be able to create an HCP cluster and manage pull secrets · suite rp-api-compat-all/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_pullsecret.go:159]: Unexpected error: <errors.joinError \| 0xc000d94420>: VerifyBasicAccess failed: failed to list services: Get "https://api.w6n9h9m3j0o9j4g.fhvr.j3408384.hcp.osadev.cloud:443/api/v1/namespaces/default/services": tls: failed to verify certificate: x509: certificate signed by unknown authority VerifyAllAPIServicesAvailable failed: failed to list all APIServices: Get "https://api.w6n9h9m3j0o9j4g.fhvr.j3408384.hcp.osadev.cloud:443/apis/apiregistration.k8s.io/v1/apiservices": tls: failed to verify certificate: x509: certificate signed by unknown authority { errs: [ <fmt.wrapError \| 0xc0008a45c0>{ msg: "VerifyBasicAccess failed: failed to list services: Get \"https://api.w6n9h9m3j0o9j4g.fhvr.j3408384.hcp.osadev.cloud:443/api/v1/namespaces/default/services\": tls: failed to verify certificate: x509: certificate signed by unknown authority", err: <fmt.wrapError \| 0xc0008a45a0>{ msg: "failed to list services: Get \"https://api.w6n9h9m3j0o9j4g.fhvr.j3408384.hcp.osadev.cloud:443/api/v1/namespaces/default/services\": tls: failed to verify certificate: x509: certificate signed by unknown authority", err: <url.Error \| 0xc000e8cb40>{ Op: "Get", URL: "https://api.w6n9h9m3j0o9j4g.fhvr.j3408384.hcp.osadev.cloud:443/api/v1/namespaces/default/services", Err: <tls.CertificateVerificationError \| 0xc000e8cb10>{ UnverifiedCertificates: [ { Raw: [48, 130, 3, 143, 48, 130, 2, 119, 160, 3, 2, 1, 2, 2, 16, 78, 129, 37, 29, 95, 171, 76, 164, 135, 197, 26, 11, 124, 42, 206, 101, 48, 13, 6, 9, 42, 134, 72, 134, 247, 13, 1, 1, 11, 5, 0, 48, 36, 49, 34, 48, 32, 6, 3, 85, 4, 3, 19, 25, 114, 101, 115, 101, 114, 118, 101, 100, 46, 104, 99, 112, 46, 111, 115, 97, 100, 101, 118, 46, 99, 108, 111, 117, 100, 48, 30, 23, 13, 50, 54, 48, 51, 49, 49, 49, 56, 52, 51, 49, 53, 90, 23, 13, 50, 54, 48, 57, 49, 49, 49, 56, 53, 51, 49, 53, 90, 48, 36, 49, 34, 48, 32, 6, 3, 85, 4, 3, 19, 25, 114, 101, 115, 101, 114, 118, 101, 100, 46, 104, 99, 112, 46, 111, 115, 97, 100, 101, 118, 46, 99, 108, 111, 117, 100, 48, 130, 1, 34, 48, 13, 6, 9, 42, 134, 72, 134, 247, 13, 1, 1, 1, 5, 0, 3, 130, 1, 15, 0, 48, 130, 1, 10, 2, 130, 1, 1, 0, 164, 55, 97, 71, 202, 158, 162, 92, 203, 165, 200, 89, 141, 70, 129, 85, 88, 202, 45, 124, 164, 7, 7, 151, 80, 113, 129, 92, 36, 242, 84, 219, 206, 184, 88, 135, 78, 56, 190, 196, 67, 102, 237, 64, 151, 12, 151, 109, 74, 187, 98, 80, 91, 104, 211, 99, 65, 1, 100, 206, 132, 235, 243, 232, 106, 0, 186, 34, 210, 158, 251, 59, 98, 177, 73, 199, 231, 49, 247, 59, 162, 13, 208, 160, 66, 94, 209, 55, 182, 19, 22, 147, 137, 8, 4, 191, 93, 129, 241, 255, 251, 152, 65, 128, 193, 218, 56, 111, 207, 243, 4, 33, 64, 153, 173, 2, 15, 175, 219, 175, 93, 242, 136, 14, 224, 237, 173, 53, 2, 40, 45, 168, 63, 135, 152, 193, 228, 60, 244, 140, 143, 59, 76, 192, 80, 158, 219, 224, 138, 165, 62, 70, 169, 249, 26, 23, 198, 146, 32, 160, 177, 172, 147, 145, 115, 105, 94, 107, 108, 197, 247, 146, 183, 90, 2, 15, 141, 21, 228, 3, 0, 73, 123, 55, 126, 21, 242, 108, 83, 211, 251, 26, 201, 253, 231, 73, 124, 124, 209, 255, 220, 148, 130, 142, 97, 26, 78, 248, 85, 92, 132, 72, 113, 132, 114, 220, 247, 170, 199, 213, 116, 14, 105, 207, 52, 244, 143, 37, 186, 223, 134, 58, 131, 236, 26, 158, 81, 108, 231, 47, 175, 241, 188, 237, 115, 54, 224, 228, 209, 221, 195, 54, 0, 6, 238, 209, 2, 3, 1, 0, 1, 163, 129, 188, 48, 129, 185, 48, 14, 6, 3, 85, 29, 15, 1, 1, 255, 4, 4, 3, 2, 5, 160, 48, 9, 6, 3, 85, 29, 19, 4, 2, 48, 0, 48, 29, 6, 3, 85, 29, 37, 4, 22, 48, 20, 6, 8, 43, 6, 1, 5, 5, 7, 3, 1, 6, 8, 43, 6, 1, 5, 5, 7, 3, 2, 48, 61, 6, 3, 85, 29, 17, 4, 54, 48, 52, 130, 50, 97, 112, 105, 46, 119, 54, 110, 57, 104, 57, 109, 51, 106, 48, 111, 57, 106, 52, 103, 46, 102, 104, 118, 114, 46, 106, 51, 52, 48, 56, 51, 56, 52, 46, 104, 99, 112, 46, 111, 115, 97, 100, 101, 118, 46, 99, 108, 111, 117, 100, 48, 31, 6, 3, 85, 29, 35, 4, 24, 48, 22, 128, 20, 49, 109, 193, 42, 117, 44, 124, 194, 184, 211, 251, 54, 204, 112, 74, 27, 87, 154, 151, 215, 48, 29, 6, 3, 85, 29, 14, 4, 22, 4, 20, 49, 109, 193, 42, 117, 44, 124, 194, 184, 211, 251, 54, 204, 112, 74, 27, 87, 154, 151, 215, 48, 13, 6, 9, 42, 134, 72, 134, 247, 13, 1, 1, 11, 5, 0, 3, 130, 1, 1, 0, 56, 51, 147, 135, 206, 111, 13, 246, 108, 167, 202, 189, 195, 164, 241, 109, 18, 98, 120, 241, 245, 112, 223, 157, 177, 191, 164, 219, 54, 3... Gomega truncated this representation as it exceeds 'format.MaxLength'. Consider having the object provide a custom 'GomegaStringer' representation or adjust the parameters in Gomega's 'format' package. Learn more here: https://onsi.github.io/gomega/#adjusting-output occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_pullsecret.go:159]: Unexpected error: <errors.joinError \| 0xc000d94420>: VerifyBasicAccess failed: failed to list services: Get "https://api.w6n9h9m3j0o9j4g.fhvr.j3408384.hcp.osadev.cloud:443/api/v1/namespaces/default/services": tls: failed to verify certificate: x509: certificate signed by unknown authority VerifyAllAPIServicesAvailable failed: failed to list all APIServices: Get "https://api.w6n9h9m3j0o9j4g.fhvr.j3408384.hcp.osadev.cloud:443/apis/apiregistration.k8s.io/v1/apiservices": tls: failed to verify certificate: x509: certificate signed by unknown authority { errs: [ <fmt.wrapError \| 0xc0008a45c0>{ msg: "VerifyBasicAccess failed: failed to list services: Get \"https://api.w6n9h9m3j0o9j4g.fhvr.j3408384.hcp.osadev.cloud:443/api/v1/namespaces/default/services\": tls: failed to verify certificate: x509: certificate signed by unknown authority", err: <fmt.wrapError \| 0xc0008a45a0>{ msg: "failed to list services: Get \"https://api.w6n9h9m3j0o9j4g.fhvr.j3408384.hcp.osadev.cloud:443/api/v1/namespaces/default/services\": tls: failed to verify certificate: x509: certificate signed by unknown authority", err: <url.Error \| 0xc000e8cb40>{ Op: "Get", URL: "https://api.w6n9h9m3j0o9j4g.fhvr.j3408384.hcp.osadev.cloud:443/api/v1/namespaces/default/services", Err: <tls.CertificateVerificationError \| 0xc000e8cb10>{ UnverifiedCertificates: [ { Raw: [48, 130, 3, 143, 48, 130, 2, 119, 160, 3, 2, 1, 2, 2, 16, 78, 129, 37, 29, 95, 171, 76, 164, 135, 197, 26, 11, 124, 42, 206, 101, 48, 13, 6, 9, 42, 134, 72, 134, 247, 13, 1, 1, 11, 5, 0, 48, 36, 49, 34, 48, 32, 6, 3, 85, 4, 3, 19, 25, 114, 101, 115, 101, 114, 118, 101, 100, 46, 104, 99, 112, 46, 111, 115, 97, 100, 101, 118, 46, 99, 108, 111, 117, 100, 48, 30, 23, 13, 50, 54, 48, 51, 49, 49, 49, 56, 52, 51, 49, 53, 90, 23, 13, 50, 54, 48, 57, 49, 49, 49, 56, 53, 51, 49, 53, 90, 48, 36, 49, 34, 48, 32, 6, 3, 85, 4, 3, 19, 25, 114, 101, 115, 101, 114, 118, 101, 100, 46, 104, 99, 112, 46, 111, 115, 97, 100, 101, 118, 46, 99, 108, 111, 117, 100, 48, 130, 1, 34, 48, 13, 6, 9, 42, 134, 72, 134, 247, 13, 1, 1, 1, 5, 0, 3, 130, 1, 15, 0, 48, 130, 1, 10, 2, 130, 1, 1, 0, 164, 55, 97, 71, 202, 158, 162, 92, 203, 165, 200, 89, 141, 70, 129, 85, 88, 202, 45, 124, 164, 7, 7, 151, 80, 113, 129, 92, 36, 242, 84, 219, 206, 184, 88, 135, 78, 56, 190, 196, 67, 102, 237, 64, 151, 12, 151, 109, 74, 187, 98, 80, 91, 104, 211, 99, 65, 1, 100, 206, 132, 235, 243, 232, 106, 0, 186, 34, 210, 158, 251, 59, 98, 177, 73, 199, 231, 49, 247, 59, 162, 13, 208, 160, 66, 94, 209, 55, 182, 19, 22, 147, 137, 8, 4, 191, 93, 129, 241, 255, 251, 152, 65, 128, 193, 218, 56, 111, 207, 243, 4, 33, 64, 153, 173, 2, 15, 175, 219, 175, 93, 242, 136, 14, 224, 237, 173, 53, 2, 40, 45, 168, 63, 135, 152, 193, 228, 60, 244, 140, 143, 59, 76, 192, 80, 158, 219, 224, 138, 165, 62, 70, 169, 249, 26, 23, 198, 146, 32, 160, 177, 172, 147, 145, 115, 105, 94, 107, 108, 197, 247, 146, 183, 90, 2, 15, 141, 21, 228, 3, 0, 73, 123, 55, 126, 21, 242, 108, 83, 211, 251, 26, 201, 253, 231, 73, 124, 124, 209, 255, 220, 148, 130, 142, 97, 26, 78, 248, 85, 92, 132, 72, 113, 132, 114, 220, 247, 170, 199, 213, 116, 14, 105, 207, 52, 244, 143, 37, 186, 223, 134, 58, 131, 236, 26, 158, 81, 108, 231, 47, 175, 241, 188, 237, 115, 54, 224, 228, 209, 221, 195, 54, 0, 6, 238, 209, 2, 3, 1, 0, 1, 163, 129, 188, 48, 129, 185, 48, 14, 6, 3, 85, 29, 15, 1, 1, 255, 4, 4, 3, 2, 5, 160, 48, 9, 6, 3, 85, 29, 19, 4, 2, 48, 0, 48, 29, 6, 3, 85, 29, 37, 4, 22, 48, 20, 6, 8, 43, 6, 1, 5, 5, 7, 3, 1, 6, 8, 43, 6, 1, 5, 5, 7, 3, 2, 48, 61, 6, 3, 85, 29, 17, 4, 54, 48, 52, 130, 50, 97, 112, 105, 46, 119, 54, 110, 57, 104, 57, 109, 51, 106, 48, 111, 57, 106, 52, 103, 46, 102, 104, 118, 114, 46, 106, 51, 52, 48, 56, 51, 56, 52, 46, 104, 99, 112, 46, 111, 115, 97, 100, 101, 118, 46, 99, 108, 111, 117, 100, 48, 31, 6, 3, 85, 29, 35, 4, 24, 48, 22, 128, 20, 49, 109, 193, 42, 117, 44, 124, 194, 184, 211, 251, 54, 204, 112, 74, 27, 87, 154, 151, 215, 48, 29, 6, 3, 85, 29, 14, 4, 22, 4, 20, 49, 109, 193, 42, 117, 44, 124, 194, 184, 211, 251, 54, 204, 112, 74, 27, 87, 154, 151, 215, 48, 13, 6, 9, 42, 134, 72, 134, 247, 13, 1, 1, 11, 5, 0, 3, 130, 1, 1, 0, 56, 51, 147, 135, 206, 111, 13, 246, 108, 167, 202, 189, 195, 164, 241, 109, 18, 98, 120, 241, 245, 112, 223, 157, 177, 191, 164, 219, 54, 3... Gomega truncated this representation as it exceeds 'format.MaxLength'. Consider having the object provide a custom 'GomegaStringer' representation or adjust the parameters in Gomega's 'format' package. Learn more here: https://onsi.github.io/gomega/#adjusting-output occurred tls: failed to verify certificate: x509: certificate signed by unknown authority 17:55:23 UTC · failed at e2e · test Customer should be able to create an HCP cluster · suite rp-api-compat-all/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create.go:104]: Unexpected error: <errors.joinError \| 0xc0004e30f8>: VerifyBasicAccess failed: failed to list services: Get "https://api.x2e0o8x5r0f9m1n.ktsh.j3408384.hcp.osadev.cloud:443/api/v1/namespaces/default/services": tls: failed to verify certificate: x509: certificate signed by unknown authority VerifyAllAPIServicesAvailable failed: failed to list all APIServices: Get "https://api.x2e0o8x5r0f9m1n.ktsh.j3408384.hcp.osadev.cloud:443/apis/apiregistration.k8s.io/v1/apiservices": tls: failed to verify certificate: x509: certificate signed by unknown authority { errs: [ <fmt.wrapError \| 0xc000a0dea0>{ msg: "VerifyBasicAccess failed: failed to list services: Get \"https://api.x2e0o8x5r0f9m1n.ktsh.j3408384.hcp.osadev.cloud:443/api/v1/namespaces/default/services\": tls: failed to verify certificate: x509: certificate signed by unknown authority", err: <fmt.wrapError \| 0xc000a0de80>{ msg: "failed to list services: Get \"https://api.x2e0o8x5r0f9m1n.ktsh.j3408384.hcp.osadev.cloud:443/api/v1/namespaces/default/services\": tls: failed to verify certificate: x509: certificate signed by unknown authority", err: <url.Error \| 0xc000a1ce70>{ Op: "Get", URL: "https://api.x2e0o8x5r0f9m1n.ktsh.j3408384.hcp.osadev.cloud:443/api/v1/namespaces/default/services", Err: <tls.CertificateVerificationError \| 0xc000a1cc60>{ UnverifiedCertificates: [ { Raw: [48, 130, 3, 143, 48, 130, 2, 119, 160, 3, 2, 1, 2, 2, 16, 88, 155, 21, 149, 5, 212, 76, 227, 136, 65, 76, 124, 168, 154, 187, 31, 48, 13, 6, 9, 42, 134, 72, 134, 247, 13, 1, 1, 11, 5, 0, 48, 36, 49, 34, 48, 32, 6, 3, 85, 4, 3, 19, 25, 114, 101, 115, 101, 114, 118, 101, 100, 46, 104, 99, 112, 46, 111, 115, 97, 100, 101, 118, 46, 99, 108, 111, 117, 100, 48, 30, 23, 13, 50, 54, 48, 51, 49, 49, 49, 56, 52, 51, 49, 55, 90, 23, 13, 50, 54, 48, 57, 49, 49, 49, 56, 53, 51, 49, 55, 90, 48, 36, 49, 34, 48, 32, 6, 3, 85, 4, 3, 19, 25, 114, 101, 115, 101, 114, 118, 101, 100, 46, 104, 99, 112, 46, 111, 115, 97, 100, 101, 118, 46, 99, 108, 111, 117, 100, 48, 130, 1, 34, 48, 13, 6, 9, 42, 134, 72, 134, 247, 13, 1, 1, 1, 5, 0, 3, 130, 1, 15, 0, 48, 130, 1, 10, 2, 130, 1, 1, 0, 214, 241, 85, 243, 30, 159, 38, 20, 142, 6, 66, 186, 87, 53, 186, 198, 14, 166, 172, 113, 93, 105, 26, 249, 124, 162, 189, 95, 8, 66, 192, 183, 242, 83, 219, 250, 165, 107, 163, 23, 254, 75, 58, 171, 234, 231, 50, 231, 40, 21, 90, 25, 146, 185, 80, 60, 73, 3, 94, 84, 117, 190, 106, 50, 131, 19, 168, 180, 172, 138, 224, 210, 193, 99, 192, 9, 248, 103, 25, 185, 165, 183, 36, 97, 212, 247, 37, 184, 55, 10, 56, 218, 170, 31, 228, 7, 91, 167, 47, 246, 74, 248, 224, 74, 113, 149, 48, 69, 23, 50, 247, 138, 174, 158, 134, 41, 103, 120, 205, 185, 214, 135, 194, 156, 78, 178, 68, 14, 128, 183, 154, 197, 124, 9, 29, 165, 205, 104, 185, 6, 132, 220, 172, 32, 235, 8, 230, 98, 35, 181, 13, 216, 158, 236, 88, 70, 96, 132, 74, 203, 132, 145, 253, 89, 92, 209, 49, 210, 82, 218, 127, 127, 59, 52, 108, 143, 145, 90, 135, 194, 135, 23, 204, 89, 238, 175, 195, 113, 131, 47, 9, 94, 82, 93, 138, 191, 99, 79, 143, 65, 239, 47, 154, 207, 169, 21, 91, 2, 108, 34, 55, 223, 175, 195, 53, 145, 60, 218, 71, 34, 115, 72, 214, 65, 226, 20, 35, 179, 56, 183, 107, 52, 30, 3, 170, 133, 108, 226, 6, 204, 94, 55, 7, 74, 199, 130, 252, 190, 51, 8, 94, 212, 109, 95, 240, 121, 2, 3, 1, 0, 1, 163, 129, 188, 48, 129, 185, 48, 14, 6, 3, 85, 29, 15, 1, 1, 255, 4, 4, 3, 2, 5, 160, 48, 9, 6, 3, 85, 29, 19, 4, 2, 48, 0, 48, 29, 6, 3, 85, 29, 37, 4, 22, 48, 20, 6, 8, 43, 6, 1, 5, 5, 7, 3, 1, 6, 8, 43, 6, 1, 5, 5, 7, 3, 2, 48, 61, 6, 3, 85, 29, 17, 4, 54, 48, 52, 130, 50, 97, 112, 105, 46, 120, 50, 101, 48, 111, 56, 120, 53, 114, 48, 102, 57, 109, 49, 110, 46, 107, 116, 115, 104, 46, 106, 51, 52, 48, 56, 51, 56, 52, 46, 104, 99, 112, 46, 111, 115, 97, 100, 101, 118, 46, 99, 108, 111, 117, 100, 48, 31, 6, 3, 85, 29, 35, 4, 24, 48, 22, 128, 20, 130, 209, 89, 232, 35, 206, 104, 124, 66, 189, 65, 239, 13, 157, 14, 45, 33, 137, 91, 119, 48, 29, 6, 3, 85, 29, 14, 4, 22, 4, 20, 130, 209, 89, 232, 35, 206, 104, 124, 66, 189, 65, 239, 13, 157, 14, 45, 33, 137, 91, 119, 48, 13, 6, 9, 42, 134, 72, 134, 247, 13, 1, 1, 11, 5, 0, 3, 130, 1, 1, 0, 33, 12, 204, 122, 139, 37, 129, 165, 94, 103, 157, 80, 211, 140, 41, 102, 31, 118, 203, 148, 179, 115, 121, 135, 75, 52, 114, 173, 107, 197, 26, 51, 134, 196... Gomega truncated this representation as it exceeds 'format.MaxLength'. Consider having the object provide a custom 'GomegaStringer' representation or adjust the parameters in Gomega's 'format' package. Learn more here: https://onsi.github.io/gomega/#adjusting-output occurred fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create.go:104]: Unexpected error: <errors.joinError \| 0xc0004e30f8>: VerifyBasicAccess failed: failed to list services: Get "https://api.x2e0o8x5r0f9m1n.ktsh.j3408384.hcp.osadev.cloud:443/api/v1/namespaces/default/services": tls: failed to verify certificate: x509: certificate signed by unknown authority VerifyAllAPIServicesAvailable failed: failed to list all APIServices: Get "https://api.x2e0o8x5r0f9m1n.ktsh.j3408384.hcp.osadev.cloud:443/apis/apiregistration.k8s.io/v1/apiservices": tls: failed to verify certificate: x509: certificate signed by unknown authority { errs: [ <fmt.wrapError \| 0xc000a0dea0>{ msg: "VerifyBasicAccess failed: failed to list services: Get \"https://api.x2e0o8x5r0f9m1n.ktsh.j3408384.hcp.osadev.cloud:443/api/v1/namespaces/default/services\": tls: failed to verify certificate: x509: certificate signed by unknown authority", err: <fmt.wrapError \| 0xc000a0de80>{ msg: "failed to list services: Get \"https://api.x2e0o8x5r0f9m1n.ktsh.j3408384.hcp.osadev.cloud:443/api/v1/namespaces/default/services\": tls: failed to verify certificate: x509: certificate signed by unknown authority", err: <url.Error \| 0xc000a1ce70>{ Op: "Get", URL: "https://api.x2e0o8x5r0f9m1n.ktsh.j3408384.hcp.osadev.cloud:443/api/v1/namespaces/default/services", Err: <tls.CertificateVerificationError \| 0xc000a1cc60>{ UnverifiedCertificates: [ { Raw: [48, 130, 3, 143, 48, 130, 2, 119, 160, 3, 2, 1, 2, 2, 16, 88, 155, 21, 149, 5, 212, 76, 227, 136, 65, 76, 124, 168, 154, 187, 31, 48, 13, 6, 9, 42, 134, 72, 134, 247, 13, 1, 1, 11, 5, 0, 48, 36, 49, 34, 48, 32, 6, 3, 85, 4, 3, 19, 25, 114, 101, 115, 101, 114, 118, 101, 100, 46, 104, 99, 112, 46, 111, 115, 97, 100, 101, 118, 46, 99, 108, 111, 117, 100, 48, 30, 23, 13, 50, 54, 48, 51, 49, 49, 49, 56, 52, 51, 49, 55, 90, 23, 13, 50, 54, 48, 57, 49, 49, 49, 56, 53, 51, 49, 55, 90, 48, 36, 49, 34, 48, 32, 6, 3, 85, 4, 3, 19, 25, 114, 101, 115, 101, 114, 118, 101, 100, 46, 104, 99, 112, 46, 111, 115, 97, 100, 101, 118, 46, 99, 108, 111, 117, 100, 48, 130, 1, 34, 48, 13, 6, 9, 42, 134, 72, 134, 247, 13, 1, 1, 1, 5, 0, 3, 130, 1, 15, 0, 48, 130, 1, 10, 2, 130, 1, 1, 0, 214, 241, 85, 243, 30, 159, 38, 20, 142, 6, 66, 186, 87, 53, 186, 198, 14, 166, 172, 113, 93, 105, 26, 249, 124, 162, 189, 95, 8, 66, 192, 183, 242, 83, 219, 250, 165, 107, 163, 23, 254, 75, 58, 171, 234, 231, 50, 231, 40, 21, 90, 25, 146, 185, 80, 60, 73, 3, 94, 84, 117, 190, 106, 50, 131, 19, 168, 180, 172, 138, 224, 210, 193, 99, 192, 9, 248, 103, 25, 185, 165, 183, 36, 97, 212, 247, 37, 184, 55, 10, 56, 218, 170, 31, 228, 7, 91, 167, 47, 246, 74, 248, 224, 74, 113, 149, 48, 69, 23, 50, 247, 138, 174, 158, 134, 41, 103, 120, 205, 185, 214, 135, 194, 156, 78, 178, 68, 14, 128, 183, 154, 197, 124, 9, 29, 165, 205, 104, 185, 6, 132, 220, 172, 32, 235, 8, 230, 98, 35, 181, 13, 216, 158, 236, 88, 70, 96, 132, 74, 203, 132, 145, 253, 89, 92, 209, 49, 210, 82, 218, 127, 127, 59, 52, 108, 143, 145, 90, 135, 194, 135, 23, 204, 89, 238, 175, 195, 113, 131, 47, 9, 94, 82, 93, 138, 191, 99, 79, 143, 65, 239, 47, 154, 207, 169, 21, 91, 2, 108, 34, 55, 223, 175, 195, 53, 145, 60, 218, 71, 34, 115, 72, 214, 65, 226, 20, 35, 179, 56, 183, 107, 52, 30, 3, 170, 133, 108, 226, 6, 204, 94, 55, 7, 74, 199, 130, 252, 190, 51, 8, 94, 212, 109, 95, 240, 121, 2, 3, 1, 0, 1, 163, 129, 188, 48, 129, 185, 48, 14, 6, 3, 85, 29, 15, 1, 1, 255, 4, 4, 3, 2, 5, 160, 48, 9, 6, 3, 85, 29, 19, 4, 2, 48, 0, 48, 29, 6, 3, 85, 29, 37, 4, 22, 48, 20, 6, 8, 43, 6, 1, 5, 5, 7, 3, 1, 6, 8, 43, 6, 1, 5, 5, 7, 3, 2, 48, 61, 6, 3, 85, 29, 17, 4, 54, 48, 52, 130, 50, 97, 112, 105, 46, 120, 50, 101, 48, 111, 56, 120, 53, 114, 48, 102, 57, 109, 49, 110, 46, 107, 116, 115, 104, 46, 106, 51, 52, 48, 56, 51, 56, 52, 46, 104, 99, 112, 46, 111, 115, 97, 100, 101, 118, 46, 99, 108, 111, 117, 100, 48, 31, 6, 3, 85, 29, 35, 4, 24, 48, 22, 128, 20, 130, 209, 89, 232, 35, 206, 104, 124, 66, 189, 65, 239, 13, 157, 14, 45, 33, 137, 91, 119, 48, 29, 6, 3, 85, 29, 14, 4, 22, 4, 20, 130, 209, 89, 232, 35, 206, 104, 124, 66, 189, 65, 239, 13, 157, 14, 45, 33, 137, 91, 119, 48, 13, 6, 9, 42, 134, 72, 134, 247, 13, 1, 1, 11, 5, 0, 3, 130, 1, 1, 0, 33, 12, 204, 122, 139, 37, 129, 165, 94, 103, 157, 80, 211, 140, 41, 102, 31, 118, 203, 148, 179, 115, 121, 135, 75, 52, 114, 173, 107, 197, 26, 51, 134, 196... Gomega truncated this representation as it exceeds 'format.MaxLength'. Consider having the object provide a custom 'GomegaStringer' representation or adjust the parameters in Gomega's 'format' package. Learn more here: https://onsi.github.io/gomega/#adjusting-output occurred tls: failed to verify certificate: x509: certificate signed by unknown authority 17:55:23 UTC · failed at e2e · test Customer should be able to create a cluster with an external auth config and get the external auth config · suite rp-api-compat-all/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/external_auth_create.go:107]: Unexpected error: <errors.joinError \| 0xc0014560c0>: VerifyAllAPIServicesAvailable failed: failed to list all APIServices: Get "https://api.ea-cluster.x3cn.j3408384.hcp.osadev.cloud:443/apis/apiregistration.k8s.io/v1/apiservices": tls: failed to verify certificate: x509: certificate signed by unknown authority VerifyBasicAccess failed: failed to list services: Get "https://api.ea-cluster.x3cn.j3408384.hcp.osadev.cloud:443/api/v1/namespaces/default/services": tls: failed to verify certificate: x509: certificate signed by unknown authority { errs: [ <fmt.wrapError \| 0xc000882540>{ msg: "VerifyAllAPIServicesAvailable failed: failed to list all APIServices: Get \"https://api.ea-cluster.x3cn.j3408384.hcp.osadev.cloud:443/apis/apiregistration.k8s.io/v1/apiservices\": tls: failed to verify certificate: x509: certificate signed by unknown authority", err: <fmt.wrapError \| 0xc000882520>{ msg: "failed to list all APIServices: Get \"https://api.ea-cluster.x3cn.j3408384.hcp.osadev.cloud:443/apis/apiregistration.k8s.io/v1/apiservices\": tls: failed to verify certificate: x509: certificate signed by unknown authority", err: <url.Error \| 0xc000a594a0>{ Op: "Get", URL: "https://api.ea-cluster.x3cn.j3408384.hcp.osadev.cloud:443/apis/apiregistration.k8s.io/v1/apiservices", Err: <tls.CertificateVerificationError \| 0xc000a59470>{ UnverifiedCertificates: [ { Raw: [48, 130, 3, 138, 48, 130, 2, 114, 160, 3, 2, 1, 2, 2, 16, 57, 97, 241, 198, 21, 91, 69, 117, 154, 108, 27, 202, 162, 174, 31, 225, 48, 13, 6, 9, 42, 134, 72, 134, 247, 13, 1, 1, 11, 5, 0, 48, 36, 49, 34, 48, 32, 6, 3, 85, 4, 3, 19, 25, 114, 101, 115, 101, 114, 118, 101, 100, 46, 104, 99, 112, 46, 111, 115, 97, 100, 101, 118, 46, 99, 108, 111, 117, 100, 48, 30, 23, 13, 50, 54, 48, 51, 49, 49, 49, 56, 52, 51, 50, 53, 90, 23, 13, 50, 54, 48, 57, 49, 49, 49, 56, 53, 51, 50, 53, 90, 48, 36, 49, 34, 48, 32, 6, 3, 85, 4, 3, 19, 25, 114, 101, 115, 101, 114, 118, 101, 100, 46, 104, 99, 112, 46, 111, 115, 97, 100, 101, 118, 46, 99, 108, 111, 117, 100, 48, 130, 1, 34, 48, 13, 6, 9, 42, 134, 72, 134, 247, 13, 1, 1, 1, 5, 0, 3, 130, 1, 15, 0, 48, 130, 1, 10, 2, 130, 1, 1, 0, 202, 172, 252, 70, 175, 39, 161, 180, 166, 87, 213, 190, 88, 23, 205, 217, 254, 233, 82, 151, 115, 197, 117, 132, 23, 175, 62, 53, 185, 54, 45, 151, 84, 97, 41, 77, 175, 64, 181, 244, 143, 188, 9, 152, 238, 28, 123, 60, 4, 180, 82, 204, 83, 157, 252, 212, 153, 210, 69, 21, 240, 97, 201, 56, 209, 122, 16, 18, 115, 160, 97, 210, 165, 16, 131, 102, 90, 150, 239, 249, 87, 181, 168, 68, 89, 246, 75, 93, 65, 232, 49, 190, 14, 177, 63, 255, 160, 11, 74, 186, 246, 103, 198, 228, 62, 248, 23, 48, 64, 66, 118, 186, 26, 214, 16, 50, 134, 72, 177, 34, 4, 97, 93, 148, 106, 115, 145, 212, 39, 183, 42, 136, 92, 28, 131, 90, 98, 105, 169, 16, 249, 175, 163, 66, 181, 49, 125, 19, 95, 242, 93, 73, 2, 141, 37, 14, 111, 79, 225, 95, 51, 191, 97, 45, 20, 91, 93, 175, 141, 7, 216, 193, 190, 210, 133, 58, 121, 145, 154, 172, 193, 129, 108, 0, 188, 76, 235, 218, 30, 97, 61, 186, 0, 208, 96, 184, 30, 1, 75, 82, 24, 204, 28, 242, 164, 1, 54, 148, 6, 19, 181, 250, 240, 230, 144, 177, 240, 125, 83, 15, 46, 67, 69, 136, 252, 183, 163, 136, 245, 184, 167, 230, 50, 36, 19, 24, 103, 8, 147, 44, 28, 204, 174, 84, 67, 19, 236, 113, 52, 55, 78, 116, 0, 106, 215, 213, 2, 3, 1, 0, 1, 163, 129, 183, 48, 129, 180, 48, 14, 6, 3, 85, 29, 15, 1, 1, 255, 4, 4, 3, 2, 5, 160, 48, 9, 6, 3, 85, 29, 19, 4, 2, 48, 0, 48, 29, 6, 3, 85, 29, 37, 4, 22, 48, 20, 6, 8, 43, 6, 1, 5, 5, 7, 3, 1, 6, 8, 43, 6, 1, 5, 5, 7, 3, 2, 48, 56, 6, 3, 85, 29, 17, 4, 49, 48, 47, 130, 45, 97, 112, 105, 46, 101, 97, 45, 99, 108, 117, 115, 116, 101, 114, 46, 120, 51, 99, 110, 46, 106, 51, 52, 48, 56, 51, 56, 52, 46, 104, 99, 112, 46, 111, 115, 97, 100, 101, 118, 46, 99, 108, 111, 117, 100, 48, 31, 6, 3, 85, 29, 35, 4, 24, 48, 22, 128, 20, 142, 36, 86, 72, 39, 91, 36, 151, 140, 98, 89, 254, 226, 72, 42, 134, 225, 97, 112, 21, 48, 29, 6, 3, 85, 29, 14, 4, 22, 4, 20, 142, 36, 86, 72, 39, 91, 36, 151, 140, 98, 89, 254, 226, 72, 42, 134, 225, 97, 112, 21, 48, 13, 6, 9, 42, 134, 72, 134, 247, 13, 1, 1, 11, 5, 0, 3, 130, 1, 1, 0, 99, 95, 116, 110, 100, 195, 94, 178, 246, 225, 202, 3, 80, 190, 162, 28, 74, 251, 19, 197, 173, 226, 227, 57, 245, 188, 198, 39, 175, 170, 223, 88, 198, 13... Gomega truncated this representation as it exceeds 'format.MaxLength'. Consider having the object provide a custom 'GomegaStringer' representation or adjust the parameters in Gomega's 'format' package. Learn more here: https://onsi.github.io/gomega/#adjusting-output occurred fail [github.com/Azure/ARO-HCP/test/e2e/external_auth_create.go:107]: Unexpected error: <errors.joinError \| 0xc0014560c0>: VerifyAllAPIServicesAvailable failed: failed to list all APIServices: Get "https://api.ea-cluster.x3cn.j3408384.hcp.osadev.cloud:443/apis/apiregistration.k8s.io/v1/apiservices": tls: failed to verify certificate: x509: certificate signed by unknown authority VerifyBasicAccess failed: failed to list services: Get "https://api.ea-cluster.x3cn.j3408384.hcp.osadev.cloud:443/api/v1/namespaces/default/services": tls: failed to verify certificate: x509: certificate signed by unknown authority { errs: [ <fmt.wrapError \| 0xc000882540>{ msg: "VerifyAllAPIServicesAvailable failed: failed to list all APIServices: Get \"https://api.ea-cluster.x3cn.j3408384.hcp.osadev.cloud:443/apis/apiregistration.k8s.io/v1/apiservices\": tls: failed to verify certificate: x509: certificate signed by unknown authority", err: <fmt.wrapError \| 0xc000882520>{ msg: "failed to list all APIServices: Get \"https://api.ea-cluster.x3cn.j3408384.hcp.osadev.cloud:443/apis/apiregistration.k8s.io/v1/apiservices\": tls: failed to verify certificate: x509: certificate signed by unknown authority", err: <url.Error \| 0xc000a594a0>{ Op: "Get", URL: "https://api.ea-cluster.x3cn.j3408384.hcp.osadev.cloud:443/apis/apiregistration.k8s.io/v1/apiservices", Err: <tls.CertificateVerificationError \| 0xc000a59470>{ UnverifiedCertificates: [ { Raw: [48, 130, 3, 138, 48, 130, 2, 114, 160, 3, 2, 1, 2, 2, 16, 57, 97, 241, 198, 21, 91, 69, 117, 154, 108, 27, 202, 162, 174, 31, 225, 48, 13, 6, 9, 42, 134, 72, 134, 247, 13, 1, 1, 11, 5, 0, 48, 36, 49, 34, 48, 32, 6, 3, 85, 4, 3, 19, 25, 114, 101, 115, 101, 114, 118, 101, 100, 46, 104, 99, 112, 46, 111, 115, 97, 100, 101, 118, 46, 99, 108, 111, 117, 100, 48, 30, 23, 13, 50, 54, 48, 51, 49, 49, 49, 56, 52, 51, 50, 53, 90, 23, 13, 50, 54, 48, 57, 49, 49, 49, 56, 53, 51, 50, 53, 90, 48, 36, 49, 34, 48, 32, 6, 3, 85, 4, 3, 19, 25, 114, 101, 115, 101, 114, 118, 101, 100, 46, 104, 99, 112, 46, 111, 115, 97, 100, 101, 118, 46, 99, 108, 111, 117, 100, 48, 130, 1, 34, 48, 13, 6, 9, 42, 134, 72, 134, 247, 13, 1, 1, 1, 5, 0, 3, 130, 1, 15, 0, 48, 130, 1, 10, 2, 130, 1, 1, 0, 202, 172, 252, 70, 175, 39, 161, 180, 166, 87, 213, 190, 88, 23, 205, 217, 254, 233, 82, 151, 115, 197, 117, 132, 23, 175, 62, 53, 185, 54, 45, 151, 84, 97, 41, 77, 175, 64, 181, 244, 143, 188, 9, 152, 238, 28, 123, 60, 4, 180, 82, 204, 83, 157, 252, 212, 153, 210, 69, 21, 240, 97, 201, 56, 209, 122, 16, 18, 115, 160, 97, 210, 165, 16, 131, 102, 90, 150, 239, 249, 87, 181, 168, 68, 89, 246, 75, 93, 65, 232, 49, 190, 14, 177, 63, 255, 160, 11, 74, 186, 246, 103, 198, 228, 62, 248, 23, 48, 64, 66, 118, 186, 26, 214, 16, 50, 134, 72, 177, 34, 4, 97, 93, 148, 106, 115, 145, 212, 39, 183, 42, 136, 92, 28, 131, 90, 98, 105, 169, 16, 249, 175, 163, 66, 181, 49, 125, 19, 95, 242, 93, 73, 2, 141, 37, 14, 111, 79, 225, 95, 51, 191, 97, 45, 20, 91, 93, 175, 141, 7, 216, 193, 190, 210, 133, 58, 121, 145, 154, 172, 193, 129, 108, 0, 188, 76, 235, 218, 30, 97, 61, 186, 0, 208, 96, 184, 30, 1, 75, 82, 24, 204, 28, 242, 164, 1, 54, 148, 6, 19, 181, 250, 240, 230, 144, 177, 240, 125, 83, 15, 46, 67, 69, 136, 252, 183, 163, 136, 245, 184, 167, 230, 50, 36, 19, 24, 103, 8, 147, 44, 28, 204, 174, 84, 67, 19, 236, 113, 52, 55, 78, 116, 0, 106, 215, 213, 2, 3, 1, 0, 1, 163, 129, 183, 48, 129, 180, 48, 14, 6, 3, 85, 29, 15, 1, 1, 255, 4, 4, 3, 2, 5, 160, 48, 9, 6, 3, 85, 29, 19, 4, 2, 48, 0, 48, 29, 6, 3, 85, 29, 37, 4, 22, 48, 20, 6, 8, 43, 6, 1, 5, 5, 7, 3, 1, 6, 8, 43, 6, 1, 5, 5, 7, 3, 2, 48, 56, 6, 3, 85, 29, 17, 4, 49, 48, 47, 130, 45, 97, 112, 105, 46, 101, 97, 45, 99, 108, 117, 115, 116, 101, 114, 46, 120, 51, 99, 110, 46, 106, 51, 52, 48, 56, 51, 56, 52, 46, 104, 99, 112, 46, 111, 115, 97, 100, 101, 118, 46, 99, 108, 111, 117, 100, 48, 31, 6, 3, 85, 29, 35, 4, 24, 48, 22, 128, 20, 142, 36, 86, 72, 39, 91, 36, 151, 140, 98, 89, 254, 226, 72, 42, 134, 225, 97, 112, 21, 48, 29, 6, 3, 85, 29, 14, 4, 22, 4, 20, 142, 36, 86, 72, 39, 91, 36, 151, 140, 98, 89, 254, 226, 72, 42, 134, 225, 97, 112, 21, 48, 13, 6, 9, 42, 134, 72, 134, 247, 13, 1, 1, 11, 5, 0, 3, 130, 1, 1, 0, 99, 95, 116, 110, 100, 195, 94, 178, 246, 225, 202, 3, 80, 190, 162, 28, 74, 251, 19, 197, 173, 226, 227, 57, 245, 188, 198, 39, 175, 170, 223, 88, 198, 13... Gomega truncated this representation as it exceeds 'format.MaxLength'. Consider having the object provide a custom 'GomegaStringer' representation or adjust the parameters in Gomega's 'format' package. Learn more here: https://onsi.github.io/gomega/#adjusting-output occurred
17:43:38	n/a	pull-ci-Azure-ARO-HCP-main-e2e-parallel post-goodmerged PR head a36f37a · merge a36f37a	n/a	n/a	passed	#4205 (merged)	0	n/a
16:40:17	n/a	pull-ci-Azure-ARO-HCP-main-e2e-parallel post-goodmerged PR head 1892906 · merge 1892906	n/a	n/a	passed	#3820 (merged)	0	n/a
16:26:29	n/a	pull-ci-Azure-ARO-HCP-main-e2e-parallel post-goodmerged PR head e0b528c · merge e0b528c	n/a	n/a	passed	#4371 (merged)	0	n/a
16:25:52	n/a	pull-ci-Azure-ARO-HCP-main-e2e-parallel post-goodmerged PR head 244d96c · merge 244d96c	n/a	n/a	passed	#4319 (merged)	0	n/a
16:18:49	n/a	pull-ci-Azure-ARO-HCP-main-e2e-parallel post-goodmerged PR head 4655674 · merge 4655674	n/a	n/a	passed	#4405 (merged)	0	n/a
16:16:36	n/a	pull-ci-Azure-ARO-HCP-main-e2e-parallel head 9e79e14	n/a	e2e	failed	#4393 (open)	1	e2e (1) ERROR CODE: InternalServerError; detail code AuthorizationPermissionMismatch; provider Microsoft.RedHatOpenShift 16:16:36 UTC · failed at e2e · test Customer should be able to create an HCP cluster · suite rp-api-compat-all/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create.go:90]: Unexpected error: <fmt.wrapError \| 0xc000867300>: failed to create HCP cluster basic-hcp-cluster: failed waiting for cluster="basic-hcp-cluster" in resourcegroup="basic-cluster-tmtp7t" to finish creating: GET https://rp.j6682624.hcpsvc.osadev.cloud/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westus3/hcpOperationStatuses/6491613a-a2b4-411e-b2bc-43f4c885a5ef -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westus3/hcpOperationStatuses/6491613a-a2b4-411e-b2bc-43f4c885a5ef", "name": "6491613a-a2b4-411e-b2bc-43f4c885a5ef", "status": "Failed", "startTime": "2026-03-11T17:18:14.777252826Z", "endTime": "2026-03-11T17:19:49.438177405Z", "error": { "code": "InternalServerError", "message": "GET https://arohcpprowoidcj6682624.blob.core.windows.net/$web\n--------------------------------------------------------------------------------\nRESPONSE 403: 403 This request is not authorized to perform this operation using this permission.\nERROR CODE: AuthorizationPermissionMismatch\n--------------------------------------------------------------------------------\n\u003c?xml version=\"1.0\" encoding=\"utf-8\"?\u003e\u003cError\u003e\u003cCode\u003eAuthorizationPermissionMismatch\u003c/Code\u003e\u003cMessage\u003eThis request is not authorized to perform this operation using this permission.\nRequestId:0728a7cc-301e-0063-1d7b-b13d00000000\nTime:2026-03-11T17:19:44.8267301Z\u003c/Message\u003e\u003c/Error\u003e\n--------------------------------------------------------------------------------\n" } } -------------------------------------------------------------------------------- { msg: "failed to create HCP cluster basic-hcp-cluster: failed waiting for cluster=\"basic-hcp-cluster\" in resourcegroup=\"basic-cluster-tmtp7t\" to finish creating: GET https://rp.j6682624.hcpsvc.osadev.cloud/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westus3/hcpOperationStatuses/6491613a-a2b4-411e-b2bc-43f4c885a5ef\n--------------------------------------------------------------------------------\nRESPONSE 200: 200 OK\nERROR CODE: InternalServerError\n--------------------------------------------------------------------------------\n{\n \"id\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westus3/hcpOperationStatuses/6491613a-a2b4-411e-b2bc-43f4c885a5ef\",\n \"name\": \"6491613a-a2b4-411e-b2bc-43f4c885a5ef\",\n \"status\": \"Failed\",\n \"startTime\": \"2026-03-11T17:18:14.777252826Z\",\n \"endTime\": \"2026-03-11T17:19:49.438177405Z\",\n \"error\": {\n \"code\": \"InternalServerError\",\n \"message\": \"GET https://arohcpprowoidcj6682624.blob.core.windows.net/$web\\n--------------------------------------------------------------------------------\\nRESPONSE 403: 403 This request is not authorized to perform this operation using this permission.\\nERROR CODE: AuthorizationPermissionMismatch\\n--------------------------------------------------------------------------------\\n\ufeff\\u003c?xml version=\\\"1.0\\\" encoding=\\\"utf-8\\\"?\\u003e\\u003cError\\u003e\\u003cCode\\u003eAuthorizationPermissionMismatch\\u003c/Code\\u003e\\u003cMessage\\u003eThis request is not authorized to perform this operation using this permission.\\nRequestId:0728a7cc-301e-0063-1d7b-b13d00000000\\nTime:2026-03-11T17:19:44.8267301Z\\u003c/Message\\u003e\\u003c/Error\\u003e\\n--------------------------------------------------------------------------------\\n\"\n }\n}\n--------------------------------------------------------------------------------\n", err: <fmt.wrapError \| 0xc0008672e0>{ msg: "failed waiting for cluster=\"basic-hcp-cluster\" in resourcegroup=\"basic-cluster-tmtp7t\" to finish creating: GET https://rp.j6682624.hcpsvc.osadev.cloud/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westus3/hcpOperationStatuses/6491613a-a2b4-411e-b2bc-43f4c885a5ef\n--------------------------------------------------------------------------------\nRESPONSE 200: 200 OK\nERROR CODE: InternalServerError\n--------------------------------------------------------------------------------\n{\n \"id\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westus3/hcpOperationStatuses/6491613a-a2b4-411e-b2bc-43f4c885a5ef\",\n \"name\": \"6491613a-a2b4-411e-b2bc-43f4c885a5ef\",\n \"status\": \"Failed\",\n \"startTime\": \"2026-03-11T17:18:14.777252826Z\",\n \"endTime\": \"2026-03-11T17:19:49.438177405Z\",\n \"error\": {\n \"code\": \"InternalServerError\",\n \"message\": \"GET https://arohcpprowoidcj6682624.blob.core.windows.net/$web\\n--------------------------------------------------------------------------------\\nRESPONSE 403: 403 This request is not authorized to perform this operation using this permission.\\nERROR CODE: AuthorizationPermissionMismatch\\n--------------------------------------------------------------------------------\\n\ufeff\\u003c?xml version=\\\"1.0\\\" encoding=\\\"utf-8\\\"?\\u003e\\u003cError\\u003e\\u003cCode\\u003eAuthorizationPermissionMismatch\\u003c/Code\\u003e\\u003cMessage\\u003eThis request is not authorized to perform this operation using this permission.\\nRequestId:0728a7cc-301e-0063-1d7b-b13d00000000\\nTime:2026-03-11T17:19:44.8267301Z\\u003c/Message\\u003e\\u003c/Error\\u003e\\n--------------------------------------------------------------------------------\\n\"\n }\n}\n--------------------------------------------------------------------------------\n", err: <exp... Gomega truncated this representation as it exceeds 'format.MaxLength'. Consider having the object provide a custom 'GomegaStringer' representation or adjust the parameters in Gomega's 'format' package. Learn more here: https://onsi.github.io/gomega/#adjusting-output occurred fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create.go:90]: Unexpected error: <fmt.wrapError \| 0xc000867300>: failed to create HCP cluster basic-hcp-cluster: failed waiting for cluster="basic-hcp-cluster" in resourcegroup="basic-cluster-tmtp7t" to finish creating: GET https://rp.j6682624.hcpsvc.osadev.cloud/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westus3/hcpOperationStatuses/6491613a-a2b4-411e-b2bc-43f4c885a5ef -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westus3/hcpOperationStatuses/6491613a-a2b4-411e-b2bc-43f4c885a5ef", "name": "6491613a-a2b4-411e-b2bc-43f4c885a5ef", "status": "Failed", "startTime": "2026-03-11T17:18:14.777252826Z", "endTime": "2026-03-11T17:19:49.438177405Z", "error": { "code": "InternalServerError", "message": "GET https://arohcpprowoidcj6682624.blob.core.windows.net/$web\n--------------------------------------------------------------------------------\nRESPONSE 403: 403 This request is not authorized to perform this operation using this permission.\nERROR CODE: AuthorizationPermissionMismatch\n--------------------------------------------------------------------------------\n\u003c?xml version=\"1.0\" encoding=\"utf-8\"?\u003e\u003cError\u003e\u003cCode\u003eAuthorizationPermissionMismatch\u003c/Code\u003e\u003cMessage\u003eThis request is not authorized to perform this operation using this permission.\nRequestId:0728a7cc-301e-0063-1d7b-b13d00000000\nTime:2026-03-11T17:19:44.8267301Z\u003c/Message\u003e\u003c/Error\u003e\n--------------------------------------------------------------------------------\n" } } -------------------------------------------------------------------------------- { msg: "failed to create HCP cluster basic-hcp-cluster: failed waiting for cluster=\"basic-hcp-cluster\" in resourcegroup=\"basic-cluster-tmtp7t\" to finish creating: GET https://rp.j6682624.hcpsvc.osadev.cloud/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westus3/hcpOperationStatuses/6491613a-a2b4-411e-b2bc-43f4c885a5ef\n--------------------------------------------------------------------------------\nRESPONSE 200: 200 OK\nERROR CODE: InternalServerError\n--------------------------------------------------------------------------------\n{\n \"id\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westus3/hcpOperationStatuses/6491613a-a2b4-411e-b2bc-43f4c885a5ef\",\n \"name\": \"6491613a-a2b4-411e-b2bc-43f4c885a5ef\",\n \"status\": \"Failed\",\n \"startTime\": \"2026-03-11T17:18:14.777252826Z\",\n \"endTime\": \"2026-03-11T17:19:49.438177405Z\",\n \"error\": {\n \"code\": \"InternalServerError\",\n \"message\": \"GET https://arohcpprowoidcj6682624.blob.core.windows.net/$web\\n--------------------------------------------------------------------------------\\nRESPONSE 403: 403 This request is not authorized to perform this operation using this permission.\\nERROR CODE: AuthorizationPermissionMismatch\\n--------------------------------------------------------------------------------\\n\ufeff\\u003c?xml version=\\\"1.0\\\" encoding=\\\"utf-8\\\"?\\u003e\\u003cError\\u003e\\u003cCode\\u003eAuthorizationPermissionMismatch\\u003c/Code\\u003e\\u003cMessage\\u003eThis request is not authorized to perform this operation using this permission.\\nRequestId:0728a7cc-301e-0063-1d7b-b13d00000000\\nTime:2026-03-11T17:19:44.8267301Z\\u003c/Message\\u003e\\u003c/Error\\u003e\\n--------------------------------------------------------------------------------\\n\"\n }\n}\n--------------------------------------------------------------------------------\n", err: <fmt.wrapError \| 0xc0008672e0>{ msg: "failed waiting for cluster=\"basic-hcp-cluster\" in resourcegroup=\"basic-cluster-tmtp7t\" to finish creating: GET https://rp.j6682624.hcpsvc.osadev.cloud/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westus3/hcpOperationStatuses/6491613a-a2b4-411e-b2bc-43f4c885a5ef\n--------------------------------------------------------------------------------\nRESPONSE 200: 200 OK\nERROR CODE: InternalServerError\n--------------------------------------------------------------------------------\n{\n \"id\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westus3/hcpOperationStatuses/6491613a-a2b4-411e-b2bc-43f4c885a5ef\",\n \"name\": \"6491613a-a2b4-411e-b2bc-43f4c885a5ef\",\n \"status\": \"Failed\",\n \"startTime\": \"2026-03-11T17:18:14.777252826Z\",\n \"endTime\": \"2026-03-11T17:19:49.438177405Z\",\n \"error\": {\n \"code\": \"InternalServerError\",\n \"message\": \"GET https://arohcpprowoidcj6682624.blob.core.windows.net/$web\\n--------------------------------------------------------------------------------\\nRESPONSE 403: 403 This request is not authorized to perform this operation using this permission.\\nERROR CODE: AuthorizationPermissionMismatch\\n--------------------------------------------------------------------------------\\n\ufeff\\u003c?xml version=\\\"1.0\\\" encoding=\\\"utf-8\\\"?\\u003e\\u003cError\\u003e\\u003cCode\\u003eAuthorizationPermissionMismatch\\u003c/Code\\u003e\\u003cMessage\\u003eThis request is not authorized to perform this operation using this permission.\\nRequestId:0728a7cc-301e-0063-1d7b-b13d00000000\\nTime:2026-03-11T17:19:44.8267301Z\\u003c/Message\\u003e\\u003c/Error\\u003e\\n--------------------------------------------------------------------------------\\n\"\n }\n}\n--------------------------------------------------------------------------------\n", err: <exp... Gomega truncated this representation as it exceeds 'format.MaxLength'. Consider having the object provide a custom 'GomegaStringer' representation or adjust the parameters in Gomega's 'format' package. Learn more here: https://onsi.github.io/gomega/#adjusting-output occurred
16:06:44	n/a	pull-ci-Azure-ARO-HCP-main-e2e-parallel post-goodmerged PR head 9128c3e · merge 9128c3e	n/a	n/a	passed	#4383 (merged)	0	n/a
15:56:18	n/a	pull-ci-Azure-ARO-HCP-main-e2e-parallel merged PR head 11ecb87 · merge e0b528c	n/a	unknown	failed	#4371 (merged)	1	non-artifact-backed failure (no junit artifacts)
15:53:55	n/a	pull-ci-Azure-ARO-HCP-main-e2e-parallel merged PR head c182599 · merge 0d2c310	n/a	n/a	passed	#4224 (merged)	0	n/a
15:28:50	n/a	pull-ci-Azure-ARO-HCP-main-e2e-parallel head 776b260	n/a	n/a	passed	#4404 (closed)	0	n/a
15:04:58	n/a	pull-ci-Azure-ARO-HCP-main-e2e-parallel head 776b260	n/a	unknown	failed	#4404 (closed)	1	non-artifact-backed failure (no junit artifacts)
14:52:11	n/a	pull-ci-Azure-ARO-HCP-main-e2e-parallel post-goodmerged PR head 4b784ee · merge 4b784ee	n/a	unknown	failed	#4408 (merged)	1	non-artifact-backed failure (no junit artifacts)
14:44:10	n/a	pull-ci-Azure-ARO-HCP-main-e2e-parallel merged PR head 5ddc3d7 · merge 0d2c310	n/a	unknown	failed	#4224 (merged)	1	non-artifact-backed failure (no junit artifacts)
14:40:31	n/a	pull-ci-Azure-ARO-HCP-main-e2e-parallel merged PR head 3791879 · merge 80403bb	n/a	n/a	passed	#4407 (merged)	0	n/a
14:24:20	n/a	pull-ci-Azure-ARO-HCP-main-e2e-parallel post-goodmerged PR head 2cc3d47 · merge 2cc3d47	n/a	n/a	passed	#4390 (merged)	0	n/a
14:00:17	n/a	pull-ci-Azure-ARO-HCP-main-e2e-parallel post-goodmerged PR head 9128c3e · merge 9128c3e	n/a	e2e	failed	★#4383 (merged)	1	e2e (1) not to be nil 14:00:17 UTC · failed at e2e · test Authorized CIDRs Connectivity should allow API access only from authorized VM IP · suite rp-api-compat-all/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_authorized_cidrs_connectivity.go:145]: Expected <string \| 0x0>: nil not to be nil fail [github.com/Azure/ARO-HCP/test/e2e/cluster_authorized_cidrs_connectivity.go:145]: Expected <string \| 0x0>: nil not to be nil
13:49:56	n/a	pull-ci-Azure-ARO-HCP-main-e2e-parallel merged PR head 66ed4b4 · merge 8a1ae2a	n/a	e2e	failed	★#4399 (merged)	1	e2e (1) not to be nil 13:49:56 UTC · failed at e2e · test Authorized CIDRs Connectivity should allow API access only from authorized VM IP · suite rp-api-compat-all/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_authorized_cidrs_connectivity.go:145]: Expected <string \| 0x0>: nil not to be nil fail [github.com/Azure/ARO-HCP/test/e2e/cluster_authorized_cidrs_connectivity.go:145]: Expected <string \| 0x0>: nil not to be nil
13:22:28	n/a	pull-ci-Azure-ARO-HCP-main-e2e-parallel post-goodmerged PR head 4655674 · merge 4655674	n/a	unknown	failed	#4405 (merged)	1	non-artifact-backed failure (no junit artifacts)
13:05:13	n/a	pull-ci-Azure-ARO-HCP-main-e2e-parallel post-goodmerged PR head 1892906 · merge 1892906	n/a	unknown	failed	#3820 (merged)	1	non-artifact-backed failure (no junit artifacts)
12:54:33	n/a	pull-ci-Azure-ARO-HCP-main-e2e-parallel post-goodmerged PR head c117b0a · merge c117b0a	n/a	e2e	failed	★#4350 (merged)	1	e2e (1) failed to create HCP cluster <cluster>: operation result model did not match expected model for type armredhatopenshifthcp.HcpOpenShiftCluster: 12:54:33 UTC · failed at e2e · test Update HCPOpenShiftCluster Positive creates a cluster and updates tags with a PATCH request · suite rp-api-compat-all/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_update.go:160]: Unexpected error: <fmt.wrapError \| 0xc0008e53e0>: failed to create HCP cluster patch-tags-cluster: operation result model did not match expected model for type armredhatopenshifthcp.HcpOpenShiftCluster: &armredhatopenshifthcp.HcpOpenShiftCluster{ Location: &"westus3", Identity: &{Type: &"UserAssigned", UserAssignedIdentities: {"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-dev-246/providers/Microsoft.ManagedIdentity/userAssignedIdentities/cloud-controller-manager": nil, "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-dev-246/providers/Microsoft.ManagedIdentity/userAssignedIdentities/cloud-network-config": nil, "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-dev-246/providers/Microsoft.ManagedIdentity/userAssignedIdentities/cluster-api-azure": nil, "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-dev-246/providers/Microsoft.ManagedIdentity/userAssignedIdentities/control-plane": nil, ...}}, Properties: &armredhatopenshifthcp.HcpOpenShiftClusterProperties{ Platform: &{NetworkSecurityGroupID: &"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGrou"..., OperatorsAuthentication: &{UserAssignedIdentities: &{ControlPlaneOperators: {"cloud-controller-manager": &"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGrou"..., "cloud-network-config": &"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGrou"..., "cluster-api-azure": &"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGrou"..., "control-plane": &"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGrou"..., ...}, DataPlaneOperators: {"disk-csi-driver": &"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGrou"..., "file-csi-driver": &"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGrou"..., "image-registry": &"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGrou"...}, ServiceManagedIdentity: &"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGrou"...}}, SubnetID: &"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGrou"..., ManagedResourceGroup: &"patch-tags-kcmxqz--managed", ...}, API: &armredhatopenshifthcp.APIProfile{ Visibility: &"Public", - URL: &"https://api.p8m4m5l4t4k7n4e.bxyv.j6197248.hcp.osadev.cloud:443", + URL: nil, AuthorizedCIDRs: nil, }, Autoscaling: &{MaxNodeProvisionTimeSeconds: &900, MaxPodGracePeriodSeconds: &600, PodPriorityThreshold: &-10}, ClusterImageRegistry: &{State: &"Enabled"}, ... // 7 identical fields }, Tags: {"aro-hcp.experimental.cluster.size-override": &"Minimal"}, ID: &"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGrou"..., ... // 1 ignored and 2 identical fields } { msg: "failed to create HCP cluster patch-tags-cluster: operation result model did not match expected model for type armredhatopenshifthcp.HcpOpenShiftCluster:\n\u00a0\u00a0&armredhatopenshifthcp.HcpOpenShiftCluster{\n\u00a0\u00a0\tLocation: &\"westus3\",\n\u00a0\u00a0\tIdentity: &{Type: &\"UserAssigned\", UserAssignedIdentities: {\"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-dev-246/providers/Microsoft.ManagedIdentity/userAssignedIdentities/cloud-controller-manager\": nil, \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-dev-246/providers/Microsoft.ManagedIdentity/userAssignedIdentities/cloud-network-config\": nil, \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-dev-246/providers/Microsoft.ManagedIdentity/userAssignedIdentities/cluster-api-azure\": nil, \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-dev-246/providers/Microsoft.ManagedIdentity/userAssignedIdentities/control-plane\": nil, ...}},\n\u00a0\u00a0\tProperties: &armredhatopenshifthcp.HcpOpenShiftClusterProperties{\n\u00a0\u00a0\t\tPlatform: &{NetworkSecurityGroupID: &\"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGrou\"..., OperatorsAuthentication: &{UserAssignedIdentities: &{ControlPlaneOperators: {\"cloud-controller-manager\": &\"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGrou\"..., \"cloud-network-config\": &\"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGrou\"..., \"cluster-api-azure\": &\"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGrou\"..., \"control-plane\": &\"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGrou\"..., ...}, DataPlaneOperators: {\"disk-csi-driver\": &\"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGrou\"..., \"file-csi-driver\": &\"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGrou\"..., \"image-registry\": &\"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGrou\"...}, ServiceManagedIdentity: &\"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGrou\"...}}, SubnetID: &\"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGrou\"..., ManagedResourceGroup: &\"patch-tags-kcmxqz--managed\", ...},\n\u00a0\u00a0\t\tAPI: &armredhatopenshifthcp.APIProfile{\n\u00a0\u00a0\t\t\tVisibility: &\"Public\",\n-\u00a0\t\t\tURL: &\"https://api.p8m4m5l4t4k7n4e.bxyv.j6197248.hcp.osadev.cloud:443\",\n+\u00a0\t\t\tURL: nil,\n\u00a0\u00a0\t\t\tAuthorizedCIDRs: nil,\n\u00a0\u00a0\t\t},\n\u00a0\u00a0\t\tAutoscaling: &{MaxNodeProvisionTimeSeconds: &900, MaxPodGracePeriodSeconds: &600, PodPriorityThreshold: &-10},\n\u00a0\u00a0\t\tClusterImageRegistry: &{State: &\"Enabled\"},\n\u00a0\u00a0\t\t... // 7 identical fields\n\u00a0\u00a0\t},\n\u00a0\u00a0\tTags: {\"aro-hcp.experimental.cluster.size-override\": &\"Minimal\"},\n\u00a0\u00a0\tID: &\"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGrou\"...,\n\u00a0\u00a0\t... // 1 ignored and 2 identical fields\n\u00a0\u00a0}\n", err: <errors.errorString \| 0xc000de0590>{ s: "operation result model did not match expected model for type armredhatopenshifthcp.HcpOpenShiftCluster:\n\u00a0\u00a0&armredhatopenshifthcp.HcpOpenShiftCluster{\n\u00a0\u00a0\tLocation: &\"westus3\",\n\u00a0\u00a0\tIdentity: &{Type: &\"UserAssigned\", UserAssignedIdentities: {\"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-dev-246/providers/Microsoft.ManagedIdentity/userAssignedIdentities/cloud-controller-manager\": nil, \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-dev-246/providers/Microsoft.ManagedIdentity/userAssignedIdentities/cloud-network-config\": nil, \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-container... Gomega truncated this representation as it exceeds 'format.MaxLength'. Consider having the object provide a custom 'GomegaStringer' representation or adjust the parameters in Gomega's 'format' package. Learn more here: https://onsi.github.io/gomega/#adjusting-output occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_update.go:160]: Unexpected error: <fmt.wrapError \| 0xc0008e53e0>: failed to create HCP cluster patch-tags-cluster: operation result model did not match expected model for type armredhatopenshifthcp.HcpOpenShiftCluster: &armredhatopenshifthcp.HcpOpenShiftCluster{ Location: &"westus3", Identity: &{Type: &"UserAssigned", UserAssignedIdentities: {"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-dev-246/providers/Microsoft.ManagedIdentity/userAssignedIdentities/cloud-controller-manager": nil, "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-dev-246/providers/Microsoft.ManagedIdentity/userAssignedIdentities/cloud-network-config": nil, "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-dev-246/providers/Microsoft.ManagedIdentity/userAssignedIdentities/cluster-api-azure": nil, "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-dev-246/providers/Microsoft.ManagedIdentity/userAssignedIdentities/control-plane": nil, ...}}, Properties: &armredhatopenshifthcp.HcpOpenShiftClusterProperties{ Platform: &{NetworkSecurityGroupID: &"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGrou"..., OperatorsAuthentication: &{UserAssignedIdentities: &{ControlPlaneOperators: {"cloud-controller-manager": &"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGrou"..., "cloud-network-config": &"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGrou"..., "cluster-api-azure": &"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGrou"..., "control-plane": &"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGrou"..., ...}, DataPlaneOperators: {"disk-csi-driver": &"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGrou"..., "file-csi-driver": &"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGrou"..., "image-registry": &"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGrou"...}, ServiceManagedIdentity: &"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGrou"...}}, SubnetID: &"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGrou"..., ManagedResourceGroup: &"patch-tags-kcmxqz--managed", ...}, API: &armredhatopenshifthcp.APIProfile{ Visibility: &"Public", - URL: &"https://api.p8m4m5l4t4k7n4e.bxyv.j6197248.hcp.osadev.cloud:443", + URL: nil, AuthorizedCIDRs: nil, }, Autoscaling: &{MaxNodeProvisionTimeSeconds: &900, MaxPodGracePeriodSeconds: &600, PodPriorityThreshold: &-10}, ClusterImageRegistry: &{State: &"Enabled"}, ... // 7 identical fields }, Tags: {"aro-hcp.experimental.cluster.size-override": &"Minimal"}, ID: &"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGrou"..., ... // 1 ignored and 2 identical fields } { msg: "failed to create HCP cluster patch-tags-cluster: operation result model did not match expected model for type armredhatopenshifthcp.HcpOpenShiftCluster:\n\u00a0\u00a0&armredhatopenshifthcp.HcpOpenShiftCluster{\n\u00a0\u00a0\tLocation: &\"westus3\",\n\u00a0\u00a0\tIdentity: &{Type: &\"UserAssigned\", UserAssignedIdentities: {\"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-dev-246/providers/Microsoft.ManagedIdentity/userAssignedIdentities/cloud-controller-manager\": nil, \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-dev-246/providers/Microsoft.ManagedIdentity/userAssignedIdentities/cloud-network-config\": nil, \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-dev-246/providers/Microsoft.ManagedIdentity/userAssignedIdentities/cluster-api-azure\": nil, \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-dev-246/providers/Microsoft.ManagedIdentity/userAssignedIdentities/control-plane\": nil, ...}},\n\u00a0\u00a0\tProperties: &armredhatopenshifthcp.HcpOpenShiftClusterProperties{\n\u00a0\u00a0\t\tPlatform: &{NetworkSecurityGroupID: &\"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGrou\"..., OperatorsAuthentication: &{UserAssignedIdentities: &{ControlPlaneOperators: {\"cloud-controller-manager\": &\"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGrou\"..., \"cloud-network-config\": &\"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGrou\"..., \"cluster-api-azure\": &\"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGrou\"..., \"control-plane\": &\"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGrou\"..., ...}, DataPlaneOperators: {\"disk-csi-driver\": &\"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGrou\"..., \"file-csi-driver\": &\"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGrou\"..., \"image-registry\": &\"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGrou\"...}, ServiceManagedIdentity: &\"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGrou\"...}}, SubnetID: &\"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGrou\"..., ManagedResourceGroup: &\"patch-tags-kcmxqz--managed\", ...},\n\u00a0\u00a0\t\tAPI: &armredhatopenshifthcp.APIProfile{\n\u00a0\u00a0\t\t\tVisibility: &\"Public\",\n-\u00a0\t\t\tURL: &\"https://api.p8m4m5l4t4k7n4e.bxyv.j6197248.hcp.osadev.cloud:443\",\n+\u00a0\t\t\tURL: nil,\n\u00a0\u00a0\t\t\tAuthorizedCIDRs: nil,\n\u00a0\u00a0\t\t},\n\u00a0\u00a0\t\tAutoscaling: &{MaxNodeProvisionTimeSeconds: &900, MaxPodGracePeriodSeconds: &600, PodPriorityThreshold: &-10},\n\u00a0\u00a0\t\tClusterImageRegistry: &{State: &\"Enabled\"},\n\u00a0\u00a0\t\t... // 7 identical fields\n\u00a0\u00a0\t},\n\u00a0\u00a0\tTags: {\"aro-hcp.experimental.cluster.size-override\": &\"Minimal\"},\n\u00a0\u00a0\tID: &\"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGrou\"...,\n\u00a0\u00a0\t... // 1 ignored and 2 identical fields\n\u00a0\u00a0}\n", err: <errors.errorString \| 0xc000de0590>{ s: "operation result model did not match expected model for type *armredhatopenshifthcp.HcpOpenShiftCluster:\n\u00a0\u00a0&armredhatopenshifthcp.HcpOpenShiftCluster{\n\u00a0\u00a0\tLocation: &\"westus3\",\n\u00a0\u00a0\tIdentity: &{Type: &\"UserAssigned\", UserAssignedIdentities: {\"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-dev-246/providers/Microsoft.ManagedIdentity/userAssignedIdentities/cloud-controller-manager\": nil, \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-dev-246/providers/Microsoft.ManagedIdentity/userAssignedIdentities/cloud-network-config\": nil, \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-container... Gomega truncated this representation as it exceeds 'format.MaxLength'. Consider having the object provide a custom 'GomegaStringer' representation or adjust the parameters in Gomega's 'format' package. Learn more here: https://onsi.github.io/gomega/#adjusting-output occurred
12:46:45	n/a	pull-ci-Azure-ARO-HCP-main-e2e-parallel head dacf876	n/a	unknown	failed	#4391 (open)	1	non-artifact-backed failure (no junit artifacts)
12:27:57	n/a	pull-ci-Azure-ARO-HCP-main-e2e-parallel merged PR head 66ed4b4 · merge 8a1ae2a	n/a	unknown	failed	#4399 (merged)	1	non-artifact-backed failure (no junit artifacts)
11:56:35	n/a	pull-ci-Azure-ARO-HCP-main-e2e-parallel merged PR head 6a89d15 · merge 0d2c310	n/a	unknown	failed	#4224 (merged)	1	non-artifact-backed failure (no junit artifacts)
11:48:19	n/a	pull-ci-Azure-ARO-HCP-main-e2e-parallel merged PR head 810b049 · merge 4655674	n/a	unknown	failed	#4405 (merged)	1	non-artifact-backed failure (no junit artifacts)
11:21:53	n/a	pull-ci-Azure-ARO-HCP-main-e2e-parallel post-goodmerged PR head 2cc3d47 · merge 2cc3d47	n/a	e2e	failed	★#4390 (merged)	1	e2e (1) not to be nil 11:21:53 UTC · failed at e2e · test Authorized CIDRs Connectivity should allow API access only from authorized VM IP · suite rp-api-compat-all/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_authorized_cidrs_connectivity.go:145]: Expected <string \| 0x0>: nil not to be nil fail [github.com/Azure/ARO-HCP/test/e2e/cluster_authorized_cidrs_connectivity.go:145]: Expected <string \| 0x0>: nil not to be nil
11:05:58	n/a	pull-ci-Azure-ARO-HCP-main-e2e-parallel merged PR head 3f58874 · merge 138964d	n/a	n/a	passed	#4226 (merged)	0	n/a
10:14:29	n/a	pull-ci-Azure-ARO-HCP-main-e2e-parallel merged PR head 6a89d15 · merge 0d2c310	n/a	unknown	failed	#4224 (merged)	1	non-artifact-backed failure (no junit artifacts)
10:13:24	n/a	pull-ci-Azure-ARO-HCP-main-e2e-parallel head fca3d7b	n/a	n/a	passed	#4402 (closed)	0	n/a
10:12:07	n/a	pull-ci-Azure-ARO-HCP-main-e2e-parallel post-goodmerged PR head 04b8269 · merge 04b8269	n/a	n/a	passed	#4392 (merged)	0	n/a
10:06:52	n/a	pull-ci-Azure-ARO-HCP-main-e2e-parallel merged PR head 11ecb87 · merge e0b528c	n/a	e2e	failed	★#4371 (merged)	1	e2e (1) not to be nil 10:06:52 UTC · failed at e2e · test Authorized CIDRs Connectivity should allow API access only from authorized VM IP · suite rp-api-compat-all/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_authorized_cidrs_connectivity.go:145]: Expected <string \| 0x0>: nil not to be nil fail [github.com/Azure/ARO-HCP/test/e2e/cluster_authorized_cidrs_connectivity.go:145]: Expected <string \| 0x0>: nil not to be nil
08:46:29	n/a	pull-ci-Azure-ARO-HCP-main-e2e-parallel head a9b9683	n/a	n/a	passed	#4401 (closed)	0	n/a
06:53:38	n/a	pull-ci-Azure-ARO-HCP-main-e2e-parallel merged PR head cd7540d · merge 07e421b	n/a	unknown	failed	#4385 (merged)	1	non-artifact-backed failure (no junit artifacts)
06:50:20	n/a	pull-ci-Azure-ARO-HCP-main-e2e-parallel post-goodmerged PR head 04b8269 · merge 04b8269	n/a	e2e	failed	★#4392 (merged)	1	e2e (1) not to be nil 06:50:20 UTC · failed at e2e · test Authorized CIDRs Connectivity should allow API access only from authorized VM IP · suite rp-api-compat-all/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_authorized_cidrs_connectivity.go:145]: Expected <string \| 0x0>: nil not to be nil fail [github.com/Azure/ARO-HCP/test/e2e/cluster_authorized_cidrs_connectivity.go:145]: Expected <string \| 0x0>: nil not to be nil
06:02:42	n/a	pull-ci-Azure-ARO-HCP-main-e2e-parallel post-goodmerged PR head 066f02d · merge 066f02d	n/a	n/a	passed	#4400 (merged)	0	n/a
06:02:31	n/a	pull-ci-Azure-ARO-HCP-main-e2e-parallel merged PR head 60648c7 · merge 07e421b	n/a	provision	failed	⚠#4385 (merged)	1	provision (1) ensure CRDs are installed first, resource mapping not found for name: "namespace-resourceid-mapping" namespace: "arobit" from "": no matches for kind "Policy" in version "policy.open-cluster-management.io/v1" 06:02:31 UTC · failed at provision · test Run pipeline step Microsoft.Azure.ARO.HCP.Management.Infra/management/arobit · suite step graph Show raw failure time=2026-03-11T06:33:16.882Z level=INFO msg="Running step." serviceGroup=Microsoft.Azure.ARO.HCP.Management.Infra resourceGroup=management step=arobit description="Step arobit\n Kind: Helm\n" time=2026-03-11T06:33:17.533Z level=INFO msg="Resolved input values." serviceGroup=Microsoft.Azure.ARO.HCP.Management.Infra resourceGroup=management step=arobit values="map[forwarder:map[azureCloudEnvironmentName:AzurePublicCloud clusterName:prow-j9726464-mgmt-1 clusterType:mgmt dynamicConfig:false environment:dev fluentbit:map[image:map[digest:sha256:2dd901545bab3c9e216681b34fab33835c392fb5ba54149cc9a8a008df2f97af pullPolicy:IfNotPresent registry:mcr.microsoft.com repository:oss/v2/fluent/fluent-bit]] geoShortId:us kusto:map[enabled:true hostedControlPlaneLogsDatabase:HostedControlPlaneLogs ingestionEndpoint:https://ingest-hcp-dev-us-2.eastus2.kusto.windows.net serviceLogsDatabase:ServiceLogs] mdsd:map[enabled:false geneva:map[clusterLogsAccountName:placeholder clusterLogsNamespace: clusterLogsSan: clusterLogsSecretName:clusterlogs configVersion:1 environment:Test region:westus3 role:hcp-underlay-prow-j9726464-mgmt-1 rpAccountName:placeholder rpNamespace: rpSan: rpSecretName:rplogs] image:map[digest:sha256:18d5476700adcd40e9ee2bc1cb62a1150ac1b5e696d7bad6adeea4da88c615db pullPolicy:IfNotPresent registry:mcr.microsoft.com repository:geneva/distroless/mdsd]] msiClientId:eafa4c14-2ebe-49e9-a4f3-33b7b6f54ed5 region:westus3 secretProvider:map[keyVault:ah-prow-mg-j9726464-1] securityContext:map[enabled:true runAsGroup:65532 runAsUser:65532] serviceAccountName:arobit-forwarder tenantId:XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX]]" time=2026-03-11T06:33:17.533Z level=INFO msg="Applying namespaces." serviceGroup=Microsoft.Azure.ARO.HCP.Management.Infra resourceGroup=management step=arobit time=2026-03-11T06:33:17.738Z level=INFO msg="Namespace applied successfully." serviceGroup=Microsoft.Azure.ARO.HCP.Management.Infra resourceGroup=management step=arobit namespace=arobit time=2026-03-11T06:33:17.967Z level=INFO msg="Rolling out Helm release." serviceGroup=Microsoft.Azure.ARO.HCP.Management.Infra resourceGroup=management step=arobit dryRun=false time=2026-03-11T06:33:17.967Z level=INFO msg="Searching for release history..." serviceGroup=Microsoft.Azure.ARO.HCP.Management.Infra resourceGroup=management step=arobit time=2026-03-11T06:33:18.072Z level=INFO msg="No release history found, running the first release..." serviceGroup=Microsoft.Azure.ARO.HCP.Management.Infra resourceGroup=management step=arobit time=2026-03-11T06:33:20.971Z level=ERROR msg="Failed to roll out the Helm release." serviceGroup=Microsoft.Azure.ARO.HCP.Management.Infra resourceGroup=management step=arobit err="unable to build kubernetes objects from release manifest: [resource mapping not found for name: \"default\" namespace: \"arobit\" from \"\": no matches for kind \"ManagedClusterSetBinding\" in version \"cluster.open-cluster-management.io/v1beta2\"\nensure CRDs are installed first, resource mapping not found for name: \"namespace-resourceid-mapping\" namespace: \"arobit\" from \"\": no matches for kind \"Placement\" in version \"cluster.open-cluster-management.io/v1beta1\"\nensure CRDs are installed first, resource mapping not found for name: \"namespace-resourceid-mapping\" namespace: \"arobit\" from \"\": no matches for kind \"PlacementBinding\" in version \"policy.open-cluster-management.io/v1\"\nensure CRDs are installed first, resource mapping not found for name: \"namespace-resourceid-mapping\" namespace: \"arobit\" from \"\": no matches for kind \"Policy\" in version \"policy.open-cluster-management.io/v1\"\nensure CRDs are installed first]" time=2026-03-11T06:33:20.971Z level=INFO msg="Finished deploying Helm release." serviceGroup=Microsoft.Azure.ARO.HCP.Management.Infra resourceGroup=management step=arobit time=2026-03-11T06:33:20.971Z level=INFO msg="Running inline diagnostics." serviceGroup=Microsoft.Azure.ARO.HCP.Management.Infra resourceGroup=management step=arobit time=2026-03-11T06:33:21.076Z level=ERROR msg="Failed to capture diagnostics for Helm release" serviceGroup=Microsoft.Azure.ARO.HCP.Management.Infra resourceGroup=management step=arobit err="failed to get status for release arobit: release: not found" time=2026-03-11T06:33:21.076Z level=INFO msg="Deployment complete." serviceGroup=Microsoft.Azure.ARO.HCP.Management.Infra resourceGroup=management step=arobit time=2026-03-11T06:33:21.077Z level=ERROR msg="Step errored." serviceGroup=Microsoft.Azure.ARO.HCP.Management.Infra resourceGroup=management step=arobit err="error running Helm release deployment Step, failed to deploy helm release: unable to build kubernetes objects from release manifest: [resource mapping not found for name: \"default\" namespace: \"arobit\" from \"\": no matches for kind \"ManagedClusterSetBinding\" in version \"cluster.open-cluster-management.io/v1beta2\"\nensure CRDs are installed first, resource mapping not found for name: \"namespace-resourceid-mapping\" namespace: \"arobit\" from \"\": no matches for kind \"Placement\" in version \"cluster.open-cluster-management.io/v1beta1\"\nensure CRDs are installed first, resource mapping not found for name: \"namespace-resourceid-mapping\" namespace: \"arobit\" from \"\": no matches for kind \"PlacementBinding\" in version \"policy.open-cluster-management.io/v1\"\nensure CRDs are installed first, resource mapping not found for name: \"namespace-resourceid-mapping\" namespace: \"arobit\" from \"\": no matches for kind \"Policy\" in version \"policy.open-cluster-management.io/v1\"\nensure CRDs are installed first]"
05:12:50	n/a	pull-ci-Azure-ARO-HCP-main-e2e-parallel head 240cba2	n/a	e2e	failed	#4381 (closed)	1	e2e (1) timeout during GetAdminRESTConfigForHCPCluster while waiting for hcpcluster creds 05:12:50 UTC · failed at e2e · test Customer should be able to create an HCP cluster and custom node pool osDisk size · suite rp-api-compat-all/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_nodepool_osdisk.go:104]: Unexpected error: <fmt.wrapErrors \| 0xc001176030>: failed waiting for hcpCluster="hcp-cluster-np-128" in resourcegroup="clusternp128-5t2k6m" to finish getting creds, caused by: timeout '10.000000' minutes exceeded during GetAdminRESTConfigForHCPCluster for cluster hcp-cluster-np-128 in resource group clusternp128-5t2k6m, error: context deadline exceeded { msg: "failed waiting for hcpCluster=\"hcp-cluster-np-128\" in resourcegroup=\"clusternp128-5t2k6m\" to finish getting creds, caused by: timeout '10.000000' minutes exceeded during GetAdminRESTConfigForHCPCluster for cluster hcp-cluster-np-128 in resource group clusternp128-5t2k6m, error: context deadline exceeded", errs: [ <errors.errorString \| 0xc0013bde40>{ s: "timeout '10.000000' minutes exceeded during GetAdminRESTConfigForHCPCluster for cluster hcp-cluster-np-128 in resource group clusternp128-5t2k6m", }, <context.deadlineExceededError>{}, ], } occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_nodepool_osdisk.go:104]: Unexpected error: <fmt.wrapErrors \| 0xc001176030>: failed waiting for hcpCluster="hcp-cluster-np-128" in resourcegroup="clusternp128-5t2k6m" to finish getting creds, caused by: timeout '10.000000' minutes exceeded during GetAdminRESTConfigForHCPCluster for cluster hcp-cluster-np-128 in resource group clusternp128-5t2k6m, error: context deadline exceeded { msg: "failed waiting for hcpCluster=\"hcp-cluster-np-128\" in resourcegroup=\"clusternp128-5t2k6m\" to finish getting creds, caused by: timeout '10.000000' minutes exceeded during GetAdminRESTConfigForHCPCluster for cluster hcp-cluster-np-128 in resource group clusternp128-5t2k6m, error: context deadline exceeded", errs: [ <errors.errorString \| 0xc0013bde40>{ s: "timeout '10.000000' minutes exceeded during GetAdminRESTConfigForHCPCluster for cluster hcp-cluster-np-128 in resource group clusternp128-5t2k6m", }, <context.deadlineExceededError>{}, ], } occurred
03:32:08	n/a	pull-ci-Azure-ARO-HCP-main-e2e-parallel merged PR head a909a00 · merge 732c175	n/a	provision	failed	⚠#4396 (merged)	1	provision (1) ERROR CODE: DeploymentFailed; detail code InvalidRequest; detail message The current utilization does not meet the criteria for both MaxTimeSeries and MaxEventsPerMinute quota requested. Please reach the required usage threshold of 50% of desired limit before requesting an increase, or request a limit increase of up to 200% of your current usage. 03:32:08 UTC · failed at provision · test Run pipeline step Microsoft.Azure.ARO.HCP.Region/regional/infra · suite step graph Show raw failure time=2026-03-11T04:00:37.985Z level=INFO msg="Running step." serviceGroup=Microsoft.Azure.ARO.HCP.Region resourceGroup=regional step=infra description="Step infra\n Kind: ARM\n Template: templates/region.bicep\n Parameters: configurations/region.tmpl.bicepparam" time=2026-03-11T04:00:40.711Z level=DEBUG+2 msg="Starting ARM deployment" serviceGroup=Microsoft.Azure.ARO.HCP.Region resourceGroup=regional step=infra time=2026-03-11T04:00:42.561Z level=DEBUG+3 msg="Deployment started" serviceGroup=Microsoft.Azure.ARO.HCP.Region resourceGroup=regional step=infra deployment=1c222b39ee1cd02410d607351b6398e01c49223f22abd6c6cd2bb3116170a5bd time=2026-03-11T04:02:43.608Z level=ERROR msg="Step errored." serviceGroup=Microsoft.Azure.ARO.HCP.Region resourceGroup=regional step=infra err="failed to run ARM step: failed to poll deployment: failed to wait for deployment completion: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourcegroups/hcp-underlay-prow-j2991616/providers/Microsoft.Resources/deployments/1c222b39ee1cd02410d607351b6398e01c49223f22abd6c6cd2bb3116170a5bd/operationStatuses/08584284052438857153\n--------------------------------------------------------------------------------\nRESPONSE 200: 200 OK\nERROR CODE: DeploymentFailed\n--------------------------------------------------------------------------------\n{\n \"status\": \"Failed\",\n \"error\": {\n \"code\": \"DeploymentFailed\",\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\n \"details\": [\n {\n \"code\": \"Conflict\",\n \"message\": \"{\\r\\n \\\"status\\\": \\\"Failed\\\",\\r\\n \\\"error\\\": {\\r\\n \\\"code\\\": \\\"ResourceDeploymentFailure\\\",\\r\\n \\\"message\\\": \\\"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\\\",\\r\\n \\\"details\\\": [\\r\\n {\\r\\n \\\"code\\\": \\\"DeploymentFailed\\\",\\r\\n \\\"target\\\": \\\"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/hcp-underlay-prow-j2991616/providers/Microsoft.Resources/deployments/svc-monitor-ingestion-limits\\\",\\r\\n \\\"message\\\": \\\"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\\\",\\r\\n \\\"details\\\": [\\r\\n {\\r\\n \\\"code\\\": \\\"InvalidRequest\\\",\\r\\n \\\"message\\\": \\\"The current utilization does not meet the criteria for both MaxTimeSeries and MaxEventsPerMinute quota requested. Please reach the required usage threshold of 50% of desired limit before requesting an increase, or request a limit increase of up to 200% of your current usage. For more details, see https://go.microsoft.com/fwlink/?linkid=2270124\\\"\\r\\n }\\r\\n ]\\r\\n }\\r\\n ]\\r\\n }\\r\\n}\"\n },\n {\n \"code\": \"Conflict\",\n \"message\": \"{\\r\\n \\\"status\\\": \\\"Failed\\\",\\r\\n \\\"error\\\": {\\r\\n \\\"code\\\": \\\"ResourceDeploymentFailure\\\",\\r\\n \\\"message\\\": \\\"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\\\",\\r\\n \\\"details\\\": [\\r\\n {\\r\\n \\\"code\\\": \\\"DeploymentFailed\\\",\\r\\n \\\"target\\\": \\\"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/hcp-underlay-prow-j2991616/providers/Microsoft.Resources/deployments/hcp-monitor-ingestion-limits\\\",\\r\\n \\\"message\\\": \\\"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\\\",\\r\\n \\\"details\\\": [\\r\\n {\\r\\n \\\"code\\\": \\\"InvalidRequest\\\",\\r\\n \\\"message\\\": \\\"The current utilization does not meet the criteria for both MaxTimeSeries and MaxEventsPerMinute quota requested. Please reach the required usage threshold of 50% of desired limit before requesting an increase, or request a limit increase of up to 200% of your current usage. For more details, see https://go.microsoft.com/fwlink/?linkid=2270124\\\"\\r\\n }\\r\\n ]\\r\\n }\\r\\n ]\\r\\n }\\r\\n}\"\n }\n ]\n }\n}\n--------------------------------------------------------------------------------\n"
02:39:34	n/a	pull-ci-Azure-ARO-HCP-main-e2e-parallel merged PR head 9a78661 · merge 04b8269	n/a	e2e	failed	★#4392 (merged)	1	e2e (1) not to be nil 02:39:34 UTC · failed at e2e · test Authorized CIDRs Connectivity should allow API access only from authorized VM IP · suite rp-api-compat-all/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_authorized_cidrs_connectivity.go:145]: Expected <string \| 0x0>: nil not to be nil fail [github.com/Azure/ARO-HCP/test/e2e/cluster_authorized_cidrs_connectivity.go:145]: Expected <string \| 0x0>: nil not to be nil
02:22:41	n/a	pull-ci-Azure-ARO-HCP-main-e2e-parallel head f2b2a6e	n/a	provision	failed	#4393 (open)	1	provision (1) resource not ready, name: aro-hcp-backend, kind: Deployment, status: InProgress context deadline exceeded 02:22:41 UTC · failed at provision · test Run pipeline step Microsoft.Azure.ARO.HCP.RP.Backend/service/deploy · suite step graph Show raw failure time=2026-03-11T03:14:45.630Z level=INFO msg="Running step." serviceGroup=Microsoft.Azure.ARO.HCP.RP.Backend resourceGroup=service step=deploy description="Step deploy\n Kind: Helm\n" time=2026-03-11T03:14:46.311Z level=INFO msg="Resolved input values." serviceGroup=Microsoft.Azure.ARO.HCP.RP.Backend resourceGroup=service step=deploy values="map[azure:map[cloudEnvironmentName:AzurePublicCloud msiRpDataPlaneAudienceResource:https://dummy.org ocpAcrHostname:arohcpocpdev.azurecr.io ocpAcrResourceId:/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/global/providers/Microsoft.ContainerRegistry/registries/arohcpocpdev oidcIssuerBaseUrl:https://arohcpprowoidcj7107840.z1.web.core.windows.net/ oidcIssuerBlobServiceUrl:https://arohcpprowoidcj7107840.blob.core.windows.net/ tenantId:XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX tlsCertificatesConfig:map[certificatesGenerationSource:AzureKeyVault issuer:Self]] clustersService:map[namespace:clusters-service serviceAccount:clusters-service] configMap:map[backendMiClientId:4d669851-7a83-488d-92cf-6d9462b59659 databaseName:arohcpprow-rp-j7107840 databaseUrl:https://arohcpprow-rp-j7107840.documents.azure.com:443/ location:westus3] deployment:map[imageName:arohcpsvcdev.azurecr.io/ci-op-9t7yjbwi/pipeline@sha256:e6fa518117bcf50c422889d8de382d6670b9bfe51669431df90285d833fa1136 replicas:2 zoneCount:4] exitOnPanic:true fpa:map[certName:firstPartyCert2 clientId:b3cb2fab-15cb-4583-ad06-f91da9bfe2d1] maestro:map[sourceEnvironmentIdentifier:arohcpprow] miMock:map[certName:msiMockCert2 clientId:e8723db7-9b9e-46a4-9f7d-64d75c3534f0 principalId:d6b62dfa-87f5-49b3-bbcb-4a687c4faa96 tenantId:XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] pullBinding:map[registry:arohcpsvcdev.azurecr.io scope:repository:ci-op-9t7yjbwi/pipeline:pull workloadIdentityClientId:91437a38-7620-40a8-8fdc-4a5a10bfe7e0 workloadIdentityTenantId:XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] serviceAccount:map[name:backend workloadIdentityClientId:4d669851-7a83-488d-92cf-6d9462b59659 workloadIdentityTenantId:XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] serviceKeyVault:map[name:aro-hcp-dev-svc-kv] tracing:map[address:http://ingest.observability:4318 exporter:otlp]]" time=2026-03-11T03:14:46.311Z level=INFO msg="Applying namespaces." serviceGroup=Microsoft.Azure.ARO.HCP.RP.Backend resourceGroup=service step=deploy time=2026-03-11T03:14:46.560Z level=INFO msg="Namespace applied successfully." serviceGroup=Microsoft.Azure.ARO.HCP.RP.Backend resourceGroup=service step=deploy namespace=aro-hcp time=2026-03-11T03:14:46.848Z level=INFO msg="Rolling out Helm release." serviceGroup=Microsoft.Azure.ARO.HCP.RP.Backend resourceGroup=service step=deploy dryRun=false time=2026-03-11T03:14:46.848Z level=INFO msg="Searching for release history..." serviceGroup=Microsoft.Azure.ARO.HCP.RP.Backend resourceGroup=service step=deploy time=2026-03-11T03:14:46.955Z level=INFO msg="No release history found, running the first release..." serviceGroup=Microsoft.Azure.ARO.HCP.RP.Backend resourceGroup=service step=deploy time=2026-03-11T03:19:53.025Z level=ERROR msg="Failed to roll out the Helm release." serviceGroup=Microsoft.Azure.ARO.HCP.RP.Backend resourceGroup=service step=deploy err="resource not ready, name: aro-hcp-backend, kind: Deployment, status: InProgress\ncontext deadline exceeded" time=2026-03-11T03:19:53.025Z level=INFO msg="Finished deploying Helm release." serviceGroup=Microsoft.Azure.ARO.HCP.RP.Backend resourceGroup=service step=deploy time=2026-03-11T03:19:53.025Z level=INFO msg="Running inline diagnostics." serviceGroup=Microsoft.Azure.ARO.HCP.RP.Backend resourceGroup=service step=deploy time=2026-03-11T03:19:55.584Z level=INFO msg="Determined release status." serviceGroup=Microsoft.Azure.ARO.HCP.RP.Backend resourceGroup=service step=deploy release=aro-hcp-backend-dev namespace=aro-hcp status=failed description="Release \"aro-hcp-backend-dev\" failed: resource not ready, name: aro-hcp-backend, kind: Deployment, status: InProgress\ncontext deadline exceeded" values="map[azure:map[cloudEnvironmentName:AzurePublicCloud msiRpDataPlaneAudienceResource:https://dummy.org ocpAcrHostname:arohcpocpdev.azurecr.io ocpAcrResourceId:/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/global/providers/Microsoft.ContainerRegistry/registries/arohcpocpdev oidcIssuerBaseUrl:https://arohcpprowoidcj7107840.z1.web.core.windows.net/ oidcIssuerBlobServiceUrl:https://arohcpprowoidcj7107840.blob.core.windows.net/ tenantId:XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX tlsCertificatesConfig:map[certificatesGenerationSource:AzureKeyVault issuer:Self]] clustersService:map[namespace:clusters-service serviceAccount:clusters-service] configMap:map[backendMiClientId:4d669851-7a83-488d-92cf-6d9462b59659 databaseName:arohcpprow-rp-j7107840 databaseUrl:https://arohcpprow-rp-j7107840.documents.azure.com:443/ location:westus3] deployment:map[imageName:arohcpsvcdev.azurecr.io/ci-op-9t7yjbwi/pipeline@sha256:e6fa518117bcf50c422889d8de382d6670b9bfe51669431df90285d833fa1136 replicas:2 zoneCount:4] exitOnPanic:true fpa:map[certName:firstPartyCert2 clientId:b3cb2fab-15cb-4583-ad06-f91da9bfe2d1] maestro:map[sourceEnvironmentIdentifier:arohcpprow] miMock:map[certName:msiMockCert2 clientId:e8723db7-9b9e-46a4-9f7d-64d75c3534f0 principalId:d6b62dfa-87f5-49b3-bbcb-4a687c4faa96 tenantId:XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] pullBinding:map[registry:arohcpsvcdev.azurecr.io scope:repository:ci-op-9t7yjbwi/pipeline:pull workloadIdentityClientId:91437a38-7620-40a8-8fdc-4a5a10bfe7e0 workloadIdentityTenantId:XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] serviceAccount:map[name:backend workloadIdentityClientId:4d669851-7a83-488d-92cf-6d9462b59659 workloadIdentityTenantId:XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] serviceKeyVault:map[name:aro-hcp-dev-svc-kv] tracing:map[address:http://ingest.observability:4318 exporter:otlp]]" time=2026-03-11T03:19:55.587Z level=INFO msg="Kube-events kusto link for troubleshooting:" serviceGroup=Microsoft.Azure.ARO.HCP.RP.Backend resourceGroup=service step=deploy url="https://hcp-dev-us-2.eastus2.kusto.windows.net/ServiceLogs?query=H4sIAAAAAAAA/5RTwWrbQBS86yseOsWgDXacBuqSg5v2VFJM01NKD6vdqbzxap94u7Jx8McXybgWwSbudd7Mm9nHTuaRSBC5FYNI92R10kmXHldcvsCkby7YWUziQlXQHvqua/yDsEbokdhoc4BHvzIionzepiWLe9XJcViwd2abF5Rr73mjaiRxJqpSmxWC7QfCammavNjLnyBrZzA3htuQOsK73MHksPhgdEK0AKTLiJCc6UP+n/4JRpAWwmtnIQ9exzhIqeI+k1phq9a69el87kcOLrGcsD+heeDwx1WPuhma6ddWoKQNydVQpqdcqj3L/sG+P6mHthAFD/PmSkPmZxesC9Vw8/vCL2g8b2uEdNnj50YWrfcnvJrWe1Ue8YMw+/0pW7UlJCAhfu0+bMx2tFlCQN21YtJ1QyXSBgh0ZXVCB1/lN+ObOzWeqsnk53g6m9zObu+e8xFdX9NZzsfZh+lzPhplO3phF2jlgr13IUAGNeNAx3YNa/W2T9mOGuFuegxaUMN2T75sSUGeq4JqxKirbiWLhVC5HTzeIpq/AQAA//+9aZdCDAQAAA==" time=2026-03-11T03:19:55.592Z level=INFO msg="Deployment complete." serviceGroup=Microsoft.Azure.ARO.HCP.RP.Backend resourceGroup=service step=deploy time=2026-03-11T03:19:55.592Z level=ERROR msg="Step errored." serviceGroup=Microsoft.Azure.ARO.HCP.RP.Backend resourceGroup=service step=deploy err="error running Helm release deployment Step, failed to deploy helm release: resource not ready, name: aro-hcp-backend, kind: Deployment, status: InProgress\ncontext deadline exceeded"
01:48:17	n/a	pull-ci-Azure-ARO-HCP-main-e2e-parallel merged PR head 785df32 · merge 732c175	n/a	provision	failed	⚠#4396 (merged)	1	provision (1) ERROR CODE: DeploymentFailed; detail code InvalidResource; detail message Invalid metrics container payload: 'location' is missing. 01:48:17 UTC · failed at provision · test Run pipeline step Microsoft.Azure.ARO.HCP.Region/regional/infra · suite step graph Show raw failure time=2026-03-11T02:22:40.802Z level=INFO msg="Running step." serviceGroup=Microsoft.Azure.ARO.HCP.Region resourceGroup=regional step=infra description="Step infra\n Kind: ARM\n Template: templates/region.bicep\n Parameters: configurations/region.tmpl.bicepparam" time=2026-03-11T02:22:43.670Z level=DEBUG+2 msg="Starting ARM deployment" serviceGroup=Microsoft.Azure.ARO.HCP.Region resourceGroup=regional step=infra time=2026-03-11T02:22:44.871Z level=DEBUG+3 msg="Deployment started" serviceGroup=Microsoft.Azure.ARO.HCP.Region resourceGroup=regional step=infra deployment=3e02e5bd4bdbb503054749825771ac6fe920e18921f6ea12fe330fe6be51adcc time=2026-03-11T02:24:15.639Z level=ERROR msg="Step errored." serviceGroup=Microsoft.Azure.ARO.HCP.Region resourceGroup=regional step=infra err="failed to run ARM step: failed to poll deployment: failed to wait for deployment completion: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourcegroups/hcp-underlay-prow-j7588480/providers/Microsoft.Resources/deployments/3e02e5bd4bdbb503054749825771ac6fe920e18921f6ea12fe330fe6be51adcc/operationStatuses/08584284111213662348\n--------------------------------------------------------------------------------\nRESPONSE 200: 200 OK\nERROR CODE: DeploymentFailed\n--------------------------------------------------------------------------------\n{\n \"status\": \"Failed\",\n \"error\": {\n \"code\": \"DeploymentFailed\",\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\n \"details\": [\n {\n \"code\": \"Conflict\",\n \"message\": \"{\\r\\n \\\"status\\\": \\\"Failed\\\",\\r\\n \\\"error\\\": {\\r\\n \\\"code\\\": \\\"ResourceDeploymentFailure\\\",\\r\\n \\\"message\\\": \\\"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\\\",\\r\\n \\\"details\\\": [\\r\\n {\\r\\n \\\"code\\\": \\\"DeploymentFailed\\\",\\r\\n \\\"target\\\": \\\"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/hcp-underlay-prow-j7588480/providers/Microsoft.Resources/deployments/hcp-monitor-ingestion-limits\\\",\\r\\n \\\"message\\\": \\\"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\\\",\\r\\n \\\"details\\\": [\\r\\n {\\r\\n \\\"code\\\": \\\"InvalidResource\\\",\\r\\n \\\"message\\\": \\\"Invalid metrics container payload: 'location' is missing.\\\",\\r\\n \\\"details\\\": [\\r\\n {\\r\\n \\\"code\\\": \\\"InvalidResource\\\",\\r\\n \\\"target\\\": \\\"location\\\",\\r\\n \\\"message\\\": \\\"Invalid metrics container payload: 'location' is missing.\\\"\\r\\n }\\r\\n ]\\r\\n }\\r\\n ]\\r\\n }\\r\\n ]\\r\\n }\\r\\n}\"\n },\n {\n \"code\": \"Conflict\",\n \"message\": \"{\\r\\n \\\"status\\\": \\\"Failed\\\",\\r\\n \\\"error\\\": {\\r\\n \\\"code\\\": \\\"ResourceDeploymentFailure\\\",\\r\\n \\\"message\\\": \\\"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\\\",\\r\\n \\\"details\\\": [\\r\\n {\\r\\n \\\"code\\\": \\\"DeploymentFailed\\\",\\r\\n \\\"target\\\": \\\"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/hcp-underlay-prow-j7588480/providers/Microsoft.Resources/deployments/svc-monitor-ingestion-limits\\\",\\r\\n \\\"message\\\": \\\"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\\\",\\r\\n \\\"details\\\": [\\r\\n {\\r\\n \\\"code\\\": \\\"InvalidResource\\\",\\r\\n \\\"message\\\": \\\"Invalid metrics container payload: 'location' is missing.\\\",\\r\\n \\\"details\\\": [\\r\\n {\\r\\n \\\"code\\\": \\\"InvalidResource\\\",\\r\\n \\\"target\\\": \\\"location\\\",\\r\\n \\\"message\\\": \\\"Invalid metrics container payload: 'location' is missing.\\\"\\r\\n }\\r\\n ]\\r\\n }\\r\\n ]\\r\\n }\\r\\n ]\\r\\n }\\r\\n}\"\n }\n ]\n }\n}\n--------------------------------------------------------------------------------\n"
00:40:22	n/a	pull-ci-Azure-ARO-HCP-main-e2e-parallel merged PR head 9a78661 · merge 04b8269	n/a	n/a	passed	#4392 (merged)	0	n/a
00:08:37	n/a	pull-ci-Azure-ARO-HCP-main-e2e-parallel post-goodmerged PR head 506b932 · merge 506b932	n/a	n/a	passed	#4372 (merged)	0	n/a