time=2026-03-27T19:19:05.051Z level=INFO msg="Running step." serviceGroup=Microsoft.Azure.ARO.HCP.Management.Infra resourceGroup=management step=cluster description="Step cluster\n  Kind: ARM\n  Template: templates/mgmt-cluster.bicep\n  Parameters: configurations/mgmt-cluster.tmpl.bicepparam"
time=2026-03-27T19:19:05.846Z level=DEBUG+2 msg="Starting ARM deployment" serviceGroup=Microsoft.Azure.ARO.HCP.Management.Infra resourceGroup=management step=cluster
time=2026-03-27T19:19:07.381Z level=DEBUG+3 msg="Deployment started" serviceGroup=Microsoft.Azure.ARO.HCP.Management.Infra resourceGroup=management step=cluster deployment=ebfac1080e8056bda95a598be1328338af0d62013965b54fd55fc53065574571
time=2026-03-27T19:31:10.214Z level=ERROR msg="Step errored." serviceGroup=Microsoft.Azure.ARO.HCP.Management.Infra resourceGroup=management step=cluster err="failed to run ARM step: failed to poll deployment: failed to wait for deployment completion: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourcegroups/hcp-underlay-prow-j9847552-mgmt-1/providers/Microsoft.Resources/deployments/ebfac1080e8056bda95a598be1328338af0d62013965b54fd55fc53065574571/operationStatuses/08584269677389986705\n--------------------------------------------------------------------------------\nRESPONSE 200: 200 OK\nERROR CODE: DeploymentFailed\n--------------------------------------------------------------------------------\n{\n  \"status\": \"Failed\",\n  \"error\": {\n    \"code\": \"DeploymentFailed\",\n    \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\n    \"details\": [\n      {\n        \"code\": \"Conflict\",\n        \"message\": \"{\\r\\n  \\\"status\\\": \\\"Failed\\\",\\r\\n  \\\"error\\\": {\\r\\n    \\\"code\\\": \\\"ResourceDeploymentFailure\\\",\\r\\n    \\\"message\\\": \\\"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\\\",\\r\\n    \\\"details\\\": [\\r\\n      {\\r\\n        \\\"code\\\": \\\"DeploymentFailed\\\",\\r\\n        \\\"target\\\": \\\"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/hcp-underlay-prow-j9847552-mgmt-1/providers/Microsoft.Resources/deployments/maestro-consumer\\\",\\r\\n        \\\"message\\\": \\\"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\\\",\\r\\n        \\\"details\\\": [\\r\\n          {\\r\\n            \\\"code\\\": \\\"ResourceDeploymentFailure\\\",\\r\\n            \\\"target\\\": \\\"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/hcp-underlay-prow-j9847552-mgmt-1/providers/Microsoft.Resources/deployments/maestro-eg-crt-kmvtx2yprmvuq\\\",\\r\\n            \\\"message\\\": \\\"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\\\",\\r\\n            \\\"details\\\": [\\r\\n              {\\r\\n                \\\"code\\\": \\\"DeploymentFailed\\\",\\r\\n                \\\"target\\\": \\\"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/hcp-underlay-prow-j9847552-mgmt-1/providers/Microsoft.Resources/deployments/maestro-eg-crt-kmvtx2yprmvuq\\\",\\r\\n                \\\"message\\\": \\\"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\\\",\\r\\n                \\\"details\\\": [\\r\\n                  {\\r\\n                    \\\"code\\\": \\\"ResourceDeploymentFailure\\\",\\r\\n                    \\\"target\\\": \\\"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/hcp-underlay-prow-j9847552-mgmt-1/providers/Microsoft.Resources/deployments/hcp-underlay-j9847552-mgmt-1-cert\\\",\\r\\n                    \\\"message\\\": \\\"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\\\",\\r\\n                    \\\"details\\\": [\\r\\n                      {\\r\\n                        \\\"code\\\": \\\"DeploymentFailed\\\",\\r\\n                        \\\"target\\\": \\\"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/hcp-underlay-prow-j9847552-mgmt-1/providers/Microsoft.Resources/deployments/hcp-underlay-j9847552-mgmt-1-cert\\\",\\r\\n                        \\\"message\\\": \\\"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\\\",\\r\\n                        \\\"details\\\": [\\r\\n                          {\\r\\n                            \\\"code\\\": \\\"ResourceDeploymentFailure\\\",\\r\\n                            \\\"target\\\": \\\"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/hcp-underlay-prow-j9847552-mgmt-1/providers/Microsoft.Resources/deploymentScripts/newCert-hcp-underlay-j9847552-mgmt-1\\\",\\r\\n                            \\\"message\\\": \\\"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'failed'.\\\",\\r\\n                            \\\"details\\\": [\\r\\n                              {\\r\\n                                \\\"code\\\": \\\"DeploymentScriptOperationFailed\\\",\\r\\n                                \\\"message\\\": \\\"A task was canceled.\\\"\\r\\n                              }\\r\\n                            ]\\r\\n                          }\\r\\n                        ]\\r\\n                      }\\r\\n                    ]\\r\\n                  }\\r\\n                ]\\r\\n              }\\r\\n            ]\\r\\n          }\\r\\n        ]\\r\\n      }\\r\\n    ]\\r\\n  }\\r\\n}\"\n      }\n    ]\n  }\n}\n--------------------------------------------------------------------------------\n"

fail [:0]: Interrupted by User
fail [:0]: Interrupted by User

fail [github.com/Azure/ARO-HCP/test/e2e/simple_negative_cases.go:234]: Unexpected error:
    <*errors.joinError | 0xc001180600>: 
    invalid node pool version update: expected error but none occurred
    updating immutable fields: expected error containing keywords 'forbidden field immutable', got: PUT https://rp.j2393088.hcpsvc.osadev.cloud/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-negative-tests-7wccc9/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/negative-tests-cluster/nodePools/np-1
    --------------------------------------------------------------------------------
    RESPONSE 409: 409 Conflict
    ERROR CODE: Conflict
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "Conflict",
        "message": "Cannot update resource while resource is \"accepted\"",
        "target": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-negative-tests-7wccc9/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/negative-tests-cluster/nodePools/np-1"
      }
    }
    --------------------------------------------------------------------------------
    
    {
        errs: [
            <*errors.errorString | 0xc000955650>{
                s: "invalid node pool version update: expected error but none occurred",
            },
            <*errors.errorString | 0xc000e49960>{
                s: "updating immutable fields: expected error containing keywords 'forbidden field immutable', got: PUT https://rp.j2393088.hcpsvc.osadev.cloud/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-negative-tests-7wccc9/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/negative-tests-cluster/nodePools/np-1\n--------------------------------------------------------------------------------\nRESPONSE 409: 409 Conflict\nERROR CODE: Conflict\n--------------------------------------------------------------------------------\n{\n  \"error\": {\n    \"code\": \"Conflict\",\n    \"message\": \"Cannot update resource while resource is \\\"accepted\\\"\",\n    \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-negative-tests-7wccc9/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/negative-tests-cluster/nodePools/np-1\"\n  }\n}\n--------------------------------------------------------------------------------\n",
            },
        ],
    }
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/simple_negative_cases.go:234]: Unexpected error:
    <*errors.joinError | 0xc001180600>: 
    invalid node pool version update: expected error but none occurred
    updating immutable fields: expected error containing keywords 'forbidden field immutable', got: PUT https://rp.j2393088.hcpsvc.osadev.cloud/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-negative-tests-7wccc9/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/negative-tests-cluster/nodePools/np-1
    --------------------------------------------------------------------------------
    RESPONSE 409: 409 Conflict
    ERROR CODE: Conflict
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "Conflict",
        "message": "Cannot update resource while resource is \"accepted\"",
        "target": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-negative-tests-7wccc9/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/negative-tests-cluster/nodePools/np-1"
      }
    }
    --------------------------------------------------------------------------------
    
    {
        errs: [
            <*errors.errorString | 0xc000955650>{
                s: "invalid node pool version update: expected error but none occurred",
            },
            <*errors.errorString | 0xc000e49960>{
                s: "updating immutable fields: expected error containing keywords 'forbidden field immutable', got: PUT https://rp.j2393088.hcpsvc.osadev.cloud/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-negative-tests-7wccc9/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/negative-tests-cluster/nodePools/np-1\n--------------------------------------------------------------------------------\nRESPONSE 409: 409 Conflict\nERROR CODE: Conflict\n--------------------------------------------------------------------------------\n{\n  \"error\": {\n    \"code\": \"Conflict\",\n    \"message\": \"Cannot update resource while resource is \\\"accepted\\\"\",\n    \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-negative-tests-7wccc9/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/negative-tests-cluster/nodePools/np-1\"\n  }\n}\n--------------------------------------------------------------------------------\n",
            },
        ],
    }
occurred

fail [github.com/Azure/ARO-HCP/test/e2e/idms_lifecycle.go:255]: Unexpected error:
    <*fmt.wrapErrors | 0xc000d60f60>: 
    failed waiting for hcpCluster="idms-e2e-hcp-cluster" in resourcegroup="idms-zmsvbg" to finish updating, caused by: timeout '10.000000' minutes exceeded during UpdateHCPCluster20251223 for cluster idms-e2e-hcp-cluster in resource group idms-zmsvbg, error: context deadline exceeded
    {
        msg: "failed waiting for hcpCluster=\"idms-e2e-hcp-cluster\" in resourcegroup=\"idms-zmsvbg\" to finish updating, caused by: timeout '10.000000' minutes exceeded during UpdateHCPCluster20251223 for cluster idms-e2e-hcp-cluster in resource group idms-zmsvbg, error: context deadline exceeded",
        errs: [
            <*errors.errorString | 0xc0009b4060>{
                s: "timeout '10.000000' minutes exceeded during UpdateHCPCluster20251223 for cluster idms-e2e-hcp-cluster in resource group idms-zmsvbg",
            },
            <context.deadlineExceededError>{},
        ],
    }
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/idms_lifecycle.go:255]: Unexpected error:
    <*fmt.wrapErrors | 0xc000d60f60>: 
    failed waiting for hcpCluster="idms-e2e-hcp-cluster" in resourcegroup="idms-zmsvbg" to finish updating, caused by: timeout '10.000000' minutes exceeded during UpdateHCPCluster20251223 for cluster idms-e2e-hcp-cluster in resource group idms-zmsvbg, error: context deadline exceeded
    {
        msg: "failed waiting for hcpCluster=\"idms-e2e-hcp-cluster\" in resourcegroup=\"idms-zmsvbg\" to finish updating, caused by: timeout '10.000000' minutes exceeded during UpdateHCPCluster20251223 for cluster idms-e2e-hcp-cluster in resource group idms-zmsvbg, error: context deadline exceeded",
        errs: [
            <*errors.errorString | 0xc0009b4060>{
                s: "timeout '10.000000' minutes exceeded during UpdateHCPCluster20251223 for cluster idms-e2e-hcp-cluster in resource group idms-zmsvbg",
            },
            <context.deadlineExceededError>{},
        ],
    }
occurred

fail [:0]: Interrupted by User
fail [:0]: Interrupted by User

fail [github.com/Azure/ARO-HCP/test/e2e/cluster_authorized_cidrs_connectivity.go:185]: Unexpected error:
    <*fmt.wrapErrors | 0xc000bc2e10>: 
    failed waiting for hcpCluster="cidr-connectivity-test" in resourcegroup="e2e-cidr-connectivity-hfjx5x" to finish getting creds, caused by: timeout '10.000000' minutes exceeded during GetAdminRESTConfigForHCPCluster for cluster cidr-connectivity-test in resource group e2e-cidr-connectivity-hfjx5x, error: context deadline exceeded
    {
        msg: "failed waiting for hcpCluster=\"cidr-connectivity-test\" in resourcegroup=\"e2e-cidr-connectivity-hfjx5x\" to finish getting creds, caused by: timeout '10.000000' minutes exceeded during GetAdminRESTConfigForHCPCluster for cluster cidr-connectivity-test in resource group e2e-cidr-connectivity-hfjx5x, error: context deadline exceeded",
        errs: [
            <*errors.errorString | 0xc00020cac0>{
                s: "timeout '10.000000' minutes exceeded during GetAdminRESTConfigForHCPCluster for cluster cidr-connectivity-test in resource group e2e-cidr-connectivity-hfjx5x",
            },
            <context.deadlineExceededError>{},
        ],
    }
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_authorized_cidrs_connectivity.go:185]: Unexpected error:
    <*fmt.wrapErrors | 0xc000bc2e10>: 
    failed waiting for hcpCluster="cidr-connectivity-test" in resourcegroup="e2e-cidr-connectivity-hfjx5x" to finish getting creds, caused by: timeout '10.000000' minutes exceeded during GetAdminRESTConfigForHCPCluster for cluster cidr-connectivity-test in resource group e2e-cidr-connectivity-hfjx5x, error: context deadline exceeded
    {
        msg: "failed waiting for hcpCluster=\"cidr-connectivity-test\" in resourcegroup=\"e2e-cidr-connectivity-hfjx5x\" to finish getting creds, caused by: timeout '10.000000' minutes exceeded during GetAdminRESTConfigForHCPCluster for cluster cidr-connectivity-test in resource group e2e-cidr-connectivity-hfjx5x, error: context deadline exceeded",
        errs: [
            <*errors.errorString | 0xc00020cac0>{
                s: "timeout '10.000000' minutes exceeded during GetAdminRESTConfigForHCPCluster for cluster cidr-connectivity-test in resource group e2e-cidr-connectivity-hfjx5x",
            },
            <context.deadlineExceededError>{},
        ],
    }
occurred

fail [github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:246] A node timeout occurred and then the following failure was recorded in the timedout node before it exited:
Unexpected error:
    <*errors.joinError | 0xc001208168>: 
    failed to cleanup resource group: at least one hcp cluster failed to delete: failed waiting for hcpCluster="oidc-wi-cluster" in resourcegroup="oidc-wi-b2nl5w" to finish deleting: context canceled
    {
        errs: [
            <*fmt.wrapError | 0xc000cf1100>{
                msg: "failed to cleanup resource group: at least one hcp cluster failed to delete: failed waiting for hcpCluster=\"oidc-wi-cluster\" in resourcegroup=\"oidc-wi-b2nl5w\" to finish deleting: context canceled",
                err: <*fmt.wrapError | 0xc000cf10e0>{
                    msg: "at least one hcp cluster failed to delete: failed waiting for hcpCluster=\"oidc-wi-cluster\" in resourcegroup=\"oidc-wi-b2nl5w\" to finish deleting: context canceled",
                    err: <*fmt.wrapError | 0xc000cf10c0>{
                        msg: "failed waiting for hcpCluster=\"oidc-wi-cluster\" in resourcegroup=\"oidc-wi-b2nl5w\" to finish deleting: context canceled",
                        err: <*errors.errorString | 0xb2b17d0>{
                            s: "context canceled",
                        },
                    },
                },
            },
        ],
    }
occurred  
fail [:0]: A node timeout occurred
fail [github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:246] A node timeout occurred and then the following failure was recorded in the timedout node before it exited:
Unexpected error:
    <*errors.joinError | 0xc001208168>: 
    failed to cleanup resource group: at least one hcp cluster failed to delete: failed waiting for hcpCluster="oidc-wi-cluster" in resourcegroup="oidc-wi-b2nl5w" to finish deleting: context canceled
    {
        errs: [
            <*fmt.wrapError | 0xc000cf1100>{
                msg: "failed to cleanup resource group: at least one hcp cluster failed to delete: failed waiting for hcpCluster=\"oidc-wi-cluster\" in resourcegroup=\"oidc-wi-b2nl5w\" to finish deleting: context canceled",
                err: <*fmt.wrapError | 0xc000cf10e0>{
                    msg: "at least one hcp cluster failed to delete: failed waiting for hcpCluster=\"oidc-wi-cluster\" in resourcegroup=\"oidc-wi-b2nl5w\" to finish deleting: context canceled",
                    err: <*fmt.wrapError | 0xc000cf10c0>{
                        msg: "failed waiting for hcpCluster=\"oidc-wi-cluster\" in resourcegroup=\"oidc-wi-b2nl5w\" to finish deleting: context canceled",
                        err: <*errors.errorString | 0xb2b17d0>{
                            s: "context canceled",
                        },
                    },
                },
            },
        ],
    }
occurred  
fail [:0]: A node timeout occurred

fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_update_nodes.go:151]: Unexpected error:
    <*fmt.wrapErrors | 0xc000a293e0>: 
    failed waiting for nodepool="np-update-nodes" in cluster="np-update-nodes-hcp-cluster" resourcegroup="nodepool-update-nodes-4h255n" to finish updating, caused by: timeout '20.000000' minutes exceeded during UpdateNodePoolAndWait for nodepool np-update-nodes in cluster np-update-nodes-hcp-cluster in resource group nodepool-update-nodes-4h255n, error: context deadline exceeded
    {
        msg: "failed waiting for nodepool=\"np-update-nodes\" in cluster=\"np-update-nodes-hcp-cluster\" resourcegroup=\"nodepool-update-nodes-4h255n\" to finish updating, caused by: timeout '20.000000' minutes exceeded during UpdateNodePoolAndWait for nodepool np-update-nodes in cluster np-update-nodes-hcp-cluster in resource group nodepool-update-nodes-4h255n, error: context deadline exceeded",
        errs: [
            <*errors.errorString | 0xc000095810>{
                s: "timeout '20.000000' minutes exceeded during UpdateNodePoolAndWait for nodepool np-update-nodes in cluster np-update-nodes-hcp-cluster in resource group nodepool-update-nodes-4h255n",
            },
            <context.deadlineExceededError>{},
        ],
    }
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_update_nodes.go:151]: Unexpected error:
    <*fmt.wrapErrors | 0xc000a293e0>: 
    failed waiting for nodepool="np-update-nodes" in cluster="np-update-nodes-hcp-cluster" resourcegroup="nodepool-update-nodes-4h255n" to finish updating, caused by: timeout '20.000000' minutes exceeded during UpdateNodePoolAndWait for nodepool np-update-nodes in cluster np-update-nodes-hcp-cluster in resource group nodepool-update-nodes-4h255n, error: context deadline exceeded
    {
        msg: "failed waiting for nodepool=\"np-update-nodes\" in cluster=\"np-update-nodes-hcp-cluster\" resourcegroup=\"nodepool-update-nodes-4h255n\" to finish updating, caused by: timeout '20.000000' minutes exceeded during UpdateNodePoolAndWait for nodepool np-update-nodes in cluster np-update-nodes-hcp-cluster in resource group nodepool-update-nodes-4h255n, error: context deadline exceeded",
        errs: [
            <*errors.errorString | 0xc000095810>{
                s: "timeout '20.000000' minutes exceeded during UpdateNodePoolAndWait for nodepool np-update-nodes in cluster np-update-nodes-hcp-cluster in resource group nodepool-update-nodes-4h255n",
            },
            <context.deadlineExceededError>{},
        ],
    }
occurred

fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:110]: Unexpected error:
    <*fmt.wrapErrors | 0xc0009e4090>: 
    failed waiting for hcpCluster="cluster-zstream-4-21-vjd65z" in resourcegroup="rg-zstream-upgrade-4-21-vbxrjb" to finish getting creds, caused by: timeout '10.000000' minutes exceeded during GetAdminRESTConfigForHCPCluster for cluster cluster-zstream-4-21-vjd65z in resource group rg-zstream-upgrade-4-21-vbxrjb, error: context deadline exceeded
    {
        msg: "failed waiting for hcpCluster=\"cluster-zstream-4-21-vjd65z\" in resourcegroup=\"rg-zstream-upgrade-4-21-vbxrjb\" to finish getting creds, caused by: timeout '10.000000' minutes exceeded during GetAdminRESTConfigForHCPCluster for cluster cluster-zstream-4-21-vjd65z in resource group rg-zstream-upgrade-4-21-vbxrjb, error: context deadline exceeded",
        errs: [
            <*errors.errorString | 0xc000a802d0>{
                s: "timeout '10.000000' minutes exceeded during GetAdminRESTConfigForHCPCluster for cluster cluster-zstream-4-21-vjd65z in resource group rg-zstream-upgrade-4-21-vbxrjb",
            },
            <context.deadlineExceededError>{},
        ],
    }
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:110]: Unexpected error:
    <*fmt.wrapErrors | 0xc0009e4090>: 
    failed waiting for hcpCluster="cluster-zstream-4-21-vjd65z" in resourcegroup="rg-zstream-upgrade-4-21-vbxrjb" to finish getting creds, caused by: timeout '10.000000' minutes exceeded during GetAdminRESTConfigForHCPCluster for cluster cluster-zstream-4-21-vjd65z in resource group rg-zstream-upgrade-4-21-vbxrjb, error: context deadline exceeded
    {
        msg: "failed waiting for hcpCluster=\"cluster-zstream-4-21-vjd65z\" in resourcegroup=\"rg-zstream-upgrade-4-21-vbxrjb\" to finish getting creds, caused by: timeout '10.000000' minutes exceeded during GetAdminRESTConfigForHCPCluster for cluster cluster-zstream-4-21-vjd65z in resource group rg-zstream-upgrade-4-21-vbxrjb, error: context deadline exceeded",
        errs: [
            <*errors.errorString | 0xc000a802d0>{
                s: "timeout '10.000000' minutes exceeded during GetAdminRESTConfigForHCPCluster for cluster cluster-zstream-4-21-vjd65z in resource group rg-zstream-upgrade-4-21-vbxrjb",
            },
            <context.deadlineExceededError>{},
        ],
    }
occurred

fail [:0]: Interrupted by User
fail [:0]: Interrupted by User

fail [:0]: Interrupted by User
fail [:0]: Interrupted by User

fail [github.com/Azure/ARO-HCP/test/e2e/admin_credential_lifecycle.go:174]: Unexpected error:
    <*fmt.wrapErrors | 0xc0006377d0>: 
    failed waiting for hcpCluster="admin-cred-lifecycle-kk24qw" in resourcegroup="admin-credential-lifecycle-test-n47zt9" to finish getting creds, caused by: timeout '10.000000' minutes exceeded during GetAdminRESTConfigForHCPCluster for cluster admin-cred-lifecycle-kk24qw in resource group admin-credential-lifecycle-test-n47zt9, error: context deadline exceeded
    {
        msg: "failed waiting for hcpCluster=\"admin-cred-lifecycle-kk24qw\" in resourcegroup=\"admin-credential-lifecycle-test-n47zt9\" to finish getting creds, caused by: timeout '10.000000' minutes exceeded during GetAdminRESTConfigForHCPCluster for cluster admin-cred-lifecycle-kk24qw in resource group admin-credential-lifecycle-test-n47zt9, error: context deadline exceeded",
        errs: [
            <*errors.errorString | 0xc0007a6c10>{
                s: "timeout '10.000000' minutes exceeded during GetAdminRESTConfigForHCPCluster for cluster admin-cred-lifecycle-kk24qw in resource group admin-credential-lifecycle-test-n47zt9",
            },
            <context.deadlineExceededError>{},
        ],
    }
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/admin_credential_lifecycle.go:174]: Unexpected error:
    <*fmt.wrapErrors | 0xc0006377d0>: 
    failed waiting for hcpCluster="admin-cred-lifecycle-kk24qw" in resourcegroup="admin-credential-lifecycle-test-n47zt9" to finish getting creds, caused by: timeout '10.000000' minutes exceeded during GetAdminRESTConfigForHCPCluster for cluster admin-cred-lifecycle-kk24qw in resource group admin-credential-lifecycle-test-n47zt9, error: context deadline exceeded
    {
        msg: "failed waiting for hcpCluster=\"admin-cred-lifecycle-kk24qw\" in resourcegroup=\"admin-credential-lifecycle-test-n47zt9\" to finish getting creds, caused by: timeout '10.000000' minutes exceeded during GetAdminRESTConfigForHCPCluster for cluster admin-cred-lifecycle-kk24qw in resource group admin-credential-lifecycle-test-n47zt9, error: context deadline exceeded",
        errs: [
            <*errors.errorString | 0xc0007a6c10>{
                s: "timeout '10.000000' minutes exceeded during GetAdminRESTConfigForHCPCluster for cluster admin-cred-lifecycle-kk24qw in resource group admin-credential-lifecycle-test-n47zt9",
            },
            <context.deadlineExceededError>{},
        ],
    }
occurred

time=2026-03-27T16:55:49.963Z level=INFO msg="Running step." serviceGroup=Microsoft.Azure.ARO.HCP.Service.Infra resourceGroup=service step=cluster description="Step cluster\n  Kind: ARM\n  Template: templates/svc-cluster.bicep\n  Parameters: configurations/svc-cluster.tmpl.bicepparam"
time=2026-03-27T16:55:51.467Z level=DEBUG+2 msg="Starting ARM deployment" serviceGroup=Microsoft.Azure.ARO.HCP.Service.Infra resourceGroup=service step=cluster
time=2026-03-27T16:55:53.166Z level=DEBUG+3 msg="Deployment started" serviceGroup=Microsoft.Azure.ARO.HCP.Service.Infra resourceGroup=service step=cluster deployment=030e0412555f7d04adbd61e54aa5cefd5e9e1d5fd8b3b060b5ee4a73f2db320a
time=2026-03-27T17:08:56.231Z level=ERROR msg="Step errored." serviceGroup=Microsoft.Azure.ARO.HCP.Service.Infra resourceGroup=service step=cluster err="failed to run ARM step: failed to poll deployment: failed to wait for deployment completion: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourcegroups/hcp-underlay-prow-j1742976-svc/providers/Microsoft.Resources/deployments/030e0412555f7d04adbd61e54aa5cefd5e9e1d5fd8b3b060b5ee4a73f2db320a/operationStatuses/08584269763333001275\n--------------------------------------------------------------------------------\nRESPONSE 200: 200 OK\nERROR CODE: DeploymentFailed\n--------------------------------------------------------------------------------\n{\n  \"status\": \"Failed\",\n  \"error\": {\n    \"code\": \"DeploymentFailed\",\n    \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\n    \"details\": [\n      {\n        \"code\": \"Conflict\",\n        \"message\": \"{\\r\\n  \\\"status\\\": \\\"Failed\\\",\\r\\n  \\\"error\\\": {\\r\\n    \\\"code\\\": \\\"ResourceDeploymentFailure\\\",\\r\\n    \\\"message\\\": \\\"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\\\",\\r\\n    \\\"details\\\": [\\r\\n      {\\r\\n        \\\"code\\\": \\\"DeploymentFailed\\\",\\r\\n        \\\"target\\\": \\\"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/hcp-underlay-prow-j1742976-svc/providers/Microsoft.Resources/deployments/cluster-b4ajflcf262bs\\\",\\r\\n        \\\"message\\\": \\\"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\\\",\\r\\n        \\\"details\\\": [\\r\\n          {\\r\\n            \\\"code\\\": \\\"ResourceDeploymentFailure\\\",\\r\\n            \\\"target\\\": \\\"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/hcp-underlay-prow-j1742976-svc/providers/Microsoft.Resources/deployments/user-agent-pools\\\",\\r\\n            \\\"message\\\": \\\"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\\\",\\r\\n            \\\"details\\\": [\\r\\n              {\\r\\n                \\\"code\\\": \\\"DeploymentFailed\\\",\\r\\n                \\\"target\\\": \\\"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/hcp-underlay-prow-j1742976-svc/providers/Microsoft.Resources/deployments/user-agent-pools\\\",\\r\\n                \\\"message\\\": \\\"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\\\",\\r\\n                \\\"details\\\": [\\r\\n                  {\\r\\n                    \\\"code\\\": \\\"ResourceDeploymentFailure\\\",\\r\\n                    \\\"target\\\": \\\"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/hcp-underlay-prow-j1742976-svc/providers/Microsoft.ContainerService/managedClusters/prow-j1742976-svc/agentPools/ud8dsv62\\\",\\r\\n                    \\\"message\\\": \\\"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\\\",\\r\\n                    \\\"details\\\": [\\r\\n                      {\\r\\n                        \\\"code\\\": \\\"OverconstrainedZonalAllocationRequest\\\",\\r\\n                        \\\"message\\\": \\\"original error: Create or update VMSS /subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/hcp-underlay-prow-j1742976-svc-aks1/providers/Microsoft.Compute/virtualMachineScaleSets/aks-ud8dsv62-21047250-vmss failed. Allocation failed. VM(s) with the following constraints cannot be allocated, because the condition is too restrictive. Please remove some constraints and try again. Constraints applied are:\\\\n  - Availability Zone\\\\n  - Differencing (Ephemeral) Disks\\\\n  - Networking Constraints (such as Accelerated Networking or IPv6)\\\\n  - VM Size\\\\n. Please note that allocation failures can also arise if a proximity placement group is associated with this VMSS.  See https://learn.microsoft.com/en-us/troubleshoot/azure/azure-kubernetes/error-code-zonalallocationfailed-allocationfailed for more details.  This is not AKS controlled behavior, please ask help to VMSS team for allocation failure.  If Node Auto Provisioner (NAP) is not enabled,  consider enabling NAP to designate multiple SKUs for higher chance of success, see https://aka.ms/aks/enableNAP.  If NAP is already enabled,  consider adding additional SKUs to the NAP configuration for higher chance of success, see https://aka.ms/aks/configNAP.\\\",\\r\\n                        \\\"details\\\": [\\r\\n                          {\\r\\n                            \\\"code\\\": \\\"OverconstrainedZonalAllocationRequest\\\",\\r\\n                            \\\"target\\\": \\\"0\\\",\\r\\n                            \\\"message\\\": \\\"Allocation failed. VM(s) with the following constraints cannot be allocated, because the condition is too restrictive. Please remove some constraints and try again. Constraints applied are:\\\\n  - Availability Zone\\\\n  - Differencing (Ephemeral) Disks\\\\n  - Networking Constraints (such as Accelerated Networking or IPv6)\\\\n  - VM Size\\\\n\\\"\\r\\n                          }\\r\\n                        ]\\r\\n                      }\\r\\n                    ]\\r\\n                  }\\r\\n                ]\\r\\n              }\\r\\n            ]\\r\\n          }\\r\\n        ]\\r\\n      }\\r\\n    ]\\r\\n  }\\r\\n}\"\n      }\n    ]\n  }\n}\n--------------------------------------------------------------------------------\n"

time=2026-03-27T15:06:36.372Z level=INFO msg="Running step." serviceGroup=Microsoft.Azure.ARO.HCP.Region resourceGroup=regional step=infra description="Step infra\n  Kind: ARM\n  Template: templates/region.bicep\n  Parameters: configurations/region.tmpl.bicepparam"
time=2026-03-27T15:06:38.469Z level=DEBUG+2 msg="Starting ARM deployment" serviceGroup=Microsoft.Azure.ARO.HCP.Region resourceGroup=regional step=infra
time=2026-03-27T15:06:39.655Z level=DEBUG+3 msg="Deployment started" serviceGroup=Microsoft.Azure.ARO.HCP.Region resourceGroup=regional step=infra deployment=6b04bb802b99afb35d3a7da01b435a2fe54542974d8c55f4de7c0fd1253cb1ef
time=2026-03-27T15:12:11.166Z level=ERROR msg="Step errored." serviceGroup=Microsoft.Azure.ARO.HCP.Region resourceGroup=regional step=infra err="failed to run ARM step: failed to poll deployment: failed to wait for deployment completion: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourcegroups/hcp-underlay-prow-j8121216/providers/Microsoft.Resources/deployments/6b04bb802b99afb35d3a7da01b435a2fe54542974d8c55f4de7c0fd1253cb1ef/operationStatuses/08584269828865178334\n--------------------------------------------------------------------------------\nRESPONSE 200: 200 OK\nERROR CODE: DeploymentFailed\n--------------------------------------------------------------------------------\n{\n  \"status\": \"Failed\",\n  \"error\": {\n    \"code\": \"DeploymentFailed\",\n    \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\n    \"details\": [\n      {\n        \"code\": \"Conflict\",\n        \"message\": \"{\\r\\n  \\\"status\\\": \\\"Failed\\\",\\r\\n  \\\"error\\\": {\\r\\n    \\\"code\\\": \\\"ResourceDeploymentFailure\\\",\\r\\n    \\\"message\\\": \\\"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\\\",\\r\\n    \\\"details\\\": [\\r\\n      {\\r\\n        \\\"code\\\": \\\"DeploymentFailed\\\",\\r\\n        \\\"target\\\": \\\"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/hcp-underlay-prow-j8121216/providers/Microsoft.Resources/deployments/maestro-infra-deployment\\\",\\r\\n        \\\"message\\\": \\\"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\\\",\\r\\n        \\\"details\\\": [\\r\\n          {\\r\\n            \\\"code\\\": \\\"ResourceDeploymentFailure\\\",\\r\\n            \\\"target\\\": \\\"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/hcp-underlay-prow-j8121216/providers/Microsoft.EventGrid/namespaces/arohcp-prow-maestro-j8121216\\\",\\r\\n            \\\"message\\\": \\\"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\\\",\\r\\n            \\\"details\\\": [\\r\\n              {\\r\\n                \\\"code\\\": \\\"Internal error\\\",\\r\\n                \\\"message\\\": \\\"The operation failed due to an internal server error. The initial state of the impacted resources (if any) are restored. Please try again in few minutes. If error still persists, report cdabde50-a0ae-4404-900f-94278705eafb:3/27/2026 3:09:17 PM (UTC) to our forums for assistance or raise a support ticket .\\\"\\r\\n              }\\r\\n            ]\\r\\n          }\\r\\n        ]\\r\\n      }\\r\\n    ]\\r\\n  }\\r\\n}\"\n      }\n    ]\n  }\n}\n--------------------------------------------------------------------------------\n"

time=2026-03-27T13:50:41.577Z level=INFO msg="Running step." serviceGroup=Microsoft.Azure.ARO.HCP.Region resourceGroup=regional step=infra description="Step infra\n  Kind: ARM\n  Template: templates/region.bicep\n  Parameters: configurations/region.tmpl.bicepparam"
time=2026-03-27T13:50:43.752Z level=DEBUG+2 msg="Starting ARM deployment" serviceGroup=Microsoft.Azure.ARO.HCP.Region resourceGroup=regional step=infra
time=2026-03-27T13:50:45.176Z level=DEBUG+3 msg="Deployment started" serviceGroup=Microsoft.Azure.ARO.HCP.Region resourceGroup=regional step=infra deployment=6be2ba751dc97e52c6d9bfdb5b511a47d6811cd308995a068a3f14585b18f93b
time=2026-03-27T13:53:46.117Z level=ERROR msg="Step errored." serviceGroup=Microsoft.Azure.ARO.HCP.Region resourceGroup=regional step=infra err="failed to run ARM step: failed to poll deployment: failed to wait for deployment completion: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourcegroups/hcp-underlay-prow-j6628864/providers/Microsoft.Resources/deployments/6be2ba751dc97e52c6d9bfdb5b511a47d6811cd308995a068a3f14585b18f93b/operationStatuses/08584269874412386907\n--------------------------------------------------------------------------------\nRESPONSE 200: 200 OK\nERROR CODE: DeploymentFailed\n--------------------------------------------------------------------------------\n{\n  \"status\": \"Failed\",\n  \"error\": {\n    \"code\": \"DeploymentFailed\",\n    \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\n    \"details\": [\n      {\n        \"code\": \"Conflict\",\n        \"message\": \"{\\r\\n  \\\"status\\\": \\\"Failed\\\",\\r\\n  \\\"error\\\": {\\r\\n    \\\"code\\\": \\\"ResourceDeploymentFailure\\\",\\r\\n    \\\"message\\\": \\\"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\\\",\\r\\n    \\\"details\\\": [\\r\\n      {\\r\\n        \\\"code\\\": \\\"DeploymentFailed\\\",\\r\\n        \\\"target\\\": \\\"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/hcp-underlay-prow-j6628864/providers/Microsoft.Resources/deployments/maestro-infra-deployment\\\",\\r\\n        \\\"message\\\": \\\"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\\\",\\r\\n        \\\"details\\\": [\\r\\n          {\\r\\n            \\\"code\\\": \\\"ResourceDeploymentFailure\\\",\\r\\n            \\\"target\\\": \\\"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/hcp-underlay-prow-j6628864/providers/Microsoft.EventGrid/namespaces/arohcp-prow-maestro-j6628864\\\",\\r\\n            \\\"message\\\": \\\"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\\\",\\r\\n            \\\"details\\\": [\\r\\n              {\\r\\n                \\\"code\\\": \\\"Internal error\\\",\\r\\n                \\\"message\\\": \\\"The operation failed due to an internal server error. The initial state of the impacted resources (if any) are restored. Please try again in few minutes. If error still persists, report 39fe6ca5-272a-4487-8e34-a1281db54110:3/27/2026 1:52:06 PM (UTC) to our forums for assistance or raise a support ticket .\\\"\\r\\n              }\\r\\n            ]\\r\\n          }\\r\\n        ]\\r\\n      }\\r\\n    ]\\r\\n  }\\r\\n}\"\n      }\n    ]\n  }\n}\n--------------------------------------------------------------------------------\n"

time=2026-03-27T13:07:17.428Z level=INFO msg="Running step." serviceGroup=Microsoft.Azure.ARO.HCP.Management.Infra resourceGroup=management step=nsp description="Step nsp\n  Kind: ARM\n  Template: templates/mgmt-nsp.bicep\n  Parameters: configurations/mgmt-nsp.tmpl.bicepparam"
time=2026-03-27T13:07:17.868Z level=DEBUG+2 msg="Starting ARM deployment" serviceGroup=Microsoft.Azure.ARO.HCP.Management.Infra resourceGroup=management step=nsp
time=2026-03-27T13:07:18.596Z level=DEBUG+3 msg="Deployment started" serviceGroup=Microsoft.Azure.ARO.HCP.Management.Infra resourceGroup=management step=nsp deployment=d16a041e49e06e1f6d817ab604587a607675dab554c68cbf6dfe492508b4735f
time=2026-03-27T13:08:48.986Z level=ERROR msg="Step errored." serviceGroup=Microsoft.Azure.ARO.HCP.Management.Infra resourceGroup=management step=nsp err="failed to run ARM step: failed to poll deployment: failed to wait for deployment completion: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourcegroups/hcp-underlay-prow-j6090624-mgmt-1/providers/Microsoft.Resources/deployments/d16a041e49e06e1f6d817ab604587a607675dab554c68cbf6dfe492508b4735f/operationStatuses/08584269900473610651\n--------------------------------------------------------------------------------\nRESPONSE 200: 200 OK\nERROR CODE: DeploymentFailed\n--------------------------------------------------------------------------------\n{\n  \"status\": \"Failed\",\n  \"error\": {\n    \"code\": \"DeploymentFailed\",\n    \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\n    \"details\": [\n      {\n        \"code\": \"Conflict\",\n        \"message\": \"{\\r\\n  \\\"status\\\": \\\"Failed\\\",\\r\\n  \\\"error\\\": {\\r\\n    \\\"code\\\": \\\"ResourceDeploymentFailure\\\",\\r\\n    \\\"message\\\": \\\"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\\\",\\r\\n    \\\"details\\\": [\\r\\n      {\\r\\n        \\\"code\\\": \\\"DeploymentFailed\\\",\\r\\n        \\\"target\\\": \\\"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/hcp-underlay-prow-j6090624-mgmt-1/providers/Microsoft.Resources/deployments/profile-expinhr3wegwq\\\",\\r\\n        \\\"message\\\": \\\"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\\\",\\r\\n        \\\"details\\\": [\\r\\n          {\\r\\n            \\\"code\\\": \\\"ResourceDeploymentFailure\\\",\\r\\n            \\\"target\\\": \\\"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/hcp-underlay-prow-j6090624-mgmt-1/providers/Microsoft.Network/networkSecurityPerimeters/nsp-j6090624-mgmt-1/resourceAssociations/nsp-j6090624-mgmt-1-ra-5f8c9afe-2b58-50af-bcb9-ea31ab3d5179\\\",\\r\\n            \\\"message\\\": \\\"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\\\",\\r\\n            \\\"details\\\": [\\r\\n              {\\r\\n                \\\"code\\\": \\\"ServiceUnavailable\\\",\\r\\n                \\\"message\\\": \\\"Error while processing request for association /subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/hcp-underlay-prow-j6090624-mgmt-1/providers/Microsoft.Network/networkSecurityPerimeters/nsp-j6090624-mgmt-1/resourceAssociations/nsp-j6090624-mgmt-1-ra-5f8c9afe-2b58-50af-bcb9-ea31ab3d5179, Error message: The service is unavailable now. Please retry the request later.\\\"\\r\\n              }\\r\\n            ]\\r\\n          }\\r\\n        ]\\r\\n      }\\r\\n    ]\\r\\n  }\\r\\n}\"\n      }\n    ]\n  }\n}\n--------------------------------------------------------------------------------\n"

time=2026-03-27T12:12:30.979Z level=INFO msg="Running step." serviceGroup=Microsoft.Azure.ARO.HCP.Region resourceGroup=regional step=infra description="Step infra\n  Kind: ARM\n  Template: templates/region.bicep\n  Parameters: configurations/region.tmpl.bicepparam"
time=2026-03-27T12:12:33.010Z level=DEBUG+2 msg="Starting ARM deployment" serviceGroup=Microsoft.Azure.ARO.HCP.Region resourceGroup=regional step=infra
time=2026-03-27T12:12:34.380Z level=DEBUG+3 msg="Deployment started" serviceGroup=Microsoft.Azure.ARO.HCP.Region resourceGroup=regional step=infra deployment=5ad762b00291321f96cc50f58624231bd6cdcebe8741580f3f82f1cf16401693
time=2026-03-27T12:15:05.312Z level=ERROR msg="Step errored." serviceGroup=Microsoft.Azure.ARO.HCP.Region resourceGroup=regional step=infra err="failed to run ARM step: failed to poll deployment: failed to wait for deployment completion: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourcegroups/hcp-underlay-prow-j3974784/providers/Microsoft.Resources/deployments/5ad762b00291321f96cc50f58624231bd6cdcebe8741580f3f82f1cf16401693/operationStatuses/08584269933320118051\n--------------------------------------------------------------------------------\nRESPONSE 200: 200 OK\nERROR CODE: DeploymentFailed\n--------------------------------------------------------------------------------\n{\n  \"status\": \"Failed\",\n  \"error\": {\n    \"code\": \"DeploymentFailed\",\n    \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\n    \"details\": [\n      {\n        \"code\": \"Conflict\",\n        \"message\": \"{\\r\\n  \\\"status\\\": \\\"Failed\\\",\\r\\n  \\\"error\\\": {\\r\\n    \\\"code\\\": \\\"ResourceDeploymentFailure\\\",\\r\\n    \\\"message\\\": \\\"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\\\",\\r\\n    \\\"details\\\": [\\r\\n      {\\r\\n        \\\"code\\\": \\\"DeploymentFailed\\\",\\r\\n        \\\"target\\\": \\\"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/hcp-underlay-prow-j3974784/providers/Microsoft.Resources/deployments/maestro-infra-deployment\\\",\\r\\n        \\\"message\\\": \\\"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\\\",\\r\\n        \\\"details\\\": [\\r\\n          {\\r\\n            \\\"code\\\": \\\"ResourceDeploymentFailure\\\",\\r\\n            \\\"target\\\": \\\"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/hcp-underlay-prow-j3974784/providers/Microsoft.EventGrid/namespaces/arohcp-prow-maestro-j3974784\\\",\\r\\n            \\\"message\\\": \\\"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\\\",\\r\\n            \\\"details\\\": [\\r\\n              {\\r\\n                \\\"code\\\": \\\"Internal error\\\",\\r\\n                \\\"message\\\": \\\"The operation failed due to an internal server error. The initial state of the impacted resources (if any) are restored. Please try again in few minutes. If error still persists, report 9f26347b-934c-4bcd-a70a-9dbc2b6b0253:3/27/2026 12:14:54 PM (UTC) to our forums for assistance or raise a support ticket .\\\"\\r\\n              }\\r\\n            ]\\r\\n          }\\r\\n        ]\\r\\n      }\\r\\n    ]\\r\\n  }\\r\\n}\"\n      }\n    ]\n  }\n}\n--------------------------------------------------------------------------------\n"

time=2026-03-27T11:34:10.047Z level=INFO msg="Running step." serviceGroup=Microsoft.Azure.ARO.HCP.Management.Infra resourceGroup=management step=nsp description="Step nsp\n  Kind: ARM\n  Template: templates/mgmt-nsp.bicep\n  Parameters: configurations/mgmt-nsp.tmpl.bicepparam"
time=2026-03-27T11:34:10.620Z level=DEBUG+2 msg="Starting ARM deployment" serviceGroup=Microsoft.Azure.ARO.HCP.Management.Infra resourceGroup=management step=nsp
time=2026-03-27T11:34:11.320Z level=DEBUG+3 msg="Deployment started" serviceGroup=Microsoft.Azure.ARO.HCP.Management.Infra resourceGroup=management step=nsp deployment=7b3e71b5bd4dbed737a5f0fd102127d346b93c4808ef9226ab4edba648c35d06
time=2026-03-27T11:35:11.625Z level=ERROR msg="Step errored." serviceGroup=Microsoft.Azure.ARO.HCP.Management.Infra resourceGroup=management step=nsp err="failed to run ARM step: failed to poll deployment: failed to wait for deployment completion: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourcegroups/hcp-underlay-prow-j9361664-mgmt-1/providers/Microsoft.Resources/deployments/7b3e71b5bd4dbed737a5f0fd102127d346b93c4808ef9226ab4edba648c35d06/operationStatuses/08584269956345969368\n--------------------------------------------------------------------------------\nRESPONSE 200: 200 OK\nERROR CODE: DeploymentFailed\n--------------------------------------------------------------------------------\n{\n  \"status\": \"Failed\",\n  \"error\": {\n    \"code\": \"DeploymentFailed\",\n    \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\n    \"details\": [\n      {\n        \"code\": \"Conflict\",\n        \"message\": \"{\\r\\n  \\\"status\\\": \\\"Failed\\\",\\r\\n  \\\"error\\\": {\\r\\n    \\\"code\\\": \\\"ResourceDeploymentFailure\\\",\\r\\n    \\\"message\\\": \\\"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\\\",\\r\\n    \\\"details\\\": [\\r\\n      {\\r\\n        \\\"code\\\": \\\"DeploymentFailed\\\",\\r\\n        \\\"target\\\": \\\"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/hcp-underlay-prow-j9361664-mgmt-1/providers/Microsoft.Resources/deployments/profile-duhqqw2zn5gxa\\\",\\r\\n        \\\"message\\\": \\\"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\\\",\\r\\n        \\\"details\\\": [\\r\\n          {\\r\\n            \\\"code\\\": \\\"ResourceDeploymentFailure\\\",\\r\\n            \\\"target\\\": \\\"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/hcp-underlay-prow-j9361664-mgmt-1/providers/Microsoft.Network/networkSecurityPerimeters/nsp-j9361664-mgmt-1/resourceAssociations/nsp-j9361664-mgmt-1-ra-f67c4493-c44d-5369-ad4a-1db781032435\\\",\\r\\n            \\\"message\\\": \\\"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\\\",\\r\\n            \\\"details\\\": [\\r\\n              {\\r\\n                \\\"code\\\": \\\"ServiceUnavailable\\\",\\r\\n                \\\"message\\\": \\\"Error while processing request for association /subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/hcp-underlay-prow-j9361664-mgmt-1/providers/Microsoft.Network/networkSecurityPerimeters/nsp-j9361664-mgmt-1/resourceAssociations/nsp-j9361664-mgmt-1-ra-f67c4493-c44d-5369-ad4a-1db781032435, Error message: The service is unavailable now. Please retry the request later.\\\"\\r\\n              }\\r\\n            ]\\r\\n          }\\r\\n        ]\\r\\n      }\\r\\n    ]\\r\\n  }\\r\\n}\"\n      }\n    ]\n  }\n}\n--------------------------------------------------------------------------------\n"

time=2026-03-27T08:56:40.881Z level=INFO msg="Running step." serviceGroup=Microsoft.Azure.ARO.HCP.Service.Infra resourceGroup=service step=cluster description="Step cluster\n  Kind: ARM\n  Template: templates/svc-cluster.bicep\n  Parameters: configurations/svc-cluster.tmpl.bicepparam"
time=2026-03-27T08:56:42.307Z level=DEBUG+2 msg="Starting ARM deployment" serviceGroup=Microsoft.Azure.ARO.HCP.Service.Infra resourceGroup=service step=cluster
time=2026-03-27T08:56:44.504Z level=DEBUG+3 msg="Deployment started" serviceGroup=Microsoft.Azure.ARO.HCP.Service.Infra resourceGroup=service step=cluster deployment=c37be19bb954b8b72ce6236bb195a49bfafdd97e7dbf2d5eb7f7756455bb09ac
time=2026-03-27T09:08:17.636Z level=ERROR msg="Step errored." serviceGroup=Microsoft.Azure.ARO.HCP.Service.Infra resourceGroup=service step=cluster err="failed to run ARM step: failed to poll deployment: failed to wait for deployment completion: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourcegroups/hcp-underlay-prow-j5164032-svc/providers/Microsoft.Resources/deployments/c37be19bb954b8b72ce6236bb195a49bfafdd97e7dbf2d5eb7f7756455bb09ac/operationStatuses/08584270050820657013\n--------------------------------------------------------------------------------\nRESPONSE 200: 200 OK\nERROR CODE: DeploymentFailed\n--------------------------------------------------------------------------------\n{\n  \"status\": \"Failed\",\n  \"error\": {\n    \"code\": \"DeploymentFailed\",\n    \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\n    \"details\": [\n      {\n        \"code\": \"Conflict\",\n        \"message\": \"{\\r\\n  \\\"status\\\": \\\"Failed\\\",\\r\\n  \\\"error\\\": {\\r\\n    \\\"code\\\": \\\"ResourceDeploymentFailure\\\",\\r\\n    \\\"message\\\": \\\"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\\\",\\r\\n    \\\"details\\\": [\\r\\n      {\\r\\n        \\\"code\\\": \\\"DeploymentFailed\\\",\\r\\n        \\\"target\\\": \\\"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/hcp-underlay-prow-j5164032-svc/providers/Microsoft.Resources/deployments/cluster-pnhfgkbajis54\\\",\\r\\n        \\\"message\\\": \\\"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\\\",\\r\\n        \\\"details\\\": [\\r\\n          {\\r\\n            \\\"code\\\": \\\"ResourceDeploymentFailure\\\",\\r\\n            \\\"target\\\": \\\"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/hcp-underlay-prow-j5164032-svc/providers/Microsoft.Resources/deployments/user-agent-pools\\\",\\r\\n            \\\"message\\\": \\\"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\\\",\\r\\n            \\\"details\\\": [\\r\\n              {\\r\\n                \\\"code\\\": \\\"DeploymentFailed\\\",\\r\\n                \\\"target\\\": \\\"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/hcp-underlay-prow-j5164032-svc/providers/Microsoft.Resources/deployments/user-agent-pools\\\",\\r\\n                \\\"message\\\": \\\"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\\\",\\r\\n                \\\"details\\\": [\\r\\n                  {\\r\\n                    \\\"code\\\": \\\"BadRequest\\\",\\r\\n                    \\\"target\\\": \\\"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/hcp-underlay-prow-j5164032-svc/providers/Microsoft.ContainerService/managedClusters/prow-j5164032-svc/agentPools/usd8sv62\\\",\\r\\n                    \\\"message\\\": \\\"{\\\\r\\\\n  \\\\\\\"code\\\\\\\": \\\\\\\"VMSizeDoesNotSupportEphemeralOS\\\\\\\",\\\\r\\\\n  \\\\\\\"details\\\\\\\": null,\\\\r\\\\n  \\\\\\\"message\\\\\\\": \\\\\\\"The Virtual Machine size Standard_D8s_v6 does not support Ephemeral OS disk.\\\\\\\",\\\\r\\\\n  \\\\\\\"subcode\\\\\\\": \\\\\\\"\\\\\\\"\\\\r\\\\n}\\\"\\r\\n                  },\\r\\n                  {\\r\\n                    \\\"code\\\": \\\"BadRequest\\\",\\r\\n                    \\\"target\\\": \\\"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/hcp-underlay-prow-j5164032-svc/providers/Microsoft.ContainerService/managedClusters/prow-j5164032-svc/agentPools/usd8sv61\\\",\\r\\n                    \\\"message\\\": \\\"{\\\\r\\\\n  \\\\\\\"code\\\\\\\": \\\\\\\"VMSizeDoesNotSupportEphemeralOS\\\\\\\",\\\\r\\\\n  \\\\\\\"details\\\\\\\": null,\\\\r\\\\n  \\\\\\\"message\\\\\\\": \\\\\\\"The Virtual Machine size Standard_D8s_v6 does not support Ephemeral OS disk.\\\\\\\",\\\\r\\\\n  \\\\\\\"subcode\\\\\\\": \\\\\\\"\\\\\\\"\\\\r\\\\n}\\\"\\r\\n                  },\\r\\n                  {\\r\\n                    \\\"code\\\": \\\"BadRequest\\\",\\r\\n                    \\\"target\\\": \\\"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/hcp-underlay-prow-j5164032-svc/providers/Microsoft.ContainerService/managedClusters/prow-j5164032-svc/agentPools/usd8sv63\\\",\\r\\n                    \\\"message\\\": \\\"{\\\\r\\\\n  \\\\\\\"code\\\\\\\": \\\\\\\"VMSizeDoesNotSupportEphemeralOS\\\\\\\",\\\\r\\\\n  \\\\\\\"details\\\\\\\": null,\\\\r\\\\n  \\\\\\\"message\\\\\\\": \\\\\\\"The Virtual Machine size Standard_D8s_v6 does not support Ephemeral OS disk.\\\\\\\",\\\\r\\\\n  \\\\\\\"subcode\\\\\\\": \\\\\\\"\\\\\\\"\\\\r\\\\n}\\\"\\r\\n                  }\\r\\n                ]\\r\\n              }\\r\\n            ]\\r\\n          }\\r\\n        ]\\r\\n      }\\r\\n    ]\\r\\n  }\\r\\n}\"\n      }\n    ]\n  }\n}\n--------------------------------------------------------------------------------\n"

time=2026-03-27T07:03:07.850Z level=INFO msg="Running step." serviceGroup=Microsoft.Azure.ARO.HCP.SessionGate resourceGroup=global step=mirror-image description="Step mirror-image\n  Kind: ImageMirror\n  From {<nil> sessiongate.image.registry <nil>}:{<nil> sessiongate.image.repository <nil>}@{<nil> sessiongate.image.digest <nil>} to {<nil> acr.svc.name <nil>}\n"
time=2026-03-27T07:03:10.439Z level=ERROR msg="Step errored." serviceGroup=Microsoft.Azure.ARO.HCP.SessionGate resourceGroup=global step=mirror-image err="error running Image Mirror Step, failed to execute shell command: Checking USE_OC_LOGIN_REGISTRIES: registry.build05.ci.openshift.org registry.build05.ci.openshift.org registry.build05.ci.openshift.org registry.build05.ci.openshift.org\nSetting up registry authentication for CI source registry.\ninfo: Using registry public hostname registry.build05.ci.openshift.org\nSaved credentials for registry.build05.ci.openshift.org into /tmp/tmp.kWAyib6f7J/containers/auth.json\nLogging into target ACR arohcpsvcdev.\nLogin Succeeded\nMirroring image registry.build05.ci.openshift.org/ci-op-46rclhn2/pipeline@sha256:88e924de288539a9fed728b0359e0f818914537bfee27d713c28505e17d27593 to arohcpsvcdev.azurecr.io/ci-op-46rclhn2/pipeline:88e924de288539a9fed728b0359e0f818914537bfee27d713c28505e17d27593.\nThe image will still be available under it's original digest sha256:88e924de288539a9fed728b0359e0f818914537bfee27d713c28505e17d27593 in the target registry.\nError: Get \"https://registry.build05.ci.openshift.org/v2/ci-op-46rclhn2/pipeline/blobs/sha256:93e3f7447420757b055fed9b7beb4211e24fa0393cac6adcf81c50b1ad4a44fe\": read tcp 172.24.211.114:48578->18.116.110.190:443: read: connection reset by peer\n exit status 1"

time=2026-03-27T06:56:41.083Z level=INFO msg="Running step." serviceGroup=Microsoft.Azure.ARO.HCP.SessionGate resourceGroup=service step=deploy description="Step deploy\n Kind: Helm\n"
time=2026-03-27T06:56:41.591Z level=INFO msg="Resolved input values." serviceGroup=Microsoft.Azure.ARO.HCP.SessionGate resourceGroup=service step=deploy values="map[affinity:map[] auth:map[audience:6dae42f8-4368-4678-94ff-3960e28e3630 authority:https://sts.windows.net/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] azure:map[clientId:89e1e7a5-df6a-4580-ad52-95f7059215f3 cloudEnvironmentName:AzurePublicCloud tenantId:XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] credsKeyVault:map[name:aro-hcp-dev-svc-kv secret:sessiongate-cert-prow-j3767552] deployment:map[replicas:2 requests:map[cpu:250m memory:512Mi]] httpRoute:map[gateway:map[name:ops-ingress-gateway namespace:aks-istio-ingress sectionName:sessiongate-https] host:sessiongate.j3767552.hcpsvc.osadev.cloud] image:map[digest:sha256:646f4826e848ab0833b51037e38c5a0869e4d2bcd4defb8725ae94ae82a0ee66 registry:arohcpsvcdev.azurecr.io repository:ci-op-6xklzdls/pipeline] leaderElection:map[leaseDuration:15s renewDeadline:10s retryPeriod:2s] logLevel:2 mise:map[deploy:false policyLabel:Session Gate] nodeSelector:map[] pullBinding:map[registry:arohcpsvcdev.azurecr.io scope:repository:ci-op-6xklzdls/pipeline:pull workloadIdentityClientId:77622d9f-9d39-430c-bcbb-1b69bf035b1e workloadIdentityTenantId:XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] service:map[name:sessiongate port:8080] serviceAccount:map[name:sessiongate workloadIdentityClientId:92133d4f-b986-4e3c-8d23-ebd6b649e36c workloadIdentityTenantId:XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] tolerations:[]]"
time=2026-03-27T06:56:41.591Z level=INFO msg="Applying namespaces." serviceGroup=Microsoft.Azure.ARO.HCP.SessionGate resourceGroup=service step=deploy
time=2026-03-27T06:56:42.037Z level=INFO msg="Namespace applied successfully." serviceGroup=Microsoft.Azure.ARO.HCP.SessionGate resourceGroup=service step=deploy namespace=sessiongate
time=2026-03-27T06:56:42.324Z level=INFO msg="Rolling out Helm release." serviceGroup=Microsoft.Azure.ARO.HCP.SessionGate resourceGroup=service step=deploy dryRun=false
time=2026-03-27T06:56:42.324Z level=INFO msg="Searching for release history..." serviceGroup=Microsoft.Azure.ARO.HCP.SessionGate resourceGroup=service step=deploy
time=2026-03-27T06:56:42.436Z level=INFO msg="No release history found, running the first release..." serviceGroup=Microsoft.Azure.ARO.HCP.SessionGate resourceGroup=service step=deploy
time=2026-03-27T06:56:48.769Z level=ERROR msg="Failed to roll out the Helm release." serviceGroup=Microsoft.Azure.ARO.HCP.SessionGate resourceGroup=service step=deploy err="Internal error occurred: failed calling webhook \"validation.istio.io\": failed to call webhook: Post \"https://istiod-asm-1-26.aks-istio-system.svc:443/validate?timeout=10s\": proxy error from localhost:9443 while dialing 10.128.64.10:15017, code 500: 500 Internal Server Error"
time=2026-03-27T06:56:48.769Z level=INFO msg="Finished deploying Helm release." serviceGroup=Microsoft.Azure.ARO.HCP.SessionGate resourceGroup=service step=deploy
time=2026-03-27T06:56:48.769Z level=INFO msg="Running inline diagnostics." serviceGroup=Microsoft.Azure.ARO.HCP.SessionGate resourceGroup=service step=deploy
time=2026-03-27T06:56:50.562Z level=INFO msg="Determined release status." serviceGroup=Microsoft.Azure.ARO.HCP.SessionGate resourceGroup=service step=deploy release=sessiongate namespace=sessiongate status=failed description="Release \"sessiongate\" failed: Internal error occurred: failed calling webhook \"validation.istio.io\": failed to call webhook: Post \"https://istiod-asm-1-26.aks-istio-system.svc:443/validate?timeout=10s\": proxy error from localhost:9443 while dialing 10.128.64.10:15017, code 500: 500 Internal Server Error" values="map[affinity:map[] auth:map[audience:6dae42f8-4368-4678-94ff-3960e28e3630 authority:https://sts.windows.net/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] azure:map[clientId:89e1e7a5-df6a-4580-ad52-95f7059215f3 cloudEnvironmentName:AzurePublicCloud tenantId:XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] credsKeyVault:map[name:aro-hcp-dev-svc-kv secret:sessiongate-cert-prow-j3767552] deployment:map[replicas:2 requests:map[cpu:250m memory:512Mi]] httpRoute:map[gateway:map[name:ops-ingress-gateway namespace:aks-istio-ingress sectionName:sessiongate-https] host:sessiongate.j3767552.hcpsvc.osadev.cloud] image:map[digest:sha256:646f4826e848ab0833b51037e38c5a0869e4d2bcd4defb8725ae94ae82a0ee66 registry:arohcpsvcdev.azurecr.io repository:ci-op-6xklzdls/pipeline] leaderElection:map[leaseDuration:15s renewDeadline:10s retryPeriod:2s] logLevel:2 mise:map[deploy:false policyLabel:Session Gate] nodeSelector:map[] pullBinding:map[registry:arohcpsvcdev.azurecr.io scope:repository:ci-op-6xklzdls/pipeline:pull workloadIdentityClientId:77622d9f-9d39-430c-bcbb-1b69bf035b1e workloadIdentityTenantId:XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] service:map[name:sessiongate port:8080] serviceAccount:map[name:sessiongate workloadIdentityClientId:92133d4f-b986-4e3c-8d23-ebd6b649e36c workloadIdentityTenantId:XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] tolerations:[]]"
time=2026-03-27T06:56:50.567Z level=INFO msg="Kube-events kusto link for troubleshooting:" serviceGroup=Microsoft.Azure.ARO.HCP.SessionGate resourceGroup=service step=deploy url="https://hcp-dev-us-2.eastus2.kusto.windows.net/ServiceLogs?query=H4sIAAAAAAAA/6SUTU8bPRDH7/sprJyIFEcoz1NaUXGgUKlSpSoCTlQ9ON4/uwavZzszBoH48JUJL9sQCG1vzuQ3/3ndqSLUMIQye4jZM7VTp24RsUWLc3j9GlK9K8ohNROzNH1zHR5NuES6s0jv/IN5/L0yxpjRMfgyeIwmZiQQCZQap89/LuH9rC1xuHEaKM0pBn9dSBcjXdkOysGLlX/USaRtSM0Lrl9OTuZHlNdIr8UPsih1R/fdO8RZSKEEHeAyHfhNHZNtfT91N5kx9dQV8lEtZlHwEcXVePb+bTuXXIMOSYeOxeNTSPXzuqynpEwxgu1iLfFnGhGuBltE+FLnBs1D9JGuH7Ld3M4XeevBGs6CL2/GGUNa8N1QL8QG0UA2pIYh8rgEnuc5xkFJfY5xUxPwM0O0LBCSlnC/D7NZxr+DrMvavqBzDM/QOdNlqMEH0ZW8Vibq+1fzv/9y9r2n/Nb+rducp+H9ldPqxFdFqh8fq4u8ACco5HO5BVLdmqsWDKOhg6jrerOAXgHJbNVOUcxbo9n2bMdu/2dn70+2d3bf7ez+Pzsdjc10al5jPpyOxuPq1pxTSOYipHovpAQeXDBK5ulwDS/W6qmqbk3PVP59SnRieqqX8NtEJiZSMzEdRFxTJIlrsFlcD4qvIf5XAAAA//86ZdTgZwUAAA=="
time=2026-03-27T06:56:50.570Z level=INFO msg="Deployment complete." serviceGroup=Microsoft.Azure.ARO.HCP.SessionGate resourceGroup=service step=deploy
time=2026-03-27T06:56:50.570Z level=ERROR msg="Step errored." serviceGroup=Microsoft.Azure.ARO.HCP.SessionGate resourceGroup=service step=deploy err="error running Helm release deployment Step, failed to deploy helm release: Internal error occurred: failed calling webhook \"validation.istio.io\": failed to call webhook: Post \"https://istiod-asm-1-26.aks-istio-system.svc:443/validate?timeout=10s\": proxy error from localhost:9443 while dialing 10.128.64.10:15017, code 500: 500 Internal Server Error"