Executive Status
EnvEnvironment partition: dev, int, stg, or prod. | GoalConfigured success target and run scope for this environment. INT/STG/PROD use all E2E job runs. DEV uses post-good runs plus Tide batches: runs on the final commit of a PR that later merged, or batch retests of PRs that passed their individual e2e-parallel checks. | RunsNumber of job runs in the selected goal definition for this environment. | SuccessSuccess rate on the selected goal definition: (runs - failed runs) / runs * 100. | Gap vs targetDifference in percentage points between current success and the configured target rate. | Change vs prevHow much the success rate changed compared with the immediately preceding equal-length window, using the same goal definition as this row. | Provision successProvisioning here means infrastructure setup before E2E tests start. A run counts as provision-successful if it got past provisioning, even if it later failed during E2E. Runs that failed for 'Other' CI-infrastructure reasons are excluded because those runs never reached the provision step. DEV rows use post-good runs plus Tide batches. INT/STG/PROD show n/a because provisioning is not part of those environments. | Provision change vs prevChange in provision success, in percentage points, compared with the previous equal-length window. Uses the same provision-success definition as the Provision success column. DEV only; INT/STG/PROD show n/a. | E2E successAmong runs that reached E2E execution, this shows how many completed successfully. Provision failures and 'Other' CI-infrastructure failures are excluded because those runs never reached the E2E step. DEV rows use only runs after the final push to a PR that later merged. INT/STG/PROD use all E2E job runs. | E2E success vs prevChange in E2E success, in percentage points, compared with the previous equal-length window. Uses the same E2E-success definition as the E2E success column. |
|---|---|---|---|---|---|---|---|---|---|
| DEV | 95% - Post-good runs + Tide batches | 192 | 57.81% (off track) | -37.19pp | n/a | 75.53% (142/188) | n/a | 78.17% (111/142) | n/a |
| INT | 95% - All E2E job runs | 34 | 58.82% (off track) | -36.18pp | n/a | n/a | n/a | 58.82% (20/34) | n/a |
| STG | 95% - All E2E job runs | 9 | 100.00% (on track) | +5.00pp | n/a | n/a | n/a | 100.00% (9/9) | n/a |
| PROD | 95% - All E2E job runs | 9 | 33.33% (off track) | -61.67pp | n/a | n/a | n/a | 33.33% (3/9) | n/a |
Environment: DEV
E2E Jobs
454
Success Rate
38.33%
Provision success
Provisioning here means infrastructure setup before E2E tests start. A run counts as provision-successful if it got past provisioning, even if it later failed during E2E. Runs that failed for 'Other' CI-infrastructure reasons are excluded because those runs never reached the provision step.68.07% (243/357)
E2E success
Among runs that reached E2E execution, this shows how many completed successfully. Provision failures and 'Other' CI-infrastructure failures are excluded because those runs never reached the E2E step.71.60% (174/243)
E2E Jobs (post-good + batches)
These PR-regression-filtered DEV metrics include runs on the final commit of a PR that later merged, plus Tide batch retests of PRs that passed their individual e2e-parallel checks. Each run counts once. This card shows the number of runs in that DEV-only goal basis.192
Success Rate (post-good + batches)
These PR-regression-filtered DEV metrics include runs on the final commit of a PR that later merged, plus Tide batch retests of PRs that passed their individual e2e-parallel checks. Each run counts once. This card shows the overall success rate for that DEV-only goal basis.57.81%
Provision success (post-good + batches)
These PR-regression-filtered DEV metrics include runs on the final commit of a PR that later merged, plus Tide batch retests of PRs that passed their individual e2e-parallel checks. Each run counts once. Provisioning here means infrastructure setup before E2E tests start. A run counts as provision-successful if it got past provisioning, even if it later failed during E2E. Runs that failed for 'Other' CI-infrastructure reasons are excluded because those runs never reached the provision step.75.53% (142/188)
E2E success (post-good + batches)
These PR-regression-filtered DEV metrics include runs on the final commit of a PR that later merged, plus Tide batch retests of PRs that passed their individual e2e-parallel checks. Each run counts once. Among runs that reached E2E execution, this shows how many completed successfully. Provision failures and 'Other' CI-infrastructure failures are excluded because those runs never reached the E2E step.78.17% (111/142)
Daily Run Outcomes
Successful runs
Provision failures
E2E failures
Other failures
Daily Run Outcomes for DEV Goal Basis (post-good + batches)
Successful runs (post-good + batches)
Provision failures
E2E failures
Other failures
| Pass rate | Runs | Date (UTC) | Suite | Test |
|---|---|---|---|---|
| 57.89% | 19 | rp-api-compat-all/parallel | Engineering should be able to retrieve expected metrics from the /metrics endpoint | |
| 74.44% | 313 | step graph | Run pipeline step Microsoft.Azure.ARO.HCP.Region/regional/infra | |
| 94.69% | 113 | rp-api-compat-all/parallel | Customer should not be able to perform various invalid operations on cluster resources |
Up to 50 failure patterns are loaded (minimum 1.00% impact), with 10 shown by default. Click column headers to re-sort.
Jump to the full Failure Patterns view for this window
| Failure patternThe canonical description of a recurring CI failure, extracted and normalized from raw logs. | Failed atThe stage of the job run where this failure occurred: 'provision' (environment setup, DEV only), 'e2e' (test suite execution), or 'other' (CI infrastructure issues that did not produce a failure pattern). | Number of distinct job runs where this failure pattern was detected. | Percentage of all job runs in this environment affected by this failure pattern during the selected window. | Classification of this failure pattern: Regression (likely caused by a specific PR), Flake (intermittent failure spread across days), Noise (low-quality or generic pattern), or Indeterminate. | TrendShows daily activity for this failure pattern in a trailing window anchored to the selected end date. The sparkline covers at least 7 days and at most 14 days, depending on the current window size. | Also inOther environments where the same failure pattern was also detected during the selected window. | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Error response from registry: failed to resolve sha256:<hex>: GET "<url>": unauthorized: Please login to the Red Hat Registry using your Customer Portal credentials. Further instr...full failure pattern: Error response from registry: failed to resolve sha256:<hex>: GET "<url>": unauthorized: Please login to the Red Hat Registry using your Customer Portal credentials. Further instructions can be found here: <url> Signal: IndeterminateSignal: Indeterminate — no prior history; active 2 day(s) — New failure pattern — no prior history | provision | 21 | 4.63%21 of 454 job runs affected | IndeterminateSignal: Indeterminate — no prior history; active 2 day(s) — New failure pattern — no prior history | Show trend detailsFeb 16: 0 · Feb 17: 0 · Feb 18: 0 · Feb 19: 0 · Feb 20: 0 · Feb 21: 0 · Feb 22: 0 · Feb 23: 0 · Feb 24: 1 · Feb 25: 20 · Feb 26: 0 · Feb 27: 0 · Feb 28: 0 · Mar 1: 0 | none | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Full failure examples (3)time=2026-02-25T22:37:27.977Z level=INFO msg="Running step." serviceGroup=Microsoft.Azure.ARO.HCP.Velero resourceGroup=global step=mirror-azure-plugin-image description="Step mirror-azure-plugin-image\n Kind: ImageMirror\n From {<nil> velero.azurePlugin.registry <nil>}:{<nil> velero.azurePlugin.repository <nil>}@{<nil> velero.azurePlugin.digest <nil>} to {<nil> acr.svc.name <nil>}\n"
time=2026-02-25T22:37:30.343Z level=ERROR msg="Step errored." serviceGroup=Microsoft.Azure.ARO.HCP.Velero resourceGroup=global step=mirror-azure-plugin-image err="error running Image Mirror Step, failed to execute shell command: Checking USE_OC_LOGIN_REGISTRIES: registry.build06.ci.openshift.org registry.build06.ci.openshift.org registry.build06.ci.openshift.org registry.build06.ci.openshift.org\nFetch pull secret for source registry registry.redhat.io from arohcpdev-global KV.\nLogging into target ACR arohcpsvcdev.\nLogin Succeeded\nMirroring image registry.redhat.io/oadp/oadp-velero-plugin-for-microsoft-azure-rhel9@sha256:47a1a93a04208748c29073c17855aeca8f72ff2025d7f404b054abd375c8172e to arohcpsvcdev.azurecr.io/oadp/oadp-velero-plugin-for-microsoft-azure-rhel9:47a1a93a04208748c29073c17855aeca8f72ff2025d7f404b054abd375c8172e.\nThe image will still be available under it's original digest sha256:47a1a93a04208748c29073c17855aeca8f72ff2025d7f404b054abd375c8172e in the target registry.\nError response from registry: failed to resolve sha256:47a1a93a04208748c29073c17855aeca8f72ff2025d7f404b054abd375c8172e: GET \"https://registry.redhat.io/v2/oadp/oadp-velero-plugin-for-microsoft-azure-rhel9/manifests/sha256:47a1a93a04208748c29073c17855aeca8f72ff2025d7f404b054abd375c8172e\": unauthorized: Please login to the Red Hat Registry using your Customer Portal credentials. Further instructions can be found here: https://access.redhat.com/RegistryAuthentication\n exit status 1"time=2026-02-25T22:44:52.734Z level=INFO msg="Running step." serviceGroup=Microsoft.Azure.ARO.HCP.Velero resourceGroup=global step=mirror-hypershift-plugin-image description="Step mirror-hypershift-plugin-image\n Kind: ImageMirror\n From {<nil> velero.hypershiftPlugin.registry <nil>}:{<nil> velero.hypershiftPlugin.repository <nil>}@{<nil> velero.hypershiftPlugin.digest <nil>} to {<nil> acr.svc.name <nil>}\n"
time=2026-02-25T22:44:55.660Z level=ERROR msg="Step errored." serviceGroup=Microsoft.Azure.ARO.HCP.Velero resourceGroup=global step=mirror-hypershift-plugin-image err="error running Image Mirror Step, failed to execute shell command: Checking USE_OC_LOGIN_REGISTRIES: registry.build06.ci.openshift.org registry.build06.ci.openshift.org registry.build06.ci.openshift.org registry.build06.ci.openshift.org\nFetch pull secret for source registry registry.redhat.io from arohcpdev-global KV.\nLogging into target ACR arohcpsvcdev.\nLogin Succeeded\nMirroring image registry.redhat.io/oadp/oadp-hypershift-velero-plugin-rhel9@sha256:f47f2f9099b6ad235ea1187556dc165aef725bcbe2b818ab62bb186ad485961e to arohcpsvcdev.azurecr.io/oadp/oadp-hypershift-velero-plugin-rhel9:f47f2f9099b6ad235ea1187556dc165aef725bcbe2b818ab62bb186ad485961e.\nThe image will still be available under it's original digest sha256:f47f2f9099b6ad235ea1187556dc165aef725bcbe2b818ab62bb186ad485961e in the target registry.\nError response from registry: failed to resolve sha256:f47f2f9099b6ad235ea1187556dc165aef725bcbe2b818ab62bb186ad485961e: GET \"https://registry.redhat.io/v2/oadp/oadp-hypershift-velero-plugin-rhel9/manifests/sha256:f47f2f9099b6ad235ea1187556dc165aef725bcbe2b818ab62bb186ad485961e\": unauthorized: Please login to the Red Hat Registry using your Customer Portal credentials. Further instructions can be found here: https://access.redhat.com/RegistryAuthentication\n exit status 1"time=2026-02-25T22:27:52.583Z level=INFO msg="Running step." serviceGroup=Microsoft.Azure.ARO.HCP.Velero resourceGroup=global step=mirror-hypershift-plugin-image description="Step mirror-hypershift-plugin-image\n Kind: ImageMirror\n From {<nil> velero.hypershiftPlugin.registry <nil>}:{<nil> velero.hypershiftPlugin.repository <nil>}@{<nil> velero.hypershiftPlugin.digest <nil>} to {<nil> acr.svc.name <nil>}\n"
time=2026-02-25T22:27:55.317Z level=ERROR msg="Step errored." serviceGroup=Microsoft.Azure.ARO.HCP.Velero resourceGroup=global step=mirror-hypershift-plugin-image err="error running Image Mirror Step, failed to execute shell command: Checking USE_OC_LOGIN_REGISTRIES: registry.build06.ci.openshift.org registry.build06.ci.openshift.org registry.build06.ci.openshift.org registry.build06.ci.openshift.org\nFetch pull secret for source registry registry.redhat.io from arohcpdev-global KV.\nLogging into target ACR arohcpsvcdev.\nLogin Succeeded\nMirroring image registry.redhat.io/oadp/oadp-hypershift-velero-plugin-rhel9@sha256:f47f2f9099b6ad235ea1187556dc165aef725bcbe2b818ab62bb186ad485961e to arohcpsvcdev.azurecr.io/oadp/oadp-hypershift-velero-plugin-rhel9:f47f2f9099b6ad235ea1187556dc165aef725bcbe2b818ab62bb186ad485961e.\nThe image will still be available under it's original digest sha256:f47f2f9099b6ad235ea1187556dc165aef725bcbe2b818ab62bb186ad485961e in the target registry.\nError response from registry: failed to resolve sha256:f47f2f9099b6ad235ea1187556dc165aef725bcbe2b818ab62bb186ad485961e: GET \"https://registry.redhat.io/v2/oadp/oadp-hypershift-velero-plugin-rhel9/manifests/sha256:f47f2f9099b6ad235ea1187556dc165aef725bcbe2b818ab62bb186ad485961e\": unauthorized: Please login to the Red Hat Registry using your Customer Portal credentials. Further instructions can be found here: https://access.redhat.com/RegistryAuthentication\n exit status 1"Contributing tests (3)
Affected runs (21) | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
ERROR CODE: DeploymentFailed; detail code Conflict; provider Microsoft.EventGridfull failure pattern: ERROR CODE: DeploymentFailed; detail code Conflict; provider Microsoft.EventGrid Signal: IndeterminateSignal: Indeterminate — no prior history; active 5 day(s) — New failure pattern — no prior history | provision | 14 | 3.08%14 of 454 job runs affected | IndeterminateSignal: Indeterminate — no prior history; active 5 day(s) — New failure pattern — no prior history | Show trend detailsFeb 16: 0 · Feb 17: 0 · Feb 18: 0 · Feb 19: 0 · Feb 20: 0 · Feb 21: 0 · Feb 22: 0 · Feb 23: 3 · Feb 24: 2 · Feb 25: 2 · Feb 26: 1 · Feb 27: 6 · Feb 28: 0 · Mar 1: 0 | none | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Full failure examples (3)time=2026-02-27T10:18:03.270Z level=INFO msg="Running step." serviceGroup=Microsoft.Azure.ARO.HCP.Region resourceGroup=regional step=infra description="Step infra\n Kind: ARM\n Template: templates/region.bicep\n Parameters: configurations/region.tmpl.bicepparam"
time=2026-02-27T10:18:05.590Z level=DEBUG+2 msg="Starting ARM deployment" serviceGroup=Microsoft.Azure.ARO.HCP.Region resourceGroup=regional step=infra
time=2026-02-27T10:18:06.914Z level=DEBUG+3 msg="Deployment started" serviceGroup=Microsoft.Azure.ARO.HCP.Region resourceGroup=regional step=infra deployment=69f7c4622f62e632a295b0706e1b549ee46bbc7afd1957cf0bf75b6ec569d88f
time=2026-02-27T10:19:37.640Z level=ERROR msg="Step errored." serviceGroup=Microsoft.Azure.ARO.HCP.Region resourceGroup=regional step=infra err="failed to run ARM step: failed to poll deployment: failed to wait for deployment completion: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourcegroups/hcp-underlay-prow-j4590592/providers/Microsoft.Resources/deployments/69f7c4622f62e632a295b0706e1b549ee46bbc7afd1957cf0bf75b6ec569d88f/operationStatuses/08584294193994438873\n--------------------------------------------------------------------------------\nRESPONSE 200: 200 OK\nERROR CODE: DeploymentFailed\n--------------------------------------------------------------------------------\n{\n \"status\": \"Failed\",\n \"error\": {\n \"code\": \"DeploymentFailed\",\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\n \"details\": [\n {\n \"code\": \"Conflict\",\n \"message\": \"{\\r\\n \\\"status\\\": \\\"Failed\\\",\\r\\n \\\"error\\\": {\\r\\n \\\"code\\\": \\\"ResourceDeploymentFailure\\\",\\r\\n \\\"message\\\": \\\"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\\\",\\r\\n \\\"details\\\": [\\r\\n {\\r\\n \\\"code\\\": \\\"DeploymentFailed\\\",\\r\\n \\\"target\\\": \\\"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/hcp-underlay-prow-j4590592/providers/Microsoft.Resources/deployments/maestro-infra-deployment\\\",\\r\\n \\\"message\\\": \\\"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\\\",\\r\\n \\\"details\\\": [\\r\\n {\\r\\n \\\"code\\\": \\\"ResourceDeploymentFailure\\\",\\r\\n \\\"target\\\": \\\"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/hcp-underlay-prow-j4590592/providers/Microsoft.EventGrid/namespaces/arohcp-prow-maestro-j4590592\\\",\\r\\n \\\"message\\\": \\\"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\\\",\\r\\n \\\"details\\\": [\\r\\n {\\r\\n \\\"code\\\": \\\"Internal error\\\",\\r\\n \\\"message\\\": \\\"The operation failed due to an internal server error. The initial state of the impacted resources (if any) are restored. Please try again in few minutes. If error still persists, report f74de8ff-463b-4c97-957c-66d3c55c1ee9:2/27/2026 10:19:16 AM (UTC) to our forums for assistance or raise a support ticket .\\\"\\r\\n }\\r\\n ]\\r\\n }\\r\\n ]\\r\\n }\\r\\n ]\\r\\n }\\r\\n}\"\n }\n ]\n }\n}\n--------------------------------------------------------------------------------\n"time=2026-02-27T09:21:26.881Z level=INFO msg="Running step." serviceGroup=Microsoft.Azure.ARO.HCP.Region resourceGroup=regional step=infra description="Step infra\n Kind: ARM\n Template: templates/region.bicep\n Parameters: configurations/region.tmpl.bicepparam"
time=2026-02-27T09:21:28.911Z level=DEBUG+2 msg="Starting ARM deployment" serviceGroup=Microsoft.Azure.ARO.HCP.Region resourceGroup=regional step=infra
time=2026-02-27T09:21:30.092Z level=DEBUG+3 msg="Deployment started" serviceGroup=Microsoft.Azure.ARO.HCP.Region resourceGroup=regional step=infra deployment=903e5da09cc4363dc3f7a8aecd3551dc9c37a85e03edb087e386fe3654bd03e7
time=2026-02-27T09:24:01.350Z level=ERROR msg="Step errored." serviceGroup=Microsoft.Azure.ARO.HCP.Region resourceGroup=regional step=infra err="failed to run ARM step: failed to poll deployment: failed to wait for deployment completion: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourcegroups/hcp-underlay-prow-j1239808/providers/Microsoft.Resources/deployments/903e5da09cc4363dc3f7a8aecd3551dc9c37a85e03edb087e386fe3654bd03e7/operationStatuses/08584294227961177353\n--------------------------------------------------------------------------------\nRESPONSE 200: 200 OK\nERROR CODE: DeploymentFailed\n--------------------------------------------------------------------------------\n{\n \"status\": \"Failed\",\n \"error\": {\n \"code\": \"DeploymentFailed\",\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\n \"details\": [\n {\n \"code\": \"Conflict\",\n \"message\": \"{\\r\\n \\\"status\\\": \\\"Failed\\\",\\r\\n \\\"error\\\": {\\r\\n \\\"code\\\": \\\"ResourceDeploymentFailure\\\",\\r\\n \\\"message\\\": \\\"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\\\",\\r\\n \\\"details\\\": [\\r\\n {\\r\\n \\\"code\\\": \\\"DeploymentFailed\\\",\\r\\n \\\"target\\\": \\\"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/hcp-underlay-prow-j1239808/providers/Microsoft.Resources/deployments/maestro-infra-deployment\\\",\\r\\n \\\"message\\\": \\\"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\\\",\\r\\n \\\"details\\\": [\\r\\n {\\r\\n \\\"code\\\": \\\"ResourceDeploymentFailure\\\",\\r\\n \\\"target\\\": \\\"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/hcp-underlay-prow-j1239808/providers/Microsoft.EventGrid/namespaces/arohcp-prow-maestro-j1239808\\\",\\r\\n \\\"message\\\": \\\"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\\\",\\r\\n \\\"details\\\": [\\r\\n {\\r\\n \\\"code\\\": \\\"Internal error\\\",\\r\\n \\\"message\\\": \\\"The operation failed due to an internal server error. The initial state of the impacted resources (if any) are restored. Please try again in few minutes. If error still persists, report 40f70052-a2dd-42a6-a66c-497fbccd4a65:2/27/2026 9:23:27 AM (UTC) to our forums for assistance or raise a support ticket .\\\"\\r\\n }\\r\\n ]\\r\\n }\\r\\n ]\\r\\n }\\r\\n ]\\r\\n }\\r\\n}\"\n }\n ]\n }\n}\n--------------------------------------------------------------------------------\n"time=2026-02-27T09:17:15.254Z level=INFO msg="Running step." serviceGroup=Microsoft.Azure.ARO.HCP.Region resourceGroup=regional step=infra description="Step infra\n Kind: ARM\n Template: templates/region.bicep\n Parameters: configurations/region.tmpl.bicepparam"
time=2026-02-27T09:17:17.545Z level=DEBUG+2 msg="Starting ARM deployment" serviceGroup=Microsoft.Azure.ARO.HCP.Region resourceGroup=regional step=infra
time=2026-02-27T09:17:18.654Z level=DEBUG+3 msg="Deployment started" serviceGroup=Microsoft.Azure.ARO.HCP.Region resourceGroup=regional step=infra deployment=0e5c0b3f5daaf867f38418e66ba56810e77d5509018e92c85ec9694eed14164e
time=2026-02-27T09:18:49.476Z level=ERROR msg="Step errored." serviceGroup=Microsoft.Azure.ARO.HCP.Region resourceGroup=regional step=infra err="failed to run ARM step: failed to poll deployment: failed to wait for deployment completion: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourcegroups/hcp-underlay-prow-j1485184/providers/Microsoft.Resources/deployments/0e5c0b3f5daaf867f38418e66ba56810e77d5509018e92c85ec9694eed14164e/operationStatuses/08584294230474711808\n--------------------------------------------------------------------------------\nRESPONSE 200: 200 OK\nERROR CODE: DeploymentFailed\n--------------------------------------------------------------------------------\n{\n \"status\": \"Failed\",\n \"error\": {\n \"code\": \"DeploymentFailed\",\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\n \"details\": [\n {\n \"code\": \"Conflict\",\n \"message\": \"{\\r\\n \\\"status\\\": \\\"Failed\\\",\\r\\n \\\"error\\\": {\\r\\n \\\"code\\\": \\\"ResourceDeploymentFailure\\\",\\r\\n \\\"message\\\": \\\"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\\\",\\r\\n \\\"details\\\": [\\r\\n {\\r\\n \\\"code\\\": \\\"DeploymentFailed\\\",\\r\\n \\\"target\\\": \\\"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/hcp-underlay-prow-j1485184/providers/Microsoft.Resources/deployments/maestro-infra-deployment\\\",\\r\\n \\\"message\\\": \\\"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\\\",\\r\\n \\\"details\\\": [\\r\\n {\\r\\n \\\"code\\\": \\\"ResourceDeploymentFailure\\\",\\r\\n \\\"target\\\": \\\"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/hcp-underlay-prow-j1485184/providers/Microsoft.EventGrid/namespaces/arohcp-prow-maestro-j1485184\\\",\\r\\n \\\"message\\\": \\\"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\\\",\\r\\n \\\"details\\\": [\\r\\n {\\r\\n \\\"code\\\": \\\"Internal error\\\",\\r\\n \\\"message\\\": \\\"The operation failed due to an internal server error. The initial state of the impacted resources (if any) are restored. Please try again in few minutes. If error still persists, report 35a4baf1-1851-4eae-aad8-262b78081e79:2/27/2026 9:18:28 AM (UTC) to our forums for assistance or raise a support ticket .\\\"\\r\\n }\\r\\n ]\\r\\n }\\r\\n ]\\r\\n }\\r\\n ]\\r\\n }\\r\\n}\"\n }\n ]\n }\n}\n--------------------------------------------------------------------------------\n"Contributing tests (1)
Affected runs (14) | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Error response from registry: generic storage error: unknown_storage_failurefull failure pattern: Error response from registry: generic storage error: unknown_storage_failure Signal: IndeterminateSignal: Indeterminate — no prior history; active 4 day(s) — New failure pattern — no prior history | provision | 14 | 3.08%14 of 454 job runs affected | IndeterminateSignal: Indeterminate — no prior history; active 4 day(s) — New failure pattern — no prior history | Show trend detailsFeb 16: 0 · Feb 17: 0 · Feb 18: 0 · Feb 19: 0 · Feb 20: 0 · Feb 21: 0 · Feb 22: 0 · Feb 23: 1 · Feb 24: 0 · Feb 25: 1 · Feb 26: 3 · Feb 27: 9 · Feb 28: 0 · Mar 1: 0 | none | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Full failure examples (3)time=2026-02-27T20:53:18.564Z level=INFO msg="Running step." serviceGroup=Microsoft.Azure.ARO.HCP.SessionGate resourceGroup=global step=mirror-image description="Step mirror-image\n Kind: ImageMirror\n From {<nil> sessiongate.image.registry <nil>}:{<nil> sessiongate.image.repository <nil>}@{<nil> sessiongate.image.digest <nil>} to {<nil> acr.svc.name <nil>}\n"
time=2026-02-27T20:53:38.885Z level=ERROR msg="Step errored." serviceGroup=Microsoft.Azure.ARO.HCP.SessionGate resourceGroup=global step=mirror-image err="error running Image Mirror Step, failed to execute shell command: Checking USE_OC_LOGIN_REGISTRIES: registry.build06.ci.openshift.org registry.build06.ci.openshift.org registry.build06.ci.openshift.org registry.build06.ci.openshift.org\nSetting up registry authentication for CI source registry.\ninfo: Using registry public hostname registry.build06.ci.openshift.org\nSaved credentials for registry.build06.ci.openshift.org into /tmp/tmp.4TIpASyiGq/containers/auth.json\nLogging into target ACR arohcpsvcdev.\nLogin Succeeded\nMirroring image registry.build06.ci.openshift.org/ci-op-9q4dtjgy/pipeline@sha256:57661c7cefbf66d8260d9118c9d7ec05c3dd9a3b3f010b89058783c7c8d85f36 to arohcpsvcdev.azurecr.io/ci-op-9q4dtjgy/pipeline:57661c7cefbf66d8260d9118c9d7ec05c3dd9a3b3f010b89058783c7c8d85f36.\nThe image will still be available under it's original digest sha256:57661c7cefbf66d8260d9118c9d7ec05c3dd9a3b3f010b89058783c7c8d85f36 in the target registry.\nError response from registry: generic storage error: unknown_storage_failure\n exit status 1"time=2026-02-27T18:28:42.331Z level=INFO msg="Running step." serviceGroup=Microsoft.Azure.ARO.HCP.SessionGate resourceGroup=global step=mirror-image description="Step mirror-image\n Kind: ImageMirror\n From {<nil> sessiongate.image.registry <nil>}:{<nil> sessiongate.image.repository <nil>}@{<nil> sessiongate.image.digest <nil>} to {<nil> acr.svc.name <nil>}\n"
time=2026-02-27T18:29:17.612Z level=ERROR msg="Step errored." serviceGroup=Microsoft.Azure.ARO.HCP.SessionGate resourceGroup=global step=mirror-image err="error running Image Mirror Step, failed to execute shell command: Checking USE_OC_LOGIN_REGISTRIES: registry.build06.ci.openshift.org registry.build06.ci.openshift.org registry.build06.ci.openshift.org registry.build06.ci.openshift.org\nSetting up registry authentication for CI source registry.\ninfo: Using registry public hostname registry.build06.ci.openshift.org\nSaved credentials for registry.build06.ci.openshift.org into /tmp/tmp.SZzAf9qcGP/containers/auth.json\nLogging into target ACR arohcpsvcdev.\nLogin Succeeded\nMirroring image registry.build06.ci.openshift.org/ci-op-ndhix1pm/pipeline@sha256:3df5e467028465e193fd400e34fca415dc7d9609e11dbc7c33ca69edb14a7e7d to arohcpsvcdev.azurecr.io/ci-op-ndhix1pm/pipeline:3df5e467028465e193fd400e34fca415dc7d9609e11dbc7c33ca69edb14a7e7d.\nThe image will still be available under it's original digest sha256:3df5e467028465e193fd400e34fca415dc7d9609e11dbc7c33ca69edb14a7e7d in the target registry.\nError response from registry: generic storage error: unknown_storage_failure\n exit status 1"time=2026-02-27T17:27:05.933Z level=INFO msg="Running step." serviceGroup=Microsoft.Azure.ARO.HCP.SessionGate resourceGroup=global step=mirror-image description="Step mirror-image\n Kind: ImageMirror\n From {<nil> sessiongate.image.registry <nil>}:{<nil> sessiongate.image.repository <nil>}@{<nil> sessiongate.image.digest <nil>} to {<nil> acr.svc.name <nil>}\n"
time=2026-02-27T17:27:35.176Z level=ERROR msg="Step errored." serviceGroup=Microsoft.Azure.ARO.HCP.SessionGate resourceGroup=global step=mirror-image err="error running Image Mirror Step, failed to execute shell command: Checking USE_OC_LOGIN_REGISTRIES: registry.build06.ci.openshift.org registry.build06.ci.openshift.org registry.build06.ci.openshift.org registry.build06.ci.openshift.org\nSetting up registry authentication for CI source registry.\ninfo: Using registry public hostname registry.build06.ci.openshift.org\nSaved credentials for registry.build06.ci.openshift.org into /tmp/tmp.CaGoNhcOSj/containers/auth.json\nLogging into target ACR arohcpsvcdev.\nLogin Succeeded\nMirroring image registry.build06.ci.openshift.org/ci-op-d7502wz7/pipeline@sha256:1161d97dfab7c0d32936bfe9ac3421c623832b1a0b40803a3c69bc1d5f8668db to arohcpsvcdev.azurecr.io/ci-op-d7502wz7/pipeline:1161d97dfab7c0d32936bfe9ac3421c623832b1a0b40803a3c69bc1d5f8668db.\nThe image will still be available under it's original digest sha256:1161d97dfab7c0d32936bfe9ac3421c623832b1a0b40803a3c69bc1d5f8668db in the target registry.\nError response from registry: generic storage error: unknown_storage_failure\n exit status 1"Contributing tests (1)
Affected runs (14) | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
cluster operators not available: monitoring (Available=Unknown, Degraded=Unknown, Progressing=True, Upgradeable=Unknown)full failure pattern: cluster operators not available: monitoring (Available=Unknown, Degraded=Unknown, Progressing=True, Upgradeable=Unknown) Signal: IndeterminateSignal: Indeterminate — no prior history; active 2 day(s) — New failure pattern — no prior history | e2e | 12 | 2.64%12 of 454 job runs affected | IndeterminateSignal: Indeterminate — no prior history; active 2 day(s) — New failure pattern — no prior history | Show trend detailsFeb 16: 0 · Feb 17: 0 · Feb 18: 0 · Feb 19: 0 · Feb 20: 0 · Feb 21: 0 · Feb 22: 0 · Feb 23: 9 · Feb 24: 3 · Feb 25: 0 · Feb 26: 0 · Feb 27: 0 · Feb 28: 0 · Mar 1: 0 | none | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Full failure examples (3)fail [github.com/Azure/ARO-HCP/test/e2e/external_auth_create.go:242]: Unexpected error:
<*errors.errorString | 0xc00091f2b0>:
cluster operators not available: monitoring (Available=Unknown, Degraded=Unknown, Progressing=True, Upgradeable=Unknown)
{
s: "cluster operators not available: monitoring (Available=Unknown, Degraded=Unknown, Progressing=True, Upgradeable=Unknown)",
}
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/external_auth_create.go:242]: Unexpected error:
<*errors.errorString | 0xc00091f2b0>:
cluster operators not available: monitoring (Available=Unknown, Degraded=Unknown, Progressing=True, Upgradeable=Unknown)
{
s: "cluster operators not available: monitoring (Available=Unknown, Degraded=Unknown, Progressing=True, Upgradeable=Unknown)",
}
occurredfail [github.com/Azure/ARO-HCP/test/e2e/external_auth_create.go:242]: Unexpected error:
<*errors.errorString | 0xc0003873c0>:
cluster operators not available: monitoring (Available=Unknown, Degraded=Unknown, Progressing=True, Upgradeable=Unknown)
{
s: "cluster operators not available: monitoring (Available=Unknown, Degraded=Unknown, Progressing=True, Upgradeable=Unknown)",
}
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/external_auth_create.go:242]: Unexpected error:
<*errors.errorString | 0xc0003873c0>:
cluster operators not available: monitoring (Available=Unknown, Degraded=Unknown, Progressing=True, Upgradeable=Unknown)
{
s: "cluster operators not available: monitoring (Available=Unknown, Degraded=Unknown, Progressing=True, Upgradeable=Unknown)",
}
occurredfail [github.com/Azure/ARO-HCP/test/e2e/external_auth_create.go:242]: Unexpected error:
<*errors.errorString | 0xc0004c0e30>:
cluster operators not available: monitoring (Available=Unknown, Degraded=Unknown, Progressing=True, Upgradeable=Unknown)
{
s: "cluster operators not available: monitoring (Available=Unknown, Degraded=Unknown, Progressing=True, Upgradeable=Unknown)",
}
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/external_auth_create.go:242]: Unexpected error:
<*errors.errorString | 0xc0004c0e30>:
cluster operators not available: monitoring (Available=Unknown, Degraded=Unknown, Progressing=True, Upgradeable=Unknown)
{
s: "cluster operators not available: monitoring (Available=Unknown, Degraded=Unknown, Progressing=True, Upgradeable=Unknown)",
}
occurredContributing tests (1)
Affected runs (12) | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
timeout during CreateHCPClusterAndWait; context deadline exceededfull failure pattern: timeout during CreateHCPClusterAndWait; context deadline exceeded Signal: IndeterminateSignal: Indeterminate — no prior history; active 7 day(s) — New failure pattern — no prior history | e2e | 11 | 2.42%11 of 454 job runs affected | IndeterminateSignal: Indeterminate — no prior history; active 7 day(s) — New failure pattern — no prior history | Show trend detailsFeb 16: 0 · Feb 17: 0 · Feb 18: 0 · Feb 19: 0 · Feb 20: 0 · Feb 21: 0 · Feb 22: 0 · Feb 23: 1 · Feb 24: 17 · Feb 25: 12 · Feb 26: 40 · Feb 27: 28 · Feb 28: 30 · Mar 1: 13 | INT | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Full failure examples (3)fail [github.com/Azure/ARO-HCP/test/e2e/cluster_update.go:160]: Unexpected error:
<*fmt.wrapErrors | 0xc0012176b0>:
failed to create HCP cluster patch-tags-cluster, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster patch-tags-cluster in resource group patch-tags-rpd8lp, error: failed waiting for cluster="patch-tags-cluster" in resourcegroup="patch-tags-rpd8lp" to finish creating, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster patch-tags-cluster in resource group patch-tags-rpd8lp, error: context deadline exceeded
{
msg: "failed to create HCP cluster patch-tags-cluster, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster patch-tags-cluster in resource group patch-tags-rpd8lp, error: failed waiting for cluster=\"patch-tags-cluster\" in resourcegroup=\"patch-tags-rpd8lp\" to finish creating, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster patch-tags-cluster in resource group patch-tags-rpd8lp, error: context deadline exceeded",
errs: [
<*errors.errorString | 0xc00117ac60>{
s: "timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster patch-tags-cluster in resource group patch-tags-rpd8lp",
},
<*fmt.wrapErrors | 0xc001217680>{
msg: "failed waiting for cluster=\"patch-tags-cluster\" in resourcegroup=\"patch-tags-rpd8lp\" to finish creating, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster patch-tags-cluster in resource group patch-tags-rpd8lp, error: context deadline exceeded",
errs: [
<*errors.errorString | 0xc00117ac60>{
s: "timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster patch-tags-cluster in resource group patch-tags-rpd8lp",
},
<context.deadlineExceededError>{},
],
},
],
}
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_update.go:160]: Unexpected error:
<*fmt.wrapErrors | 0xc0012176b0>:
failed to create HCP cluster patch-tags-cluster, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster patch-tags-cluster in resource group patch-tags-rpd8lp, error: failed waiting for cluster="patch-tags-cluster" in resourcegroup="patch-tags-rpd8lp" to finish creating, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster patch-tags-cluster in resource group patch-tags-rpd8lp, error: context deadline exceeded
{
msg: "failed to create HCP cluster patch-tags-cluster, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster patch-tags-cluster in resource group patch-tags-rpd8lp, error: failed waiting for cluster=\"patch-tags-cluster\" in resourcegroup=\"patch-tags-rpd8lp\" to finish creating, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster patch-tags-cluster in resource group patch-tags-rpd8lp, error: context deadline exceeded",
errs: [
<*errors.errorString | 0xc00117ac60>{
s: "timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster patch-tags-cluster in resource group patch-tags-rpd8lp",
},
<*fmt.wrapErrors | 0xc001217680>{
msg: "failed waiting for cluster=\"patch-tags-cluster\" in resourcegroup=\"patch-tags-rpd8lp\" to finish creating, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster patch-tags-cluster in resource group patch-tags-rpd8lp, error: context deadline exceeded",
errs: [
<*errors.errorString | 0xc00117ac60>{
s: "timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster patch-tags-cluster in resource group patch-tags-rpd8lp",
},
<context.deadlineExceededError>{},
],
},
],
}
occurredfail [github.com/Azure/ARO-HCP/test/e2e/admin_api.go:91]: Unexpected error:
<*fmt.wrapErrors | 0xc0014ca8a0>:
failed to create HCP cluster sre-hcp-cluster, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster sre-hcp-cluster in resource group admin-api-breakglass-bm7zt9, error: failed waiting for cluster="sre-hcp-cluster" in resourcegroup="admin-api-breakglass-bm7zt9" to finish creating, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster sre-hcp-cluster in resource group admin-api-breakglass-bm7zt9, error: context deadline exceeded
{
msg: "failed to create HCP cluster sre-hcp-cluster, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster sre-hcp-cluster in resource group admin-api-breakglass-bm7zt9, error: failed waiting for cluster=\"sre-hcp-cluster\" in resourcegroup=\"admin-api-breakglass-bm7zt9\" to finish creating, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster sre-hcp-cluster in resource group admin-api-breakglass-bm7zt9, error: context deadline exceeded",
errs: [
<*errors.errorString | 0xc001363080>{
s: "timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster sre-hcp-cluster in resource group admin-api-breakglass-bm7zt9",
},
<*fmt.wrapErrors | 0xc0014ca870>{
msg: "failed waiting for cluster=\"sre-hcp-cluster\" in resourcegroup=\"admin-api-breakglass-bm7zt9\" to finish creating, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster sre-hcp-cluster in resource group admin-api-breakglass-bm7zt9, error: context deadline exceeded",
errs: [
<*errors.errorString | 0xc001363080>{
s: "timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster sre-hcp-cluster in resource group admin-api-breakglass-bm7zt9",
},
<context.deadlineExceededError>{},
],
},
],
}
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/admin_api.go:91]: Unexpected error:
<*fmt.wrapErrors | 0xc0014ca8a0>:
failed to create HCP cluster sre-hcp-cluster, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster sre-hcp-cluster in resource group admin-api-breakglass-bm7zt9, error: failed waiting for cluster="sre-hcp-cluster" in resourcegroup="admin-api-breakglass-bm7zt9" to finish creating, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster sre-hcp-cluster in resource group admin-api-breakglass-bm7zt9, error: context deadline exceeded
{
msg: "failed to create HCP cluster sre-hcp-cluster, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster sre-hcp-cluster in resource group admin-api-breakglass-bm7zt9, error: failed waiting for cluster=\"sre-hcp-cluster\" in resourcegroup=\"admin-api-breakglass-bm7zt9\" to finish creating, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster sre-hcp-cluster in resource group admin-api-breakglass-bm7zt9, error: context deadline exceeded",
errs: [
<*errors.errorString | 0xc001363080>{
s: "timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster sre-hcp-cluster in resource group admin-api-breakglass-bm7zt9",
},
<*fmt.wrapErrors | 0xc0014ca870>{
msg: "failed waiting for cluster=\"sre-hcp-cluster\" in resourcegroup=\"admin-api-breakglass-bm7zt9\" to finish creating, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster sre-hcp-cluster in resource group admin-api-breakglass-bm7zt9, error: context deadline exceeded",
errs: [
<*errors.errorString | 0xc001363080>{
s: "timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster sre-hcp-cluster in resource group admin-api-breakglass-bm7zt9",
},
<context.deadlineExceededError>{},
],
},
],
}
occurredfail [github.com/Azure/ARO-HCP/test/e2e/external_auth_list_and_verify.go:84]: Unexpected error:
<*fmt.wrapErrors | 0xc001181f50>:
failed to create HCP cluster ea-list26gbt7, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster ea-list26gbt7 in resource group ea-list-jrflvj, error: failed waiting for cluster="ea-list26gbt7" in resourcegroup="ea-list-jrflvj" to finish creating, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster ea-list26gbt7 in resource group ea-list-jrflvj, error: context deadline exceeded
{
msg: "failed to create HCP cluster ea-list26gbt7, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster ea-list26gbt7 in resource group ea-list-jrflvj, error: failed waiting for cluster=\"ea-list26gbt7\" in resourcegroup=\"ea-list-jrflvj\" to finish creating, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster ea-list26gbt7 in resource group ea-list-jrflvj, error: context deadline exceeded",
errs: [
<*errors.errorString | 0xc00101ac90>{
s: "timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster ea-list26gbt7 in resource group ea-list-jrflvj",
},
<*fmt.wrapErrors | 0xc001181f20>{
msg: "failed waiting for cluster=\"ea-list26gbt7\" in resourcegroup=\"ea-list-jrflvj\" to finish creating, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster ea-list26gbt7 in resource group ea-list-jrflvj, error: context deadline exceeded",
errs: [
<*errors.errorString | 0xc00101ac90>{
s: "timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster ea-list26gbt7 in resource group ea-list-jrflvj",
},
<context.deadlineExceededError>{},
],
},
],
}
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/external_auth_list_and_verify.go:84]: Unexpected error:
<*fmt.wrapErrors | 0xc001181f50>:
failed to create HCP cluster ea-list26gbt7, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster ea-list26gbt7 in resource group ea-list-jrflvj, error: failed waiting for cluster="ea-list26gbt7" in resourcegroup="ea-list-jrflvj" to finish creating, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster ea-list26gbt7 in resource group ea-list-jrflvj, error: context deadline exceeded
{
msg: "failed to create HCP cluster ea-list26gbt7, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster ea-list26gbt7 in resource group ea-list-jrflvj, error: failed waiting for cluster=\"ea-list26gbt7\" in resourcegroup=\"ea-list-jrflvj\" to finish creating, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster ea-list26gbt7 in resource group ea-list-jrflvj, error: context deadline exceeded",
errs: [
<*errors.errorString | 0xc00101ac90>{
s: "timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster ea-list26gbt7 in resource group ea-list-jrflvj",
},
<*fmt.wrapErrors | 0xc001181f20>{
msg: "failed waiting for cluster=\"ea-list26gbt7\" in resourcegroup=\"ea-list-jrflvj\" to finish creating, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster ea-list26gbt7 in resource group ea-list-jrflvj, error: context deadline exceeded",
errs: [
<*errors.errorString | 0xc00101ac90>{
s: "timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster ea-list26gbt7 in resource group ea-list-jrflvj",
},
<context.deadlineExceededError>{},
],
},
],
}
occurredContributing tests (18)
Affected runs (11) | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
ERROR: Invalid base64-encoded string: number of data characters (1325) cannot be 1 more than a multiple of 4full failure pattern: ERROR: Invalid base64-encoded string: number of data characters (1325) cannot be 1 more than a multiple of 4 Signal: IndeterminateSignal: Indeterminate — no prior history; active 2 day(s) — New failure pattern — no prior history | provision | 10 | 2.20%10 of 454 job runs affected | IndeterminateSignal: Indeterminate — no prior history; active 2 day(s) — New failure pattern — no prior history | Show trend detailsFeb 16: 0 · Feb 17: 0 · Feb 18: 0 · Feb 19: 0 · Feb 20: 0 · Feb 21: 0 · Feb 22: 0 · Feb 23: 0 · Feb 24: 1 · Feb 25: 9 · Feb 26: 0 · Feb 27: 0 · Feb 28: 0 · Mar 1: 0 | none | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Full failure examples (3)time=2026-02-25T03:54:32.710Z level=INFO msg="Running step." serviceGroup=Microsoft.Azure.ARO.HCP.SecretSyncController resourceGroup=global step=mirror-image description="Step mirror-image\n Kind: ImageMirror\n From {<nil> secretSyncController.image.registry <nil>}:{<nil> secretSyncController.image.repository <nil>}@{<nil> secretSyncController.image.digest <nil>} to {<nil> acr.svc.name <nil>}\n"
time=2026-02-25T03:54:34.346Z level=ERROR msg="Step errored." serviceGroup=Microsoft.Azure.ARO.HCP.SecretSyncController resourceGroup=global step=mirror-image err="error running Image Mirror Step, failed to execute shell command: Checking USE_OC_LOGIN_REGISTRIES: registry.build06.ci.openshift.org registry.build06.ci.openshift.org registry.build06.ci.openshift.org registry.build06.ci.openshift.org\nFetch pull secret for source registry registry.k8s.io from arohcpdev-global KV.\nERROR: Invalid base64-encoded string: number of data characters (1325) cannot be 1 more than a multiple of 4\n exit status 1"time=2026-02-25T03:03:01.221Z level=INFO msg="Running step." serviceGroup=Microsoft.Azure.ARO.HCP.Velero resourceGroup=global step=mirror-velero-image description="Step mirror-velero-image\n Kind: ImageMirror\n From {<nil> velero.server.registry <nil>}:{<nil> velero.server.repository <nil>}@{<nil> velero.server.digest <nil>} to {<nil> acr.svc.name <nil>}\n"
time=2026-02-25T03:03:02.633Z level=ERROR msg="Step errored." serviceGroup=Microsoft.Azure.ARO.HCP.Velero resourceGroup=global step=mirror-velero-image err="error running Image Mirror Step, failed to execute shell command: Checking USE_OC_LOGIN_REGISTRIES: registry.build06.ci.openshift.org registry.build06.ci.openshift.org registry.build06.ci.openshift.org registry.build06.ci.openshift.org\nFetch pull secret for source registry quay.io from arohcpdev-global KV.\nERROR: Invalid base64-encoded string: number of data characters (1325) cannot be 1 more than a multiple of 4\n exit status 1"time=2026-02-25T02:17:47.824Z level=INFO msg="Running step." serviceGroup=Microsoft.Azure.ARO.HCP.Maestro.Server resourceGroup=global step=mirror-image description="Step mirror-image\n Kind: ImageMirror\n From {<nil> maestro.image.registry <nil>}:{<nil> maestro.image.repository <nil>}@{<nil> maestro.image.digest <nil>} to {<nil> acr.svc.name <nil>}\n"
time=2026-02-25T02:17:49.114Z level=ERROR msg="Step errored." serviceGroup=Microsoft.Azure.ARO.HCP.Maestro.Server resourceGroup=global step=mirror-image err="error running Image Mirror Step, failed to execute shell command: Checking USE_OC_LOGIN_REGISTRIES: registry.build06.ci.openshift.org registry.build06.ci.openshift.org registry.build06.ci.openshift.org registry.build06.ci.openshift.org\nFetch pull secret for source registry quay.io from arohcpdev-global KV.\nERROR: Invalid base64-encoded string: number of data characters (1325) cannot be 1 more than a multiple of 4\n exit status 1"Contributing tests (5)
Affected runs (10) | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Cluster should become ready within 45 minutesfull failure pattern: Cluster should become ready within 45 minutes Signal: IndeterminateSignal: Indeterminate — no prior history; active 6 day(s) — New failure pattern — no prior history | e2e | 10 | 2.20%10 of 454 job runs affected | IndeterminateSignal: Indeterminate — no prior history; active 6 day(s) — New failure pattern — no prior history | Show trend detailsFeb 16: 0 · Feb 17: 0 · Feb 18: 0 · Feb 19: 0 · Feb 20: 0 · Feb 21: 0 · Feb 22: 0 · Feb 23: 0 · Feb 24: 1 · Feb 25: 1 · Feb 26: 3 · Feb 27: 2 · Feb 28: 2 · Mar 1: 1 | INT | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Full failure examples (2)fail [github.com/Azure/ARO-HCP/test/e2e/admin_credential_lifecycle.go:158]: Timed out after 2700.000s.
Cluster should become ready within 45 minutes
Expected
<bool>: false
to be true
fail [github.com/Azure/ARO-HCP/test/e2e/admin_credential_lifecycle.go:158]: Timed out after 2700.000s.
Cluster should become ready within 45 minutes
Expected
<bool>: false
to be truefail [github.com/Azure/ARO-HCP/test/e2e/admin_credential_lifecycle.go:158]: Timed out after 2700.001s.
Cluster should become ready within 45 minutes
Expected
<bool>: false
to be true
fail [github.com/Azure/ARO-HCP/test/e2e/admin_credential_lifecycle.go:158]: Timed out after 2700.001s.
Cluster should become ready within 45 minutes
Expected
<bool>: false
to be trueContributing tests (1)
Affected runs (10) | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Deserializaion Error: no output from commandfull failure pattern: Deserializaion Error: no output from command Signal: NoiseSignal: Noise — source deserialization/no-output error — New failure pattern — no prior history | e2e | 8 | 1.76%8 of 454 job runs affected | NoiseSignal: Noise — source deserialization/no-output error — New failure pattern — no prior history | Show trend detailsFeb 16: 0 · Feb 17: 0 · Feb 18: 0 · Feb 19: 0 · Feb 20: 0 · Feb 21: 0 · Feb 22: 0 · Feb 23: 0 · Feb 24: 0 · Feb 25: 1 · Feb 26: 2 · Feb 27: 3 · Feb 28: 2 · Mar 1: 1 | PROD | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Full failure examples (3)oller.go:284
> github.com/Azure/ARO-HCP/test/e2e.init.func16.2({0x7efc482480a0, 0xc000f84900})
/opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/clusters_sharing_resgroup.go:126
|
| By("waiting for first cluster to complete creation")
> _, err = poller1.PollUntilDone(ctx, &runtime.PollUntilDoneOptions{
| Frequency: framework.StandardPollInterval,
| })
github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func2({0x82213c0?, 0xc000f84900?})
/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/node.go:465
github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func3()
/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:901
github.com/onsi/ginkgo/v2/internal.(*Suite).runNode in goroutine 82
/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:888
------------------------------
[FAILED] in [It] - /opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/clusters_sharing_resgroup.go:129 @ 03/02/26 01:12:24.343
"ts"="2026-03-02 01:12:24.343554" "level"=0 "msg"="Commiting timing metadata."
"ts"="2026-03-02 01:12:24.343662" "level"=0 "msg"="Dumping deployment operations." "deployment"="customer-infra-basic-hcp-cluster-6x2ll7" "resourceGroup"="customer-rg-l6jc7m"
"ts"="2026-03-02 01:12:25.069532" "level"=0 "msg"="Dumping deployment operations." "deployment"="mi-basic-hcp-cluster-6x2ll7" "resourceGroup"=""
"ts"="2026-03-02 01:12:25.731588" "level"=0 "msg"="Dumping deployment operations." "deployment"="customer-infra-basic-hcp-cluster2-k58v2n" "resourceGroup"="customer-rg-l6jc7m"
"ts"="2026-03-02 01:12:25.928872" "level"=0 "msg"="Dumping deployment operations." "deployment"="mi-basic-hcp-cluster2-k58v2n" "resourceGroup"=""
"ts"="2026-03-02 01:12:26.878002" "level"=0 "msg"="Wrote timing metadata" "path"="/tmp/secret/timing-metadata-b8ed2189eb62b4145bd17832fa47015d3fc1ad56caa60a743b61b98540ac41f3.yaml.gz"
"ts"="2026-03-02 01:12:26.878099" "level"=0 "msg"="Wrote timing metadata" "path"="/logs/artifacts/test-timing/timing-metadata-b8ed2189eb62b4145bd17832fa47015d3fc1ad56caa60a743b61b98540ac41f3.yaml.gz"
"ts"="2026-03-02 01:12:26.878161" "level"=0 "msg"="collecting debug info"
"ts"="2026-03-02 01:12:26.879196" "level"=0 "msg"="collecting deployments" "resourceGroup"="aro-hcp-test-msi-containers-dev-33"
"ts"="2026-03-02 01:12:26.879211" "level"=0 "msg"="collecting deployments" "resourceGroup"="aro-hcp-test-msi-containers-dev-64"
"ts"="2026-03-02 01:12:26.879250" "level"=0 "msg"="collecting deployments" "resourceGroup"="aro-hcp-test-msi-containers-dev-89"
"ts"="2026-03-02 01:12:26.879223" "level"=0 "msg"="collecting deployments" "resourceGroup"="customer-rg-l6jc7m"
"ts"="2026-03-02 01:12:27.389788" "level"=0 "msg"="collecting operations" "resourceGroup"="aro-hcp-test-msi-containers-dev-33" "deployment"="pooledMsiScopedAssignments"
"ts"="2026-03-02 01:12:27.762702" "level"=0 "msg"="collecting operations" "resourceGroup"="aro-hcp-test-msi-containers-dev-64" "deployment"="pooledMsiScopedAssignments"
"ts"="2026-03-02 01:12:28.110722" "level"=0 "msg"="collecting operations" "resourceGroup"="aro-hcp-test-msi-containers-dev-89" "deployment"="pooledMsiScopedAssignments"
"ts"="2026-03-02 01:12:28.593089" "level"=0 "msg"="collecting operations" "resourceGroup"="customer-rg-l6jc7m" "deployment"="pooledNonMsiScopedAssignments-mooiin3edme32"
"ts"="2026-03-02 01:12:29.006960" "level"=0 "msg"="collecting operations" "resourceGroup"="aro-hcp-test-msi-containers-dev-33" "deployment"="msi-deployment-33"
"ts"="2026-03-02 01:12:29.234149" "level"=0 "msg"="collecting operations" "resourceGroup"="aro-hcp-test-msi-containers-dev-64" "deployment"="msi-deployment-64"
"ts"="2026-03-02 01:12:29.513903" "level"=0 "msg"="collecting operations" "resourceGroup"="aro-hcp-test-msi-containers-dev-89" "deployment"="msi-deployment-89"
"ts"="2026-03-02 01:12:29.986132" "level"=0 "msg"="collecting operations" "resourceGroup"="customer-rg-l6jc7m" "deployment"="customer-infra-basic-hcp-cluster2-k58v2n"
"ts"="2026-03-02 01:12:31.335703" "level"=0 "msg"="collecting operations" "resourceGroup"="customer-rg-l6jc7m" "deployment"="pooledNonMsiScopedAssignments-eqbgltuyzhpg4"
"ts"="2026-03-02 01:12:31.795402" "level"=0 "msg"="collecting operations" "resourceGroup"="customer-rg-l6jc7m" "deployment"="customer-infra-basic-hcp-cluster-6x2ll7"
"ts"="2026-03-02 01:12:32.294274" "level"=0 "msg"="finished collecting debug info"
"ts"="2026-03-02 01:12:32.294402" "level"=0 "msg"="deleting created resources"
"ts"="2026-03-02 01:12:32.294503" "level"=0 "msg"="deleting all hcp clusters in resource group" "resourceGroup"="customer-rg-l6jc7m"
Command Error: signal: killed
Deserializaion Error: no output from command
oller.go:284
> github.com/Azure/ARO-HCP/test/e2e.init.func16.2({0x7efc482480a0, 0xc000f84900})
/opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/clusters_sharing_resgroup.go:126
|
| By("waiting for first cluster to complete creation")
> _, err = poller1.PollUntilDone(ctx, &runtime.PollUntilDoneOptions{
| Frequency: framework.StandardPollInterval,
| })
github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func2({0x82213c0?, 0xc000f84900?})
/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/node.go:465
github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func3()
/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:901
github.com/onsi/ginkgo/v2/internal.(*Suite).runNode in goroutine 82
/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:888
------------------------------
[FAILED] in [It] - /opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/clusters_sharing_resgroup.go:129 @ 03/02/26 01:12:24.343
"ts"="2026-03-02 01:12:24.343554" "level"=0 "msg"="Commiting timing metadata."
"ts"="2026-03-02 01:12:24.343662" "level"=0 "msg"="Dumping deployment operations." "deployment"="customer-infra-basic-hcp-cluster-6x2ll7" "resourceGroup"="customer-rg-l6jc7m"
"ts"="2026-03-02 01:12:25.069532" "level"=0 "msg"="Dumping deployment operations." "deployment"="mi-basic-hcp-cluster-6x2ll7" "resourceGroup"=""
"ts"="2026-03-02 01:12:25.731588" "level"=0 "msg"="Dumping deployment operations." "deployment"="customer-infra-basic-hcp-cluster2-k58v2n" "resourceGroup"="customer-rg-l6jc7m"
"ts"="2026-03-02 01:12:25.928872" "level"=0 "msg"="Dumping deployment operations." "deployment"="mi-basic-hcp-cluster2-k58v2n" "resourceGroup"=""
"ts"="2026-03-02 01:12:26.878002" "level"=0 "msg"="Wrote timing metadata" "path"="/tmp/secret/timing-metadata-b8ed2189eb62b4145bd17832fa47015d3fc1ad56caa60a743b61b98540ac41f3.yaml.gz"
"ts"="2026-03-02 01:12:26.878099" "level"=0 "msg"="Wrote timing metadata" "path"="/logs/artifacts/test-timing/timing-metadata-b8ed2189eb62b4145bd17832fa47015d3fc1ad56caa60a743b61b98540ac41f3.yaml.gz"
"ts"="2026-03-02 01:12:26.878161" "level"=0 "msg"="collecting debug info"
"ts"="2026-03-02 01:12:26.879196" "level"=0 "msg"="collecting deployments" "resourceGroup"="aro-hcp-test-msi-containers-dev-33"
"ts"="2026-03-02 01:12:26.879211" "level"=0 "msg"="collecting deployments" "resourceGroup"="aro-hcp-test-msi-containers-dev-64"
"ts"="2026-03-02 01:12:26.879250" "level"=0 "msg"="collecting deployments" "resourceGroup"="aro-hcp-test-msi-containers-dev-89"
"ts"="2026-03-02 01:12:26.879223" "level"=0 "msg"="collecting deployments" "resourceGroup"="customer-rg-l6jc7m"
"ts"="2026-03-02 01:12:27.389788" "level"=0 "msg"="collecting operations" "resourceGroup"="aro-hcp-test-msi-containers-dev-33" "deployment"="pooledMsiScopedAssignments"
"ts"="2026-03-02 01:12:27.762702" "level"=0 "msg"="collecting operations" "resourceGroup"="aro-hcp-test-msi-containers-dev-64" "deployment"="pooledMsiScopedAssignments"
"ts"="2026-03-02 01:12:28.110722" "level"=0 "msg"="collecting operations" "resourceGroup"="aro-hcp-test-msi-containers-dev-89" "deployment"="pooledMsiScopedAssignments"
"ts"="2026-03-02 01:12:28.593089" "level"=0 "msg"="collecting operations" "resourceGroup"="customer-rg-l6jc7m" "deployment"="pooledNonMsiScopedAssignments-mooiin3edme32"
"ts"="2026-03-02 01:12:29.006960" "level"=0 "msg"="collecting operations" "resourceGroup"="aro-hcp-test-msi-containers-dev-33" "deployment"="msi-deployment-33"
"ts"="2026-03-02 01:12:29.234149" "level"=0 "msg"="collecting operations" "resourceGroup"="aro-hcp-test-msi-containers-dev-64" "deployment"="msi-deployment-64"
"ts"="2026-03-02 01:12:29.513903" "level"=0 "msg"="collecting operations" "resourceGroup"="aro-hcp-test-msi-containers-dev-89" "deployment"="msi-deployment-89"
"ts"="2026-03-02 01:12:29.986132" "level"=0 "msg"="collecting operations" "resourceGroup"="customer-rg-l6jc7m" "deployment"="customer-infra-basic-hcp-cluster2-k58v2n"
"ts"="2026-03-02 01:12:31.335703" "level"=0 "msg"="collecting operations" "resourceGroup"="customer-rg-l6jc7m" "deployment"="pooledNonMsiScopedAssignments-eqbgltuyzhpg4"
"ts"="2026-03-02 01:12:31.795402" "level"=0 "msg"="collecting operations" "resourceGroup"="customer-rg-l6jc7m" "deployment"="customer-infra-basic-hcp-cluster-6x2ll7"
"ts"="2026-03-02 01:12:32.294274" "level"=0 "msg"="finished collecting debug info"
"ts"="2026-03-02 01:12:32.294402" "level"=0 "msg"="deleting created resources"
"ts"="2026-03-02 01:12:32.294503" "level"=0 "msg"="deleting all hcp clusters in resource group" "resourceGroup"="customer-rg-l6jc7m"
Command Error: signal: killed
Deserializaion Error: no output from command.go:284
> github.com/Azure/ARO-HCP/test/e2e.init.func16.2({0x7fe009a29a60, 0xc000a2cf60})
/opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/clusters_sharing_resgroup.go:126
|
| By("waiting for first cluster to complete creation")
> _, err = poller1.PollUntilDone(ctx, &runtime.PollUntilDoneOptions{
| Frequency: framework.StandardPollInterval,
| })
github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func2({0x82213c0?, 0xc000a2cf60?})
/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/node.go:465
github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func3()
/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:901
github.com/onsi/ginkgo/v2/internal.(*Suite).runNode in goroutine 106
/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:888
------------------------------
[FAILED] in [It] - /opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/clusters_sharing_resgroup.go:129 @ 02/28/26 13:44:09.572
"ts"="2026-02-28 13:44:09.572241" "level"=0 "msg"="Commiting timing metadata."
"ts"="2026-02-28 13:44:09.572322" "level"=0 "msg"="Dumping deployment operations." "deployment"="customer-infra-basic-hcp-cluster-kdcxp9" "resourceGroup"="customer-rg-29mf77"
"ts"="2026-02-28 13:44:10.199648" "level"=0 "msg"="Dumping deployment operations." "deployment"="mi-basic-hcp-cluster-kdcxp9" "resourceGroup"=""
"ts"="2026-02-28 13:44:10.938987" "level"=0 "msg"="Dumping deployment operations." "deployment"="customer-infra-basic-hcp-cluster2-c9cpvm" "resourceGroup"="customer-rg-29mf77"
"ts"="2026-02-28 13:44:11.055310" "level"=0 "msg"="Dumping deployment operations." "deployment"="mi-basic-hcp-cluster2-c9cpvm" "resourceGroup"=""
"ts"="2026-02-28 13:44:11.807286" "level"=0 "msg"="Wrote timing metadata" "path"="/tmp/secret/timing-metadata-b8ed2189eb62b4145bd17832fa47015d3fc1ad56caa60a743b61b98540ac41f3.yaml.gz"
"ts"="2026-02-28 13:44:11.807399" "level"=0 "msg"="Wrote timing metadata" "path"="/logs/artifacts/test-timing/timing-metadata-b8ed2189eb62b4145bd17832fa47015d3fc1ad56caa60a743b61b98540ac41f3.yaml.gz"
"ts"="2026-02-28 13:44:11.807469" "level"=0 "msg"="collecting debug info"
"ts"="2026-02-28 13:44:11.808524" "level"=0 "msg"="collecting deployments" "resourceGroup"="aro-hcp-test-msi-containers-dev-275"
"ts"="2026-02-28 13:44:11.808532" "level"=0 "msg"="collecting deployments" "resourceGroup"="aro-hcp-test-msi-containers-dev-67"
"ts"="2026-02-28 13:44:11.808584" "level"=0 "msg"="collecting deployments" "resourceGroup"="aro-hcp-test-msi-containers-dev-16"
"ts"="2026-02-28 13:44:11.808549" "level"=0 "msg"="collecting deployments" "resourceGroup"="customer-rg-29mf77"
"ts"="2026-02-28 13:44:12.372883" "level"=0 "msg"="collecting operations" "resourceGroup"="aro-hcp-test-msi-containers-dev-275" "deployment"="pooledMsiScopedAssignments"
"ts"="2026-02-28 13:44:12.723750" "level"=0 "msg"="collecting operations" "resourceGroup"="aro-hcp-test-msi-containers-dev-67" "deployment"="pooledMsiScopedAssignments"
"ts"="2026-02-28 13:44:13.061745" "level"=0 "msg"="collecting operations" "resourceGroup"="aro-hcp-test-msi-containers-dev-16" "deployment"="pooledMsiScopedAssignments"
"ts"="2026-02-28 13:44:13.446336" "level"=0 "msg"="collecting operations" "resourceGroup"="customer-rg-29mf77" "deployment"="pooledNonMsiScopedAssignments-mooiin3edme32"
"ts"="2026-02-28 13:44:13.836912" "level"=0 "msg"="collecting operations" "resourceGroup"="aro-hcp-test-msi-containers-dev-275" "deployment"="msi-deployment-275"
"ts"="2026-02-28 13:44:14.191876" "level"=0 "msg"="collecting operations" "resourceGroup"="aro-hcp-test-msi-containers-dev-67" "deployment"="msi-deployment-67"
"ts"="2026-02-28 13:44:14.521905" "level"=0 "msg"="collecting operations" "resourceGroup"="aro-hcp-test-msi-containers-dev-16" "deployment"="msi-deployment-16"
"ts"="2026-02-28 13:44:14.978794" "level"=0 "msg"="collecting operations" "resourceGroup"="customer-rg-29mf77" "deployment"="customer-infra-basic-hcp-cluster2-c9cpvm"
"ts"="2026-02-28 13:44:16.431858" "level"=0 "msg"="collecting operations" "resourceGroup"="customer-rg-29mf77" "deployment"="pooledNonMsiScopedAssignments-eqbgltuyzhpg4"
"ts"="2026-02-28 13:44:16.902541" "level"=0 "msg"="collecting operations" "resourceGroup"="customer-rg-29mf77" "deployment"="customer-infra-basic-hcp-cluster-kdcxp9"
"ts"="2026-02-28 13:44:17.410312" "level"=0 "msg"="finished collecting debug info"
"ts"="2026-02-28 13:44:17.410446" "level"=0 "msg"="deleting created resources"
"ts"="2026-02-28 13:44:17.410565" "level"=0 "msg"="deleting all hcp clusters in resource group" "resourceGroup"="customer-rg-29mf77"
Command Error: signal: killed
Deserializaion Error: no output from command
.go:284
> github.com/Azure/ARO-HCP/test/e2e.init.func16.2({0x7fe009a29a60, 0xc000a2cf60})
/opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/clusters_sharing_resgroup.go:126
|
| By("waiting for first cluster to complete creation")
> _, err = poller1.PollUntilDone(ctx, &runtime.PollUntilDoneOptions{
| Frequency: framework.StandardPollInterval,
| })
github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func2({0x82213c0?, 0xc000a2cf60?})
/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/node.go:465
github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func3()
/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:901
github.com/onsi/ginkgo/v2/internal.(*Suite).runNode in goroutine 106
/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:888
------------------------------
[FAILED] in [It] - /opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/clusters_sharing_resgroup.go:129 @ 02/28/26 13:44:09.572
"ts"="2026-02-28 13:44:09.572241" "level"=0 "msg"="Commiting timing metadata."
"ts"="2026-02-28 13:44:09.572322" "level"=0 "msg"="Dumping deployment operations." "deployment"="customer-infra-basic-hcp-cluster-kdcxp9" "resourceGroup"="customer-rg-29mf77"
"ts"="2026-02-28 13:44:10.199648" "level"=0 "msg"="Dumping deployment operations." "deployment"="mi-basic-hcp-cluster-kdcxp9" "resourceGroup"=""
"ts"="2026-02-28 13:44:10.938987" "level"=0 "msg"="Dumping deployment operations." "deployment"="customer-infra-basic-hcp-cluster2-c9cpvm" "resourceGroup"="customer-rg-29mf77"
"ts"="2026-02-28 13:44:11.055310" "level"=0 "msg"="Dumping deployment operations." "deployment"="mi-basic-hcp-cluster2-c9cpvm" "resourceGroup"=""
"ts"="2026-02-28 13:44:11.807286" "level"=0 "msg"="Wrote timing metadata" "path"="/tmp/secret/timing-metadata-b8ed2189eb62b4145bd17832fa47015d3fc1ad56caa60a743b61b98540ac41f3.yaml.gz"
"ts"="2026-02-28 13:44:11.807399" "level"=0 "msg"="Wrote timing metadata" "path"="/logs/artifacts/test-timing/timing-metadata-b8ed2189eb62b4145bd17832fa47015d3fc1ad56caa60a743b61b98540ac41f3.yaml.gz"
"ts"="2026-02-28 13:44:11.807469" "level"=0 "msg"="collecting debug info"
"ts"="2026-02-28 13:44:11.808524" "level"=0 "msg"="collecting deployments" "resourceGroup"="aro-hcp-test-msi-containers-dev-275"
"ts"="2026-02-28 13:44:11.808532" "level"=0 "msg"="collecting deployments" "resourceGroup"="aro-hcp-test-msi-containers-dev-67"
"ts"="2026-02-28 13:44:11.808584" "level"=0 "msg"="collecting deployments" "resourceGroup"="aro-hcp-test-msi-containers-dev-16"
"ts"="2026-02-28 13:44:11.808549" "level"=0 "msg"="collecting deployments" "resourceGroup"="customer-rg-29mf77"
"ts"="2026-02-28 13:44:12.372883" "level"=0 "msg"="collecting operations" "resourceGroup"="aro-hcp-test-msi-containers-dev-275" "deployment"="pooledMsiScopedAssignments"
"ts"="2026-02-28 13:44:12.723750" "level"=0 "msg"="collecting operations" "resourceGroup"="aro-hcp-test-msi-containers-dev-67" "deployment"="pooledMsiScopedAssignments"
"ts"="2026-02-28 13:44:13.061745" "level"=0 "msg"="collecting operations" "resourceGroup"="aro-hcp-test-msi-containers-dev-16" "deployment"="pooledMsiScopedAssignments"
"ts"="2026-02-28 13:44:13.446336" "level"=0 "msg"="collecting operations" "resourceGroup"="customer-rg-29mf77" "deployment"="pooledNonMsiScopedAssignments-mooiin3edme32"
"ts"="2026-02-28 13:44:13.836912" "level"=0 "msg"="collecting operations" "resourceGroup"="aro-hcp-test-msi-containers-dev-275" "deployment"="msi-deployment-275"
"ts"="2026-02-28 13:44:14.191876" "level"=0 "msg"="collecting operations" "resourceGroup"="aro-hcp-test-msi-containers-dev-67" "deployment"="msi-deployment-67"
"ts"="2026-02-28 13:44:14.521905" "level"=0 "msg"="collecting operations" "resourceGroup"="aro-hcp-test-msi-containers-dev-16" "deployment"="msi-deployment-16"
"ts"="2026-02-28 13:44:14.978794" "level"=0 "msg"="collecting operations" "resourceGroup"="customer-rg-29mf77" "deployment"="customer-infra-basic-hcp-cluster2-c9cpvm"
"ts"="2026-02-28 13:44:16.431858" "level"=0 "msg"="collecting operations" "resourceGroup"="customer-rg-29mf77" "deployment"="pooledNonMsiScopedAssignments-eqbgltuyzhpg4"
"ts"="2026-02-28 13:44:16.902541" "level"=0 "msg"="collecting operations" "resourceGroup"="customer-rg-29mf77" "deployment"="customer-infra-basic-hcp-cluster-kdcxp9"
"ts"="2026-02-28 13:44:17.410312" "level"=0 "msg"="finished collecting debug info"
"ts"="2026-02-28 13:44:17.410446" "level"=0 "msg"="deleting created resources"
"ts"="2026-02-28 13:44:17.410565" "level"=0 "msg"="deleting all hcp clusters in resource group" "resourceGroup"="customer-rg-29mf77"
Command Error: signal: killed
Deserializaion Error: no output from command:284
> github.com/Azure/ARO-HCP/test/e2e.init.func16.2({0x7f50e423e660, 0xc000df70b0})
/opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/clusters_sharing_resgroup.go:126
|
| By("waiting for first cluster to complete creation")
> _, err = poller1.PollUntilDone(ctx, &runtime.PollUntilDoneOptions{
| Frequency: framework.StandardPollInterval,
| })
github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func2({0x82213c0?, 0xc000df70b0?})
/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/node.go:465
github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func3()
/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:901
github.com/onsi/ginkgo/v2/internal.(*Suite).runNode in goroutine 94
/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:888
------------------------------
[FAILED] in [It] - /opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/clusters_sharing_resgroup.go:129 @ 02/28/26 10:41:38.759
"ts"="2026-02-28 10:41:38.760059" "level"=0 "msg"="Commiting timing metadata."
"ts"="2026-02-28 10:41:38.760158" "level"=0 "msg"="Dumping deployment operations." "deployment"="customer-infra-basic-hcp-cluster-w9t9q7" "resourceGroup"="customer-rg-79tnhj"
"ts"="2026-02-28 10:41:39.321838" "level"=0 "msg"="Dumping deployment operations." "deployment"="mi-basic-hcp-cluster-w9t9q7" "resourceGroup"=""
"ts"="2026-02-28 10:41:40.384891" "level"=0 "msg"="Dumping deployment operations." "deployment"="customer-infra-basic-hcp-cluster2-t6dlsm" "resourceGroup"="customer-rg-79tnhj"
"ts"="2026-02-28 10:41:40.546646" "level"=0 "msg"="Dumping deployment operations." "deployment"="mi-basic-hcp-cluster2-t6dlsm" "resourceGroup"=""
"ts"="2026-02-28 10:41:41.764729" "level"=0 "msg"="Wrote timing metadata" "path"="/tmp/secret/timing-metadata-b8ed2189eb62b4145bd17832fa47015d3fc1ad56caa60a743b61b98540ac41f3.yaml.gz"
"ts"="2026-02-28 10:41:41.764840" "level"=0 "msg"="Wrote timing metadata" "path"="/logs/artifacts/test-timing/timing-metadata-b8ed2189eb62b4145bd17832fa47015d3fc1ad56caa60a743b61b98540ac41f3.yaml.gz"
"ts"="2026-02-28 10:41:41.764916" "level"=0 "msg"="collecting debug info"
"ts"="2026-02-28 10:41:41.765947" "level"=0 "msg"="collecting deployments" "resourceGroup"="aro-hcp-test-msi-containers-dev-113"
"ts"="2026-02-28 10:41:41.765967" "level"=0 "msg"="collecting deployments" "resourceGroup"="aro-hcp-test-msi-containers-dev-249"
"ts"="2026-02-28 10:41:41.765986" "level"=0 "msg"="collecting deployments" "resourceGroup"="aro-hcp-test-msi-containers-dev-14"
"ts"="2026-02-28 10:41:41.765947" "level"=0 "msg"="collecting deployments" "resourceGroup"="customer-rg-79tnhj"
"ts"="2026-02-28 10:41:42.522257" "level"=0 "msg"="collecting operations" "resourceGroup"="aro-hcp-test-msi-containers-dev-113" "deployment"="pooledMsiScopedAssignments"
"ts"="2026-02-28 10:41:42.665793" "level"=0 "msg"="collecting operations" "resourceGroup"="aro-hcp-test-msi-containers-dev-249" "deployment"="pooledMsiScopedAssignments"
"ts"="2026-02-28 10:41:43.018536" "level"=0 "msg"="collecting operations" "resourceGroup"="aro-hcp-test-msi-containers-dev-14" "deployment"="pooledMsiScopedAssignments"
"ts"="2026-02-28 10:41:43.543205" "level"=0 "msg"="collecting operations" "resourceGroup"="customer-rg-79tnhj" "deployment"="pooledNonMsiScopedAssignments-mooiin3edme32"
"ts"="2026-02-28 10:41:43.742639" "level"=0 "msg"="collecting operations" "resourceGroup"="aro-hcp-test-msi-containers-dev-113" "deployment"="msi-deployment-113"
"ts"="2026-02-28 10:41:44.109135" "level"=0 "msg"="collecting operations" "resourceGroup"="aro-hcp-test-msi-containers-dev-249" "deployment"="msi-deployment-249"
"ts"="2026-02-28 10:41:44.537345" "level"=0 "msg"="collecting operations" "resourceGroup"="aro-hcp-test-msi-containers-dev-14" "deployment"="msi-deployment-14"
"ts"="2026-02-28 10:41:44.846232" "level"=0 "msg"="collecting operations" "resourceGroup"="customer-rg-79tnhj" "deployment"="customer-infra-basic-hcp-cluster2-t6dlsm"
"ts"="2026-02-28 10:41:46.295132" "level"=0 "msg"="collecting operations" "resourceGroup"="customer-rg-79tnhj" "deployment"="pooledNonMsiScopedAssignments-eqbgltuyzhpg4"
"ts"="2026-02-28 10:41:46.878822" "level"=0 "msg"="collecting operations" "resourceGroup"="customer-rg-79tnhj" "deployment"="customer-infra-basic-hcp-cluster-w9t9q7"
"ts"="2026-02-28 10:41:47.362416" "level"=0 "msg"="finished collecting debug info"
"ts"="2026-02-28 10:41:47.362560" "level"=0 "msg"="deleting created resources"
"ts"="2026-02-28 10:41:47.362673" "level"=0 "msg"="deleting all hcp clusters in resource group" "resourceGroup"="customer-rg-79tnhj"
Command Error: signal: killed
Deserializaion Error: no output from command
:284
> github.com/Azure/ARO-HCP/test/e2e.init.func16.2({0x7f50e423e660, 0xc000df70b0})
/opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/clusters_sharing_resgroup.go:126
|
| By("waiting for first cluster to complete creation")
> _, err = poller1.PollUntilDone(ctx, &runtime.PollUntilDoneOptions{
| Frequency: framework.StandardPollInterval,
| })
github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func2({0x82213c0?, 0xc000df70b0?})
/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/node.go:465
github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func3()
/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:901
github.com/onsi/ginkgo/v2/internal.(*Suite).runNode in goroutine 94
/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:888
------------------------------
[FAILED] in [It] - /opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/clusters_sharing_resgroup.go:129 @ 02/28/26 10:41:38.759
"ts"="2026-02-28 10:41:38.760059" "level"=0 "msg"="Commiting timing metadata."
"ts"="2026-02-28 10:41:38.760158" "level"=0 "msg"="Dumping deployment operations." "deployment"="customer-infra-basic-hcp-cluster-w9t9q7" "resourceGroup"="customer-rg-79tnhj"
"ts"="2026-02-28 10:41:39.321838" "level"=0 "msg"="Dumping deployment operations." "deployment"="mi-basic-hcp-cluster-w9t9q7" "resourceGroup"=""
"ts"="2026-02-28 10:41:40.384891" "level"=0 "msg"="Dumping deployment operations." "deployment"="customer-infra-basic-hcp-cluster2-t6dlsm" "resourceGroup"="customer-rg-79tnhj"
"ts"="2026-02-28 10:41:40.546646" "level"=0 "msg"="Dumping deployment operations." "deployment"="mi-basic-hcp-cluster2-t6dlsm" "resourceGroup"=""
"ts"="2026-02-28 10:41:41.764729" "level"=0 "msg"="Wrote timing metadata" "path"="/tmp/secret/timing-metadata-b8ed2189eb62b4145bd17832fa47015d3fc1ad56caa60a743b61b98540ac41f3.yaml.gz"
"ts"="2026-02-28 10:41:41.764840" "level"=0 "msg"="Wrote timing metadata" "path"="/logs/artifacts/test-timing/timing-metadata-b8ed2189eb62b4145bd17832fa47015d3fc1ad56caa60a743b61b98540ac41f3.yaml.gz"
"ts"="2026-02-28 10:41:41.764916" "level"=0 "msg"="collecting debug info"
"ts"="2026-02-28 10:41:41.765947" "level"=0 "msg"="collecting deployments" "resourceGroup"="aro-hcp-test-msi-containers-dev-113"
"ts"="2026-02-28 10:41:41.765967" "level"=0 "msg"="collecting deployments" "resourceGroup"="aro-hcp-test-msi-containers-dev-249"
"ts"="2026-02-28 10:41:41.765986" "level"=0 "msg"="collecting deployments" "resourceGroup"="aro-hcp-test-msi-containers-dev-14"
"ts"="2026-02-28 10:41:41.765947" "level"=0 "msg"="collecting deployments" "resourceGroup"="customer-rg-79tnhj"
"ts"="2026-02-28 10:41:42.522257" "level"=0 "msg"="collecting operations" "resourceGroup"="aro-hcp-test-msi-containers-dev-113" "deployment"="pooledMsiScopedAssignments"
"ts"="2026-02-28 10:41:42.665793" "level"=0 "msg"="collecting operations" "resourceGroup"="aro-hcp-test-msi-containers-dev-249" "deployment"="pooledMsiScopedAssignments"
"ts"="2026-02-28 10:41:43.018536" "level"=0 "msg"="collecting operations" "resourceGroup"="aro-hcp-test-msi-containers-dev-14" "deployment"="pooledMsiScopedAssignments"
"ts"="2026-02-28 10:41:43.543205" "level"=0 "msg"="collecting operations" "resourceGroup"="customer-rg-79tnhj" "deployment"="pooledNonMsiScopedAssignments-mooiin3edme32"
"ts"="2026-02-28 10:41:43.742639" "level"=0 "msg"="collecting operations" "resourceGroup"="aro-hcp-test-msi-containers-dev-113" "deployment"="msi-deployment-113"
"ts"="2026-02-28 10:41:44.109135" "level"=0 "msg"="collecting operations" "resourceGroup"="aro-hcp-test-msi-containers-dev-249" "deployment"="msi-deployment-249"
"ts"="2026-02-28 10:41:44.537345" "level"=0 "msg"="collecting operations" "resourceGroup"="aro-hcp-test-msi-containers-dev-14" "deployment"="msi-deployment-14"
"ts"="2026-02-28 10:41:44.846232" "level"=0 "msg"="collecting operations" "resourceGroup"="customer-rg-79tnhj" "deployment"="customer-infra-basic-hcp-cluster2-t6dlsm"
"ts"="2026-02-28 10:41:46.295132" "level"=0 "msg"="collecting operations" "resourceGroup"="customer-rg-79tnhj" "deployment"="pooledNonMsiScopedAssignments-eqbgltuyzhpg4"
"ts"="2026-02-28 10:41:46.878822" "level"=0 "msg"="collecting operations" "resourceGroup"="customer-rg-79tnhj" "deployment"="customer-infra-basic-hcp-cluster-w9t9q7"
"ts"="2026-02-28 10:41:47.362416" "level"=0 "msg"="finished collecting debug info"
"ts"="2026-02-28 10:41:47.362560" "level"=0 "msg"="deleting created resources"
"ts"="2026-02-28 10:41:47.362673" "level"=0 "msg"="deleting all hcp clusters in resource group" "resourceGroup"="customer-rg-79tnhj"
Command Error: signal: killed
Deserializaion Error: no output from commandContributing tests (2)
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
All ClusterOperators should report Available=True, but these are not available: [monitoring (Available=Unknown)]full failure pattern: All ClusterOperators should report Available=True, but these are not available: [monitoring (Available=Unknown)] Signal: IndeterminateSignal: Indeterminate — no prior history; active 2 day(s) — New failure pattern — no prior history | e2e | 8 | 1.76%8 of 454 job runs affected | IndeterminateSignal: Indeterminate — no prior history; active 2 day(s) — New failure pattern — no prior history | Show trend detailsFeb 16: 0 · Feb 17: 0 · Feb 18: 0 · Feb 19: 0 · Feb 20: 0 · Feb 21: 0 · Feb 22: 0 · Feb 23: 1 · Feb 24: 7 · Feb 25: 0 · Feb 26: 0 · Feb 27: 0 · Feb 28: 0 · Mar 1: 0 | none | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Full failure examples (3)fail [github.com/Azure/ARO-HCP/test/e2e/cluster_authorized_cidrs_connectivity.go:390]: Timed out after 600.005s.
The function passed to Eventually failed at /opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/cluster_authorized_cidrs_connectivity.go:389 with:
All ClusterOperators should report Available=True, but these are not available: [monitoring (Available=Unknown)]
Expected
<[]string | len:1, cap:1>: [
"monitoring (Available=Unknown)",
]
to be empty
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_authorized_cidrs_connectivity.go:390]: Timed out after 600.005s.
The function passed to Eventually failed at /opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/cluster_authorized_cidrs_connectivity.go:389 with:
All ClusterOperators should report Available=True, but these are not available: [monitoring (Available=Unknown)]
Expected
<[]string | len:1, cap:1>: [
"monitoring (Available=Unknown)",
]
to be emptyfail [github.com/Azure/ARO-HCP/test/e2e/cluster_authorized_cidrs_connectivity.go:390]: Timed out after 600.000s.
The function passed to Eventually failed at /opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/cluster_authorized_cidrs_connectivity.go:389 with:
All ClusterOperators should report Available=True, but these are not available: [monitoring (Available=Unknown)]
Expected
<[]string | len:1, cap:1>: [
"monitoring (Available=Unknown)",
]
to be empty
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_authorized_cidrs_connectivity.go:390]: Timed out after 600.000s.
The function passed to Eventually failed at /opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/cluster_authorized_cidrs_connectivity.go:389 with:
All ClusterOperators should report Available=True, but these are not available: [monitoring (Available=Unknown)]
Expected
<[]string | len:1, cap:1>: [
"monitoring (Available=Unknown)",
]
to be emptyfail [github.com/Azure/ARO-HCP/test/e2e/cluster_authorized_cidrs_connectivity.go:390]: Timed out after 600.006s.
The function passed to Eventually failed at /opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/cluster_authorized_cidrs_connectivity.go:389 with:
All ClusterOperators should report Available=True, but these are not available: [monitoring (Available=Unknown)]
Expected
<[]string | len:1, cap:1>: [
"monitoring (Available=Unknown)",
]
to be empty
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_authorized_cidrs_connectivity.go:390]: Timed out after 600.006s.
The function passed to Eventually failed at /opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/cluster_authorized_cidrs_connectivity.go:389 with:
All ClusterOperators should report Available=True, but these are not available: [monitoring (Available=Unknown)]
Expected
<[]string | len:1, cap:1>: [
"monitoring (Available=Unknown)",
]
to be emptyContributing tests (1)
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Interrupted by Userfull failure pattern: Interrupted by User Signal: IndeterminateSignal: Indeterminate — no prior history; active 5 day(s) — New failure pattern — no prior history | e2e | 6 | 1.32%6 of 454 job runs affected | IndeterminateSignal: Indeterminate — no prior history; active 5 day(s) — New failure pattern — no prior history | Show trend detailsFeb 16: 0 · Feb 17: 0 · Feb 18: 0 · Feb 19: 0 · Feb 20: 0 · Feb 21: 0 · Feb 22: 0 · Feb 23: 2 · Feb 24: 1 · Feb 25: 1 · Feb 26: 3 · Feb 27: 0 · Feb 28: 1 · Mar 1: 0 | INT, PROD | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Full failure examples (3)fail [:0]: Interrupted by User fail [:0]: Interrupted by User fail [github.com/Azure/ARO-HCP/test/e2e/arm64_nodepool.go:35]: Interrupted by User fail [github.com/Azure/ARO-HCP/test/e2e/arm64_nodepool.go:35]: Interrupted by User fail [github.com/Azure/ARO-HCP/test/e2e/clusters_sharing_resgroup.go:35]: Interrupted by User fail [github.com/Azure/ARO-HCP/test/e2e/clusters_sharing_resgroup.go:35]: Interrupted by User Contributing tests (6)
Affected runs (6) | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
failed to gather logs: failed to execute cluster id query: failed to execute query: failed to execute query: Op(OpQuery): Kind(KInternal): Error while getting token : Get "<url>full failure pattern: failed to gather logs: failed to execute cluster id query: failed to execute query: failed to execute query: Op(OpQuery): Kind(KInternal): Error while getting token : Get "<url> Signal: IndeterminateSignal: Indeterminate — no prior history; active 1 day(s) — New failure pattern — no prior history | e2e | 6 | 1.32%6 of 454 job runs affected | IndeterminateSignal: Indeterminate — no prior history; active 1 day(s) — New failure pattern — no prior history | Show trend detailsFeb 16: 0 · Feb 17: 0 · Feb 18: 0 · Feb 19: 0 · Feb 20: 0 · Feb 21: 0 · Feb 22: 0 · Feb 23: 0 · Feb 24: 6 · Feb 25: 0 · Feb 26: 0 · Feb 27: 0 · Feb 28: 0 · Mar 1: 0 | none | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Full failure examples (3)fail [github.com/Azure/ARO-HCP/test/e2e/kusto_logs_present.go:94]: Timed out after 600.014s.
Expected success, but got an error:
<*fmt.wrapError | 0xc000546440>:
failed to gather logs: failed to execute cluster id query: failed to execute query: failed to execute query: Op(OpQuery): Kind(KInternal): Error while getting token : Get "https://hcp-dev-us.westus3.kusto.windows.net/v1/rest/auth/metadata": dial tcp: lookup hcp-dev-us.westus3.kusto.windows.net on 172.30.0.10:53: no such host
{
msg: "failed to gather logs: failed to execute cluster id query: failed to execute query: failed to execute query: Op(OpQuery): Kind(KInternal): Error while getting token : Get \"https://hcp-dev-us.westus3.kusto.windows.net/v1/rest/auth/metadata\": dial tcp: lookup hcp-dev-us.westus3.kusto.windows.net on 172.30.0.10:53: no such host",
err: <*fmt.wrapError | 0xc000546420>{
msg: "failed to execute cluster id query: failed to execute query: failed to execute query: Op(OpQuery): Kind(KInternal): Error while getting token : Get \"https://hcp-dev-us.westus3.kusto.windows.net/v1/rest/auth/metadata\": dial tcp: lookup hcp-dev-us.westus3.kusto.windows.net on 172.30.0.10:53: no such host",
err: <*fmt.wrapError | 0xc000546400>{
msg: "failed to execute query: failed to execute query: Op(OpQuery): Kind(KInternal): Error while getting token : Get \"https://hcp-dev-us.westus3.kusto.windows.net/v1/rest/auth/metadata\": dial tcp: lookup hcp-dev-us.westus3.kusto.windows.net on 172.30.0.10:53: no such host",
err: <*fmt.wrapError | 0xc0005463e0>{
msg: "failed to execute query: Op(OpQuery): Kind(KInternal): Error while getting token : Get \"https://hcp-dev-us.westus3.kusto.windows.net/v1/rest/auth/metadata\": dial tcp: lookup hcp-dev-us.westus3.kusto.windows.net on 172.30.0.10:53: no such host",
err: <*errors.Error | 0xc000706af0>{
Op: 1,
Kind: 2,
Err: <*errors.errorString | 0xc000488360>{
s: "Error while getting token : Get \"https://hcp-dev-us.westus3.kusto.windows.net/v1/rest/auth/metadata\": dial tcp: lookup hcp-dev-us.westus3.kusto.windows.net on 172.30.0.10:53: no such host",
},
restErrMsg: nil,
decoded: nil,
permanent: false,
inner: nil,
},
},
},
},
}
fail [github.com/Azure/ARO-HCP/test/e2e/kusto_logs_present.go:94]: Timed out after 600.014s.
Expected success, but got an error:
<*fmt.wrapError | 0xc000546440>:
failed to gather logs: failed to execute cluster id query: failed to execute query: failed to execute query: Op(OpQuery): Kind(KInternal): Error while getting token : Get "https://hcp-dev-us.westus3.kusto.windows.net/v1/rest/auth/metadata": dial tcp: lookup hcp-dev-us.westus3.kusto.windows.net on 172.30.0.10:53: no such host
{
msg: "failed to gather logs: failed to execute cluster id query: failed to execute query: failed to execute query: Op(OpQuery): Kind(KInternal): Error while getting token : Get \"https://hcp-dev-us.westus3.kusto.windows.net/v1/rest/auth/metadata\": dial tcp: lookup hcp-dev-us.westus3.kusto.windows.net on 172.30.0.10:53: no such host",
err: <*fmt.wrapError | 0xc000546420>{
msg: "failed to execute cluster id query: failed to execute query: failed to execute query: Op(OpQuery): Kind(KInternal): Error while getting token : Get \"https://hcp-dev-us.westus3.kusto.windows.net/v1/rest/auth/metadata\": dial tcp: lookup hcp-dev-us.westus3.kusto.windows.net on 172.30.0.10:53: no such host",
err: <*fmt.wrapError | 0xc000546400>{
msg: "failed to execute query: failed to execute query: Op(OpQuery): Kind(KInternal): Error while getting token : Get \"https://hcp-dev-us.westus3.kusto.windows.net/v1/rest/auth/metadata\": dial tcp: lookup hcp-dev-us.westus3.kusto.windows.net on 172.30.0.10:53: no such host",
err: <*fmt.wrapError | 0xc0005463e0>{
msg: "failed to execute query: Op(OpQuery): Kind(KInternal): Error while getting token : Get \"https://hcp-dev-us.westus3.kusto.windows.net/v1/rest/auth/metadata\": dial tcp: lookup hcp-dev-us.westus3.kusto.windows.net on 172.30.0.10:53: no such host",
err: <*errors.Error | 0xc000706af0>{
Op: 1,
Kind: 2,
Err: <*errors.errorString | 0xc000488360>{
s: "Error while getting token : Get \"https://hcp-dev-us.westus3.kusto.windows.net/v1/rest/auth/metadata\": dial tcp: lookup hcp-dev-us.westus3.kusto.windows.net on 172.30.0.10:53: no such host",
},
restErrMsg: nil,
decoded: nil,
permanent: false,
inner: nil,
},
},
},
},
}fail [github.com/Azure/ARO-HCP/test/e2e/kusto_logs_present.go:94]: Timed out after 600.021s.
Expected success, but got an error:
<*fmt.wrapError | 0xc0008202a0>:
failed to gather logs: failed to execute cluster id query: failed to execute query: failed to execute query: Op(OpQuery): Kind(KInternal): Error while getting token : Get "https://hcp-dev-us.westus3.kusto.windows.net/v1/rest/auth/metadata": dial tcp: lookup hcp-dev-us.westus3.kusto.windows.net on 172.30.0.10:53: no such host
{
msg: "failed to gather logs: failed to execute cluster id query: failed to execute query: failed to execute query: Op(OpQuery): Kind(KInternal): Error while getting token : Get \"https://hcp-dev-us.westus3.kusto.windows.net/v1/rest/auth/metadata\": dial tcp: lookup hcp-dev-us.westus3.kusto.windows.net on 172.30.0.10:53: no such host",
err: <*fmt.wrapError | 0xc000820280>{
msg: "failed to execute cluster id query: failed to execute query: failed to execute query: Op(OpQuery): Kind(KInternal): Error while getting token : Get \"https://hcp-dev-us.westus3.kusto.windows.net/v1/rest/auth/metadata\": dial tcp: lookup hcp-dev-us.westus3.kusto.windows.net on 172.30.0.10:53: no such host",
err: <*fmt.wrapError | 0xc000820260>{
msg: "failed to execute query: failed to execute query: Op(OpQuery): Kind(KInternal): Error while getting token : Get \"https://hcp-dev-us.westus3.kusto.windows.net/v1/rest/auth/metadata\": dial tcp: lookup hcp-dev-us.westus3.kusto.windows.net on 172.30.0.10:53: no such host",
err: <*fmt.wrapError | 0xc000820240>{
msg: "failed to execute query: Op(OpQuery): Kind(KInternal): Error while getting token : Get \"https://hcp-dev-us.westus3.kusto.windows.net/v1/rest/auth/metadata\": dial tcp: lookup hcp-dev-us.westus3.kusto.windows.net on 172.30.0.10:53: no such host",
err: <*errors.Error | 0xc000668550>{
Op: 1,
Kind: 2,
Err: <*errors.errorString | 0xc0008e4150>{
s: "Error while getting token : Get \"https://hcp-dev-us.westus3.kusto.windows.net/v1/rest/auth/metadata\": dial tcp: lookup hcp-dev-us.westus3.kusto.windows.net on 172.30.0.10:53: no such host",
},
restErrMsg: nil,
decoded: nil,
permanent: false,
inner: nil,
},
},
},
},
}
fail [github.com/Azure/ARO-HCP/test/e2e/kusto_logs_present.go:94]: Timed out after 600.021s.
Expected success, but got an error:
<*fmt.wrapError | 0xc0008202a0>:
failed to gather logs: failed to execute cluster id query: failed to execute query: failed to execute query: Op(OpQuery): Kind(KInternal): Error while getting token : Get "https://hcp-dev-us.westus3.kusto.windows.net/v1/rest/auth/metadata": dial tcp: lookup hcp-dev-us.westus3.kusto.windows.net on 172.30.0.10:53: no such host
{
msg: "failed to gather logs: failed to execute cluster id query: failed to execute query: failed to execute query: Op(OpQuery): Kind(KInternal): Error while getting token : Get \"https://hcp-dev-us.westus3.kusto.windows.net/v1/rest/auth/metadata\": dial tcp: lookup hcp-dev-us.westus3.kusto.windows.net on 172.30.0.10:53: no such host",
err: <*fmt.wrapError | 0xc000820280>{
msg: "failed to execute cluster id query: failed to execute query: failed to execute query: Op(OpQuery): Kind(KInternal): Error while getting token : Get \"https://hcp-dev-us.westus3.kusto.windows.net/v1/rest/auth/metadata\": dial tcp: lookup hcp-dev-us.westus3.kusto.windows.net on 172.30.0.10:53: no such host",
err: <*fmt.wrapError | 0xc000820260>{
msg: "failed to execute query: failed to execute query: Op(OpQuery): Kind(KInternal): Error while getting token : Get \"https://hcp-dev-us.westus3.kusto.windows.net/v1/rest/auth/metadata\": dial tcp: lookup hcp-dev-us.westus3.kusto.windows.net on 172.30.0.10:53: no such host",
err: <*fmt.wrapError | 0xc000820240>{
msg: "failed to execute query: Op(OpQuery): Kind(KInternal): Error while getting token : Get \"https://hcp-dev-us.westus3.kusto.windows.net/v1/rest/auth/metadata\": dial tcp: lookup hcp-dev-us.westus3.kusto.windows.net on 172.30.0.10:53: no such host",
err: <*errors.Error | 0xc000668550>{
Op: 1,
Kind: 2,
Err: <*errors.errorString | 0xc0008e4150>{
s: "Error while getting token : Get \"https://hcp-dev-us.westus3.kusto.windows.net/v1/rest/auth/metadata\": dial tcp: lookup hcp-dev-us.westus3.kusto.windows.net on 172.30.0.10:53: no such host",
},
restErrMsg: nil,
decoded: nil,
permanent: false,
inner: nil,
},
},
},
},
}fail [github.com/Azure/ARO-HCP/test/e2e/kusto_logs_present.go:94]: Timed out after 600.003s.
Expected success, but got an error:
<*fmt.wrapError | 0xc00070e360>:
failed to gather logs: failed to execute cluster id query: failed to execute query: failed to execute query: Op(OpQuery): Kind(KInternal): Error while getting token : Get "https://hcp-dev-us.westus3.kusto.windows.net/v1/rest/auth/metadata": dial tcp: lookup hcp-dev-us.westus3.kusto.windows.net on 172.30.0.10:53: no such host
{
msg: "failed to gather logs: failed to execute cluster id query: failed to execute query: failed to execute query: Op(OpQuery): Kind(KInternal): Error while getting token : Get \"https://hcp-dev-us.westus3.kusto.windows.net/v1/rest/auth/metadata\": dial tcp: lookup hcp-dev-us.westus3.kusto.windows.net on 172.30.0.10:53: no such host",
err: <*fmt.wrapError | 0xc00070e320>{
msg: "failed to execute cluster id query: failed to execute query: failed to execute query: Op(OpQuery): Kind(KInternal): Error while getting token : Get \"https://hcp-dev-us.westus3.kusto.windows.net/v1/rest/auth/metadata\": dial tcp: lookup hcp-dev-us.westus3.kusto.windows.net on 172.30.0.10:53: no such host",
err: <*fmt.wrapError | 0xc00070e300>{
msg: "failed to execute query: failed to execute query: Op(OpQuery): Kind(KInternal): Error while getting token : Get \"https://hcp-dev-us.westus3.kusto.windows.net/v1/rest/auth/metadata\": dial tcp: lookup hcp-dev-us.westus3.kusto.windows.net on 172.30.0.10:53: no such host",
err: <*fmt.wrapError | 0xc00070e2e0>{
msg: "failed to execute query: Op(OpQuery): Kind(KInternal): Error while getting token : Get \"https://hcp-dev-us.westus3.kusto.windows.net/v1/rest/auth/metadata\": dial tcp: lookup hcp-dev-us.westus3.kusto.windows.net on 172.30.0.10:53: no such host",
err: <*errors.Error | 0xc000468a50>{
Op: 1,
Kind: 2,
Err: <*errors.errorString | 0xc0004e2550>{
s: "Error while getting token : Get \"https://hcp-dev-us.westus3.kusto.windows.net/v1/rest/auth/metadata\": dial tcp: lookup hcp-dev-us.westus3.kusto.windows.net on 172.30.0.10:53: no such host",
},
restErrMsg: nil,
decoded: nil,
permanent: false,
inner: nil,
},
},
},
},
}
fail [github.com/Azure/ARO-HCP/test/e2e/kusto_logs_present.go:94]: Timed out after 600.003s.
Expected success, but got an error:
<*fmt.wrapError | 0xc00070e360>:
failed to gather logs: failed to execute cluster id query: failed to execute query: failed to execute query: Op(OpQuery): Kind(KInternal): Error while getting token : Get "https://hcp-dev-us.westus3.kusto.windows.net/v1/rest/auth/metadata": dial tcp: lookup hcp-dev-us.westus3.kusto.windows.net on 172.30.0.10:53: no such host
{
msg: "failed to gather logs: failed to execute cluster id query: failed to execute query: failed to execute query: Op(OpQuery): Kind(KInternal): Error while getting token : Get \"https://hcp-dev-us.westus3.kusto.windows.net/v1/rest/auth/metadata\": dial tcp: lookup hcp-dev-us.westus3.kusto.windows.net on 172.30.0.10:53: no such host",
err: <*fmt.wrapError | 0xc00070e320>{
msg: "failed to execute cluster id query: failed to execute query: failed to execute query: Op(OpQuery): Kind(KInternal): Error while getting token : Get \"https://hcp-dev-us.westus3.kusto.windows.net/v1/rest/auth/metadata\": dial tcp: lookup hcp-dev-us.westus3.kusto.windows.net on 172.30.0.10:53: no such host",
err: <*fmt.wrapError | 0xc00070e300>{
msg: "failed to execute query: failed to execute query: Op(OpQuery): Kind(KInternal): Error while getting token : Get \"https://hcp-dev-us.westus3.kusto.windows.net/v1/rest/auth/metadata\": dial tcp: lookup hcp-dev-us.westus3.kusto.windows.net on 172.30.0.10:53: no such host",
err: <*fmt.wrapError | 0xc00070e2e0>{
msg: "failed to execute query: Op(OpQuery): Kind(KInternal): Error while getting token : Get \"https://hcp-dev-us.westus3.kusto.windows.net/v1/rest/auth/metadata\": dial tcp: lookup hcp-dev-us.westus3.kusto.windows.net on 172.30.0.10:53: no such host",
err: <*errors.Error | 0xc000468a50>{
Op: 1,
Kind: 2,
Err: <*errors.errorString | 0xc0004e2550>{
s: "Error while getting token : Get \"https://hcp-dev-us.westus3.kusto.windows.net/v1/rest/auth/metadata\": dial tcp: lookup hcp-dev-us.westus3.kusto.windows.net on 172.30.0.10:53: no such host",
},
restErrMsg: nil,
decoded: nil,
permanent: false,
inner: nil,
},
},
},
},
}Contributing tests (1)
Affected runs (6) | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
ERROR CODE: InternalServerError; context ; provider Microsoft.RedHatOpenShiftfull failure pattern: ERROR CODE: InternalServerError; context ; provider Microsoft.RedHatOpenShift Signal: IndeterminateSignal: Indeterminate — no prior history; active 1 day(s) — New failure pattern — no prior history | e2e | 5 | 1.10%5 of 454 job runs affected | IndeterminateSignal: Indeterminate — no prior history; active 1 day(s) — New failure pattern — no prior history | Show trend detailsFeb 16: 0 · Feb 17: 0 · Feb 18: 0 · Feb 19: 0 · Feb 20: 0 · Feb 21: 0 · Feb 22: 0 · Feb 23: 13 · Feb 24: 0 · Feb 25: 0 · Feb 26: 0 · Feb 27: 0 · Feb 28: 0 · Mar 1: 0 | none | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Full failure examples (3)fail [github.com/Azure/ARO-HCP/test/e2e/cluster_version_backlevel.go:156]: Unexpected error:
<*fmt.wrapError | 0xc0002f0320>:
failed to start credential request: POST https://rp.j5426048.hcpsvc.osadev.cloud/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-cluster-back-version-s94mmw/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cluster-ver-4-19/requestAdminCredential
--------------------------------------------------------------------------------
RESPONSE 500: 500 Internal Server Error
ERROR CODE: InternalServerError
--------------------------------------------------------------------------------
{
"error": {
"code": "InternalServerError",
"message": "Internal server error."
}
}
--------------------------------------------------------------------------------
{
msg: "failed to start credential request: POST https://rp.j5426048.hcpsvc.osadev.cloud/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-cluster-back-version-s94mmw/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cluster-ver-4-19/requestAdminCredential\n--------------------------------------------------------------------------------\nRESPONSE 500: 500 Internal Server Error\nERROR CODE: InternalServerError\n--------------------------------------------------------------------------------\n{\n \"error\": {\n \"code\": \"InternalServerError\",\n \"message\": \"Internal server error.\"\n }\n}\n--------------------------------------------------------------------------------\n",
err: <*exported.ResponseError | 0xc000908780>{
ErrorCode: "InternalServerError",
StatusCode: 500,
RawResponse: {
Status: "500 Internal Server Error",
StatusCode: 500,
Proto: "HTTP/2.0",
ProtoMajor: 2,
ProtoMinor: 0,
Header: {
"Content-Type": ["application/json"],
"X-Ms-Error-Code": [
"InternalServerError",
],
"X-Ms-Request-Id": [
"f1726be7-59f9-42d1-9146-2bd34299320f",
],
"Date": [
"Mon, 23 Feb 2026 23:42:37 GMT",
],
"Content-Length": ["107"],
"X-Envoy-Upstream-Service-Time": ["450"],
"Server": ["istio-envoy"],
},
Body: <*exported.nopClosingBytesReader | 0xc0002f0200>{
s: [123, 10, 32, 32, 32, 32, 34, 101, 114, 114, 111, 114, 34, 58, 32, 123, 10, 32, 32, 32, 32, 32, 32, 32, 32, 34, 99, 111, 100, 101, 34, 58, 32, 34, 73, 110, 116, 101, 114, 110, 97, 108, 83, 101, 114, 118, 101, 114, 69, 114, 114, 111, 114, 34, 44, 10, 32, 32, 32, 32, 32, 32, 32, 32, 34, 109, 101, 115, 115, 97, 103, 101, 34, 58, 32, 34, 73, 110, 116, 101, 114, 110, 97, 108, 32, 115, 101, 114, 118, 101, 114, 32, 101, 114, 114, 111, 114, 46, 34, 10, 32, 32, 32, 32, 125, 10, 125],
i: 0,
},
ContentLength: 107,
TransferEncoding: nil,
Close: false,
Uncompressed: false,
Trailer: nil,
Request: {
Method: "POST",
URL: {
Scheme: "https",
Opaque: "",
User: nil,
Host: "rp.j5426048.hcpsvc.osadev.cloud",
Path: "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-cluster-back-version-s94mmw/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cluster-ver-4-19/requestAdminCredential",
RawPath: "",
OmitHost: false,
ForceQuery: false,
RawQuery: "api-version=2024-06-10-preview",
Fragment: "",
RawFragment: "",
},
Proto: "HTTP/1.1",
ProtoMajor: 1,
ProtoMinor: 1,
Header: {
"X-Ms-Arm-Resource-System-Data": [
"{\"createdBy\": \"e2e-test\", \"createdByType\": \"Application\", \"createdAt\": \"2026-02-23T23:42:25Z\"}",
],
"X-Ms-Identity-Url": [
"https://dummyhost.identity.azure.net",
],
"Accept": ["application/json"],
"User-Agent": [
"azsdk-go-armredhatopenshifthcp/v0.1.0 (go1.25.7; linux)",
],
"Authori...
Gomega truncated this representation as it exceeds 'format.MaxLength'.
Consider having the object provide a custom 'GomegaStringer' representation
or adjust the parameters in Gomega's 'format' package.
Learn more here: https://onsi.github.io/gomega/#adjusting-output
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_version_backlevel.go:156]: Unexpected error:
<*fmt.wrapError | 0xc0002f0320>:
failed to start credential request: POST https://rp.j5426048.hcpsvc.osadev.cloud/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-cluster-back-version-s94mmw/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cluster-ver-4-19/requestAdminCredential
--------------------------------------------------------------------------------
RESPONSE 500: 500 Internal Server Error
ERROR CODE: InternalServerError
--------------------------------------------------------------------------------
{
"error": {
"code": "InternalServerError",
"message": "Internal server error."
}
}
--------------------------------------------------------------------------------
{
msg: "failed to start credential request: POST https://rp.j5426048.hcpsvc.osadev.cloud/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-cluster-back-version-s94mmw/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cluster-ver-4-19/requestAdminCredential\n--------------------------------------------------------------------------------\nRESPONSE 500: 500 Internal Server Error\nERROR CODE: InternalServerError\n--------------------------------------------------------------------------------\n{\n \"error\": {\n \"code\": \"InternalServerError\",\n \"message\": \"Internal server error.\"\n }\n}\n--------------------------------------------------------------------------------\n",
err: <*exported.ResponseError | 0xc000908780>{
ErrorCode: "InternalServerError",
StatusCode: 500,
RawResponse: {
Status: "500 Internal Server Error",
StatusCode: 500,
Proto: "HTTP/2.0",
ProtoMajor: 2,
ProtoMinor: 0,
Header: {
"Content-Type": ["application/json"],
"X-Ms-Error-Code": [
"InternalServerError",
],
"X-Ms-Request-Id": [
"f1726be7-59f9-42d1-9146-2bd34299320f",
],
"Date": [
"Mon, 23 Feb 2026 23:42:37 GMT",
],
"Content-Length": ["107"],
"X-Envoy-Upstream-Service-Time": ["450"],
"Server": ["istio-envoy"],
},
Body: <*exported.nopClosingBytesReader | 0xc0002f0200>{
s: [123, 10, 32, 32, 32, 32, 34, 101, 114, 114, 111, 114, 34, 58, 32, 123, 10, 32, 32, 32, 32, 32, 32, 32, 32, 34, 99, 111, 100, 101, 34, 58, 32, 34, 73, 110, 116, 101, 114, 110, 97, 108, 83, 101, 114, 118, 101, 114, 69, 114, 114, 111, 114, 34, 44, 10, 32, 32, 32, 32, 32, 32, 32, 32, 34, 109, 101, 115, 115, 97, 103, 101, 34, 58, 32, 34, 73, 110, 116, 101, 114, 110, 97, 108, 32, 115, 101, 114, 118, 101, 114, 32, 101, 114, 114, 111, 114, 46, 34, 10, 32, 32, 32, 32, 125, 10, 125],
i: 0,
},
ContentLength: 107,
TransferEncoding: nil,
Close: false,
Uncompressed: false,
Trailer: nil,
Request: {
Method: "POST",
URL: {
Scheme: "https",
Opaque: "",
User: nil,
Host: "rp.j5426048.hcpsvc.osadev.cloud",
Path: "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-cluster-back-version-s94mmw/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cluster-ver-4-19/requestAdminCredential",
RawPath: "",
OmitHost: false,
ForceQuery: false,
RawQuery: "api-version=2024-06-10-preview",
Fragment: "",
RawFragment: "",
},
Proto: "HTTP/1.1",
ProtoMajor: 1,
ProtoMinor: 1,
Header: {
"X-Ms-Arm-Resource-System-Data": [
"{\"createdBy\": \"e2e-test\", \"createdByType\": \"Application\", \"createdAt\": \"2026-02-23T23:42:25Z\"}",
],
"X-Ms-Identity-Url": [
"https://dummyhost.identity.azure.net",
],
"Accept": ["application/json"],
"User-Agent": [
"azsdk-go-armredhatopenshifthcp/v0.1.0 (go1.25.7; linux)",
],
"Authori...
Gomega truncated this representation as it exceeds 'format.MaxLength'.
Consider having the object provide a custom 'GomegaStringer' representation
or adjust the parameters in Gomega's 'format' package.
Learn more here: https://onsi.github.io/gomega/#adjusting-output
occurredfail [github.com/Azure/ARO-HCP/test/e2e/cluster_authorized_cidrs_connectivity.go:185]: Unexpected error:
<*fmt.wrapError | 0xc0008287a0>:
failed waiting for hcpCluster="cidr-connectivity-test" in resourcegroup="e2e-cidr-connectivity-ph7rvf" to finish getting creds: GET https://rp.j5426048.hcpsvc.osadev.cloud/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westus3/hcpOperationResults/a03227c3-c2cf-4efe-b8f2-9726e7f24621
--------------------------------------------------------------------------------
RESPONSE 500: 500 Internal Server Error
ERROR CODE: InternalServerError
--------------------------------------------------------------------------------
{
"error": {
"code": "InternalServerError",
"message": "Internal server error."
}
}
--------------------------------------------------------------------------------
{
msg: "failed waiting for hcpCluster=\"cidr-connectivity-test\" in resourcegroup=\"e2e-cidr-connectivity-ph7rvf\" to finish getting creds: GET https://rp.j5426048.hcpsvc.osadev.cloud/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westus3/hcpOperationResults/a03227c3-c2cf-4efe-b8f2-9726e7f24621\n--------------------------------------------------------------------------------\nRESPONSE 500: 500 Internal Server Error\nERROR CODE: InternalServerError\n--------------------------------------------------------------------------------\n{\n \"error\": {\n \"code\": \"InternalServerError\",\n \"message\": \"Internal server error.\"\n }\n}\n--------------------------------------------------------------------------------\n",
err: <*exported.ResponseError | 0xc000d88e40>{
ErrorCode: "InternalServerError",
StatusCode: 500,
RawResponse: {
Status: "500 Internal Server Error",
StatusCode: 500,
Proto: "HTTP/2.0",
ProtoMajor: 2,
ProtoMinor: 0,
Header: {
"X-Ms-Request-Id": [
"744d8a96-bb52-44f4-af79-55142559dffb",
],
"Date": [
"Mon, 23 Feb 2026 23:40:35 GMT",
],
"Content-Length": ["107"],
"X-Envoy-Upstream-Service-Time": ["3943"],
"Server": ["istio-envoy"],
"Content-Type": ["application/json"],
"X-Ms-Error-Code": [
"InternalServerError",
],
},
Body: <*exported.nopClosingBytesReader | 0xc0008286e0>{
s: [123, 10, 32, 32, 32, 32, 34, 101, 114, 114, 111, 114, 34, 58, 32, 123, 10, 32, 32, 32, 32, 32, 32, 32, 32, 34, 99, 111, 100, 101, 34, 58, 32, 34, 73, 110, 116, 101, 114, 110, 97, 108, 83, 101, 114, 118, 101, 114, 69, 114, 114, 111, 114, 34, 44, 10, 32, 32, 32, 32, 32, 32, 32, 32, 34, 109, 101, 115, 115, 97, 103, 101, 34, 58, 32, 34, 73, 110, 116, 101, 114, 110, 97, 108, 32, 115, 101, 114, 118, 101, 114, 32, 101, 114, 114, 111, 114, 46, 34, 10, 32, 32, 32, 32, 125, 10, 125],
i: 0,
},
ContentLength: 107,
TransferEncoding: nil,
Close: false,
Uncompressed: false,
Trailer: nil,
Request: {
Method: "GET",
URL: {
Scheme: "https",
Opaque: "",
User: nil,
Host: "rp.j5426048.hcpsvc.osadev.cloud",
Path: "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westus3/hcpOperationResults/a03227c3-c2cf-4efe-b8f2-9726e7f24621",
RawPath: "",
OmitHost: false,
ForceQuery: false,
RawQuery: "api-version=2024-06-10-preview",
Fragment: "",
RawFragment: "",
},
Proto: "HTTP/1.1",
ProtoMajor: 1,
ProtoMinor: 1,
Header: {
"User-Agent": [
"azsdk-go-armredhatopenshifthcp/v0.1.0 (go1.25.7; linux)",
],
"X-Ms-Arm-Resource-System-Data": [
"{\"createdBy\": \"e2e-test\", \"createdByType\": \"Application\", \"createdAt\": \"2026-02-23T23:40:05Z\"}",
],
"X-Ms-Identity-Url": [
"https://dummyhost.identity.azure.net",
],
"Authorization": ["redacted"],
...
Gomega truncated this representation as it exceeds 'format.MaxLength'.
Consider having the object provide a custom 'GomegaStringer' representation
or adjust the parameters in Gomega's 'format' package.
Learn more here: https://onsi.github.io/gomega/#adjusting-output
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_authorized_cidrs_connectivity.go:185]: Unexpected error:
<*fmt.wrapError | 0xc0008287a0>:
failed waiting for hcpCluster="cidr-connectivity-test" in resourcegroup="e2e-cidr-connectivity-ph7rvf" to finish getting creds: GET https://rp.j5426048.hcpsvc.osadev.cloud/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westus3/hcpOperationResults/a03227c3-c2cf-4efe-b8f2-9726e7f24621
--------------------------------------------------------------------------------
RESPONSE 500: 500 Internal Server Error
ERROR CODE: InternalServerError
--------------------------------------------------------------------------------
{
"error": {
"code": "InternalServerError",
"message": "Internal server error."
}
}
--------------------------------------------------------------------------------
{
msg: "failed waiting for hcpCluster=\"cidr-connectivity-test\" in resourcegroup=\"e2e-cidr-connectivity-ph7rvf\" to finish getting creds: GET https://rp.j5426048.hcpsvc.osadev.cloud/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westus3/hcpOperationResults/a03227c3-c2cf-4efe-b8f2-9726e7f24621\n--------------------------------------------------------------------------------\nRESPONSE 500: 500 Internal Server Error\nERROR CODE: InternalServerError\n--------------------------------------------------------------------------------\n{\n \"error\": {\n \"code\": \"InternalServerError\",\n \"message\": \"Internal server error.\"\n }\n}\n--------------------------------------------------------------------------------\n",
err: <*exported.ResponseError | 0xc000d88e40>{
ErrorCode: "InternalServerError",
StatusCode: 500,
RawResponse: {
Status: "500 Internal Server Error",
StatusCode: 500,
Proto: "HTTP/2.0",
ProtoMajor: 2,
ProtoMinor: 0,
Header: {
"X-Ms-Request-Id": [
"744d8a96-bb52-44f4-af79-55142559dffb",
],
"Date": [
"Mon, 23 Feb 2026 23:40:35 GMT",
],
"Content-Length": ["107"],
"X-Envoy-Upstream-Service-Time": ["3943"],
"Server": ["istio-envoy"],
"Content-Type": ["application/json"],
"X-Ms-Error-Code": [
"InternalServerError",
],
},
Body: <*exported.nopClosingBytesReader | 0xc0008286e0>{
s: [123, 10, 32, 32, 32, 32, 34, 101, 114, 114, 111, 114, 34, 58, 32, 123, 10, 32, 32, 32, 32, 32, 32, 32, 32, 34, 99, 111, 100, 101, 34, 58, 32, 34, 73, 110, 116, 101, 114, 110, 97, 108, 83, 101, 114, 118, 101, 114, 69, 114, 114, 111, 114, 34, 44, 10, 32, 32, 32, 32, 32, 32, 32, 32, 34, 109, 101, 115, 115, 97, 103, 101, 34, 58, 32, 34, 73, 110, 116, 101, 114, 110, 97, 108, 32, 115, 101, 114, 118, 101, 114, 32, 101, 114, 114, 111, 114, 46, 34, 10, 32, 32, 32, 32, 125, 10, 125],
i: 0,
},
ContentLength: 107,
TransferEncoding: nil,
Close: false,
Uncompressed: false,
Trailer: nil,
Request: {
Method: "GET",
URL: {
Scheme: "https",
Opaque: "",
User: nil,
Host: "rp.j5426048.hcpsvc.osadev.cloud",
Path: "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westus3/hcpOperationResults/a03227c3-c2cf-4efe-b8f2-9726e7f24621",
RawPath: "",
OmitHost: false,
ForceQuery: false,
RawQuery: "api-version=2024-06-10-preview",
Fragment: "",
RawFragment: "",
},
Proto: "HTTP/1.1",
ProtoMajor: 1,
ProtoMinor: 1,
Header: {
"User-Agent": [
"azsdk-go-armredhatopenshifthcp/v0.1.0 (go1.25.7; linux)",
],
"X-Ms-Arm-Resource-System-Data": [
"{\"createdBy\": \"e2e-test\", \"createdByType\": \"Application\", \"createdAt\": \"2026-02-23T23:40:05Z\"}",
],
"X-Ms-Identity-Url": [
"https://dummyhost.identity.azure.net",
],
"Authorization": ["redacted"],
...
Gomega truncated this representation as it exceeds 'format.MaxLength'.
Consider having the object provide a custom 'GomegaStringer' representation
or adjust the parameters in Gomega's 'format' package.
Learn more here: https://onsi.github.io/gomega/#adjusting-output
occurredfail [github.com/Azure/ARO-HCP/test/e2e/nodepool_update_nodes.go:92]: Unexpected error:
<*fmt.wrapError | 0xc000199000>:
failed waiting for hcpCluster="np-update-nodes-hcp-cluster" in resourcegroup="nodepool-update-nodes-stz27g" to finish getting creds: GET https://rp.j5426048.hcpsvc.osadev.cloud/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westus3/hcpOperationResults/c6979eba-31d6-4230-aec4-396b24b8277d
--------------------------------------------------------------------------------
RESPONSE 500: 500 Internal Server Error
ERROR CODE: InternalServerError
--------------------------------------------------------------------------------
{
"error": {
"code": "InternalServerError",
"message": "Internal server error."
}
}
--------------------------------------------------------------------------------
{
msg: "failed waiting for hcpCluster=\"np-update-nodes-hcp-cluster\" in resourcegroup=\"nodepool-update-nodes-stz27g\" to finish getting creds: GET https://rp.j5426048.hcpsvc.osadev.cloud/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westus3/hcpOperationResults/c6979eba-31d6-4230-aec4-396b24b8277d\n--------------------------------------------------------------------------------\nRESPONSE 500: 500 Internal Server Error\nERROR CODE: InternalServerError\n--------------------------------------------------------------------------------\n{\n \"error\": {\n \"code\": \"InternalServerError\",\n \"message\": \"Internal server error.\"\n }\n}\n--------------------------------------------------------------------------------\n",
err: <*exported.ResponseError | 0xc000bf64e0>{
ErrorCode: "InternalServerError",
StatusCode: 500,
RawResponse: {
Status: "500 Internal Server Error",
StatusCode: 500,
Proto: "HTTP/2.0",
ProtoMajor: 2,
ProtoMinor: 0,
Header: {
"Content-Type": ["application/json"],
"X-Ms-Error-Code": [
"InternalServerError",
],
"X-Ms-Request-Id": [
"eb1ebdf1-825a-48e9-bf00-f8e13c300546",
],
"Date": [
"Mon, 23 Feb 2026 23:40:38 GMT",
],
"Content-Length": ["107"],
"X-Envoy-Upstream-Service-Time": ["3713"],
"Server": ["istio-envoy"],
},
Body: <*exported.nopClosingBytesReader | 0xc000198c60>{
s: [123, 10, 32, 32, 32, 32, 34, 101, 114, 114, 111, 114, 34, 58, 32, 123, 10, 32, 32, 32, 32, 32, 32, 32, 32, 34, 99, 111, 100, 101, 34, 58, 32, 34, 73, 110, 116, 101, 114, 110, 97, 108, 83, 101, 114, 118, 101, 114, 69, 114, 114, 111, 114, 34, 44, 10, 32, 32, 32, 32, 32, 32, 32, 32, 34, 109, 101, 115, 115, 97, 103, 101, 34, 58, 32, 34, 73, 110, 116, 101, 114, 110, 97, 108, 32, 115, 101, 114, 118, 101, 114, 32, 101, 114, 114, 111, 114, 46, 34, 10, 32, 32, 32, 32, 125, 10, 125],
i: 0,
},
ContentLength: 107,
TransferEncoding: nil,
Close: false,
Uncompressed: false,
Trailer: nil,
Request: {
Method: "GET",
URL: {
Scheme: "https",
Opaque: "",
User: nil,
Host: "rp.j5426048.hcpsvc.osadev.cloud",
Path: "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westus3/hcpOperationResults/c6979eba-31d6-4230-aec4-396b24b8277d",
RawPath: "",
OmitHost: false,
ForceQuery: false,
RawQuery: "api-version=2024-06-10-preview",
Fragment: "",
RawFragment: "",
},
Proto: "HTTP/1.1",
ProtoMajor: 1,
ProtoMinor: 1,
Header: {
"User-Agent": [
"azsdk-go-armredhatopenshifthcp/v0.1.0 (go1.25.7; linux)",
],
"X-Ms-Arm-Resource-System-Data": [
"{\"createdBy\": \"e2e-test\", \"createdByType\": \"Application\", \"createdAt\": \"2026-02-23T23:40:09Z\"}",
],
"X-Ms-Identity-Url": [
"https://dummyhost.identity.azure.net",
],
"Authorization": ["redacted"...
Gomega truncated this representation as it exceeds 'format.MaxLength'.
Consider having the object provide a custom 'GomegaStringer' representation
or adjust the parameters in Gomega's 'format' package.
Learn more here: https://onsi.github.io/gomega/#adjusting-output
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_update_nodes.go:92]: Unexpected error:
<*fmt.wrapError | 0xc000199000>:
failed waiting for hcpCluster="np-update-nodes-hcp-cluster" in resourcegroup="nodepool-update-nodes-stz27g" to finish getting creds: GET https://rp.j5426048.hcpsvc.osadev.cloud/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westus3/hcpOperationResults/c6979eba-31d6-4230-aec4-396b24b8277d
--------------------------------------------------------------------------------
RESPONSE 500: 500 Internal Server Error
ERROR CODE: InternalServerError
--------------------------------------------------------------------------------
{
"error": {
"code": "InternalServerError",
"message": "Internal server error."
}
}
--------------------------------------------------------------------------------
{
msg: "failed waiting for hcpCluster=\"np-update-nodes-hcp-cluster\" in resourcegroup=\"nodepool-update-nodes-stz27g\" to finish getting creds: GET https://rp.j5426048.hcpsvc.osadev.cloud/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westus3/hcpOperationResults/c6979eba-31d6-4230-aec4-396b24b8277d\n--------------------------------------------------------------------------------\nRESPONSE 500: 500 Internal Server Error\nERROR CODE: InternalServerError\n--------------------------------------------------------------------------------\n{\n \"error\": {\n \"code\": \"InternalServerError\",\n \"message\": \"Internal server error.\"\n }\n}\n--------------------------------------------------------------------------------\n",
err: <*exported.ResponseError | 0xc000bf64e0>{
ErrorCode: "InternalServerError",
StatusCode: 500,
RawResponse: {
Status: "500 Internal Server Error",
StatusCode: 500,
Proto: "HTTP/2.0",
ProtoMajor: 2,
ProtoMinor: 0,
Header: {
"Content-Type": ["application/json"],
"X-Ms-Error-Code": [
"InternalServerError",
],
"X-Ms-Request-Id": [
"eb1ebdf1-825a-48e9-bf00-f8e13c300546",
],
"Date": [
"Mon, 23 Feb 2026 23:40:38 GMT",
],
"Content-Length": ["107"],
"X-Envoy-Upstream-Service-Time": ["3713"],
"Server": ["istio-envoy"],
},
Body: <*exported.nopClosingBytesReader | 0xc000198c60>{
s: [123, 10, 32, 32, 32, 32, 34, 101, 114, 114, 111, 114, 34, 58, 32, 123, 10, 32, 32, 32, 32, 32, 32, 32, 32, 34, 99, 111, 100, 101, 34, 58, 32, 34, 73, 110, 116, 101, 114, 110, 97, 108, 83, 101, 114, 118, 101, 114, 69, 114, 114, 111, 114, 34, 44, 10, 32, 32, 32, 32, 32, 32, 32, 32, 34, 109, 101, 115, 115, 97, 103, 101, 34, 58, 32, 34, 73, 110, 116, 101, 114, 110, 97, 108, 32, 115, 101, 114, 118, 101, 114, 32, 101, 114, 114, 111, 114, 46, 34, 10, 32, 32, 32, 32, 125, 10, 125],
i: 0,
},
ContentLength: 107,
TransferEncoding: nil,
Close: false,
Uncompressed: false,
Trailer: nil,
Request: {
Method: "GET",
URL: {
Scheme: "https",
Opaque: "",
User: nil,
Host: "rp.j5426048.hcpsvc.osadev.cloud",
Path: "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westus3/hcpOperationResults/c6979eba-31d6-4230-aec4-396b24b8277d",
RawPath: "",
OmitHost: false,
ForceQuery: false,
RawQuery: "api-version=2024-06-10-preview",
Fragment: "",
RawFragment: "",
},
Proto: "HTTP/1.1",
ProtoMajor: 1,
ProtoMinor: 1,
Header: {
"User-Agent": [
"azsdk-go-armredhatopenshifthcp/v0.1.0 (go1.25.7; linux)",
],
"X-Ms-Arm-Resource-System-Data": [
"{\"createdBy\": \"e2e-test\", \"createdByType\": \"Application\", \"createdAt\": \"2026-02-23T23:40:09Z\"}",
],
"X-Ms-Identity-Url": [
"https://dummyhost.identity.azure.net",
],
"Authorization": ["redacted"...
Gomega truncated this representation as it exceeds 'format.MaxLength'.
Consider having the object provide a custom 'GomegaStringer' representation
or adjust the parameters in Gomega's 'format' package.
Learn more here: https://onsi.github.io/gomega/#adjusting-output
occurredContributing tests (7)
Affected runs (5) | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
ERROR CODE: DeploymentFailed; detail code ResourceNotFound; detail message The Resource 'Microsoft.Kusto/clusters/<resource>' under resource group '<resource-group>' was not found...full failure pattern: ERROR CODE: DeploymentFailed; detail code ResourceNotFound; detail message The Resource 'Microsoft.Kusto/clusters/<resource>' under resource group '<resource-group>' was not found. For more details please go to <url>; provider Microsoft.Kusto Signal: IndeterminateSignal: Indeterminate — no prior history; active 1 day(s) — New failure pattern — no prior history | provision | 5 | 1.10%5 of 454 job runs affected | IndeterminateSignal: Indeterminate — no prior history; active 1 day(s) — New failure pattern — no prior history | Show trend detailsFeb 16: 0 · Feb 17: 0 · Feb 18: 0 · Feb 19: 0 · Feb 20: 0 · Feb 21: 0 · Feb 22: 0 · Feb 23: 0 · Feb 24: 5 · Feb 25: 0 · Feb 26: 0 · Feb 27: 0 · Feb 28: 0 · Mar 1: 0 | none | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Full failure examples (3)time=2026-02-24T12:07:50.250Z level=INFO msg="Running step." serviceGroup=Microsoft.Azure.ARO.HCP.Service.Infra resourceGroup=kusto step=output description="Step output\n Kind: ARM\n Template: templates/kusto-lookup.bicep\n Parameters: configurations/kusto-lookup.tmpl.bicepparam"
time=2026-02-24T12:07:51.175Z level=DEBUG+2 msg="Waiting for existing deployment to complete" serviceGroup=Microsoft.Azure.ARO.HCP.Service.Infra resourceGroup=kusto step=output deployment=ef2bb3355bd54675e90926b2ffd43526b0e1367038533fb09dcdc29269b1ddf2
time=2026-02-24T12:07:51.175Z level=ERROR msg="Step errored." serviceGroup=Microsoft.Azure.ARO.HCP.Service.Infra resourceGroup=kusto step=output err="failed to run ARM step: failed to poll previously-existing deployment: pre-existing deployment failed: {\"code\":\"DeploymentFailed\",\"details\":[{\"code\":\"ResourceNotFound\",\"message\":\"The Resource 'Microsoft.Kusto/clusters/hcp-dev-us' under resource group 'hcp-kusto-us' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix\"}],\"message\":\"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\"}"time=2026-02-24T12:11:25.801Z level=INFO msg="Running step." serviceGroup=Microsoft.Azure.ARO.HCP.Service.Infra resourceGroup=kusto step=output description="Step output\n Kind: ARM\n Template: templates/kusto-lookup.bicep\n Parameters: configurations/kusto-lookup.tmpl.bicepparam"
time=2026-02-24T12:11:26.577Z level=DEBUG+2 msg="Waiting for existing deployment to complete" serviceGroup=Microsoft.Azure.ARO.HCP.Service.Infra resourceGroup=kusto step=output deployment=ef2bb3355bd54675e90926b2ffd43526b0e1367038533fb09dcdc29269b1ddf2
time=2026-02-24T12:11:26.577Z level=ERROR msg="Step errored." serviceGroup=Microsoft.Azure.ARO.HCP.Service.Infra resourceGroup=kusto step=output err="failed to run ARM step: failed to poll previously-existing deployment: pre-existing deployment failed: {\"code\":\"DeploymentFailed\",\"details\":[{\"code\":\"ResourceNotFound\",\"message\":\"The Resource 'Microsoft.Kusto/clusters/hcp-dev-us' under resource group 'hcp-kusto-us' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix\"}],\"message\":\"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\"}"time=2026-02-24T12:03:16.907Z level=INFO msg="Running step." serviceGroup=Microsoft.Azure.ARO.HCP.Service.Infra resourceGroup=kusto step=output description="Step output\n Kind: ARM\n Template: templates/kusto-lookup.bicep\n Parameters: configurations/kusto-lookup.tmpl.bicepparam"
time=2026-02-24T12:03:18.043Z level=DEBUG+2 msg="Waiting for existing deployment to complete" serviceGroup=Microsoft.Azure.ARO.HCP.Service.Infra resourceGroup=kusto step=output deployment=ef2bb3355bd54675e90926b2ffd43526b0e1367038533fb09dcdc29269b1ddf2
time=2026-02-24T12:03:18.043Z level=ERROR msg="Step errored." serviceGroup=Microsoft.Azure.ARO.HCP.Service.Infra resourceGroup=kusto step=output err="failed to run ARM step: failed to poll previously-existing deployment: pre-existing deployment failed: {\"code\":\"DeploymentFailed\",\"details\":[{\"code\":\"ResourceNotFound\",\"message\":\"The Resource 'Microsoft.Kusto/clusters/hcp-dev-us' under resource group 'hcp-kusto-us' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix\"}],\"message\":\"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\"}"Contributing tests (2)
Affected runs (5) | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Environment: INT
E2E Jobs
34
Success Rate
58.82%
E2E success
Among runs that reached E2E execution, this shows how many completed successfully. Provision failures and 'Other' CI-infrastructure failures are excluded because those runs never reached the E2E step.58.82% (20/34)
Daily Run Outcomes
Successful runs
Provision failures
E2E failures
Other failures
| Pass rate | Runs | Date (UTC) | Suite | Test |
|---|---|---|---|---|
| 69.70% | 33 | integration/parallel | Customer should be able to create an HCP cluster | |
| 69.70% | 33 | integration/parallel | Customer should be able to create an HCP cluster with custom autoscaling | |
| 72.73% | 33 | integration/parallel | Customer should be able to create an HCP cluster and custom node pool osDisk size | |
| 72.73% | 33 | integration/parallel | Customer should be able to create an HCP cluster with back-level version 4.19 | |
| 72.73% | 33 | integration/parallel | Customer should be able to create node pools with ARM64-based VMs |
Up to 50 failure patterns are loaded (minimum 1.00% impact), with 10 shown by default. Click column headers to re-sort.
Jump to the full Failure Patterns view for this window
| Failure patternThe canonical description of a recurring CI failure, extracted and normalized from raw logs. | Failed atThe stage of the job run where this failure occurred: 'provision' (environment setup, DEV only), 'e2e' (test suite execution), or 'other' (CI infrastructure issues that did not produce a failure pattern). | Number of distinct job runs where this failure pattern was detected. | Percentage of all job runs in this environment affected by this failure pattern during the selected window. | Classification of this failure pattern: Regression (likely caused by a specific PR), Flake (intermittent failure spread across days), Noise (low-quality or generic pattern), or Indeterminate. | TrendShows daily activity for this failure pattern in a trailing window anchored to the selected end date. The sparkline covers at least 7 days and at most 14 days, depending on the current window size. | Also inOther environments where the same failure pattern was also detected during the selected window. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Interrupted by Userfull failure pattern: Interrupted by User Signal: IndeterminateSignal: Indeterminate — no prior history; active 3 day(s) — New failure pattern — no prior history | e2e | 8 | 23.53%8 of 34 job runs affected | IndeterminateSignal: Indeterminate — no prior history; active 3 day(s) — New failure pattern — no prior history | Show trend detailsFeb 16: 0 · Feb 17: 0 · Feb 18: 0 · Feb 19: 0 · Feb 20: 0 · Feb 21: 0 · Feb 22: 0 · Feb 23: 0 · Feb 24: 0 · Feb 25: 1 · Feb 26: 0 · Feb 27: 0 · Feb 28: 21 · Mar 1: 7 | DEV, PROD | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Full failure examples (3)fail [github.com/Azure/ARO-HCP/test/e2e/cluster_update.go:120]: Interrupted by User fail [github.com/Azure/ARO-HCP/test/e2e/cluster_update.go:120]: Interrupted by User fail [github.com/Azure/ARO-HCP/test/e2e/external_auth_create.go:47]: Interrupted by User fail [github.com/Azure/ARO-HCP/test/e2e/external_auth_create.go:47]: Interrupted by User fail [github.com/Azure/ARO-HCP/test/e2e/clusters_sharing_resgroup.go:35]: Interrupted by User fail [github.com/Azure/ARO-HCP/test/e2e/clusters_sharing_resgroup.go:35]: Interrupted by User Contributing tests (17)
Affected runs (8)
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
timeout during CreateHCPClusterAndWait; context deadline exceededfull failure pattern: timeout during CreateHCPClusterAndWait; context deadline exceeded Signal: IndeterminateSignal: Indeterminate — no prior history; active 2 day(s) — New failure pattern — no prior history | e2e | 7 | 20.59%7 of 34 job runs affected | IndeterminateSignal: Indeterminate — no prior history; active 2 day(s) — New failure pattern — no prior history | Show trend detailsFeb 16: 0 · Feb 17: 0 · Feb 18: 0 · Feb 19: 0 · Feb 20: 0 · Feb 21: 0 · Feb 22: 0 · Feb 23: 0 · Feb 24: 0 · Feb 25: 0 · Feb 26: 0 · Feb 27: 0 · Feb 28: 70 · Mar 1: 29 | DEV | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Full failure examples (3)fail [github.com/Azure/ARO-HCP/test/e2e/cluster_tls_endpoints.go:83]: Unexpected error:
<*fmt.wrapErrors | 0xc0010223c0>:
failed to create HCP cluster tls-endpoint-hcp-cluster, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster tls-endpoint-hcp-cluster in resource group tls-endpoint-cluster-4wthdg, error: failed waiting for cluster="tls-endpoint-hcp-cluster" in resourcegroup="tls-endpoint-cluster-4wthdg" to finish creating, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster tls-endpoint-hcp-cluster in resource group tls-endpoint-cluster-4wthdg, error: context deadline exceeded
{
msg: "failed to create HCP cluster tls-endpoint-hcp-cluster, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster tls-endpoint-hcp-cluster in resource group tls-endpoint-cluster-4wthdg, error: failed waiting for cluster=\"tls-endpoint-hcp-cluster\" in resourcegroup=\"tls-endpoint-cluster-4wthdg\" to finish creating, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster tls-endpoint-hcp-cluster in resource group tls-endpoint-cluster-4wthdg, error: context deadline exceeded",
errs: [
<*errors.errorString | 0xc000d35d00>{
s: "timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster tls-endpoint-hcp-cluster in resource group tls-endpoint-cluster-4wthdg",
},
<*fmt.wrapErrors | 0xc001022390>{
msg: "failed waiting for cluster=\"tls-endpoint-hcp-cluster\" in resourcegroup=\"tls-endpoint-cluster-4wthdg\" to finish creating, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster tls-endpoint-hcp-cluster in resource group tls-endpoint-cluster-4wthdg, error: context deadline exceeded",
errs: [
<*errors.errorString | 0xc000d35d00>{
s: "timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster tls-endpoint-hcp-cluster in resource group tls-endpoint-cluster-4wthdg",
},
<context.deadlineExceededError>{},
],
},
],
}
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_tls_endpoints.go:83]: Unexpected error:
<*fmt.wrapErrors | 0xc0010223c0>:
failed to create HCP cluster tls-endpoint-hcp-cluster, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster tls-endpoint-hcp-cluster in resource group tls-endpoint-cluster-4wthdg, error: failed waiting for cluster="tls-endpoint-hcp-cluster" in resourcegroup="tls-endpoint-cluster-4wthdg" to finish creating, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster tls-endpoint-hcp-cluster in resource group tls-endpoint-cluster-4wthdg, error: context deadline exceeded
{
msg: "failed to create HCP cluster tls-endpoint-hcp-cluster, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster tls-endpoint-hcp-cluster in resource group tls-endpoint-cluster-4wthdg, error: failed waiting for cluster=\"tls-endpoint-hcp-cluster\" in resourcegroup=\"tls-endpoint-cluster-4wthdg\" to finish creating, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster tls-endpoint-hcp-cluster in resource group tls-endpoint-cluster-4wthdg, error: context deadline exceeded",
errs: [
<*errors.errorString | 0xc000d35d00>{
s: "timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster tls-endpoint-hcp-cluster in resource group tls-endpoint-cluster-4wthdg",
},
<*fmt.wrapErrors | 0xc001022390>{
msg: "failed waiting for cluster=\"tls-endpoint-hcp-cluster\" in resourcegroup=\"tls-endpoint-cluster-4wthdg\" to finish creating, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster tls-endpoint-hcp-cluster in resource group tls-endpoint-cluster-4wthdg, error: context deadline exceeded",
errs: [
<*errors.errorString | 0xc000d35d00>{
s: "timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster tls-endpoint-hcp-cluster in resource group tls-endpoint-cluster-4wthdg",
},
<context.deadlineExceededError>{},
],
},
],
}
occurredfail [github.com/Azure/ARO-HCP/test/e2e/external_auth_list_and_verify.go:84]: Unexpected error:
<*fmt.wrapErrors | 0xc000877d70>:
failed to create HCP cluster ea-listppngfh, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster ea-listppngfh in resource group ea-list-hmpn7q, error: failed waiting for cluster="ea-listppngfh" in resourcegroup="ea-list-hmpn7q" to finish creating, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster ea-listppngfh in resource group ea-list-hmpn7q, error: context deadline exceeded
{
msg: "failed to create HCP cluster ea-listppngfh, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster ea-listppngfh in resource group ea-list-hmpn7q, error: failed waiting for cluster=\"ea-listppngfh\" in resourcegroup=\"ea-list-hmpn7q\" to finish creating, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster ea-listppngfh in resource group ea-list-hmpn7q, error: context deadline exceeded",
errs: [
<*errors.errorString | 0xc0000914d0>{
s: "timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster ea-listppngfh in resource group ea-list-hmpn7q",
},
<*fmt.wrapErrors | 0xc000877d40>{
msg: "failed waiting for cluster=\"ea-listppngfh\" in resourcegroup=\"ea-list-hmpn7q\" to finish creating, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster ea-listppngfh in resource group ea-list-hmpn7q, error: context deadline exceeded",
errs: [
<*errors.errorString | 0xc0000914d0>{
s: "timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster ea-listppngfh in resource group ea-list-hmpn7q",
},
<context.deadlineExceededError>{},
],
},
],
}
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/external_auth_list_and_verify.go:84]: Unexpected error:
<*fmt.wrapErrors | 0xc000877d70>:
failed to create HCP cluster ea-listppngfh, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster ea-listppngfh in resource group ea-list-hmpn7q, error: failed waiting for cluster="ea-listppngfh" in resourcegroup="ea-list-hmpn7q" to finish creating, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster ea-listppngfh in resource group ea-list-hmpn7q, error: context deadline exceeded
{
msg: "failed to create HCP cluster ea-listppngfh, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster ea-listppngfh in resource group ea-list-hmpn7q, error: failed waiting for cluster=\"ea-listppngfh\" in resourcegroup=\"ea-list-hmpn7q\" to finish creating, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster ea-listppngfh in resource group ea-list-hmpn7q, error: context deadline exceeded",
errs: [
<*errors.errorString | 0xc0000914d0>{
s: "timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster ea-listppngfh in resource group ea-list-hmpn7q",
},
<*fmt.wrapErrors | 0xc000877d40>{
msg: "failed waiting for cluster=\"ea-listppngfh\" in resourcegroup=\"ea-list-hmpn7q\" to finish creating, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster ea-listppngfh in resource group ea-list-hmpn7q, error: context deadline exceeded",
errs: [
<*errors.errorString | 0xc0000914d0>{
s: "timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster ea-listppngfh in resource group ea-list-hmpn7q",
},
<context.deadlineExceededError>{},
],
},
],
}
occurredfail [github.com/Azure/ARO-HCP/test/e2e/arm64_nodepool.go:90]: Unexpected error:
<*fmt.wrapErrors | 0xc00120a960>:
failed to create HCP cluster arm64-vm-hcp-cluster, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster arm64-vm-hcp-cluster in resource group arm64-vm-cluster-5hmftr, error: failed waiting for cluster="arm64-vm-hcp-cluster" in resourcegroup="arm64-vm-cluster-5hmftr" to finish creating, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster arm64-vm-hcp-cluster in resource group arm64-vm-cluster-5hmftr, error: context deadline exceeded
{
msg: "failed to create HCP cluster arm64-vm-hcp-cluster, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster arm64-vm-hcp-cluster in resource group arm64-vm-cluster-5hmftr, error: failed waiting for cluster=\"arm64-vm-hcp-cluster\" in resourcegroup=\"arm64-vm-cluster-5hmftr\" to finish creating, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster arm64-vm-hcp-cluster in resource group arm64-vm-cluster-5hmftr, error: context deadline exceeded",
errs: [
<*errors.errorString | 0xc0011fc6c0>{
s: "timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster arm64-vm-hcp-cluster in resource group arm64-vm-cluster-5hmftr",
},
<*fmt.wrapErrors | 0xc00120a930>{
msg: "failed waiting for cluster=\"arm64-vm-hcp-cluster\" in resourcegroup=\"arm64-vm-cluster-5hmftr\" to finish creating, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster arm64-vm-hcp-cluster in resource group arm64-vm-cluster-5hmftr, error: context deadline exceeded",
errs: [
<*errors.errorString | 0xc0011fc6c0>{
s: "timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster arm64-vm-hcp-cluster in resource group arm64-vm-cluster-5hmftr",
},
<context.deadlineExceededError>{},
],
},
],
}
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/arm64_nodepool.go:90]: Unexpected error:
<*fmt.wrapErrors | 0xc00120a960>:
failed to create HCP cluster arm64-vm-hcp-cluster, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster arm64-vm-hcp-cluster in resource group arm64-vm-cluster-5hmftr, error: failed waiting for cluster="arm64-vm-hcp-cluster" in resourcegroup="arm64-vm-cluster-5hmftr" to finish creating, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster arm64-vm-hcp-cluster in resource group arm64-vm-cluster-5hmftr, error: context deadline exceeded
{
msg: "failed to create HCP cluster arm64-vm-hcp-cluster, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster arm64-vm-hcp-cluster in resource group arm64-vm-cluster-5hmftr, error: failed waiting for cluster=\"arm64-vm-hcp-cluster\" in resourcegroup=\"arm64-vm-cluster-5hmftr\" to finish creating, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster arm64-vm-hcp-cluster in resource group arm64-vm-cluster-5hmftr, error: context deadline exceeded",
errs: [
<*errors.errorString | 0xc0011fc6c0>{
s: "timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster arm64-vm-hcp-cluster in resource group arm64-vm-cluster-5hmftr",
},
<*fmt.wrapErrors | 0xc00120a930>{
msg: "failed waiting for cluster=\"arm64-vm-hcp-cluster\" in resourcegroup=\"arm64-vm-cluster-5hmftr\" to finish creating, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster arm64-vm-hcp-cluster in resource group arm64-vm-cluster-5hmftr, error: context deadline exceeded",
errs: [
<*errors.errorString | 0xc0011fc6c0>{
s: "timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster arm64-vm-hcp-cluster in resource group arm64-vm-cluster-5hmftr",
},
<context.deadlineExceededError>{},
],
},
],
}
occurredContributing tests (17)
Affected runs (7)
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
failed waiting for deployment "aro-hcp-demo" in resourcegroup="<resource-group>" to finish: context deadline exceededfull failure pattern: failed waiting for deployment "aro-hcp-demo" in resourcegroup="<resource-group>" to finish: context deadline exceeded Signal: IndeterminateSignal: Indeterminate — no prior history; active 2 day(s) — New failure pattern — no prior history | e2e | 7 | 20.59%7 of 34 job runs affected | IndeterminateSignal: Indeterminate — no prior history; active 2 day(s) — New failure pattern — no prior history | Show trend detailsFeb 16: 0 · Feb 17: 0 · Feb 18: 0 · Feb 19: 0 · Feb 20: 0 · Feb 21: 0 · Feb 22: 0 · Feb 23: 0 · Feb 24: 0 · Feb 25: 0 · Feb 26: 0 · Feb 27: 0 · Feb 28: 5 · Mar 1: 2 | none | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Full failure examples (3)fail [github.com/Azure/ARO-HCP/test/e2e/gpu_nodepools_create_delete.go:96]: Unexpected error:
<*fmt.wrapError | 0xc001468380>:
failed waiting for deployment "aro-hcp-demo" in resourcegroup="gpu-nodepools-NC4asT4v3-fkx485" to finish: context deadline exceeded
{
msg: "failed waiting for deployment \"aro-hcp-demo\" in resourcegroup=\"gpu-nodepools-NC4asT4v3-fkx485\" to finish: context deadline exceeded",
err: <context.deadlineExceededError>{},
}
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/gpu_nodepools_create_delete.go:96]: Unexpected error:
<*fmt.wrapError | 0xc001468380>:
failed waiting for deployment "aro-hcp-demo" in resourcegroup="gpu-nodepools-NC4asT4v3-fkx485" to finish: context deadline exceeded
{
msg: "failed waiting for deployment \"aro-hcp-demo\" in resourcegroup=\"gpu-nodepools-NC4asT4v3-fkx485\" to finish: context deadline exceeded",
err: <context.deadlineExceededError>{},
}
occurredfail [github.com/Azure/ARO-HCP/test/e2e/gpu_nodepools_create_delete.go:96]: Unexpected error:
<*fmt.wrapError | 0xc000e9a220>:
failed waiting for deployment "aro-hcp-demo" in resourcegroup="gpu-nodepools-NC4asT4v3-kkpjc6" to finish: context deadline exceeded
{
msg: "failed waiting for deployment \"aro-hcp-demo\" in resourcegroup=\"gpu-nodepools-NC4asT4v3-kkpjc6\" to finish: context deadline exceeded",
err: <context.deadlineExceededError>{},
}
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/gpu_nodepools_create_delete.go:96]: Unexpected error:
<*fmt.wrapError | 0xc000e9a220>:
failed waiting for deployment "aro-hcp-demo" in resourcegroup="gpu-nodepools-NC4asT4v3-kkpjc6" to finish: context deadline exceeded
{
msg: "failed waiting for deployment \"aro-hcp-demo\" in resourcegroup=\"gpu-nodepools-NC4asT4v3-kkpjc6\" to finish: context deadline exceeded",
err: <context.deadlineExceededError>{},
}
occurredfail [github.com/Azure/ARO-HCP/test/e2e/gpu_nodepools_create_delete.go:96]: Unexpected error:
<*fmt.wrapError | 0xc0014122e0>:
failed waiting for deployment "aro-hcp-demo" in resourcegroup="gpu-nodepools-NC4asT4v3-dhsjcz" to finish: context deadline exceeded
{
msg: "failed waiting for deployment \"aro-hcp-demo\" in resourcegroup=\"gpu-nodepools-NC4asT4v3-dhsjcz\" to finish: context deadline exceeded",
err: <context.deadlineExceededError>{},
}
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/gpu_nodepools_create_delete.go:96]: Unexpected error:
<*fmt.wrapError | 0xc0014122e0>:
failed waiting for deployment "aro-hcp-demo" in resourcegroup="gpu-nodepools-NC4asT4v3-dhsjcz" to finish: context deadline exceeded
{
msg: "failed waiting for deployment \"aro-hcp-demo\" in resourcegroup=\"gpu-nodepools-NC4asT4v3-dhsjcz\" to finish: context deadline exceeded",
err: <context.deadlineExceededError>{},
}
occurredContributing tests (1)
Affected runs (7)
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Cluster should become ready within 45 minutesfull failure pattern: Cluster should become ready within 45 minutes Signal: IndeterminateSignal: Indeterminate — no prior history; active 2 day(s) — New failure pattern — no prior history | e2e | 6 | 17.65%6 of 34 job runs affected | IndeterminateSignal: Indeterminate — no prior history; active 2 day(s) — New failure pattern — no prior history | Show trend detailsFeb 16: 0 · Feb 17: 0 · Feb 18: 0 · Feb 19: 0 · Feb 20: 0 · Feb 21: 0 · Feb 22: 0 · Feb 23: 0 · Feb 24: 0 · Feb 25: 0 · Feb 26: 0 · Feb 27: 0 · Feb 28: 4 · Mar 1: 2 | DEV | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Full failure examples (3)fail [github.com/Azure/ARO-HCP/test/e2e/admin_credential_lifecycle.go:158]: Timed out after 2700.001s.
Cluster should become ready within 45 minutes
Expected
<bool>: false
to be true
fail [github.com/Azure/ARO-HCP/test/e2e/admin_credential_lifecycle.go:158]: Timed out after 2700.001s.
Cluster should become ready within 45 minutes
Expected
<bool>: false
to be truefail [github.com/Azure/ARO-HCP/test/e2e/admin_credential_lifecycle.go:158]: Timed out after 2700.000s.
Cluster should become ready within 45 minutes
Expected
<bool>: false
to be true
fail [github.com/Azure/ARO-HCP/test/e2e/admin_credential_lifecycle.go:158]: Timed out after 2700.000s.
Cluster should become ready within 45 minutes
Expected
<bool>: false
to be truefail [github.com/Azure/ARO-HCP/test/e2e/admin_credential_lifecycle.go:158]: Timed out after 2700.344s.
Cluster should become ready within 45 minutes
Expected
<bool>: false
to be true
fail [github.com/Azure/ARO-HCP/test/e2e/admin_credential_lifecycle.go:158]: Timed out after 2700.344s.
Cluster should become ready within 45 minutes
Expected
<bool>: false
to be trueContributing tests (1)
Affected runs (6)
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
failed to create HCP cluster <cluster>: operation result model did not match expected model for type *armredhatopenshifthcp.HcpOpenShiftCluster:full failure pattern: failed to create HCP cluster <cluster>: operation result model did not match expected model for type *armredhatopenshifthcp.HcpOpenShiftCluster: Signal: IndeterminateSignal: Indeterminate — no prior history; active 1 day(s) — New failure pattern — no prior history | e2e | 2 | 5.88%2 of 34 job runs affected | IndeterminateSignal: Indeterminate — no prior history; active 1 day(s) — New failure pattern — no prior history | Show trend detailsFeb 16: 0 · Feb 17: 0 · Feb 18: 0 · Feb 19: 0 · Feb 20: 0 · Feb 21: 0 · Feb 22: 0 · Feb 23: 0 · Feb 24: 0 · Feb 25: 0 · Feb 26: 0 · Feb 27: 2 · Feb 28: 0 · Mar 1: 0 | none | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Full failure examples (2)fail [github.com/Azure/ARO-HCP/test/e2e/cluster_authorized_cidrs_connectivity.go:133]: Unexpected error:
<*fmt.wrapError | 0xc00031e180>:
failed to create HCP cluster cidr-connectivity-test: operation result model did not match expected model for type *armredhatopenshifthcp.HcpOpenShiftCluster:
&armredhatopenshifthcp.HcpOpenShiftCluster{
Location: &"uksouth",
Identity: &{Type: &"UserAssigned", UserAssignedIdentities: {"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-32/providers/Microsoft.ManagedIdentity/userAssignedIdentities/cloud-controller-manager": &{ClientID: &"f2e4769e-d3c2-498d-92b9-3e6d24cd2d7a", PrincipalID: &"a5c120f7-50c3-4cf6-84b4-57517f60630d"}, "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-32/providers/Microsoft.ManagedIdentity/userAssignedIdentities/cloud-network-config": &{ClientID: &"f2e4769e-d3c2-498d-92b9-3e6d24cd2d7a", PrincipalID: &"a5c120f7-50c3-4cf6-84b4-57517f60630d"}, "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-32/providers/Microsoft.ManagedIdentity/userAssignedIdentities/cluster-api-azure": &{ClientID: &"f2e4769e-d3c2-498d-92b9-3e6d24cd2d7a", PrincipalID: &"a5c120f7-50c3-4cf6-84b4-57517f60630d"}, "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-32/providers/Microsoft.ManagedIdentity/userAssignedIdentities/control-plane": &{ClientID: &"f2e4769e-d3c2-498d-92b9-3e6d24cd2d7a", PrincipalID: &"a5c120f7-50c3-4cf6-84b4-57517f60630d"}, ...}},
Properties: &armredhatopenshifthcp.HcpOpenShiftClusterProperties{
... // 8 identical fields
Version: &{ChannelGroup: &"stable", ID: &"4.20"},
Console: nil,
- ProvisioningState: &"Succeeded",
+ ProvisioningState: &"Provisioning",
},
Tags: {"aro-hcp.experimental.cluster.size-override": &"Minimal"},
ID: &"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGrou"...,
... // 1 ignored and 2 identical fields
}
{
msg: "failed to create HCP cluster cidr-connectivity-test: operation result model did not match expected model for type *armredhatopenshifthcp.HcpOpenShiftCluster:\n &armredhatopenshifthcp.HcpOpenShiftCluster{\n \tLocation: &\"uksouth\",\n \tIdentity: &{Type: &\"UserAssigned\", UserAssignedIdentities: {\"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-32/providers/Microsoft.ManagedIdentity/userAssignedIdentities/cloud-controller-manager\": &{ClientID: &\"f2e4769e-d3c2-498d-92b9-3e6d24cd2d7a\", PrincipalID: &\"a5c120f7-50c3-4cf6-84b4-57517f60630d\"}, \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-32/providers/Microsoft.ManagedIdentity/userAssignedIdentities/cloud-network-config\": &{ClientID: &\"f2e4769e-d3c2-498d-92b9-3e6d24cd2d7a\", PrincipalID: &\"a5c120f7-50c3-4cf6-84b4-57517f60630d\"}, \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-32/providers/Microsoft.ManagedIdentity/userAssignedIdentities/cluster-api-azure\": &{ClientID: &\"f2e4769e-d3c2-498d-92b9-3e6d24cd2d7a\", PrincipalID: &\"a5c120f7-50c3-4cf6-84b4-57517f60630d\"}, \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-32/providers/Microsoft.ManagedIdentity/userAssignedIdentities/control-plane\": &{ClientID: &\"f2e4769e-d3c2-498d-92b9-3e6d24cd2d7a\", PrincipalID: &\"a5c120f7-50c3-4cf6-84b4-57517f60630d\"}, ...}},\n \tProperties: &armredhatopenshifthcp.HcpOpenShiftClusterProperties{\n \t\t... // 8 identical fields\n \t\tVersion: &{ChannelGroup: &\"stable\", ID: &\"4.20\"},\n \t\tConsole: nil,\n- \t\tProvisioningState: &\"Succeeded\",\n+ \t\tProvisioningState: &\"Provisioning\",\n \t},\n \tTags: {\"aro-hcp.experimental.cluster.size-override\": &\"Minimal\"},\n \tID: &\"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGrou\"...,\n \t... // 1 ignored and 2 identical fields\n }\n",
err: <*errors.errorString | 0xc0008dc6f0>{
s: "operation result model did not match expected model for type *armredhatopenshifthcp.HcpOpenShiftCluster:\n &armredhatopenshifthcp.HcpOpenShiftCluster{\n \tLocation: &\"uksouth\",\n \tIdentity: &{Type: &\"UserAssigned\", UserAssignedIdentities: {\"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-32/providers/Microsoft.ManagedIdentity/userAssignedIdentities/cloud-controller-manager\": &{ClientID: &\"f2e4769e-d3c2-498d-92b9-3e6d24cd2d7a\", PrincipalID: &\"a5c120f7-50c3-4cf6-84b4-57517f60630d\"}, \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-32/providers/Microsoft.ManagedIdentity/userAssignedIdentities/cloud-network-config\": &{ClientID: &\"f2e4769e-d3c2-498d-92b9-3e6d24cd2d7a\", PrincipalID: &\"a5c120f7-50c3-4cf6-84b4-57517f60630d\"}, \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-32/providers/Microsoft.ManagedIdentity/userAssignedIdentities/cluster-api-azure\": &{ClientID: &\"f2e4769e-d3c2-498d-92b9-3e6d24cd2d7a\", PrincipalID: &\"a5c120f7-50c3-4cf6-84b4-57517f60630d\"}, \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-32/providers/Microsoft.ManagedIdentity/userAssignedIdentities/control-plane\": &{ClientID: &\"f2e4769e-d3c2-498d-92b9-3e6d24cd2d7a\", PrincipalID: &\"a5c120f7-50c3-4cf6-84b4-57517f60630d\"}, ...}},\n \tProperties: &armredhatopenshifthcp.HcpOpenShiftClusterProperties{\n \t\t... // 8 identical fields\n \t\tVersion: &{ChannelGroup: &\"stable\", ID: &\"4.20\"},\n \t\tConsole: nil,\n- \t\tProvisioningState: &\"Succeeded\",\n+ \t\tProvisioningState: &\"Provisioning\",\n \t},\n \tTags: {\"aro-hcp.experimental.cluster.size-override\": &\"Minimal\"},\n \tID: &\"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGrou\"......
Gomega truncated this representation as it exceeds 'format.MaxLength'.
Consider having the object provide a custom 'GomegaStringer' representation
or adjust the parameters in Gomega's 'format' package.
Learn more here: https://onsi.github.io/gomega/#adjusting-output
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_authorized_cidrs_connectivity.go:133]: Unexpected error:
<*fmt.wrapError | 0xc00031e180>:
failed to create HCP cluster cidr-connectivity-test: operation result model did not match expected model for type *armredhatopenshifthcp.HcpOpenShiftCluster:
&armredhatopenshifthcp.HcpOpenShiftCluster{
Location: &"uksouth",
Identity: &{Type: &"UserAssigned", UserAssignedIdentities: {"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-32/providers/Microsoft.ManagedIdentity/userAssignedIdentities/cloud-controller-manager": &{ClientID: &"f2e4769e-d3c2-498d-92b9-3e6d24cd2d7a", PrincipalID: &"a5c120f7-50c3-4cf6-84b4-57517f60630d"}, "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-32/providers/Microsoft.ManagedIdentity/userAssignedIdentities/cloud-network-config": &{ClientID: &"f2e4769e-d3c2-498d-92b9-3e6d24cd2d7a", PrincipalID: &"a5c120f7-50c3-4cf6-84b4-57517f60630d"}, "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-32/providers/Microsoft.ManagedIdentity/userAssignedIdentities/cluster-api-azure": &{ClientID: &"f2e4769e-d3c2-498d-92b9-3e6d24cd2d7a", PrincipalID: &"a5c120f7-50c3-4cf6-84b4-57517f60630d"}, "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-32/providers/Microsoft.ManagedIdentity/userAssignedIdentities/control-plane": &{ClientID: &"f2e4769e-d3c2-498d-92b9-3e6d24cd2d7a", PrincipalID: &"a5c120f7-50c3-4cf6-84b4-57517f60630d"}, ...}},
Properties: &armredhatopenshifthcp.HcpOpenShiftClusterProperties{
... // 8 identical fields
Version: &{ChannelGroup: &"stable", ID: &"4.20"},
Console: nil,
- ProvisioningState: &"Succeeded",
+ ProvisioningState: &"Provisioning",
},
Tags: {"aro-hcp.experimental.cluster.size-override": &"Minimal"},
ID: &"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGrou"...,
... // 1 ignored and 2 identical fields
}
{
msg: "failed to create HCP cluster cidr-connectivity-test: operation result model did not match expected model for type *armredhatopenshifthcp.HcpOpenShiftCluster:\n &armredhatopenshifthcp.HcpOpenShiftCluster{\n \tLocation: &\"uksouth\",\n \tIdentity: &{Type: &\"UserAssigned\", UserAssignedIdentities: {\"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-32/providers/Microsoft.ManagedIdentity/userAssignedIdentities/cloud-controller-manager\": &{ClientID: &\"f2e4769e-d3c2-498d-92b9-3e6d24cd2d7a\", PrincipalID: &\"a5c120f7-50c3-4cf6-84b4-57517f60630d\"}, \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-32/providers/Microsoft.ManagedIdentity/userAssignedIdentities/cloud-network-config\": &{ClientID: &\"f2e4769e-d3c2-498d-92b9-3e6d24cd2d7a\", PrincipalID: &\"a5c120f7-50c3-4cf6-84b4-57517f60630d\"}, \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-32/providers/Microsoft.ManagedIdentity/userAssignedIdentities/cluster-api-azure\": &{ClientID: &\"f2e4769e-d3c2-498d-92b9-3e6d24cd2d7a\", PrincipalID: &\"a5c120f7-50c3-4cf6-84b4-57517f60630d\"}, \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-32/providers/Microsoft.ManagedIdentity/userAssignedIdentities/control-plane\": &{ClientID: &\"f2e4769e-d3c2-498d-92b9-3e6d24cd2d7a\", PrincipalID: &\"a5c120f7-50c3-4cf6-84b4-57517f60630d\"}, ...}},\n \tProperties: &armredhatopenshifthcp.HcpOpenShiftClusterProperties{\n \t\t... // 8 identical fields\n \t\tVersion: &{ChannelGroup: &\"stable\", ID: &\"4.20\"},\n \t\tConsole: nil,\n- \t\tProvisioningState: &\"Succeeded\",\n+ \t\tProvisioningState: &\"Provisioning\",\n \t},\n \tTags: {\"aro-hcp.experimental.cluster.size-override\": &\"Minimal\"},\n \tID: &\"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGrou\"...,\n \t... // 1 ignored and 2 identical fields\n }\n",
err: <*errors.errorString | 0xc0008dc6f0>{
s: "operation result model did not match expected model for type *armredhatopenshifthcp.HcpOpenShiftCluster:\n &armredhatopenshifthcp.HcpOpenShiftCluster{\n \tLocation: &\"uksouth\",\n \tIdentity: &{Type: &\"UserAssigned\", UserAssignedIdentities: {\"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-32/providers/Microsoft.ManagedIdentity/userAssignedIdentities/cloud-controller-manager\": &{ClientID: &\"f2e4769e-d3c2-498d-92b9-3e6d24cd2d7a\", PrincipalID: &\"a5c120f7-50c3-4cf6-84b4-57517f60630d\"}, \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-32/providers/Microsoft.ManagedIdentity/userAssignedIdentities/cloud-network-config\": &{ClientID: &\"f2e4769e-d3c2-498d-92b9-3e6d24cd2d7a\", PrincipalID: &\"a5c120f7-50c3-4cf6-84b4-57517f60630d\"}, \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-32/providers/Microsoft.ManagedIdentity/userAssignedIdentities/cluster-api-azure\": &{ClientID: &\"f2e4769e-d3c2-498d-92b9-3e6d24cd2d7a\", PrincipalID: &\"a5c120f7-50c3-4cf6-84b4-57517f60630d\"}, \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-32/providers/Microsoft.ManagedIdentity/userAssignedIdentities/control-plane\": &{ClientID: &\"f2e4769e-d3c2-498d-92b9-3e6d24cd2d7a\", PrincipalID: &\"a5c120f7-50c3-4cf6-84b4-57517f60630d\"}, ...}},\n \tProperties: &armredhatopenshifthcp.HcpOpenShiftClusterProperties{\n \t\t... // 8 identical fields\n \t\tVersion: &{ChannelGroup: &\"stable\", ID: &\"4.20\"},\n \t\tConsole: nil,\n- \t\tProvisioningState: &\"Succeeded\",\n+ \t\tProvisioningState: &\"Provisioning\",\n \t},\n \tTags: {\"aro-hcp.experimental.cluster.size-override\": &\"Minimal\"},\n \tID: &\"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGrou\"......
Gomega truncated this representation as it exceeds 'format.MaxLength'.
Consider having the object provide a custom 'GomegaStringer' representation
or adjust the parameters in Gomega's 'format' package.
Learn more here: https://onsi.github.io/gomega/#adjusting-output
occurredfail [github.com/Azure/ARO-HCP/test/e2e/cluster_update.go:160]: Unexpected error:
<*fmt.wrapError | 0xc000800f80>:
failed to create HCP cluster patch-tags-cluster: operation result model did not match expected model for type *armredhatopenshifthcp.HcpOpenShiftCluster:
&armredhatopenshifthcp.HcpOpenShiftCluster{
Location: &"uksouth",
Identity: &{Type: &"UserAssigned", UserAssignedIdentities: {"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-26/providers/Microsoft.ManagedIdentity/userAssignedIdentities/cloud-controller-manager": &{ClientID: &"f2e4769e-d3c2-498d-92b9-3e6d24cd2d7a", PrincipalID: &"a5c120f7-50c3-4cf6-84b4-57517f60630d"}, "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-26/providers/Microsoft.ManagedIdentity/userAssignedIdentities/cloud-network-config": &{ClientID: &"f2e4769e-d3c2-498d-92b9-3e6d24cd2d7a", PrincipalID: &"a5c120f7-50c3-4cf6-84b4-57517f60630d"}, "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-26/providers/Microsoft.ManagedIdentity/userAssignedIdentities/cluster-api-azure": &{ClientID: &"f2e4769e-d3c2-498d-92b9-3e6d24cd2d7a", PrincipalID: &"a5c120f7-50c3-4cf6-84b4-57517f60630d"}, "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-26/providers/Microsoft.ManagedIdentity/userAssignedIdentities/control-plane": &{ClientID: &"f2e4769e-d3c2-498d-92b9-3e6d24cd2d7a", PrincipalID: &"a5c120f7-50c3-4cf6-84b4-57517f60630d"}, ...}},
Properties: &armredhatopenshifthcp.HcpOpenShiftClusterProperties{
... // 8 identical fields
Version: &{ChannelGroup: &"stable", ID: &"4.20"},
Console: nil,
- ProvisioningState: &"Succeeded",
+ ProvisioningState: &"Provisioning",
},
Tags: {"aro-hcp.experimental.cluster.size-override": &"Minimal"},
ID: &"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGrou"...,
... // 1 ignored and 2 identical fields
}
{
msg: "failed to create HCP cluster patch-tags-cluster: operation result model did not match expected model for type *armredhatopenshifthcp.HcpOpenShiftCluster:\n &armredhatopenshifthcp.HcpOpenShiftCluster{\n \tLocation: &\"uksouth\",\n \tIdentity: &{Type: &\"UserAssigned\", UserAssignedIdentities: {\"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-26/providers/Microsoft.ManagedIdentity/userAssignedIdentities/cloud-controller-manager\": &{ClientID: &\"f2e4769e-d3c2-498d-92b9-3e6d24cd2d7a\", PrincipalID: &\"a5c120f7-50c3-4cf6-84b4-57517f60630d\"}, \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-26/providers/Microsoft.ManagedIdentity/userAssignedIdentities/cloud-network-config\": &{ClientID: &\"f2e4769e-d3c2-498d-92b9-3e6d24cd2d7a\", PrincipalID: &\"a5c120f7-50c3-4cf6-84b4-57517f60630d\"}, \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-26/providers/Microsoft.ManagedIdentity/userAssignedIdentities/cluster-api-azure\": &{ClientID: &\"f2e4769e-d3c2-498d-92b9-3e6d24cd2d7a\", PrincipalID: &\"a5c120f7-50c3-4cf6-84b4-57517f60630d\"}, \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-26/providers/Microsoft.ManagedIdentity/userAssignedIdentities/control-plane\": &{ClientID: &\"f2e4769e-d3c2-498d-92b9-3e6d24cd2d7a\", PrincipalID: &\"a5c120f7-50c3-4cf6-84b4-57517f60630d\"}, ...}},\n \tProperties: &armredhatopenshifthcp.HcpOpenShiftClusterProperties{\n \t\t... // 8 identical fields\n \t\tVersion: &{ChannelGroup: &\"stable\", ID: &\"4.20\"},\n \t\tConsole: nil,\n- \t\tProvisioningState: &\"Succeeded\",\n+ \t\tProvisioningState: &\"Provisioning\",\n \t},\n \tTags: {\"aro-hcp.experimental.cluster.size-override\": &\"Minimal\"},\n \tID: &\"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGrou\"...,\n \t... // 1 ignored and 2 identical fields\n }\n",
err: <*errors.errorString | 0xc0007cf6c0>{
s: "operation result model did not match expected model for type *armredhatopenshifthcp.HcpOpenShiftCluster:\n &armredhatopenshifthcp.HcpOpenShiftCluster{\n \tLocation: &\"uksouth\",\n \tIdentity: &{Type: &\"UserAssigned\", UserAssignedIdentities: {\"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-26/providers/Microsoft.ManagedIdentity/userAssignedIdentities/cloud-controller-manager\": &{ClientID: &\"f2e4769e-d3c2-498d-92b9-3e6d24cd2d7a\", PrincipalID: &\"a5c120f7-50c3-4cf6-84b4-57517f60630d\"}, \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-26/providers/Microsoft.ManagedIdentity/userAssignedIdentities/cloud-network-config\": &{ClientID: &\"f2e4769e-d3c2-498d-92b9-3e6d24cd2d7a\", PrincipalID: &\"a5c120f7-50c3-4cf6-84b4-57517f60630d\"}, \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-26/providers/Microsoft.ManagedIdentity/userAssignedIdentities/cluster-api-azure\": &{ClientID: &\"f2e4769e-d3c2-498d-92b9-3e6d24cd2d7a\", PrincipalID: &\"a5c120f7-50c3-4cf6-84b4-57517f60630d\"}, \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-26/providers/Microsoft.ManagedIdentity/userAssignedIdentities/control-plane\": &{ClientID: &\"f2e4769e-d3c2-498d-92b9-3e6d24cd2d7a\", PrincipalID: &\"a5c120f7-50c3-4cf6-84b4-57517f60630d\"}, ...}},\n \tProperties: &armredhatopenshifthcp.HcpOpenShiftClusterProperties{\n \t\t... // 8 identical fields\n \t\tVersion: &{ChannelGroup: &\"stable\", ID: &\"4.20\"},\n \t\tConsole: nil,\n- \t\tProvisioningState: &\"Succeeded\",\n+ \t\tProvisioningState: &\"Provisioning\",\n \t},\n \tTags: {\"aro-hcp.experimental.cluster.size-override\": &\"Minimal\"},\n \tID: &\"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGrou\"...,\n ...
Gomega truncated this representation as it exceeds 'format.MaxLength'.
Consider having the object provide a custom 'GomegaStringer' representation
or adjust the parameters in Gomega's 'format' package.
Learn more here: https://onsi.github.io/gomega/#adjusting-output
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_update.go:160]: Unexpected error:
<*fmt.wrapError | 0xc000800f80>:
failed to create HCP cluster patch-tags-cluster: operation result model did not match expected model for type *armredhatopenshifthcp.HcpOpenShiftCluster:
&armredhatopenshifthcp.HcpOpenShiftCluster{
Location: &"uksouth",
Identity: &{Type: &"UserAssigned", UserAssignedIdentities: {"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-26/providers/Microsoft.ManagedIdentity/userAssignedIdentities/cloud-controller-manager": &{ClientID: &"f2e4769e-d3c2-498d-92b9-3e6d24cd2d7a", PrincipalID: &"a5c120f7-50c3-4cf6-84b4-57517f60630d"}, "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-26/providers/Microsoft.ManagedIdentity/userAssignedIdentities/cloud-network-config": &{ClientID: &"f2e4769e-d3c2-498d-92b9-3e6d24cd2d7a", PrincipalID: &"a5c120f7-50c3-4cf6-84b4-57517f60630d"}, "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-26/providers/Microsoft.ManagedIdentity/userAssignedIdentities/cluster-api-azure": &{ClientID: &"f2e4769e-d3c2-498d-92b9-3e6d24cd2d7a", PrincipalID: &"a5c120f7-50c3-4cf6-84b4-57517f60630d"}, "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-26/providers/Microsoft.ManagedIdentity/userAssignedIdentities/control-plane": &{ClientID: &"f2e4769e-d3c2-498d-92b9-3e6d24cd2d7a", PrincipalID: &"a5c120f7-50c3-4cf6-84b4-57517f60630d"}, ...}},
Properties: &armredhatopenshifthcp.HcpOpenShiftClusterProperties{
... // 8 identical fields
Version: &{ChannelGroup: &"stable", ID: &"4.20"},
Console: nil,
- ProvisioningState: &"Succeeded",
+ ProvisioningState: &"Provisioning",
},
Tags: {"aro-hcp.experimental.cluster.size-override": &"Minimal"},
ID: &"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGrou"...,
... // 1 ignored and 2 identical fields
}
{
msg: "failed to create HCP cluster patch-tags-cluster: operation result model did not match expected model for type *armredhatopenshifthcp.HcpOpenShiftCluster:\n &armredhatopenshifthcp.HcpOpenShiftCluster{\n \tLocation: &\"uksouth\",\n \tIdentity: &{Type: &\"UserAssigned\", UserAssignedIdentities: {\"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-26/providers/Microsoft.ManagedIdentity/userAssignedIdentities/cloud-controller-manager\": &{ClientID: &\"f2e4769e-d3c2-498d-92b9-3e6d24cd2d7a\", PrincipalID: &\"a5c120f7-50c3-4cf6-84b4-57517f60630d\"}, \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-26/providers/Microsoft.ManagedIdentity/userAssignedIdentities/cloud-network-config\": &{ClientID: &\"f2e4769e-d3c2-498d-92b9-3e6d24cd2d7a\", PrincipalID: &\"a5c120f7-50c3-4cf6-84b4-57517f60630d\"}, \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-26/providers/Microsoft.ManagedIdentity/userAssignedIdentities/cluster-api-azure\": &{ClientID: &\"f2e4769e-d3c2-498d-92b9-3e6d24cd2d7a\", PrincipalID: &\"a5c120f7-50c3-4cf6-84b4-57517f60630d\"}, \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-26/providers/Microsoft.ManagedIdentity/userAssignedIdentities/control-plane\": &{ClientID: &\"f2e4769e-d3c2-498d-92b9-3e6d24cd2d7a\", PrincipalID: &\"a5c120f7-50c3-4cf6-84b4-57517f60630d\"}, ...}},\n \tProperties: &armredhatopenshifthcp.HcpOpenShiftClusterProperties{\n \t\t... // 8 identical fields\n \t\tVersion: &{ChannelGroup: &\"stable\", ID: &\"4.20\"},\n \t\tConsole: nil,\n- \t\tProvisioningState: &\"Succeeded\",\n+ \t\tProvisioningState: &\"Provisioning\",\n \t},\n \tTags: {\"aro-hcp.experimental.cluster.size-override\": &\"Minimal\"},\n \tID: &\"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGrou\"...,\n \t... // 1 ignored and 2 identical fields\n }\n",
err: <*errors.errorString | 0xc0007cf6c0>{
s: "operation result model did not match expected model for type *armredhatopenshifthcp.HcpOpenShiftCluster:\n &armredhatopenshifthcp.HcpOpenShiftCluster{\n \tLocation: &\"uksouth\",\n \tIdentity: &{Type: &\"UserAssigned\", UserAssignedIdentities: {\"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-26/providers/Microsoft.ManagedIdentity/userAssignedIdentities/cloud-controller-manager\": &{ClientID: &\"f2e4769e-d3c2-498d-92b9-3e6d24cd2d7a\", PrincipalID: &\"a5c120f7-50c3-4cf6-84b4-57517f60630d\"}, \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-26/providers/Microsoft.ManagedIdentity/userAssignedIdentities/cloud-network-config\": &{ClientID: &\"f2e4769e-d3c2-498d-92b9-3e6d24cd2d7a\", PrincipalID: &\"a5c120f7-50c3-4cf6-84b4-57517f60630d\"}, \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-26/providers/Microsoft.ManagedIdentity/userAssignedIdentities/cluster-api-azure\": &{ClientID: &\"f2e4769e-d3c2-498d-92b9-3e6d24cd2d7a\", PrincipalID: &\"a5c120f7-50c3-4cf6-84b4-57517f60630d\"}, \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-26/providers/Microsoft.ManagedIdentity/userAssignedIdentities/control-plane\": &{ClientID: &\"f2e4769e-d3c2-498d-92b9-3e6d24cd2d7a\", PrincipalID: &\"a5c120f7-50c3-4cf6-84b4-57517f60630d\"}, ...}},\n \tProperties: &armredhatopenshifthcp.HcpOpenShiftClusterProperties{\n \t\t... // 8 identical fields\n \t\tVersion: &{ChannelGroup: &\"stable\", ID: &\"4.20\"},\n \t\tConsole: nil,\n- \t\tProvisioningState: &\"Succeeded\",\n+ \t\tProvisioningState: &\"Provisioning\",\n \t},\n \tTags: {\"aro-hcp.experimental.cluster.size-override\": &\"Minimal\"},\n \tID: &\"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGrou\"...,\n ...
Gomega truncated this representation as it exceeds 'format.MaxLength'.
Consider having the object provide a custom 'GomegaStringer' representation
or adjust the parameters in Gomega's 'format' package.
Learn more here: https://onsi.github.io/gomega/#adjusting-output
occurredContributing tests (2)
Affected runs (2)
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
ERROR CODE: Conflict; detail message Cannot update resource while resource is "provisioning"; provider Microsoft.RedHatOpenShiftfull failure pattern: ERROR CODE: Conflict; detail message Cannot update resource while resource is "provisioning"; provider Microsoft.RedHatOpenShift Signal: IndeterminateSignal: Indeterminate — no prior history; active 1 day(s) — New failure pattern — no prior history | e2e | 1 | 2.94%1 of 34 job runs affected | IndeterminateSignal: Indeterminate — no prior history; active 1 day(s) — New failure pattern — no prior history | Show trend detailsFeb 16: 0 · Feb 17: 0 · Feb 18: 0 · Feb 19: 0 · Feb 20: 0 · Feb 21: 0 · Feb 22: 0 · Feb 23: 0 · Feb 24: 0 · Feb 25: 0 · Feb 26: 0 · Feb 27: 0 · Feb 28: 1 · Mar 1: 0 | none | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Full failure examples (1)fail [github.com/Azure/ARO-HCP/test/e2e/cluster_pullsecret.go:112]: Unexpected error:
<*fmt.wrapError | 0xc00047c640>:
failed to create NodePool np-1: failed starting nodepool creation "np-1" for cluster "pullsecret-hcp-cluster" in resourcegroup="pullsecret-test-q2kh8p": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/pullsecret-test-q2kh8p/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/pullsecret-hcp-cluster/nodePools/np-1
--------------------------------------------------------------------------------
RESPONSE 409: 409 Conflict
ERROR CODE: Conflict
--------------------------------------------------------------------------------
{
"error": {
"code": "Conflict",
"message": "Cannot update resource while resource is \"provisioning\"",
"target": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/pullsecret-test-q2kh8p/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/pullsecret-hcp-cluster/nodePools/np-1"
}
}
--------------------------------------------------------------------------------
{
msg: "failed to create NodePool np-1: failed starting nodepool creation \"np-1\" for cluster \"pullsecret-hcp-cluster\" in resourcegroup=\"pullsecret-test-q2kh8p\": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/pullsecret-test-q2kh8p/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/pullsecret-hcp-cluster/nodePools/np-1\n--------------------------------------------------------------------------------\nRESPONSE 409: 409 Conflict\nERROR CODE: Conflict\n--------------------------------------------------------------------------------\n{\n \"error\": {\n \"code\": \"Conflict\",\n \"message\": \"Cannot update resource while resource is \\\"provisioning\\\"\",\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/pullsecret-test-q2kh8p/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/pullsecret-hcp-cluster/nodePools/np-1\"\n }\n}\n--------------------------------------------------------------------------------\n",
err: <*fmt.wrapError | 0xc00047c620>{
msg: "failed starting nodepool creation \"np-1\" for cluster \"pullsecret-hcp-cluster\" in resourcegroup=\"pullsecret-test-q2kh8p\": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/pullsecret-test-q2kh8p/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/pullsecret-hcp-cluster/nodePools/np-1\n--------------------------------------------------------------------------------\nRESPONSE 409: 409 Conflict\nERROR CODE: Conflict\n--------------------------------------------------------------------------------\n{\n \"error\": {\n \"code\": \"Conflict\",\n \"message\": \"Cannot update resource while resource is \\\"provisioning\\\"\",\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/pullsecret-test-q2kh8p/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/pullsecret-hcp-cluster/nodePools/np-1\"\n }\n}\n--------------------------------------------------------------------------------\n",
err: <*exported.ResponseError | 0xc000da2840>{
ErrorCode: "Conflict",
StatusCode: 409,
RawResponse: {
Status: "409 Conflict",
StatusCode: 409,
Proto: "HTTP/2.0",
ProtoMajor: 2,
ProtoMinor: 0,
Header: {
"Cache-Control": ["no-cache"],
"Content-Length": ["337"],
"X-Ms-Error-Code": ["Conflict"],
"X-Ms-Request-Id": [
"4ed37b5a-b4c0-4770-a28e-c6490ea6c312",
],
"X-Msedge-Ref": [
"Ref A: 43729C327091453782F1D7DDE5C7DAA5 Ref B: CHI261140503060 Ref C: 2026-02-28T04:05:08Z",
],
"X-Envoy-Upstream-Service-Time": ["147"],
"X-Ms-Ratelimit-Remaining-Subscription-Global-Writes": ["2999"],
"X-Ms-Correlation-Request-Id": [
"8145a09b-5ba1-4659-a364-95e4ef0bdd6a",
],
"X-Ms-Routing-Request-Id": [
"UKSOUTH:20260228T040509Z:8145a09b-5ba1-4659-a364-95e4ef0bdd6a",
],
"Strict-Transport-Security": [
"max-age=31536000; includeSubDomains",
],
"X-Cache": ["CONFIG_NOCACHE"],
"Content-Type": ["application/json"],
"X-Ms-Ratelimit-Remaining-Subscription-Writes": ["199"],
"Date": [
"Sat, 28 Feb 2026 04:05:08 GMT",
],
"Pragma": ["no-cache"],
"Expires"...
Gomega truncated this representation as it exceeds 'format.MaxLength'.
Consider having the object provide a custom 'GomegaStringer' representation
or adjust the parameters in Gomega's 'format' package.
Learn more here: https://onsi.github.io/gomega/#adjusting-output
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_pullsecret.go:112]: Unexpected error:
<*fmt.wrapError | 0xc00047c640>:
failed to create NodePool np-1: failed starting nodepool creation "np-1" for cluster "pullsecret-hcp-cluster" in resourcegroup="pullsecret-test-q2kh8p": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/pullsecret-test-q2kh8p/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/pullsecret-hcp-cluster/nodePools/np-1
--------------------------------------------------------------------------------
RESPONSE 409: 409 Conflict
ERROR CODE: Conflict
--------------------------------------------------------------------------------
{
"error": {
"code": "Conflict",
"message": "Cannot update resource while resource is \"provisioning\"",
"target": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/pullsecret-test-q2kh8p/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/pullsecret-hcp-cluster/nodePools/np-1"
}
}
--------------------------------------------------------------------------------
{
msg: "failed to create NodePool np-1: failed starting nodepool creation \"np-1\" for cluster \"pullsecret-hcp-cluster\" in resourcegroup=\"pullsecret-test-q2kh8p\": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/pullsecret-test-q2kh8p/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/pullsecret-hcp-cluster/nodePools/np-1\n--------------------------------------------------------------------------------\nRESPONSE 409: 409 Conflict\nERROR CODE: Conflict\n--------------------------------------------------------------------------------\n{\n \"error\": {\n \"code\": \"Conflict\",\n \"message\": \"Cannot update resource while resource is \\\"provisioning\\\"\",\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/pullsecret-test-q2kh8p/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/pullsecret-hcp-cluster/nodePools/np-1\"\n }\n}\n--------------------------------------------------------------------------------\n",
err: <*fmt.wrapError | 0xc00047c620>{
msg: "failed starting nodepool creation \"np-1\" for cluster \"pullsecret-hcp-cluster\" in resourcegroup=\"pullsecret-test-q2kh8p\": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/pullsecret-test-q2kh8p/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/pullsecret-hcp-cluster/nodePools/np-1\n--------------------------------------------------------------------------------\nRESPONSE 409: 409 Conflict\nERROR CODE: Conflict\n--------------------------------------------------------------------------------\n{\n \"error\": {\n \"code\": \"Conflict\",\n \"message\": \"Cannot update resource while resource is \\\"provisioning\\\"\",\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/pullsecret-test-q2kh8p/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/pullsecret-hcp-cluster/nodePools/np-1\"\n }\n}\n--------------------------------------------------------------------------------\n",
err: <*exported.ResponseError | 0xc000da2840>{
ErrorCode: "Conflict",
StatusCode: 409,
RawResponse: {
Status: "409 Conflict",
StatusCode: 409,
Proto: "HTTP/2.0",
ProtoMajor: 2,
ProtoMinor: 0,
Header: {
"Cache-Control": ["no-cache"],
"Content-Length": ["337"],
"X-Ms-Error-Code": ["Conflict"],
"X-Ms-Request-Id": [
"4ed37b5a-b4c0-4770-a28e-c6490ea6c312",
],
"X-Msedge-Ref": [
"Ref A: 43729C327091453782F1D7DDE5C7DAA5 Ref B: CHI261140503060 Ref C: 2026-02-28T04:05:08Z",
],
"X-Envoy-Upstream-Service-Time": ["147"],
"X-Ms-Ratelimit-Remaining-Subscription-Global-Writes": ["2999"],
"X-Ms-Correlation-Request-Id": [
"8145a09b-5ba1-4659-a364-95e4ef0bdd6a",
],
"X-Ms-Routing-Request-Id": [
"UKSOUTH:20260228T040509Z:8145a09b-5ba1-4659-a364-95e4ef0bdd6a",
],
"Strict-Transport-Security": [
"max-age=31536000; includeSubDomains",
],
"X-Cache": ["CONFIG_NOCACHE"],
"Content-Type": ["application/json"],
"X-Ms-Ratelimit-Remaining-Subscription-Writes": ["199"],
"Date": [
"Sat, 28 Feb 2026 04:05:08 GMT",
],
"Pragma": ["no-cache"],
"Expires"...
Gomega truncated this representation as it exceeds 'format.MaxLength'.
Consider having the object provide a custom 'GomegaStringer' representation
or adjust the parameters in Gomega's 'format' package.
Learn more here: https://onsi.github.io/gomega/#adjusting-output
occurredContributing tests (1)
Affected runs (1)
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
VerifyAllAPIServicesAvailable failed: failed to list all APIServices: Unauthorizedfull failure pattern: VerifyAllAPIServicesAvailable failed: failed to list all APIServices: Unauthorized Signal: IndeterminateSignal: Indeterminate — no prior history; active 1 day(s) — New failure pattern — no prior history | e2e | 1 | 2.94%1 of 34 job runs affected | IndeterminateSignal: Indeterminate — no prior history; active 1 day(s) — New failure pattern — no prior history | Show trend detailsFeb 16: 0 · Feb 17: 0 · Feb 18: 0 · Feb 19: 0 · Feb 20: 0 · Feb 21: 0 · Feb 22: 0 · Feb 23: 1 · Feb 24: 0 · Feb 25: 0 · Feb 26: 0 · Feb 27: 0 · Feb 28: 0 · Mar 1: 0 | none | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Full failure examples (1)fail [github.com/Azure/ARO-HCP/test/e2e/admin_credential_lifecycle.go:258]: New admin credentials should work after revocation
Expected success, but got an error:
<*errors.joinError | 0xc000e0d7b8>:
VerifyAllAPIServicesAvailable failed: failed to list all APIServices: Unauthorized
VerifyBasicAccess failed: failed to list services: Unauthorized
{
errs: [
<*fmt.wrapError | 0xc0004caca0>{
msg: "VerifyAllAPIServicesAvailable failed: failed to list all APIServices: Unauthorized",
err: <*fmt.wrapError | 0xc0004caaa0>{
msg: "failed to list all APIServices: Unauthorized",
err: <*errors.StatusError | 0xc000922140>{
ErrStatus: {
TypeMeta: {Kind: "", APIVersion: ""},
ListMeta: {
SelfLink: "",
ResourceVersion: "",
Continue: "",
RemainingItemCount: nil,
},
Status: "Failure",
Message: "Unauthorized",
Reason: "Unauthorized",
Details: nil,
Code: 401,
},
},
},
},
<*fmt.wrapError | 0xc0006b3bc0>{
msg: "VerifyBasicAccess failed: failed to list services: Unauthorized",
err: <*fmt.wrapError | 0xc0006b3ba0>{
msg: "failed to list services: Unauthorized",
err: <*errors.StatusError | 0xc000793860>{
ErrStatus: {
TypeMeta: {Kind: "", APIVersion: ""},
ListMeta: {
SelfLink: "",
ResourceVersion: "",
Continue: "",
RemainingItemCount: nil,
},
Status: "Failure",
Message: "Unauthorized",
Reason: "Unauthorized",
Details: nil,
Code: 401,
},
},
},
},
],
}
fail [github.com/Azure/ARO-HCP/test/e2e/admin_credential_lifecycle.go:258]: New admin credentials should work after revocation
Expected success, but got an error:
<*errors.joinError | 0xc000e0d7b8>:
VerifyAllAPIServicesAvailable failed: failed to list all APIServices: Unauthorized
VerifyBasicAccess failed: failed to list services: Unauthorized
{
errs: [
<*fmt.wrapError | 0xc0004caca0>{
msg: "VerifyAllAPIServicesAvailable failed: failed to list all APIServices: Unauthorized",
err: <*fmt.wrapError | 0xc0004caaa0>{
msg: "failed to list all APIServices: Unauthorized",
err: <*errors.StatusError | 0xc000922140>{
ErrStatus: {
TypeMeta: {Kind: "", APIVersion: ""},
ListMeta: {
SelfLink: "",
ResourceVersion: "",
Continue: "",
RemainingItemCount: nil,
},
Status: "Failure",
Message: "Unauthorized",
Reason: "Unauthorized",
Details: nil,
Code: 401,
},
},
},
},
<*fmt.wrapError | 0xc0006b3bc0>{
msg: "VerifyBasicAccess failed: failed to list services: Unauthorized",
err: <*fmt.wrapError | 0xc0006b3ba0>{
msg: "failed to list services: Unauthorized",
err: <*errors.StatusError | 0xc000793860>{
ErrStatus: {
TypeMeta: {Kind: "", APIVersion: ""},
ListMeta: {
SelfLink: "",
ResourceVersion: "",
Continue: "",
RemainingItemCount: nil,
},
Status: "Failure",
Message: "Unauthorized",
Reason: "Unauthorized",
Details: nil,
Code: 401,
},
},
},
},
],
}Contributing tests (1)
Affected runs (1)
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
VerifyBasicAccess failed: failed to list services: Unauthorizedfull failure pattern: VerifyBasicAccess failed: failed to list services: Unauthorized Signal: IndeterminateSignal: Indeterminate — no prior history; active 1 day(s) — New failure pattern — no prior history | e2e | 1 | 2.94%1 of 34 job runs affected | IndeterminateSignal: Indeterminate — no prior history; active 1 day(s) — New failure pattern — no prior history | Show trend detailsFeb 16: 0 · Feb 17: 0 · Feb 18: 0 · Feb 19: 0 · Feb 20: 0 · Feb 21: 0 · Feb 22: 0 · Feb 23: 0 · Feb 24: 0 · Feb 25: 0 · Feb 26: 1 · Feb 27: 0 · Feb 28: 0 · Mar 1: 0 | none | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Full failure examples (1)fail [github.com/Azure/ARO-HCP/test/e2e/admin_credential_lifecycle.go:258]: New admin credentials should work after revocation
Expected success, but got an error:
<*errors.joinError | 0xc00129b740>:
VerifyBasicAccess failed: failed to list services: Unauthorized
VerifyAllAPIServicesAvailable failed: failed to list all APIServices: Unauthorized
{
errs: [
<*fmt.wrapError | 0xc000c8e7e0>{
msg: "VerifyBasicAccess failed: failed to list services: Unauthorized",
err: <*fmt.wrapError | 0xc000c8e7a0>{
msg: "failed to list services: Unauthorized",
err: <*errors.StatusError | 0xc00108c280>{
ErrStatus: {
TypeMeta: {Kind: "", APIVersion: ""},
ListMeta: {
SelfLink: "",
ResourceVersion: "",
Continue: "",
RemainingItemCount: nil,
},
Status: "Failure",
Message: "Unauthorized",
Reason: "Unauthorized",
Details: nil,
Code: 401,
},
},
},
},
<*fmt.wrapError | 0xc000831c60>{
msg: "VerifyAllAPIServicesAvailable failed: failed to list all APIServices: Unauthorized",
err: <*fmt.wrapError | 0xc000831c40>{
msg: "failed to list all APIServices: Unauthorized",
err: <*errors.StatusError | 0xc000a797c0>{
ErrStatus: {
TypeMeta: {Kind: "", APIVersion: ""},
ListMeta: {
SelfLink: "",
ResourceVersion: "",
Continue: "",
RemainingItemCount: nil,
},
Status: "Failure",
Message: "Unauthorized",
Reason: "Unauthorized",
Details: nil,
Code: 401,
},
},
},
},
],
}
fail [github.com/Azure/ARO-HCP/test/e2e/admin_credential_lifecycle.go:258]: New admin credentials should work after revocation
Expected success, but got an error:
<*errors.joinError | 0xc00129b740>:
VerifyBasicAccess failed: failed to list services: Unauthorized
VerifyAllAPIServicesAvailable failed: failed to list all APIServices: Unauthorized
{
errs: [
<*fmt.wrapError | 0xc000c8e7e0>{
msg: "VerifyBasicAccess failed: failed to list services: Unauthorized",
err: <*fmt.wrapError | 0xc000c8e7a0>{
msg: "failed to list services: Unauthorized",
err: <*errors.StatusError | 0xc00108c280>{
ErrStatus: {
TypeMeta: {Kind: "", APIVersion: ""},
ListMeta: {
SelfLink: "",
ResourceVersion: "",
Continue: "",
RemainingItemCount: nil,
},
Status: "Failure",
Message: "Unauthorized",
Reason: "Unauthorized",
Details: nil,
Code: 401,
},
},
},
},
<*fmt.wrapError | 0xc000831c60>{
msg: "VerifyAllAPIServicesAvailable failed: failed to list all APIServices: Unauthorized",
err: <*fmt.wrapError | 0xc000831c40>{
msg: "failed to list all APIServices: Unauthorized",
err: <*errors.StatusError | 0xc000a797c0>{
ErrStatus: {
TypeMeta: {Kind: "", APIVersion: ""},
ListMeta: {
SelfLink: "",
ResourceVersion: "",
Continue: "",
RemainingItemCount: nil,
},
Status: "Failure",
Message: "Unauthorized",
Reason: "Unauthorized",
Details: nil,
Code: 401,
},
},
},
},
],
}Contributing tests (1)
Affected runs (1)
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
operation result model did not match expected model for type *armredhatopenshifthcp.ExternalAuth:full failure pattern: operation result model did not match expected model for type *armredhatopenshifthcp.ExternalAuth: Signal: IndeterminateSignal: Indeterminate — no prior history; active 1 day(s) — New failure pattern — no prior history | e2e | 1 | 2.94%1 of 34 job runs affected | IndeterminateSignal: Indeterminate — no prior history; active 1 day(s) — New failure pattern — no prior history | Show trend detailsFeb 16: 0 · Feb 17: 0 · Feb 18: 0 · Feb 19: 0 · Feb 20: 0 · Feb 21: 0 · Feb 22: 0 · Feb 23: 0 · Feb 24: 0 · Feb 25: 0 · Feb 26: 0 · Feb 27: 1 · Feb 28: 0 · Mar 1: 0 | none | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Full failure examples (1)fail [github.com/Azure/ARO-HCP/test/e2e/external_auth_create.go:172]: Unexpected error:
<*errors.errorString | 0xc000980da0>:
operation result model did not match expected model for type *armredhatopenshifthcp.ExternalAuth:
&armredhatopenshifthcp.ExternalAuth{
Properties: &armredhatopenshifthcp.ExternalAuthProperties{
... // 2 identical fields
Clients: {&{ClientID: &"7404448a-dc7a-4158-9184-effd2edcc2f0", Component: &{AuthClientNamespace: &"openshift-console", Name: &"console"}, Type: &"Confidential"}, &{ClientID: &"7404448a-dc7a-4158-9184-effd2edcc2f0", Component: &{AuthClientNamespace: &"openshift-console", Name: &"cli"}, Type: &"Public"}},
Condition: nil,
- ProvisioningState: &"Succeeded",
+ ProvisioningState: &"Accepted",
},
ID: &"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGrou"...,
Name: &"external-auth",
... // 1 ignored and 1 identical fields
}
{
s: "operation result model did not match expected model for type *armredhatopenshifthcp.ExternalAuth:\n\u00a0\u00a0&armredhatopenshifthcp.ExternalAuth{\n\u00a0\u00a0\tProperties: &armredhatopenshifthcp.ExternalAuthProperties{\n\u00a0\u00a0\t\t... // 2 identical fields\n\u00a0\u00a0\t\tClients: {&{ClientID: &\"7404448a-dc7a-4158-9184-effd2edcc2f0\", Component: &{AuthClientNamespace: &\"openshift-console\", Name: &\"console\"}, Type: &\"Confidential\"}, &{ClientID: &\"7404448a-dc7a-4158-9184-effd2edcc2f0\", Component: &{AuthClientNamespace: &\"openshift-console\", Name: &\"cli\"}, Type: &\"Public\"}},\n\u00a0\u00a0\t\tCondition: nil,\n-\u00a0\t\tProvisioningState: &\"Succeeded\",\n+\u00a0\t\tProvisioningState: &\"Accepted\",\n\u00a0\u00a0\t},\n\u00a0\u00a0\tID: &\"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGrou\"...,\n\u00a0\u00a0\tName: &\"external-auth\",\n\u00a0\u00a0\t... // 1 ignored and 1 identical fields\n\u00a0\u00a0}\n",
}
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/external_auth_create.go:172]: Unexpected error:
<*errors.errorString | 0xc000980da0>:
operation result model did not match expected model for type *armredhatopenshifthcp.ExternalAuth:
&armredhatopenshifthcp.ExternalAuth{
Properties: &armredhatopenshifthcp.ExternalAuthProperties{
... // 2 identical fields
Clients: {&{ClientID: &"7404448a-dc7a-4158-9184-effd2edcc2f0", Component: &{AuthClientNamespace: &"openshift-console", Name: &"console"}, Type: &"Confidential"}, &{ClientID: &"7404448a-dc7a-4158-9184-effd2edcc2f0", Component: &{AuthClientNamespace: &"openshift-console", Name: &"cli"}, Type: &"Public"}},
Condition: nil,
- ProvisioningState: &"Succeeded",
+ ProvisioningState: &"Accepted",
},
ID: &"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGrou"...,
Name: &"external-auth",
... // 1 ignored and 1 identical fields
}
{
s: "operation result model did not match expected model for type *armredhatopenshifthcp.ExternalAuth:\n\u00a0\u00a0&armredhatopenshifthcp.ExternalAuth{\n\u00a0\u00a0\tProperties: &armredhatopenshifthcp.ExternalAuthProperties{\n\u00a0\u00a0\t\t... // 2 identical fields\n\u00a0\u00a0\t\tClients: {&{ClientID: &\"7404448a-dc7a-4158-9184-effd2edcc2f0\", Component: &{AuthClientNamespace: &\"openshift-console\", Name: &\"console\"}, Type: &\"Confidential\"}, &{ClientID: &\"7404448a-dc7a-4158-9184-effd2edcc2f0\", Component: &{AuthClientNamespace: &\"openshift-console\", Name: &\"cli\"}, Type: &\"Public\"}},\n\u00a0\u00a0\t\tCondition: nil,\n-\u00a0\t\tProvisioningState: &\"Succeeded\",\n+\u00a0\t\tProvisioningState: &\"Accepted\",\n\u00a0\u00a0\t},\n\u00a0\u00a0\tID: &\"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGrou\"...,\n\u00a0\u00a0\tName: &\"external-auth\",\n\u00a0\u00a0\t... // 1 ignored and 1 identical fields\n\u00a0\u00a0}\n",
}
occurredContributing tests (1)
Affected runs (1)
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Environment: STG
E2E Jobs
9
Success Rate
100.00%
E2E success
Among runs that reached E2E execution, this shows how many completed successfully. Provision failures and 'Other' CI-infrastructure failures are excluded because those runs never reached the E2E step.100.00% (9/9)
Daily Run Outcomes
Successful runs
Provision failures
E2E failures
Other failures
| Pass rate | Runs | Date (UTC) | Suite | Test |
|---|---|---|---|---|
| 56.25% | 16 | stage/parallel | Authorized CIDRs Connectivity should allow API access only from authorized VM IP | |
| 56.25% | 16 | stage/parallel | Customer should be able to create an HCP cluster and manage pull secrets | |
| 56.25% | 16 | stage/parallel | Customer should be able to create node pools with ARM64-based VMs | |
| 62.50% | 16 | stage/parallel | Customer should be able to create a cluster with an external auth config and get the external auth config | |
| 62.50% | 16 | stage/parallel | Customer should be able to create a cluster with default autoscaling and a nodepool with autoscaling enabled |
Up to 50 failure patterns are loaded (minimum 1.00% impact), with 10 shown by default. Click column headers to re-sort.
Jump to the full Failure Patterns view for this window
No failure patterns available for this environment in the selected window.
Environment: PROD
E2E Jobs
9
Success Rate
33.33%
E2E success
Among runs that reached E2E execution, this shows how many completed successfully. Provision failures and 'Other' CI-infrastructure failures are excluded because those runs never reached the E2E step.33.33% (3/9)
Daily Run Outcomes
Successful runs
Provision failures
E2E failures
Other failures
| Pass rate | Runs | Date (UTC) | Suite | Test |
|---|---|---|---|---|
| 66.67% | 24 | prod/parallel | Authorized CIDRs Connectivity should allow API access only from authorized VM IP | |
| 66.67% | 24 | prod/parallel | Customer should be able to create a cluster with an external auth config and get the external auth config | |
| 66.67% | 24 | prod/parallel | Customer should be able to create an HCP cluster and custom node pool osDisk size | |
| 66.67% | 24 | prod/parallel | Customer should be able to create an HCP cluster and manage pull secrets | |
| 66.67% | 24 | prod/parallel | Customer should create an HCP cluster and validate TLS certificates |
Up to 50 failure patterns are loaded (minimum 1.00% impact), with 10 shown by default. Click column headers to re-sort.
Jump to the full Failure Patterns view for this window
| Failure patternThe canonical description of a recurring CI failure, extracted and normalized from raw logs. | Failed atThe stage of the job run where this failure occurred: 'provision' (environment setup, DEV only), 'e2e' (test suite execution), or 'other' (CI infrastructure issues that did not produce a failure pattern). | Number of distinct job runs where this failure pattern was detected. | Percentage of all job runs in this environment affected by this failure pattern during the selected window. | Classification of this failure pattern: Regression (likely caused by a specific PR), Flake (intermittent failure spread across days), Noise (low-quality or generic pattern), or Indeterminate. | TrendShows daily activity for this failure pattern in a trailing window anchored to the selected end date. The sparkline covers at least 7 days and at most 14 days, depending on the current window size. | Also inOther environments where the same failure pattern was also detected during the selected window. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
CreateNodePoolFromParam timed out after <minutes> minutes; context deadline exceededfull failure pattern: CreateNodePoolFromParam timed out after <minutes> minutes; context deadline exceeded Signal: IndeterminateSignal: Indeterminate — no prior history; active 4 day(s) — New failure pattern — no prior history | e2e | 4 | 44.44%4 of 9 job runs affected | IndeterminateSignal: Indeterminate — no prior history; active 4 day(s) — New failure pattern — no prior history | Show trend detailsFeb 16: 0 · Feb 17: 0 · Feb 18: 0 · Feb 19: 0 · Feb 20: 0 · Feb 21: 0 · Feb 22: 0 · Feb 23: 0 · Feb 24: 0 · Feb 25: 0 · Feb 26: 8 · Feb 27: 5 · Feb 28: 7 · Mar 1: 6 | DEV | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Full failure examples (3)fail [github.com/Azure/ARO-HCP/test/e2e/cluster_pullsecret.go:112]: Unexpected error:
<*fmt.wrapErrors | 0xc0011cc060>:
failed to create NodePool np-1, caused by: timeout '45.000000' minutes exceeded during CreateNodePoolFromParam for node pool np-1 in resource group pullsecret-test-b4m6br, error: failed waiting for nodepool="np-1" for cluster "pullsecret-hcp-cluster" in resourcegroup="pullsecret-test-b4m6br" to finish creating: context deadline exceeded
{
msg: "failed to create NodePool np-1, caused by: timeout '45.000000' minutes exceeded during CreateNodePoolFromParam for node pool np-1 in resource group pullsecret-test-b4m6br, error: failed waiting for nodepool=\"np-1\" for cluster \"pullsecret-hcp-cluster\" in resourcegroup=\"pullsecret-test-b4m6br\" to finish creating: context deadline exceeded",
errs: [
<*errors.errorString | 0xc000b8c910>{
s: "timeout '45.000000' minutes exceeded during CreateNodePoolFromParam for node pool np-1 in resource group pullsecret-test-b4m6br",
},
<*fmt.wrapError | 0xc00080a6e0>{
msg: "failed waiting for nodepool=\"np-1\" for cluster \"pullsecret-hcp-cluster\" in resourcegroup=\"pullsecret-test-b4m6br\" to finish creating: context deadline exceeded",
err: <context.deadlineExceededError>{},
},
],
}
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_pullsecret.go:112]: Unexpected error:
<*fmt.wrapErrors | 0xc0011cc060>:
failed to create NodePool np-1, caused by: timeout '45.000000' minutes exceeded during CreateNodePoolFromParam for node pool np-1 in resource group pullsecret-test-b4m6br, error: failed waiting for nodepool="np-1" for cluster "pullsecret-hcp-cluster" in resourcegroup="pullsecret-test-b4m6br" to finish creating: context deadline exceeded
{
msg: "failed to create NodePool np-1, caused by: timeout '45.000000' minutes exceeded during CreateNodePoolFromParam for node pool np-1 in resource group pullsecret-test-b4m6br, error: failed waiting for nodepool=\"np-1\" for cluster \"pullsecret-hcp-cluster\" in resourcegroup=\"pullsecret-test-b4m6br\" to finish creating: context deadline exceeded",
errs: [
<*errors.errorString | 0xc000b8c910>{
s: "timeout '45.000000' minutes exceeded during CreateNodePoolFromParam for node pool np-1 in resource group pullsecret-test-b4m6br",
},
<*fmt.wrapError | 0xc00080a6e0>{
msg: "failed waiting for nodepool=\"np-1\" for cluster \"pullsecret-hcp-cluster\" in resourcegroup=\"pullsecret-test-b4m6br\" to finish creating: context deadline exceeded",
err: <context.deadlineExceededError>{},
},
],
}
occurredfail [github.com/Azure/ARO-HCP/test/e2e/cluster_tls_endpoints.go:114]: Unexpected error:
<*fmt.wrapErrors | 0xc000dd08a0>:
failed to create NodePool np-1, caused by: timeout '45.000000' minutes exceeded during CreateNodePoolFromParam for node pool np-1 in resource group tls-endpoint-cluster-5694vf, error: failed waiting for nodepool="np-1" for cluster "tls-endpoint-hcp-cluster" in resourcegroup="tls-endpoint-cluster-5694vf" to finish creating: context deadline exceeded
{
msg: "failed to create NodePool np-1, caused by: timeout '45.000000' minutes exceeded during CreateNodePoolFromParam for node pool np-1 in resource group tls-endpoint-cluster-5694vf, error: failed waiting for nodepool=\"np-1\" for cluster \"tls-endpoint-hcp-cluster\" in resourcegroup=\"tls-endpoint-cluster-5694vf\" to finish creating: context deadline exceeded",
errs: [
<*errors.errorString | 0xc0009e2330>{
s: "timeout '45.000000' minutes exceeded during CreateNodePoolFromParam for node pool np-1 in resource group tls-endpoint-cluster-5694vf",
},
<*fmt.wrapError | 0xc000e04740>{
msg: "failed waiting for nodepool=\"np-1\" for cluster \"tls-endpoint-hcp-cluster\" in resourcegroup=\"tls-endpoint-cluster-5694vf\" to finish creating: context deadline exceeded",
err: <context.deadlineExceededError>{},
},
],
}
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_tls_endpoints.go:114]: Unexpected error:
<*fmt.wrapErrors | 0xc000dd08a0>:
failed to create NodePool np-1, caused by: timeout '45.000000' minutes exceeded during CreateNodePoolFromParam for node pool np-1 in resource group tls-endpoint-cluster-5694vf, error: failed waiting for nodepool="np-1" for cluster "tls-endpoint-hcp-cluster" in resourcegroup="tls-endpoint-cluster-5694vf" to finish creating: context deadline exceeded
{
msg: "failed to create NodePool np-1, caused by: timeout '45.000000' minutes exceeded during CreateNodePoolFromParam for node pool np-1 in resource group tls-endpoint-cluster-5694vf, error: failed waiting for nodepool=\"np-1\" for cluster \"tls-endpoint-hcp-cluster\" in resourcegroup=\"tls-endpoint-cluster-5694vf\" to finish creating: context deadline exceeded",
errs: [
<*errors.errorString | 0xc0009e2330>{
s: "timeout '45.000000' minutes exceeded during CreateNodePoolFromParam for node pool np-1 in resource group tls-endpoint-cluster-5694vf",
},
<*fmt.wrapError | 0xc000e04740>{
msg: "failed waiting for nodepool=\"np-1\" for cluster \"tls-endpoint-hcp-cluster\" in resourcegroup=\"tls-endpoint-cluster-5694vf\" to finish creating: context deadline exceeded",
err: <context.deadlineExceededError>{},
},
],
}
occurredfail [github.com/Azure/ARO-HCP/test/e2e/external_auth_create.go:120]: Unexpected error:
<*fmt.wrapErrors | 0xc000cd2f00>:
failed to create NodePool ea-np-1, caused by: timeout '45.000000' minutes exceeded during CreateNodePoolFromParam for node pool ea-np-1 in resource group external-auth-cluster-dsqvqh, error: failed waiting for nodepool="ea-np-1" for cluster "ea-cluster" in resourcegroup="external-auth-cluster-dsqvqh" to finish creating: context deadline exceeded
{
msg: "failed to create NodePool ea-np-1, caused by: timeout '45.000000' minutes exceeded during CreateNodePoolFromParam for node pool ea-np-1 in resource group external-auth-cluster-dsqvqh, error: failed waiting for nodepool=\"ea-np-1\" for cluster \"ea-cluster\" in resourcegroup=\"external-auth-cluster-dsqvqh\" to finish creating: context deadline exceeded",
errs: [
<*errors.errorString | 0xc001423500>{
s: "timeout '45.000000' minutes exceeded during CreateNodePoolFromParam for node pool ea-np-1 in resource group external-auth-cluster-dsqvqh",
},
<*fmt.wrapError | 0xc000a015a0>{
msg: "failed waiting for nodepool=\"ea-np-1\" for cluster \"ea-cluster\" in resourcegroup=\"external-auth-cluster-dsqvqh\" to finish creating: context deadline exceeded",
err: <context.deadlineExceededError>{},
},
],
}
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/external_auth_create.go:120]: Unexpected error:
<*fmt.wrapErrors | 0xc000cd2f00>:
failed to create NodePool ea-np-1, caused by: timeout '45.000000' minutes exceeded during CreateNodePoolFromParam for node pool ea-np-1 in resource group external-auth-cluster-dsqvqh, error: failed waiting for nodepool="ea-np-1" for cluster "ea-cluster" in resourcegroup="external-auth-cluster-dsqvqh" to finish creating: context deadline exceeded
{
msg: "failed to create NodePool ea-np-1, caused by: timeout '45.000000' minutes exceeded during CreateNodePoolFromParam for node pool ea-np-1 in resource group external-auth-cluster-dsqvqh, error: failed waiting for nodepool=\"ea-np-1\" for cluster \"ea-cluster\" in resourcegroup=\"external-auth-cluster-dsqvqh\" to finish creating: context deadline exceeded",
errs: [
<*errors.errorString | 0xc001423500>{
s: "timeout '45.000000' minutes exceeded during CreateNodePoolFromParam for node pool ea-np-1 in resource group external-auth-cluster-dsqvqh",
},
<*fmt.wrapError | 0xc000a015a0>{
msg: "failed waiting for nodepool=\"ea-np-1\" for cluster \"ea-cluster\" in resourcegroup=\"external-auth-cluster-dsqvqh\" to finish creating: context deadline exceeded",
err: <context.deadlineExceededError>{},
},
],
}
occurredContributing tests (8)
Affected runs (4)
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
CreateOrUpdateExternalAuthAndWait timed out after <minutes> minutes; context deadline exceededfull failure pattern: CreateOrUpdateExternalAuthAndWait timed out after <minutes> minutes; context deadline exceeded Signal: IndeterminateSignal: Indeterminate — no prior history; active 4 day(s) — New failure pattern — no prior history | e2e | 4 | 44.44%4 of 9 job runs affected | IndeterminateSignal: Indeterminate — no prior history; active 4 day(s) — New failure pattern — no prior history | Show trend detailsFeb 16: 0 · Feb 17: 0 · Feb 18: 0 · Feb 19: 0 · Feb 20: 0 · Feb 21: 0 · Feb 22: 0 · Feb 23: 0 · Feb 24: 0 · Feb 25: 0 · Feb 26: 1 · Feb 27: 1 · Feb 28: 1 · Mar 1: 1 | none | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Full failure examples (3)fail [github.com/Azure/ARO-HCP/test/e2e/external_auth_list_and_verify.go:118]: Unexpected error:
<*fmt.wrapErrors | 0xc001106de0>:
failed waiting for external auth "ea-list" in resourcegroup="ea-list-6jmnbl" for cluster="ea-listr2rlb7" to finish, caused by: timeout '15.000000' minutes exceeded during CreateOrUpdateExternalAuthAndWait for external auth ea-list in cluster ea-listr2rlb7 in resource group ea-list-6jmnbl, error: context deadline exceeded
{
msg: "failed waiting for external auth \"ea-list\" in resourcegroup=\"ea-list-6jmnbl\" for cluster=\"ea-listr2rlb7\" to finish, caused by: timeout '15.000000' minutes exceeded during CreateOrUpdateExternalAuthAndWait for external auth ea-list in cluster ea-listr2rlb7 in resource group ea-list-6jmnbl, error: context deadline exceeded",
errs: [
<*errors.errorString | 0xc0008d0390>{
s: "timeout '15.000000' minutes exceeded during CreateOrUpdateExternalAuthAndWait for external auth ea-list in cluster ea-listr2rlb7 in resource group ea-list-6jmnbl",
},
<context.deadlineExceededError>{},
],
}
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/external_auth_list_and_verify.go:118]: Unexpected error:
<*fmt.wrapErrors | 0xc001106de0>:
failed waiting for external auth "ea-list" in resourcegroup="ea-list-6jmnbl" for cluster="ea-listr2rlb7" to finish, caused by: timeout '15.000000' minutes exceeded during CreateOrUpdateExternalAuthAndWait for external auth ea-list in cluster ea-listr2rlb7 in resource group ea-list-6jmnbl, error: context deadline exceeded
{
msg: "failed waiting for external auth \"ea-list\" in resourcegroup=\"ea-list-6jmnbl\" for cluster=\"ea-listr2rlb7\" to finish, caused by: timeout '15.000000' minutes exceeded during CreateOrUpdateExternalAuthAndWait for external auth ea-list in cluster ea-listr2rlb7 in resource group ea-list-6jmnbl, error: context deadline exceeded",
errs: [
<*errors.errorString | 0xc0008d0390>{
s: "timeout '15.000000' minutes exceeded during CreateOrUpdateExternalAuthAndWait for external auth ea-list in cluster ea-listr2rlb7 in resource group ea-list-6jmnbl",
},
<context.deadlineExceededError>{},
],
}
occurredfail [github.com/Azure/ARO-HCP/test/e2e/external_auth_list_and_verify.go:118]: Unexpected error:
<*fmt.wrapErrors | 0xc0014545a0>:
failed waiting for external auth "ea-list" in resourcegroup="ea-list-lfg8pd" for cluster="ea-list45lzgb" to finish, caused by: timeout '15.000000' minutes exceeded during CreateOrUpdateExternalAuthAndWait for external auth ea-list in cluster ea-list45lzgb in resource group ea-list-lfg8pd, error: context deadline exceeded
{
msg: "failed waiting for external auth \"ea-list\" in resourcegroup=\"ea-list-lfg8pd\" for cluster=\"ea-list45lzgb\" to finish, caused by: timeout '15.000000' minutes exceeded during CreateOrUpdateExternalAuthAndWait for external auth ea-list in cluster ea-list45lzgb in resource group ea-list-lfg8pd, error: context deadline exceeded",
errs: [
<*errors.errorString | 0xc000954420>{
s: "timeout '15.000000' minutes exceeded during CreateOrUpdateExternalAuthAndWait for external auth ea-list in cluster ea-list45lzgb in resource group ea-list-lfg8pd",
},
<context.deadlineExceededError>{},
],
}
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/external_auth_list_and_verify.go:118]: Unexpected error:
<*fmt.wrapErrors | 0xc0014545a0>:
failed waiting for external auth "ea-list" in resourcegroup="ea-list-lfg8pd" for cluster="ea-list45lzgb" to finish, caused by: timeout '15.000000' minutes exceeded during CreateOrUpdateExternalAuthAndWait for external auth ea-list in cluster ea-list45lzgb in resource group ea-list-lfg8pd, error: context deadline exceeded
{
msg: "failed waiting for external auth \"ea-list\" in resourcegroup=\"ea-list-lfg8pd\" for cluster=\"ea-list45lzgb\" to finish, caused by: timeout '15.000000' minutes exceeded during CreateOrUpdateExternalAuthAndWait for external auth ea-list in cluster ea-list45lzgb in resource group ea-list-lfg8pd, error: context deadline exceeded",
errs: [
<*errors.errorString | 0xc000954420>{
s: "timeout '15.000000' minutes exceeded during CreateOrUpdateExternalAuthAndWait for external auth ea-list in cluster ea-list45lzgb in resource group ea-list-lfg8pd",
},
<context.deadlineExceededError>{},
],
}
occurredfail [github.com/Azure/ARO-HCP/test/e2e/external_auth_list_and_verify.go:118]: Unexpected error:
<*fmt.wrapErrors | 0xc0013c5b30>:
failed waiting for external auth "ea-list" in resourcegroup="ea-list-dwdpnj" for cluster="ea-list677v96" to finish, caused by: timeout '15.000000' minutes exceeded during CreateOrUpdateExternalAuthAndWait for external auth ea-list in cluster ea-list677v96 in resource group ea-list-dwdpnj, error: context deadline exceeded
{
msg: "failed waiting for external auth \"ea-list\" in resourcegroup=\"ea-list-dwdpnj\" for cluster=\"ea-list677v96\" to finish, caused by: timeout '15.000000' minutes exceeded during CreateOrUpdateExternalAuthAndWait for external auth ea-list in cluster ea-list677v96 in resource group ea-list-dwdpnj, error: context deadline exceeded",
errs: [
<*errors.errorString | 0xc000718220>{
s: "timeout '15.000000' minutes exceeded during CreateOrUpdateExternalAuthAndWait for external auth ea-list in cluster ea-list677v96 in resource group ea-list-dwdpnj",
},
<context.deadlineExceededError>{},
],
}
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/external_auth_list_and_verify.go:118]: Unexpected error:
<*fmt.wrapErrors | 0xc0013c5b30>:
failed waiting for external auth "ea-list" in resourcegroup="ea-list-dwdpnj" for cluster="ea-list677v96" to finish, caused by: timeout '15.000000' minutes exceeded during CreateOrUpdateExternalAuthAndWait for external auth ea-list in cluster ea-list677v96 in resource group ea-list-dwdpnj, error: context deadline exceeded
{
msg: "failed waiting for external auth \"ea-list\" in resourcegroup=\"ea-list-dwdpnj\" for cluster=\"ea-list677v96\" to finish, caused by: timeout '15.000000' minutes exceeded during CreateOrUpdateExternalAuthAndWait for external auth ea-list in cluster ea-list677v96 in resource group ea-list-dwdpnj, error: context deadline exceeded",
errs: [
<*errors.errorString | 0xc000718220>{
s: "timeout '15.000000' minutes exceeded during CreateOrUpdateExternalAuthAndWait for external auth ea-list in cluster ea-list677v96 in resource group ea-list-dwdpnj",
},
<context.deadlineExceededError>{},
],
}
occurredContributing tests (1)
Affected runs (4)
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Deserializaion Error: no output from commandfull failure pattern: Deserializaion Error: no output from command Signal: NoiseSignal: Noise — source deserialization/no-output error — New failure pattern — no prior history | e2e | 3 | 33.33%3 of 9 job runs affected | NoiseSignal: Noise — source deserialization/no-output error — New failure pattern — no prior history | Show trend detailsFeb 16: 0 · Feb 17: 0 · Feb 18: 0 · Feb 19: 0 · Feb 20: 0 · Feb 21: 0 · Feb 22: 0 · Feb 23: 0 · Feb 24: 0 · Feb 25: 0 · Feb 26: 0 · Feb 27: 2 · Feb 28: 3 · Mar 1: 2 | DEV | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Full failure examples (3)custom autoscaling (Spec Runtime: 1h29m59.892s)
/opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/cluster_autoscaling.go:36
In [It] (Node Runtime: 1h29m59.892s)
/opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/cluster_autoscaling.go:36
At [By Step] creating the node pool (Step Runtime: 43m59.075s)
/opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/cluster_autoscaling.go:112
Spec Goroutine
goroutine 96 [select]
github.com/Azure/azure-sdk-for-go/sdk/azcore/internal/shared.Delay({0x820bb58, 0xc000b26190}, 0x77f1f7b?)
/go/pkg/mod/github.com/!azure/azure-sdk-for-go/sdk/azcore@v1.21.0/internal/shared/shared.go:36
github.com/Azure/azure-sdk-for-go/sdk/azcore/runtime.(*Poller[...]).PollUntilDone(0x824f7a0, {0x820bb58, 0xc000b26190}, 0x0)
/go/pkg/mod/github.com/!azure/azure-sdk-for-go/sdk/azcore@v1.21.0/runtime/poller.go:284
github.com/Azure/ARO-HCP/test/util/framework.CreateNodePoolAndWait({0x820bbc8, 0xc00072a1c0}, 0xc000095748, {0xc00127de00, 0x1a}, {0x78273c8, 0x17}, {0x77d2688, 0x4}, {0xc00157e570, ...}, ...)
/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/hcp_helper.go:757
github.com/Azure/ARO-HCP/test/util/framework.(*perItOrDescribeTestContext).CreateNodePoolFromParam(0xc0007867e0, {0x7ffbb42ca660, 0xc000975b00}, {0xc00127de00, 0x1a}, {0x78273c8, 0x17}, {{0x77d5d51, 0x6}, {0x78273c8, ...}, ...}, ...)
/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/deployment_helper.go:384
> github.com/Azure/ARO-HCP/test/e2e.init.func6.2({0x7ffbb42ca660, 0xc000975b00})
/opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/cluster_autoscaling.go:118
| nodePoolParams.Replicas = int32(2)
|
> err = tc.CreateNodePoolFromParam(ctx,
| *resourceGroup.Name,
| customerClusterName,
github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func2({0x82213c0?, 0xc000975b00?})
/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/node.go:465
github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func3()
/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:901
github.com/onsi/ginkgo/v2/internal.(*Suite).runNode in goroutine 93
/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:888
------------------------------
[FAILED] in [It] - /opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/cluster_autoscaling.go:124 @ 03/01/26 03:41:53.656
"ts"="2026-03-01 03:41:53.656795" "level"=0 "msg"="Commiting timing metadata."
"ts"="2026-03-01 03:41:53.656865" "level"=0 "msg"="Dumping deployment operations." "deployment"="customer-infra-autoscaling-hcp-cluster-pfg4sv" "resourceGroup"="autoscaling-cluster-6mbww8"
"ts"="2026-03-01 03:41:53.882690" "level"=0 "msg"="Dumping deployment operations." "deployment"="mi-autoscaling-hcp-cluster-pfg4sv" "resourceGroup"=""
"ts"="2026-03-01 03:41:55.474450" "level"=0 "msg"="Wrote timing metadata" "path"="/tmp/secret/timing-metadata-31a71a35884d87a4a1de36eceeb324011e1c775154472da33ffc40cde64c08cb.yaml.gz"
"ts"="2026-03-01 03:41:55.474569" "level"=0 "msg"="Wrote timing metadata" "path"="/logs/artifacts/test-timing/timing-metadata-31a71a35884d87a4a1de36eceeb324011e1c775154472da33ffc40cde64c08cb.yaml.gz"
"ts"="2026-03-01 03:41:55.474638" "level"=0 "msg"="collecting debug info"
"ts"="2026-03-01 03:41:55.475586" "level"=0 "msg"="collecting deployments" "resourceGroup"="aro-hcp-test-msi-containers-prod-148"
"ts"="2026-03-01 03:41:55.475602" "level"=0 "msg"="collecting deployments" "resourceGroup"="autoscaling-cluster-6mbww8"
"ts"="2026-03-01 03:41:55.655617" "level"=0 "msg"="collecting operations" "resourceGroup"="autoscaling-cluster-6mbww8" "deployment"="pooledNonMsiScopedAssignments-e4ixwyxsfjnje"
"ts"="2026-03-01 03:41:55.814591" "level"=0 "msg"="collecting operations" "resourceGroup"="autoscaling-cluster-6mbww8" "deployment"="customer-infra-autoscaling-hcp-cluster-pfg4sv"
"ts"="2026-03-01 03:41:55.974500" "level"=0 "msg"="collecting operations" "resourceGroup"="aro-hcp-test-msi-containers-prod-148" "deployment"="pooledMsiScopedAssignments"
"ts"="2026-03-01 03:41:56.147010" "level"=0 "msg"="collecting operations" "resourceGroup"="aro-hcp-test-msi-containers-prod-148" "deployment"="msi-deployment-148"
"ts"="2026-03-01 03:41:56.300640" "level"=0 "msg"="finished collecting debug info"
"ts"="2026-03-01 03:41:56.300813" "level"=0 "msg"="deleting created resources"
"ts"="2026-03-01 03:41:56.300956" "level"=0 "msg"="deleting all hcp clusters in resource group" "resourceGroup"="autoscaling-cluster-6mbww8"
Command Error: signal: killed
Deserializaion Error: no output from command
custom autoscaling (Spec Runtime: 1h29m59.892s)
/opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/cluster_autoscaling.go:36
In [It] (Node Runtime: 1h29m59.892s)
/opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/cluster_autoscaling.go:36
At [By Step] creating the node pool (Step Runtime: 43m59.075s)
/opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/cluster_autoscaling.go:112
Spec Goroutine
goroutine 96 [select]
github.com/Azure/azure-sdk-for-go/sdk/azcore/internal/shared.Delay({0x820bb58, 0xc000b26190}, 0x77f1f7b?)
/go/pkg/mod/github.com/!azure/azure-sdk-for-go/sdk/azcore@v1.21.0/internal/shared/shared.go:36
github.com/Azure/azure-sdk-for-go/sdk/azcore/runtime.(*Poller[...]).PollUntilDone(0x824f7a0, {0x820bb58, 0xc000b26190}, 0x0)
/go/pkg/mod/github.com/!azure/azure-sdk-for-go/sdk/azcore@v1.21.0/runtime/poller.go:284
github.com/Azure/ARO-HCP/test/util/framework.CreateNodePoolAndWait({0x820bbc8, 0xc00072a1c0}, 0xc000095748, {0xc00127de00, 0x1a}, {0x78273c8, 0x17}, {0x77d2688, 0x4}, {0xc00157e570, ...}, ...)
/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/hcp_helper.go:757
github.com/Azure/ARO-HCP/test/util/framework.(*perItOrDescribeTestContext).CreateNodePoolFromParam(0xc0007867e0, {0x7ffbb42ca660, 0xc000975b00}, {0xc00127de00, 0x1a}, {0x78273c8, 0x17}, {{0x77d5d51, 0x6}, {0x78273c8, ...}, ...}, ...)
/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/deployment_helper.go:384
> github.com/Azure/ARO-HCP/test/e2e.init.func6.2({0x7ffbb42ca660, 0xc000975b00})
/opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/cluster_autoscaling.go:118
| nodePoolParams.Replicas = int32(2)
|
> err = tc.CreateNodePoolFromParam(ctx,
| *resourceGroup.Name,
| customerClusterName,
github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func2({0x82213c0?, 0xc000975b00?})
/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/node.go:465
github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func3()
/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:901
github.com/onsi/ginkgo/v2/internal.(*Suite).runNode in goroutine 93
/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:888
------------------------------
[FAILED] in [It] - /opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/cluster_autoscaling.go:124 @ 03/01/26 03:41:53.656
"ts"="2026-03-01 03:41:53.656795" "level"=0 "msg"="Commiting timing metadata."
"ts"="2026-03-01 03:41:53.656865" "level"=0 "msg"="Dumping deployment operations." "deployment"="customer-infra-autoscaling-hcp-cluster-pfg4sv" "resourceGroup"="autoscaling-cluster-6mbww8"
"ts"="2026-03-01 03:41:53.882690" "level"=0 "msg"="Dumping deployment operations." "deployment"="mi-autoscaling-hcp-cluster-pfg4sv" "resourceGroup"=""
"ts"="2026-03-01 03:41:55.474450" "level"=0 "msg"="Wrote timing metadata" "path"="/tmp/secret/timing-metadata-31a71a35884d87a4a1de36eceeb324011e1c775154472da33ffc40cde64c08cb.yaml.gz"
"ts"="2026-03-01 03:41:55.474569" "level"=0 "msg"="Wrote timing metadata" "path"="/logs/artifacts/test-timing/timing-metadata-31a71a35884d87a4a1de36eceeb324011e1c775154472da33ffc40cde64c08cb.yaml.gz"
"ts"="2026-03-01 03:41:55.474638" "level"=0 "msg"="collecting debug info"
"ts"="2026-03-01 03:41:55.475586" "level"=0 "msg"="collecting deployments" "resourceGroup"="aro-hcp-test-msi-containers-prod-148"
"ts"="2026-03-01 03:41:55.475602" "level"=0 "msg"="collecting deployments" "resourceGroup"="autoscaling-cluster-6mbww8"
"ts"="2026-03-01 03:41:55.655617" "level"=0 "msg"="collecting operations" "resourceGroup"="autoscaling-cluster-6mbww8" "deployment"="pooledNonMsiScopedAssignments-e4ixwyxsfjnje"
"ts"="2026-03-01 03:41:55.814591" "level"=0 "msg"="collecting operations" "resourceGroup"="autoscaling-cluster-6mbww8" "deployment"="customer-infra-autoscaling-hcp-cluster-pfg4sv"
"ts"="2026-03-01 03:41:55.974500" "level"=0 "msg"="collecting operations" "resourceGroup"="aro-hcp-test-msi-containers-prod-148" "deployment"="pooledMsiScopedAssignments"
"ts"="2026-03-01 03:41:56.147010" "level"=0 "msg"="collecting operations" "resourceGroup"="aro-hcp-test-msi-containers-prod-148" "deployment"="msi-deployment-148"
"ts"="2026-03-01 03:41:56.300640" "level"=0 "msg"="finished collecting debug info"
"ts"="2026-03-01 03:41:56.300813" "level"=0 "msg"="deleting created resources"
"ts"="2026-03-01 03:41:56.300956" "level"=0 "msg"="deleting all hcp clusters in resource group" "resourceGroup"="autoscaling-cluster-6mbww8"
Command Error: signal: killed
Deserializaion Error: no output from commandodepool with autoscaling enabled (Spec Runtime: 1h29m59.958s)
/opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/nodepool_autoscaling.go:36
In [It] (Node Runtime: 1h29m59.958s)
/opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/nodepool_autoscaling.go:36
At [By Step] creating the autoscaling nodepool (Step Runtime: 19m22.214s)
/opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/nodepool_autoscaling.go:95
Spec Goroutine
goroutine 103 [select]
github.com/Azure/azure-sdk-for-go/sdk/azcore/internal/shared.Delay({0x820bb58, 0xc0008fc370}, 0x77f1f7b?)
/go/pkg/mod/github.com/!azure/azure-sdk-for-go/sdk/azcore@v1.21.0/internal/shared/shared.go:36
github.com/Azure/azure-sdk-for-go/sdk/azcore/runtime.(*Poller[...]).PollUntilDone(0x824f7a0, {0x820bb58, 0xc0008fc370}, 0x0)
/go/pkg/mod/github.com/!azure/azure-sdk-for-go/sdk/azcore@v1.21.0/runtime/poller.go:284
github.com/Azure/ARO-HCP/test/util/framework.CreateNodePoolAndWait({0x820bbc8, 0xc000578930}, 0xc000095788, {0xc000f0ee10, 0x15}, {0x78183e3, 0x14}, {0x77f1f57, 0xc}, {0xc001523630, ...}, ...)
/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/hcp_helper.go:757
github.com/Azure/ARO-HCP/test/util/framework.(*perItOrDescribeTestContext).CreateNodePoolFromParam(0xc0007b37a0, {0x7f78b44bda60, 0xc000a0b200}, {0xc000f0ee10, 0x15}, {0x78183e3, 0x14}, {{0x77d5d51, 0x6}, {0x78183e3, ...}, ...}, ...)
/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/deployment_helper.go:384
> github.com/Azure/ARO-HCP/test/e2e.init.func23.2({0x7f78b44bda60, 0xc000a0b200})
/opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/nodepool_autoscaling.go:96
|
| By("creating the autoscaling nodepool")
> err = tc.CreateNodePoolFromParam(ctx,
| *resourceGroup.Name,
| customerClusterName,
github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func2({0x82213c0?, 0xc000a0b200?})
/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/node.go:465
github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func3()
/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:901
github.com/onsi/ginkgo/v2/internal.(*Suite).runNode in goroutine 100
/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:888
------------------------------
[FAILED] in [It] - /opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/nodepool_autoscaling.go:102 @ 03/01/26 03:41:54.92
"ts"="2026-03-01 03:41:54.920840" "level"=0 "msg"="Commiting timing metadata."
"ts"="2026-03-01 03:41:54.920909" "level"=0 "msg"="Dumping deployment operations." "deployment"="customer-infra-np-autoscale-cluster-q7mds5" "resourceGroup"="np-autoscaling-ntl9rp"
"ts"="2026-03-01 03:41:55.097361" "level"=0 "msg"="Dumping deployment operations." "deployment"="mi-np-autoscale-cluster-q7mds5" "resourceGroup"=""
"ts"="2026-03-01 03:41:56.239629" "level"=0 "msg"="Wrote timing metadata" "path"="/tmp/secret/timing-metadata-3ace844abea5a4d3a28ffc64801a1d716d83f775e2e8d3dceadd0cb72b88474b.yaml.gz"
"ts"="2026-03-01 03:41:56.239729" "level"=0 "msg"="Wrote timing metadata" "path"="/logs/artifacts/test-timing/timing-metadata-3ace844abea5a4d3a28ffc64801a1d716d83f775e2e8d3dceadd0cb72b88474b.yaml.gz"
"ts"="2026-03-01 03:41:56.239818" "level"=0 "msg"="collecting debug info"
"ts"="2026-03-01 03:41:56.240822" "level"=0 "msg"="collecting deployments" "resourceGroup"="aro-hcp-test-msi-containers-prod-116"
"ts"="2026-03-01 03:41:56.240836" "level"=0 "msg"="collecting deployments" "resourceGroup"="np-autoscaling-ntl9rp"
"ts"="2026-03-01 03:41:56.385178" "level"=0 "msg"="collecting operations" "resourceGroup"="aro-hcp-test-msi-containers-prod-116" "deployment"="pooledMsiScopedAssignments"
"ts"="2026-03-01 03:41:56.408437" "level"=0 "msg"="collecting operations" "resourceGroup"="np-autoscaling-ntl9rp" "deployment"="pooledNonMsiScopedAssignments-ssrwqrnhyyabs"
"ts"="2026-03-01 03:41:56.534892" "level"=0 "msg"="collecting operations" "resourceGroup"="aro-hcp-test-msi-containers-prod-116" "deployment"="msi-deployment-116"
"ts"="2026-03-01 03:41:56.745301" "level"=0 "msg"="collecting operations" "resourceGroup"="np-autoscaling-ntl9rp" "deployment"="customer-infra-np-autoscale-cluster-q7mds5"
"ts"="2026-03-01 03:41:56.911097" "level"=0 "msg"="finished collecting debug info"
"ts"="2026-03-01 03:41:56.911200" "level"=0 "msg"="deleting created resources"
"ts"="2026-03-01 03:41:56.911310" "level"=0 "msg"="deleting all hcp clusters in resource group" "resourceGroup"="np-autoscaling-ntl9rp"
Command Error: signal: killed
Deserializaion Error: no output from command
odepool with autoscaling enabled (Spec Runtime: 1h29m59.958s)
/opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/nodepool_autoscaling.go:36
In [It] (Node Runtime: 1h29m59.958s)
/opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/nodepool_autoscaling.go:36
At [By Step] creating the autoscaling nodepool (Step Runtime: 19m22.214s)
/opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/nodepool_autoscaling.go:95
Spec Goroutine
goroutine 103 [select]
github.com/Azure/azure-sdk-for-go/sdk/azcore/internal/shared.Delay({0x820bb58, 0xc0008fc370}, 0x77f1f7b?)
/go/pkg/mod/github.com/!azure/azure-sdk-for-go/sdk/azcore@v1.21.0/internal/shared/shared.go:36
github.com/Azure/azure-sdk-for-go/sdk/azcore/runtime.(*Poller[...]).PollUntilDone(0x824f7a0, {0x820bb58, 0xc0008fc370}, 0x0)
/go/pkg/mod/github.com/!azure/azure-sdk-for-go/sdk/azcore@v1.21.0/runtime/poller.go:284
github.com/Azure/ARO-HCP/test/util/framework.CreateNodePoolAndWait({0x820bbc8, 0xc000578930}, 0xc000095788, {0xc000f0ee10, 0x15}, {0x78183e3, 0x14}, {0x77f1f57, 0xc}, {0xc001523630, ...}, ...)
/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/hcp_helper.go:757
github.com/Azure/ARO-HCP/test/util/framework.(*perItOrDescribeTestContext).CreateNodePoolFromParam(0xc0007b37a0, {0x7f78b44bda60, 0xc000a0b200}, {0xc000f0ee10, 0x15}, {0x78183e3, 0x14}, {{0x77d5d51, 0x6}, {0x78183e3, ...}, ...}, ...)
/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/deployment_helper.go:384
> github.com/Azure/ARO-HCP/test/e2e.init.func23.2({0x7f78b44bda60, 0xc000a0b200})
/opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/nodepool_autoscaling.go:96
|
| By("creating the autoscaling nodepool")
> err = tc.CreateNodePoolFromParam(ctx,
| *resourceGroup.Name,
| customerClusterName,
github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func2({0x82213c0?, 0xc000a0b200?})
/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/node.go:465
github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func3()
/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:901
github.com/onsi/ginkgo/v2/internal.(*Suite).runNode in goroutine 100
/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:888
------------------------------
[FAILED] in [It] - /opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/nodepool_autoscaling.go:102 @ 03/01/26 03:41:54.92
"ts"="2026-03-01 03:41:54.920840" "level"=0 "msg"="Commiting timing metadata."
"ts"="2026-03-01 03:41:54.920909" "level"=0 "msg"="Dumping deployment operations." "deployment"="customer-infra-np-autoscale-cluster-q7mds5" "resourceGroup"="np-autoscaling-ntl9rp"
"ts"="2026-03-01 03:41:55.097361" "level"=0 "msg"="Dumping deployment operations." "deployment"="mi-np-autoscale-cluster-q7mds5" "resourceGroup"=""
"ts"="2026-03-01 03:41:56.239629" "level"=0 "msg"="Wrote timing metadata" "path"="/tmp/secret/timing-metadata-3ace844abea5a4d3a28ffc64801a1d716d83f775e2e8d3dceadd0cb72b88474b.yaml.gz"
"ts"="2026-03-01 03:41:56.239729" "level"=0 "msg"="Wrote timing metadata" "path"="/logs/artifacts/test-timing/timing-metadata-3ace844abea5a4d3a28ffc64801a1d716d83f775e2e8d3dceadd0cb72b88474b.yaml.gz"
"ts"="2026-03-01 03:41:56.239818" "level"=0 "msg"="collecting debug info"
"ts"="2026-03-01 03:41:56.240822" "level"=0 "msg"="collecting deployments" "resourceGroup"="aro-hcp-test-msi-containers-prod-116"
"ts"="2026-03-01 03:41:56.240836" "level"=0 "msg"="collecting deployments" "resourceGroup"="np-autoscaling-ntl9rp"
"ts"="2026-03-01 03:41:56.385178" "level"=0 "msg"="collecting operations" "resourceGroup"="aro-hcp-test-msi-containers-prod-116" "deployment"="pooledMsiScopedAssignments"
"ts"="2026-03-01 03:41:56.408437" "level"=0 "msg"="collecting operations" "resourceGroup"="np-autoscaling-ntl9rp" "deployment"="pooledNonMsiScopedAssignments-ssrwqrnhyyabs"
"ts"="2026-03-01 03:41:56.534892" "level"=0 "msg"="collecting operations" "resourceGroup"="aro-hcp-test-msi-containers-prod-116" "deployment"="msi-deployment-116"
"ts"="2026-03-01 03:41:56.745301" "level"=0 "msg"="collecting operations" "resourceGroup"="np-autoscaling-ntl9rp" "deployment"="customer-infra-np-autoscale-cluster-q7mds5"
"ts"="2026-03-01 03:41:56.911097" "level"=0 "msg"="finished collecting debug info"
"ts"="2026-03-01 03:41:56.911200" "level"=0 "msg"="deleting created resources"
"ts"="2026-03-01 03:41:56.911310" "level"=0 "msg"="deleting all hcp clusters in resource group" "resourceGroup"="np-autoscaling-ntl9rp"
Command Error: signal: killed
Deserializaion Error: no output from commandeanup and reporting nodes but will skip all remaining specs. Interrupt again to skip cleanup.
Here's a current progress report:
Customer should be able to create an HCP cluster with back-level version 4.19 (Spec Runtime: 1h29m59.881s)
/opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/cluster_version_backlevel.go:48
In [It] (Node Runtime: 1h29m59.881s)
/opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/cluster_version_backlevel.go:48
At [By Step] creating node pool version 4.19.7 and verifying a simple web app can run (Step Runtime: 44m14.857s)
/opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/cluster_version_backlevel.go:174
Spec Goroutine
goroutine 46 [select]
github.com/Azure/azure-sdk-for-go/sdk/azcore/internal/shared.Delay({0x820bba8, 0xc0005f2af0}, 0x77f1f7b?)
/go/pkg/mod/github.com/!azure/azure-sdk-for-go/sdk/azcore@v1.21.0/internal/shared/shared.go:36
github.com/Azure/azure-sdk-for-go/sdk/azcore/runtime.(*Poller[...]).PollUntilDone(0x824f780, {0x820bba8, 0xc0005f2af0}, 0x0)
/go/pkg/mod/github.com/!azure/azure-sdk-for-go/sdk/azcore@v1.21.0/runtime/poller.go:284
github.com/Azure/ARO-HCP/test/util/framework.CreateNodePoolAndWait({0x7fb640315f60, 0xc000e897a0}, 0xc000095d10, {0xc0013f40e0, 0x1e}, {0xc0013da330, 0x10}, {0xc000a3e680, 0xd}, {0xc000e3a4e0, ...}, ...)
/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/hcp_helper.go:757
> github.com/Azure/ARO-HCP/test/e2e.init.func15.1({0x7fb640315f60, 0xc000e897a0})
/opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/cluster_version_backlevel.go:182
| Expect(err).NotTo(HaveOccurred())
| nodePoolClient := tc.Get20240610ClientFactoryOrDie(ctx).NewNodePoolsClient()
> _, err = framework.CreateNodePoolAndWait(ctx,
| nodePoolClient,
| *resourceGroup.Name,
github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func2({0x82213a0?, 0xc000e897a0?})
/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/node.go:465
github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func3()
/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:901
github.com/onsi/ginkgo/v2/internal.(*Suite).runNode in goroutine 87
/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:888
------------------------------
[FAILED] in [It] - /opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/cluster_version_backlevel.go:190 @ 02/28/26 03:33:42.095
"ts"="2026-02-28 03:33:42.095647" "level"=0 "msg"="Commiting timing metadata."
"ts"="2026-02-28 03:33:42.095721" "level"=0 "msg"="Dumping deployment operations." "deployment"="customer-infra-cluster-ver-4-19-sqgvfd" "resourceGroup"="rg-cluster-back-version-phh9hp"
"ts"="2026-02-28 03:33:42.266832" "level"=0 "msg"="Dumping deployment operations." "deployment"="mi-cluster-ver-4-19-rhxxsh" "resourceGroup"=""
"ts"="2026-02-28 03:33:44.552401" "level"=0 "msg"="Wrote timing metadata" "path"="/tmp/secret/timing-metadata-b47f269c3c8e3f0a236ed67162dd7947c6f274b6961894c6582aefea4761a7f3.yaml.gz"
"ts"="2026-02-28 03:33:44.552493" "level"=0 "msg"="Wrote timing metadata" "path"="/logs/artifacts/test-timing/timing-metadata-b47f269c3c8e3f0a236ed67162dd7947c6f274b6961894c6582aefea4761a7f3.yaml.gz"
"ts"="2026-02-28 03:33:44.552551" "level"=0 "msg"="collecting debug info"
"ts"="2026-02-28 03:33:44.553626" "level"=0 "msg"="collecting deployments" "resourceGroup"="aro-hcp-test-msi-containers-prod-75"
"ts"="2026-02-28 03:33:44.553643" "level"=0 "msg"="collecting deployments" "resourceGroup"="rg-cluster-back-version-phh9hp"
"ts"="2026-02-28 03:33:44.694802" "level"=0 "msg"="collecting operations" "resourceGroup"="aro-hcp-test-msi-containers-prod-75" "deployment"="pooledMsiScopedAssignments"
"ts"="2026-02-28 03:33:44.701210" "level"=0 "msg"="collecting operations" "resourceGroup"="rg-cluster-back-version-phh9hp" "deployment"="pooledNonMsiScopedAssignments"
"ts"="2026-02-28 03:33:44.845805" "level"=0 "msg"="collecting operations" "resourceGroup"="rg-cluster-back-version-phh9hp" "deployment"="customer-infra-cluster-ver-4-19-sqgvfd"
"ts"="2026-02-28 03:33:44.893842" "level"=0 "msg"="collecting operations" "resourceGroup"="aro-hcp-test-msi-containers-prod-75" "deployment"="msi-deployment-75"
"ts"="2026-02-28 03:33:45.046661" "level"=0 "msg"="finished collecting debug info"
"ts"="2026-02-28 03:33:45.046802" "level"=0 "msg"="deleting created resources"
"ts"="2026-02-28 03:33:45.046928" "level"=0 "msg"="deleting all hcp clusters in resource group" "resourceGroup"="rg-cluster-back-version-phh9hp"
Command Error: signal: killed
Deserializaion Error: no output from command
eanup and reporting nodes but will skip all remaining specs. Interrupt again to skip cleanup.
Here's a current progress report:
Customer should be able to create an HCP cluster with back-level version 4.19 (Spec Runtime: 1h29m59.881s)
/opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/cluster_version_backlevel.go:48
In [It] (Node Runtime: 1h29m59.881s)
/opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/cluster_version_backlevel.go:48
At [By Step] creating node pool version 4.19.7 and verifying a simple web app can run (Step Runtime: 44m14.857s)
/opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/cluster_version_backlevel.go:174
Spec Goroutine
goroutine 46 [select]
github.com/Azure/azure-sdk-for-go/sdk/azcore/internal/shared.Delay({0x820bba8, 0xc0005f2af0}, 0x77f1f7b?)
/go/pkg/mod/github.com/!azure/azure-sdk-for-go/sdk/azcore@v1.21.0/internal/shared/shared.go:36
github.com/Azure/azure-sdk-for-go/sdk/azcore/runtime.(*Poller[...]).PollUntilDone(0x824f780, {0x820bba8, 0xc0005f2af0}, 0x0)
/go/pkg/mod/github.com/!azure/azure-sdk-for-go/sdk/azcore@v1.21.0/runtime/poller.go:284
github.com/Azure/ARO-HCP/test/util/framework.CreateNodePoolAndWait({0x7fb640315f60, 0xc000e897a0}, 0xc000095d10, {0xc0013f40e0, 0x1e}, {0xc0013da330, 0x10}, {0xc000a3e680, 0xd}, {0xc000e3a4e0, ...}, ...)
/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/hcp_helper.go:757
> github.com/Azure/ARO-HCP/test/e2e.init.func15.1({0x7fb640315f60, 0xc000e897a0})
/opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/cluster_version_backlevel.go:182
| Expect(err).NotTo(HaveOccurred())
| nodePoolClient := tc.Get20240610ClientFactoryOrDie(ctx).NewNodePoolsClient()
> _, err = framework.CreateNodePoolAndWait(ctx,
| nodePoolClient,
| *resourceGroup.Name,
github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func2({0x82213a0?, 0xc000e897a0?})
/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/node.go:465
github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func3()
/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:901
github.com/onsi/ginkgo/v2/internal.(*Suite).runNode in goroutine 87
/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:888
------------------------------
[FAILED] in [It] - /opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/cluster_version_backlevel.go:190 @ 02/28/26 03:33:42.095
"ts"="2026-02-28 03:33:42.095647" "level"=0 "msg"="Commiting timing metadata."
"ts"="2026-02-28 03:33:42.095721" "level"=0 "msg"="Dumping deployment operations." "deployment"="customer-infra-cluster-ver-4-19-sqgvfd" "resourceGroup"="rg-cluster-back-version-phh9hp"
"ts"="2026-02-28 03:33:42.266832" "level"=0 "msg"="Dumping deployment operations." "deployment"="mi-cluster-ver-4-19-rhxxsh" "resourceGroup"=""
"ts"="2026-02-28 03:33:44.552401" "level"=0 "msg"="Wrote timing metadata" "path"="/tmp/secret/timing-metadata-b47f269c3c8e3f0a236ed67162dd7947c6f274b6961894c6582aefea4761a7f3.yaml.gz"
"ts"="2026-02-28 03:33:44.552493" "level"=0 "msg"="Wrote timing metadata" "path"="/logs/artifacts/test-timing/timing-metadata-b47f269c3c8e3f0a236ed67162dd7947c6f274b6961894c6582aefea4761a7f3.yaml.gz"
"ts"="2026-02-28 03:33:44.552551" "level"=0 "msg"="collecting debug info"
"ts"="2026-02-28 03:33:44.553626" "level"=0 "msg"="collecting deployments" "resourceGroup"="aro-hcp-test-msi-containers-prod-75"
"ts"="2026-02-28 03:33:44.553643" "level"=0 "msg"="collecting deployments" "resourceGroup"="rg-cluster-back-version-phh9hp"
"ts"="2026-02-28 03:33:44.694802" "level"=0 "msg"="collecting operations" "resourceGroup"="aro-hcp-test-msi-containers-prod-75" "deployment"="pooledMsiScopedAssignments"
"ts"="2026-02-28 03:33:44.701210" "level"=0 "msg"="collecting operations" "resourceGroup"="rg-cluster-back-version-phh9hp" "deployment"="pooledNonMsiScopedAssignments"
"ts"="2026-02-28 03:33:44.845805" "level"=0 "msg"="collecting operations" "resourceGroup"="rg-cluster-back-version-phh9hp" "deployment"="customer-infra-cluster-ver-4-19-sqgvfd"
"ts"="2026-02-28 03:33:44.893842" "level"=0 "msg"="collecting operations" "resourceGroup"="aro-hcp-test-msi-containers-prod-75" "deployment"="msi-deployment-75"
"ts"="2026-02-28 03:33:45.046661" "level"=0 "msg"="finished collecting debug info"
"ts"="2026-02-28 03:33:45.046802" "level"=0 "msg"="deleting created resources"
"ts"="2026-02-28 03:33:45.046928" "level"=0 "msg"="deleting all hcp clusters in resource group" "resourceGroup"="rg-cluster-back-version-phh9hp"
Command Error: signal: killed
Deserializaion Error: no output from commandContributing tests (5)
Affected runs (3)
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
ERROR CODE: LocationNotAvailableForResourceType; detail message The provided location 'eastus2euap' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClust...full failure pattern: ERROR CODE: LocationNotAvailableForResourceType; detail message The provided location 'eastus2euap' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'uksouth,switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2'.; provider Microsoft.RedHatOpenShift Signal: IndeterminateSignal: Indeterminate — no prior history; active 1 day(s) — New failure pattern — no prior history | e2e | 2 | 22.22%2 of 9 job runs affected | IndeterminateSignal: Indeterminate — no prior history; active 1 day(s) — New failure pattern — no prior history | Show trend detailsFeb 16: 0 · Feb 17: 0 · Feb 18: 0 · Feb 19: 0 · Feb 20: 0 · Feb 21: 0 · Feb 22: 0 · Feb 23: 0 · Feb 24: 0 · Feb 25: 38 · Feb 26: 0 · Feb 27: 0 · Feb 28: 0 · Mar 1: 0 | none | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Full failure examples (3)fail [github.com/Azure/ARO-HCP/test/e2e/clusters_sharing_resgroup.go:111]: Unexpected error:
<*fmt.wrapError | 0xc00100d260>:
failed starting cluster creation "basic-hcp-cluster" in resourcegroup="customer-rg-jjg55m": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/customer-rg-jjg55m/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/basic-hcp-cluster
--------------------------------------------------------------------------------
RESPONSE 400: 400 Bad Request
ERROR CODE: LocationNotAvailableForResourceType
--------------------------------------------------------------------------------
{
"error": {
"code": "LocationNotAvailableForResourceType",
"message": "The provided location 'eastus2euap' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'uksouth,switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2'."
}
}
--------------------------------------------------------------------------------
{
msg: "failed starting cluster creation \"basic-hcp-cluster\" in resourcegroup=\"customer-rg-jjg55m\": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/customer-rg-jjg55m/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/basic-hcp-cluster\n--------------------------------------------------------------------------------\nRESPONSE 400: 400 Bad Request\nERROR CODE: LocationNotAvailableForResourceType\n--------------------------------------------------------------------------------\n{\n \"error\": {\n \"code\": \"LocationNotAvailableForResourceType\",\n \"message\": \"The provided location 'eastus2euap' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'uksouth,switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2'.\"\n }\n}\n--------------------------------------------------------------------------------\n",
err: <*exported.ResponseError | 0xc00115c930>{
ErrorCode: "LocationNotAvailableForResourceType",
StatusCode: 400,
RawResponse: {
Status: "400 Bad Request",
StatusCode: 400,
Proto: "HTTP/2.0",
ProtoMajor: 2,
ProtoMinor: 0,
Header: {
"Cache-Control": ["no-cache"],
"Content-Length": ["340"],
"X-Ms-Failure-Cause": ["gateway"],
"Strict-Transport-Security": [
"max-age=31536000; includeSubDomains",
],
"X-Content-Type-Options": ["nosniff"],
"X-Cache": ["CONFIG_NOCACHE"],
"X-Msedge-Ref": [
"Ref A: 5EFCEE0887284F3B96A38479106B14B8 Ref B: CH1AA2020603011 Ref C: 2026-02-25T23:32:31Z",
],
"Date": [
"Wed, 25 Feb 2026 23:32:30 GMT",
],
"Pragma": ["no-cache"],
"Content-Type": [
"application/json; charset=utf-8",
],
"Expires": ["-1"],
"X-Ms-Request-Id": [
"934b872d-6075-43da-b3b7-6b123a1067ff",
],
"X-Ms-Correlation-Request-Id": [
"934b872d-6075-43da-b3b7-6b123a1067ff",
],
"X-Ms-Routing-Request-Id": [
"EASTUS2EUAP:20260225T233231Z:934b872d-6075-43da-b3b7-6b123a1067ff",
],
},
Body: <*exported.nopClosingBytesReader | 0xc00100d160>{
s: "{\"error\":{\"code\":\"LocationNotAvailableForResourceType\",\"message\":\"The provided location 'eastus2euap' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'uksouth,switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2'.\"}}",
i: 0,
},
ContentLength: 340,
TransferEncoding: nil,
Close: false,
Uncompressed: false,
Trailer: nil,
Request: {
Method: "PUT",
URL: {
Scheme: "https",
Opaque: "",
User: nil,
Host: "management.azure.com",
Path: "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/customer-rg-jjg55m/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/basic-hcp-cluster",
RawPath: "",
OmitHost: false,
ForceQuery: false,
RawQuery: "a...
Gomega truncated this representation as it exceeds 'format.MaxLength'.
Consider having the object provide a custom 'GomegaStringer' representation
or adjust the parameters in Gomega's 'format' package.
Learn more here: https://onsi.github.io/gomega/#adjusting-output
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/clusters_sharing_resgroup.go:111]: Unexpected error:
<*fmt.wrapError | 0xc00100d260>:
failed starting cluster creation "basic-hcp-cluster" in resourcegroup="customer-rg-jjg55m": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/customer-rg-jjg55m/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/basic-hcp-cluster
--------------------------------------------------------------------------------
RESPONSE 400: 400 Bad Request
ERROR CODE: LocationNotAvailableForResourceType
--------------------------------------------------------------------------------
{
"error": {
"code": "LocationNotAvailableForResourceType",
"message": "The provided location 'eastus2euap' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'uksouth,switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2'."
}
}
--------------------------------------------------------------------------------
{
msg: "failed starting cluster creation \"basic-hcp-cluster\" in resourcegroup=\"customer-rg-jjg55m\": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/customer-rg-jjg55m/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/basic-hcp-cluster\n--------------------------------------------------------------------------------\nRESPONSE 400: 400 Bad Request\nERROR CODE: LocationNotAvailableForResourceType\n--------------------------------------------------------------------------------\n{\n \"error\": {\n \"code\": \"LocationNotAvailableForResourceType\",\n \"message\": \"The provided location 'eastus2euap' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'uksouth,switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2'.\"\n }\n}\n--------------------------------------------------------------------------------\n",
err: <*exported.ResponseError | 0xc00115c930>{
ErrorCode: "LocationNotAvailableForResourceType",
StatusCode: 400,
RawResponse: {
Status: "400 Bad Request",
StatusCode: 400,
Proto: "HTTP/2.0",
ProtoMajor: 2,
ProtoMinor: 0,
Header: {
"Cache-Control": ["no-cache"],
"Content-Length": ["340"],
"X-Ms-Failure-Cause": ["gateway"],
"Strict-Transport-Security": [
"max-age=31536000; includeSubDomains",
],
"X-Content-Type-Options": ["nosniff"],
"X-Cache": ["CONFIG_NOCACHE"],
"X-Msedge-Ref": [
"Ref A: 5EFCEE0887284F3B96A38479106B14B8 Ref B: CH1AA2020603011 Ref C: 2026-02-25T23:32:31Z",
],
"Date": [
"Wed, 25 Feb 2026 23:32:30 GMT",
],
"Pragma": ["no-cache"],
"Content-Type": [
"application/json; charset=utf-8",
],
"Expires": ["-1"],
"X-Ms-Request-Id": [
"934b872d-6075-43da-b3b7-6b123a1067ff",
],
"X-Ms-Correlation-Request-Id": [
"934b872d-6075-43da-b3b7-6b123a1067ff",
],
"X-Ms-Routing-Request-Id": [
"EASTUS2EUAP:20260225T233231Z:934b872d-6075-43da-b3b7-6b123a1067ff",
],
},
Body: <*exported.nopClosingBytesReader | 0xc00100d160>{
s: "{\"error\":{\"code\":\"LocationNotAvailableForResourceType\",\"message\":\"The provided location 'eastus2euap' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'uksouth,switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2'.\"}}",
i: 0,
},
ContentLength: 340,
TransferEncoding: nil,
Close: false,
Uncompressed: false,
Trailer: nil,
Request: {
Method: "PUT",
URL: {
Scheme: "https",
Opaque: "",
User: nil,
Host: "management.azure.com",
Path: "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/customer-rg-jjg55m/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/basic-hcp-cluster",
RawPath: "",
OmitHost: false,
ForceQuery: false,
RawQuery: "a...
Gomega truncated this representation as it exceeds 'format.MaxLength'.
Consider having the object provide a custom 'GomegaStringer' representation
or adjust the parameters in Gomega's 'format' package.
Learn more here: https://onsi.github.io/gomega/#adjusting-output
occurredfail [github.com/Azure/ARO-HCP/test/e2e/cluster_update.go:84]: Unexpected error:
<*fmt.wrapError | 0xc000a51ae0>:
failed to create HCP cluster patch-name-cluster: failed starting cluster creation "patch-name-cluster" in resourcegroup="patch-name-ds6vvt": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/patch-name-ds6vvt/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/patch-name-cluster
--------------------------------------------------------------------------------
RESPONSE 400: 400 Bad Request
ERROR CODE: LocationNotAvailableForResourceType
--------------------------------------------------------------------------------
{
"error": {
"code": "LocationNotAvailableForResourceType",
"message": "The provided location 'eastus2euap' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'uksouth,switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2'."
}
}
--------------------------------------------------------------------------------
{
msg: "failed to create HCP cluster patch-name-cluster: failed starting cluster creation \"patch-name-cluster\" in resourcegroup=\"patch-name-ds6vvt\": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/patch-name-ds6vvt/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/patch-name-cluster\n--------------------------------------------------------------------------------\nRESPONSE 400: 400 Bad Request\nERROR CODE: LocationNotAvailableForResourceType\n--------------------------------------------------------------------------------\n{\n \"error\": {\n \"code\": \"LocationNotAvailableForResourceType\",\n \"message\": \"The provided location 'eastus2euap' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'uksouth,switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2'.\"\n }\n}\n--------------------------------------------------------------------------------\n",
err: <*fmt.wrapError | 0xc000a51ac0>{
msg: "failed starting cluster creation \"patch-name-cluster\" in resourcegroup=\"patch-name-ds6vvt\": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/patch-name-ds6vvt/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/patch-name-cluster\n--------------------------------------------------------------------------------\nRESPONSE 400: 400 Bad Request\nERROR CODE: LocationNotAvailableForResourceType\n--------------------------------------------------------------------------------\n{\n \"error\": {\n \"code\": \"LocationNotAvailableForResourceType\",\n \"message\": \"The provided location 'eastus2euap' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'uksouth,switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2'.\"\n }\n}\n--------------------------------------------------------------------------------\n",
err: <*exported.ResponseError | 0xc00103ac00>{
ErrorCode: "LocationNotAvailableForResourceType",
StatusCode: 400,
RawResponse: {
Status: "400 Bad Request",
StatusCode: 400,
Proto: "HTTP/2.0",
ProtoMajor: 2,
ProtoMinor: 0,
Header: {
"Content-Type": [
"application/json; charset=utf-8",
],
"Expires": ["-1"],
"X-Ms-Failure-Cause": ["gateway"],
"X-Ms-Request-Id": [
"26242653-bfb8-413a-8162-b88d031d4073",
],
"X-Ms-Routing-Request-Id": [
"EASTUS2EUAP:20260225T232839Z:26242653-bfb8-413a-8162-b88d031d4073",
],
"X-Content-Type-Options": ["nosniff"],
"Pragma": ["no-cache"],
"X-Ms-Correlation-Request-Id": [
"26242653-bfb8-413a-8162-b88d031d4073",
],
"Strict-Transport-Security": [
"max-age=31536000; includeSubDomains",
],
"X-Cache": ["CONFIG_NOCACHE"],
"X-Msedge-Ref": [
"Ref A: DA2AD0AB8DCC4E9D927EC710E452B479 Ref B: CH1AA2020610039 Ref C: 2026-02-25T23:28:39Z",
],
"Date": [
"Wed, 25 Feb 2026 23:28:38 GMT",
],
"Cache-Control": ["no-cache"],
"Content-Length": ["340"],
},
...
Gomega truncated this representation as it exceeds 'format.MaxLength'.
Consider having the object provide a custom 'GomegaStringer' representation
or adjust the parameters in Gomega's 'format' package.
Learn more here: https://onsi.github.io/gomega/#adjusting-output
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_update.go:84]: Unexpected error:
<*fmt.wrapError | 0xc000a51ae0>:
failed to create HCP cluster patch-name-cluster: failed starting cluster creation "patch-name-cluster" in resourcegroup="patch-name-ds6vvt": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/patch-name-ds6vvt/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/patch-name-cluster
--------------------------------------------------------------------------------
RESPONSE 400: 400 Bad Request
ERROR CODE: LocationNotAvailableForResourceType
--------------------------------------------------------------------------------
{
"error": {
"code": "LocationNotAvailableForResourceType",
"message": "The provided location 'eastus2euap' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'uksouth,switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2'."
}
}
--------------------------------------------------------------------------------
{
msg: "failed to create HCP cluster patch-name-cluster: failed starting cluster creation \"patch-name-cluster\" in resourcegroup=\"patch-name-ds6vvt\": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/patch-name-ds6vvt/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/patch-name-cluster\n--------------------------------------------------------------------------------\nRESPONSE 400: 400 Bad Request\nERROR CODE: LocationNotAvailableForResourceType\n--------------------------------------------------------------------------------\n{\n \"error\": {\n \"code\": \"LocationNotAvailableForResourceType\",\n \"message\": \"The provided location 'eastus2euap' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'uksouth,switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2'.\"\n }\n}\n--------------------------------------------------------------------------------\n",
err: <*fmt.wrapError | 0xc000a51ac0>{
msg: "failed starting cluster creation \"patch-name-cluster\" in resourcegroup=\"patch-name-ds6vvt\": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/patch-name-ds6vvt/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/patch-name-cluster\n--------------------------------------------------------------------------------\nRESPONSE 400: 400 Bad Request\nERROR CODE: LocationNotAvailableForResourceType\n--------------------------------------------------------------------------------\n{\n \"error\": {\n \"code\": \"LocationNotAvailableForResourceType\",\n \"message\": \"The provided location 'eastus2euap' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'uksouth,switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2'.\"\n }\n}\n--------------------------------------------------------------------------------\n",
err: <*exported.ResponseError | 0xc00103ac00>{
ErrorCode: "LocationNotAvailableForResourceType",
StatusCode: 400,
RawResponse: {
Status: "400 Bad Request",
StatusCode: 400,
Proto: "HTTP/2.0",
ProtoMajor: 2,
ProtoMinor: 0,
Header: {
"Content-Type": [
"application/json; charset=utf-8",
],
"Expires": ["-1"],
"X-Ms-Failure-Cause": ["gateway"],
"X-Ms-Request-Id": [
"26242653-bfb8-413a-8162-b88d031d4073",
],
"X-Ms-Routing-Request-Id": [
"EASTUS2EUAP:20260225T232839Z:26242653-bfb8-413a-8162-b88d031d4073",
],
"X-Content-Type-Options": ["nosniff"],
"Pragma": ["no-cache"],
"X-Ms-Correlation-Request-Id": [
"26242653-bfb8-413a-8162-b88d031d4073",
],
"Strict-Transport-Security": [
"max-age=31536000; includeSubDomains",
],
"X-Cache": ["CONFIG_NOCACHE"],
"X-Msedge-Ref": [
"Ref A: DA2AD0AB8DCC4E9D927EC710E452B479 Ref B: CH1AA2020610039 Ref C: 2026-02-25T23:28:39Z",
],
"Date": [
"Wed, 25 Feb 2026 23:28:38 GMT",
],
"Cache-Control": ["no-cache"],
"Content-Length": ["340"],
},
...
Gomega truncated this representation as it exceeds 'format.MaxLength'.
Consider having the object provide a custom 'GomegaStringer' representation
or adjust the parameters in Gomega's 'format' package.
Learn more here: https://onsi.github.io/gomega/#adjusting-output
occurredfail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create.go:90]: Unexpected error:
<*fmt.wrapError | 0xc00060b640>:
failed to create HCP cluster basic-hcp-cluster: failed starting cluster creation "basic-hcp-cluster" in resourcegroup="basic-cluster-ctxbgj": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/basic-cluster-ctxbgj/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/basic-hcp-cluster
--------------------------------------------------------------------------------
RESPONSE 400: 400 Bad Request
ERROR CODE: LocationNotAvailableForResourceType
--------------------------------------------------------------------------------
{
"error": {
"code": "LocationNotAvailableForResourceType",
"message": "The provided location 'eastus2euap' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'uksouth,switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2'."
}
}
--------------------------------------------------------------------------------
{
msg: "failed to create HCP cluster basic-hcp-cluster: failed starting cluster creation \"basic-hcp-cluster\" in resourcegroup=\"basic-cluster-ctxbgj\": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/basic-cluster-ctxbgj/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/basic-hcp-cluster\n--------------------------------------------------------------------------------\nRESPONSE 400: 400 Bad Request\nERROR CODE: LocationNotAvailableForResourceType\n--------------------------------------------------------------------------------\n{\n \"error\": {\n \"code\": \"LocationNotAvailableForResourceType\",\n \"message\": \"The provided location 'eastus2euap' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'uksouth,switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2'.\"\n }\n}\n--------------------------------------------------------------------------------\n",
err: <*fmt.wrapError | 0xc00060b620>{
msg: "failed starting cluster creation \"basic-hcp-cluster\" in resourcegroup=\"basic-cluster-ctxbgj\": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/basic-cluster-ctxbgj/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/basic-hcp-cluster\n--------------------------------------------------------------------------------\nRESPONSE 400: 400 Bad Request\nERROR CODE: LocationNotAvailableForResourceType\n--------------------------------------------------------------------------------\n{\n \"error\": {\n \"code\": \"LocationNotAvailableForResourceType\",\n \"message\": \"The provided location 'eastus2euap' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'uksouth,switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2'.\"\n }\n}\n--------------------------------------------------------------------------------\n",
err: <*exported.ResponseError | 0xc0011b8990>{
ErrorCode: "LocationNotAvailableForResourceType",
StatusCode: 400,
RawResponse: {
Status: "400 Bad Request",
StatusCode: 400,
Proto: "HTTP/2.0",
ProtoMajor: 2,
ProtoMinor: 0,
Header: {
"Cache-Control": ["no-cache"],
"Pragma": ["no-cache"],
"Content-Length": ["340"],
"Content-Type": [
"application/json; charset=utf-8",
],
"X-Ms-Request-Id": [
"9f9f91af-3ec7-4e2e-9193-bf4eb02a8859",
],
"X-Ms-Correlation-Request-Id": [
"9f9f91af-3ec7-4e2e-9193-bf4eb02a8859",
],
"X-Ms-Routing-Request-Id": [
"EASTUS2EUAP:20260225T233146Z:9f9f91af-3ec7-4e2e-9193-bf4eb02a8859",
],
"Strict-Transport-Security": [
"max-age=31536000; includeSubDomains",
],
"Expires": ["-1"],
"X-Ms-Failure-Cause": ["gateway"],
"X-Content-Type-Options": ["nosniff"],
"X-Cache": ["CONFIG_NOCACHE"],
"X-Msedge-Ref": [
"Ref A: 6541A0991AC14037AEEB409D03363692 Ref B: CHI261140509054 Ref C: 2026-02-25T23:31:46Z",
],
"Date": [
"Wed, 25 Feb 2026 23:31:45 GMT",
],
},
...
Gomega truncated this representation as it exceeds 'format.MaxLength'.
Consider having the object provide a custom 'GomegaStringer' representation
or adjust the parameters in Gomega's 'format' package.
Learn more here: https://onsi.github.io/gomega/#adjusting-output
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create.go:90]: Unexpected error:
<*fmt.wrapError | 0xc00060b640>:
failed to create HCP cluster basic-hcp-cluster: failed starting cluster creation "basic-hcp-cluster" in resourcegroup="basic-cluster-ctxbgj": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/basic-cluster-ctxbgj/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/basic-hcp-cluster
--------------------------------------------------------------------------------
RESPONSE 400: 400 Bad Request
ERROR CODE: LocationNotAvailableForResourceType
--------------------------------------------------------------------------------
{
"error": {
"code": "LocationNotAvailableForResourceType",
"message": "The provided location 'eastus2euap' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'uksouth,switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2'."
}
}
--------------------------------------------------------------------------------
{
msg: "failed to create HCP cluster basic-hcp-cluster: failed starting cluster creation \"basic-hcp-cluster\" in resourcegroup=\"basic-cluster-ctxbgj\": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/basic-cluster-ctxbgj/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/basic-hcp-cluster\n--------------------------------------------------------------------------------\nRESPONSE 400: 400 Bad Request\nERROR CODE: LocationNotAvailableForResourceType\n--------------------------------------------------------------------------------\n{\n \"error\": {\n \"code\": \"LocationNotAvailableForResourceType\",\n \"message\": \"The provided location 'eastus2euap' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'uksouth,switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2'.\"\n }\n}\n--------------------------------------------------------------------------------\n",
err: <*fmt.wrapError | 0xc00060b620>{
msg: "failed starting cluster creation \"basic-hcp-cluster\" in resourcegroup=\"basic-cluster-ctxbgj\": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/basic-cluster-ctxbgj/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/basic-hcp-cluster\n--------------------------------------------------------------------------------\nRESPONSE 400: 400 Bad Request\nERROR CODE: LocationNotAvailableForResourceType\n--------------------------------------------------------------------------------\n{\n \"error\": {\n \"code\": \"LocationNotAvailableForResourceType\",\n \"message\": \"The provided location 'eastus2euap' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'uksouth,switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2'.\"\n }\n}\n--------------------------------------------------------------------------------\n",
err: <*exported.ResponseError | 0xc0011b8990>{
ErrorCode: "LocationNotAvailableForResourceType",
StatusCode: 400,
RawResponse: {
Status: "400 Bad Request",
StatusCode: 400,
Proto: "HTTP/2.0",
ProtoMajor: 2,
ProtoMinor: 0,
Header: {
"Cache-Control": ["no-cache"],
"Pragma": ["no-cache"],
"Content-Length": ["340"],
"Content-Type": [
"application/json; charset=utf-8",
],
"X-Ms-Request-Id": [
"9f9f91af-3ec7-4e2e-9193-bf4eb02a8859",
],
"X-Ms-Correlation-Request-Id": [
"9f9f91af-3ec7-4e2e-9193-bf4eb02a8859",
],
"X-Ms-Routing-Request-Id": [
"EASTUS2EUAP:20260225T233146Z:9f9f91af-3ec7-4e2e-9193-bf4eb02a8859",
],
"Strict-Transport-Security": [
"max-age=31536000; includeSubDomains",
],
"Expires": ["-1"],
"X-Ms-Failure-Cause": ["gateway"],
"X-Content-Type-Options": ["nosniff"],
"X-Cache": ["CONFIG_NOCACHE"],
"X-Msedge-Ref": [
"Ref A: 6541A0991AC14037AEEB409D03363692 Ref B: CHI261140509054 Ref C: 2026-02-25T23:31:46Z",
],
"Date": [
"Wed, 25 Feb 2026 23:31:45 GMT",
],
},
...
Gomega truncated this representation as it exceeds 'format.MaxLength'.
Consider having the object provide a custom 'GomegaStringer' representation
or adjust the parameters in Gomega's 'format' package.
Learn more here: https://onsi.github.io/gomega/#adjusting-output
occurredContributing tests (19)
Affected runs (2)
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
timeout during GetAdminRESTConfigForHCPCluster while waiting for hcpcluster credsfull failure pattern: timeout during GetAdminRESTConfigForHCPCluster while waiting for hcpcluster creds Signal: IndeterminateSignal: Indeterminate — no prior history; active 2 day(s) — New failure pattern — no prior history | e2e | 2 | 22.22%2 of 9 job runs affected | IndeterminateSignal: Indeterminate — no prior history; active 2 day(s) — New failure pattern — no prior history | Show trend detailsFeb 16: 0 · Feb 17: 0 · Feb 18: 0 · Feb 19: 0 · Feb 20: 0 · Feb 21: 0 · Feb 22: 0 · Feb 23: 0 · Feb 24: 0 · Feb 25: 0 · Feb 26: 0 · Feb 27: 3 · Feb 28: 0 · Mar 1: 3 | DEV | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Full failure examples (3)fail [github.com/Azure/ARO-HCP/test/e2e/cluster_authorized_cidrs_connectivity.go:185]: Unexpected error:
<*fmt.wrapErrors | 0xc000e5a810>:
failed waiting for hcpCluster="cidr-connectivity-test" in resourcegroup="e2e-cidr-connectivity-wwvzjd" to finish getting creds, caused by: timeout '10.000000' minutes exceeded during GetAdminRESTConfigForHCPCluster for cluster cidr-connectivity-test in resource group e2e-cidr-connectivity-wwvzjd, error: context deadline exceeded
{
msg: "failed waiting for hcpCluster=\"cidr-connectivity-test\" in resourcegroup=\"e2e-cidr-connectivity-wwvzjd\" to finish getting creds, caused by: timeout '10.000000' minutes exceeded during GetAdminRESTConfigForHCPCluster for cluster cidr-connectivity-test in resource group e2e-cidr-connectivity-wwvzjd, error: context deadline exceeded",
errs: [
<*errors.errorString | 0xc00072e2f0>{
s: "timeout '10.000000' minutes exceeded during GetAdminRESTConfigForHCPCluster for cluster cidr-connectivity-test in resource group e2e-cidr-connectivity-wwvzjd",
},
<context.deadlineExceededError>{},
],
}
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_authorized_cidrs_connectivity.go:185]: Unexpected error:
<*fmt.wrapErrors | 0xc000e5a810>:
failed waiting for hcpCluster="cidr-connectivity-test" in resourcegroup="e2e-cidr-connectivity-wwvzjd" to finish getting creds, caused by: timeout '10.000000' minutes exceeded during GetAdminRESTConfigForHCPCluster for cluster cidr-connectivity-test in resource group e2e-cidr-connectivity-wwvzjd, error: context deadline exceeded
{
msg: "failed waiting for hcpCluster=\"cidr-connectivity-test\" in resourcegroup=\"e2e-cidr-connectivity-wwvzjd\" to finish getting creds, caused by: timeout '10.000000' minutes exceeded during GetAdminRESTConfigForHCPCluster for cluster cidr-connectivity-test in resource group e2e-cidr-connectivity-wwvzjd, error: context deadline exceeded",
errs: [
<*errors.errorString | 0xc00072e2f0>{
s: "timeout '10.000000' minutes exceeded during GetAdminRESTConfigForHCPCluster for cluster cidr-connectivity-test in resource group e2e-cidr-connectivity-wwvzjd",
},
<context.deadlineExceededError>{},
],
}
occurredfail [github.com/Azure/ARO-HCP/test/e2e/admin_credential_lifecycle.go:174]: Unexpected error:
<*fmt.wrapErrors | 0xc000d1f650>:
failed waiting for hcpCluster="admin-cred-lifecycle-88jst9" in resourcegroup="admin-credential-lifecycle-test-sbtn87" to finish getting creds, caused by: timeout '10.000000' minutes exceeded during GetAdminRESTConfigForHCPCluster for cluster admin-cred-lifecycle-88jst9 in resource group admin-credential-lifecycle-test-sbtn87, error: context deadline exceeded
{
msg: "failed waiting for hcpCluster=\"admin-cred-lifecycle-88jst9\" in resourcegroup=\"admin-credential-lifecycle-test-sbtn87\" to finish getting creds, caused by: timeout '10.000000' minutes exceeded during GetAdminRESTConfigForHCPCluster for cluster admin-cred-lifecycle-88jst9 in resource group admin-credential-lifecycle-test-sbtn87, error: context deadline exceeded",
errs: [
<*errors.errorString | 0xc000d6b6f0>{
s: "timeout '10.000000' minutes exceeded during GetAdminRESTConfigForHCPCluster for cluster admin-cred-lifecycle-88jst9 in resource group admin-credential-lifecycle-test-sbtn87",
},
<context.deadlineExceededError>{},
],
}
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/admin_credential_lifecycle.go:174]: Unexpected error:
<*fmt.wrapErrors | 0xc000d1f650>:
failed waiting for hcpCluster="admin-cred-lifecycle-88jst9" in resourcegroup="admin-credential-lifecycle-test-sbtn87" to finish getting creds, caused by: timeout '10.000000' minutes exceeded during GetAdminRESTConfigForHCPCluster for cluster admin-cred-lifecycle-88jst9 in resource group admin-credential-lifecycle-test-sbtn87, error: context deadline exceeded
{
msg: "failed waiting for hcpCluster=\"admin-cred-lifecycle-88jst9\" in resourcegroup=\"admin-credential-lifecycle-test-sbtn87\" to finish getting creds, caused by: timeout '10.000000' minutes exceeded during GetAdminRESTConfigForHCPCluster for cluster admin-cred-lifecycle-88jst9 in resource group admin-credential-lifecycle-test-sbtn87, error: context deadline exceeded",
errs: [
<*errors.errorString | 0xc000d6b6f0>{
s: "timeout '10.000000' minutes exceeded during GetAdminRESTConfigForHCPCluster for cluster admin-cred-lifecycle-88jst9 in resource group admin-credential-lifecycle-test-sbtn87",
},
<context.deadlineExceededError>{},
],
}
occurredfail [github.com/Azure/ARO-HCP/test/e2e/image_registry_cluster_create.go:105]: Unexpected error:
<*fmt.wrapErrors | 0xc000efc060>:
failed waiting for hcpCluster="disabled-image-registry-hcp-cluster" in resourcegroup="disabled-image-registry-xfxfjs" to finish getting creds, caused by: timeout '10.000000' minutes exceeded during GetAdminRESTConfigForHCPCluster for cluster disabled-image-registry-hcp-cluster in resource group disabled-image-registry-xfxfjs, error: context deadline exceeded
{
msg: "failed waiting for hcpCluster=\"disabled-image-registry-hcp-cluster\" in resourcegroup=\"disabled-image-registry-xfxfjs\" to finish getting creds, caused by: timeout '10.000000' minutes exceeded during GetAdminRESTConfigForHCPCluster for cluster disabled-image-registry-hcp-cluster in resource group disabled-image-registry-xfxfjs, error: context deadline exceeded",
errs: [
<*errors.errorString | 0xc000b21af0>{
s: "timeout '10.000000' minutes exceeded during GetAdminRESTConfigForHCPCluster for cluster disabled-image-registry-hcp-cluster in resource group disabled-image-registry-xfxfjs",
},
<context.deadlineExceededError>{},
],
}
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/image_registry_cluster_create.go:105]: Unexpected error:
<*fmt.wrapErrors | 0xc000efc060>:
failed waiting for hcpCluster="disabled-image-registry-hcp-cluster" in resourcegroup="disabled-image-registry-xfxfjs" to finish getting creds, caused by: timeout '10.000000' minutes exceeded during GetAdminRESTConfigForHCPCluster for cluster disabled-image-registry-hcp-cluster in resource group disabled-image-registry-xfxfjs, error: context deadline exceeded
{
msg: "failed waiting for hcpCluster=\"disabled-image-registry-hcp-cluster\" in resourcegroup=\"disabled-image-registry-xfxfjs\" to finish getting creds, caused by: timeout '10.000000' minutes exceeded during GetAdminRESTConfigForHCPCluster for cluster disabled-image-registry-hcp-cluster in resource group disabled-image-registry-xfxfjs, error: context deadline exceeded",
errs: [
<*errors.errorString | 0xc000b21af0>{
s: "timeout '10.000000' minutes exceeded during GetAdminRESTConfigForHCPCluster for cluster disabled-image-registry-hcp-cluster in resource group disabled-image-registry-xfxfjs",
},
<context.deadlineExceededError>{},
],
}
occurredContributing tests (6)
Affected runs (2)
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
ERROR CODE: NoRegisteredProviderFound; detail message No registered resource provider found for location 'eastus2euap' and API version '2024-06-10-preview' for type 'locations/hcp...full failure pattern: ERROR CODE: NoRegisteredProviderFound; detail message No registered resource provider found for location 'eastus2euap' and API version '2024-06-10-preview' for type 'locations/hcpOpenShiftVersions'. The supported api-versions are '2024-06-10-preview'. The supported locations are 'uksouth, switzerlandnorth, canadacentral, australiaeast, westeurope, centralindia, brazilsouth, eastus2'.; provider Microsoft.RedHatOpenShift Signal: IndeterminateSignal: Indeterminate — no prior history; active 1 day(s) — New failure pattern — no prior history | e2e | 2 | 22.22%2 of 9 job runs affected | IndeterminateSignal: Indeterminate — no prior history; active 1 day(s) — New failure pattern — no prior history | Show trend detailsFeb 16: 0 · Feb 17: 0 · Feb 18: 0 · Feb 19: 0 · Feb 20: 0 · Feb 21: 0 · Feb 22: 0 · Feb 23: 0 · Feb 24: 0 · Feb 25: 2 · Feb 26: 0 · Feb 27: 0 · Feb 28: 0 · Mar 1: 0 | none | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Full failure examples (2)fail [github.com/Azure/ARO-HCP/test/e2e/cluster_versions.go:42]: Unexpected error:
<*exported.ResponseError | 0xc00098acc0>:
GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOpenShiftVersions
--------------------------------------------------------------------------------
RESPONSE 400: 400 Bad Request
ERROR CODE: NoRegisteredProviderFound
--------------------------------------------------------------------------------
{
"error": {
"code": "NoRegisteredProviderFound",
"message": "No registered resource provider found for location 'eastus2euap' and API version '2024-06-10-preview' for type 'locations/hcpOpenShiftVersions'. The supported api-versions are '2024-06-10-preview'. The supported locations are 'uksouth, switzerlandnorth, canadacentral, australiaeast, westeurope, centralindia, brazilsouth, eastus2'."
}
}
--------------------------------------------------------------------------------
{
ErrorCode: "NoRegisteredProviderFound",
StatusCode: 400,
RawResponse: {
Status: "400 Bad Request",
StatusCode: 400,
Proto: "HTTP/2.0",
ProtoMajor: 2,
ProtoMinor: 0,
Header: {
"Content-Length": ["391"],
"X-Ms-Failure-Cause": ["gateway"],
"X-Ms-Routing-Request-Id": [
"CANADACENTRAL:20260225T232729Z:f8e776ca-80b7-4689-829c-7c4b206f3557",
],
"X-Msedge-Ref": [
"Ref A: E2444D8D87FF4BA4A29D99D501BB6B2A Ref B: CHI261140502031 Ref C: 2026-02-25T23:27:29Z",
],
"Content-Type": [
"application/json; charset=utf-8",
],
"Expires": ["-1"],
"X-Ms-Request-Id": [
"f8e776ca-80b7-4689-829c-7c4b206f3557",
],
"X-Ms-Correlation-Request-Id": [
"f8e776ca-80b7-4689-829c-7c4b206f3557",
],
"Strict-Transport-Security": [
"max-age=31536000; includeSubDomains",
],
"X-Content-Type-Options": ["nosniff"],
"X-Cache": ["CONFIG_NOCACHE"],
"Date": [
"Wed, 25 Feb 2026 23:27:29 GMT",
],
"Cache-Control": ["no-cache"],
"Pragma": ["no-cache"],
},
Body: <*exported.nopClosingBytesReader | 0xc001616200>{
s: "{\"error\":{\"code\":\"NoRegisteredProviderFound\",\"message\":\"No registered resource provider found for location 'eastus2euap' and API version '2024-06-10-preview' for type 'locations/hcpOpenShiftVersions'. The supported api-versions are '2024-06-10-preview'. The supported locations are 'uksouth, switzerlandnorth, canadacentral, australiaeast, westeurope, centralindia, brazilsouth, eastus2'.\"}}",
i: 0,
},
ContentLength: 391,
TransferEncoding: nil,
Close: false,
Uncompressed: false,
Trailer: nil,
Request: {
Method: "GET",
URL: {
Scheme: "https",
Opaque: "",
User: nil,
Host: "management.azure.com",
Path: "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOpenShiftVersions",
RawPath: "",
OmitHost: false,
ForceQuery: false,
RawQuery: "api-version=2024-06-10-preview",
Fragment: "",
RawFragment: "",
},
Proto: "HTTP/1.1",
ProtoMajor: 1,
ProtoMinor: 1,
Header: {
"Accept": ["application/json"],
"User-Agent": [
"azsdk-go-armredhatopenshifthcp/v0.1.0 (go1.25.7; linux)",
],
"Authorization": ["redacted"],
},
Body: nil,
GetBody: nil,
ContentLength: 0,
TransferEncoding: nil,
Close: false,
Host: "management.azure.com",
Form: nil,
PostForm: nil,
MultipartForm: nil,
Trailer: nil,
RemoteAddr: "",
RequestURI: "",
TLS: nil,
Cancel: nil,
Response: nil,
Pattern: "",
ctx: <suppressed context>,
pat: nil,
matches: nil,
otherValues: nil,
},
TLS: {
Version: 772,
HandshakeComplete: true,
DidResume: false...
Gomega truncated this representation as it exceeds 'format.MaxLength'.
Consider having the object provide a custom 'GomegaStringer' representation
or adjust the parameters in Gomega's 'format' package.
Learn more here: https://onsi.github.io/gomega/#adjusting-output
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_versions.go:42]: Unexpected error:
<*exported.ResponseError | 0xc00098acc0>:
GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOpenShiftVersions
--------------------------------------------------------------------------------
RESPONSE 400: 400 Bad Request
ERROR CODE: NoRegisteredProviderFound
--------------------------------------------------------------------------------
{
"error": {
"code": "NoRegisteredProviderFound",
"message": "No registered resource provider found for location 'eastus2euap' and API version '2024-06-10-preview' for type 'locations/hcpOpenShiftVersions'. The supported api-versions are '2024-06-10-preview'. The supported locations are 'uksouth, switzerlandnorth, canadacentral, australiaeast, westeurope, centralindia, brazilsouth, eastus2'."
}
}
--------------------------------------------------------------------------------
{
ErrorCode: "NoRegisteredProviderFound",
StatusCode: 400,
RawResponse: {
Status: "400 Bad Request",
StatusCode: 400,
Proto: "HTTP/2.0",
ProtoMajor: 2,
ProtoMinor: 0,
Header: {
"Content-Length": ["391"],
"X-Ms-Failure-Cause": ["gateway"],
"X-Ms-Routing-Request-Id": [
"CANADACENTRAL:20260225T232729Z:f8e776ca-80b7-4689-829c-7c4b206f3557",
],
"X-Msedge-Ref": [
"Ref A: E2444D8D87FF4BA4A29D99D501BB6B2A Ref B: CHI261140502031 Ref C: 2026-02-25T23:27:29Z",
],
"Content-Type": [
"application/json; charset=utf-8",
],
"Expires": ["-1"],
"X-Ms-Request-Id": [
"f8e776ca-80b7-4689-829c-7c4b206f3557",
],
"X-Ms-Correlation-Request-Id": [
"f8e776ca-80b7-4689-829c-7c4b206f3557",
],
"Strict-Transport-Security": [
"max-age=31536000; includeSubDomains",
],
"X-Content-Type-Options": ["nosniff"],
"X-Cache": ["CONFIG_NOCACHE"],
"Date": [
"Wed, 25 Feb 2026 23:27:29 GMT",
],
"Cache-Control": ["no-cache"],
"Pragma": ["no-cache"],
},
Body: <*exported.nopClosingBytesReader | 0xc001616200>{
s: "{\"error\":{\"code\":\"NoRegisteredProviderFound\",\"message\":\"No registered resource provider found for location 'eastus2euap' and API version '2024-06-10-preview' for type 'locations/hcpOpenShiftVersions'. The supported api-versions are '2024-06-10-preview'. The supported locations are 'uksouth, switzerlandnorth, canadacentral, australiaeast, westeurope, centralindia, brazilsouth, eastus2'.\"}}",
i: 0,
},
ContentLength: 391,
TransferEncoding: nil,
Close: false,
Uncompressed: false,
Trailer: nil,
Request: {
Method: "GET",
URL: {
Scheme: "https",
Opaque: "",
User: nil,
Host: "management.azure.com",
Path: "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOpenShiftVersions",
RawPath: "",
OmitHost: false,
ForceQuery: false,
RawQuery: "api-version=2024-06-10-preview",
Fragment: "",
RawFragment: "",
},
Proto: "HTTP/1.1",
ProtoMajor: 1,
ProtoMinor: 1,
Header: {
"Accept": ["application/json"],
"User-Agent": [
"azsdk-go-armredhatopenshifthcp/v0.1.0 (go1.25.7; linux)",
],
"Authorization": ["redacted"],
},
Body: nil,
GetBody: nil,
ContentLength: 0,
TransferEncoding: nil,
Close: false,
Host: "management.azure.com",
Form: nil,
PostForm: nil,
MultipartForm: nil,
Trailer: nil,
RemoteAddr: "",
RequestURI: "",
TLS: nil,
Cancel: nil,
Response: nil,
Pattern: "",
ctx: <suppressed context>,
pat: nil,
matches: nil,
otherValues: nil,
},
TLS: {
Version: 772,
HandshakeComplete: true,
DidResume: false...
Gomega truncated this representation as it exceeds 'format.MaxLength'.
Consider having the object provide a custom 'GomegaStringer' representation
or adjust the parameters in Gomega's 'format' package.
Learn more here: https://onsi.github.io/gomega/#adjusting-output
occurredfail [github.com/Azure/ARO-HCP/test/e2e/cluster_versions.go:42]: Unexpected error:
<*exported.ResponseError | 0xc001480ed0>:
GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOpenShiftVersions
--------------------------------------------------------------------------------
RESPONSE 400: 400 Bad Request
ERROR CODE: NoRegisteredProviderFound
--------------------------------------------------------------------------------
{
"error": {
"code": "NoRegisteredProviderFound",
"message": "No registered resource provider found for location 'eastus2euap' and API version '2024-06-10-preview' for type 'locations/hcpOpenShiftVersions'. The supported api-versions are '2024-06-10-preview'. The supported locations are 'uksouth, switzerlandnorth, canadacentral, australiaeast, westeurope, centralindia, brazilsouth, eastus2'."
}
}
--------------------------------------------------------------------------------
{
ErrorCode: "NoRegisteredProviderFound",
StatusCode: 400,
RawResponse: {
Status: "400 Bad Request",
StatusCode: 400,
Proto: "HTTP/2.0",
ProtoMajor: 2,
ProtoMinor: 0,
Header: {
"Expires": ["-1"],
"X-Ms-Failure-Cause": ["gateway"],
"X-Content-Type-Options": ["nosniff"],
"Date": [
"Wed, 25 Feb 2026 22:52:04 GMT",
],
"Cache-Control": ["no-cache"],
"Content-Length": ["391"],
"X-Ms-Request-Id": [
"533d7cc8-73b5-460e-939b-480e9ca3e33a",
],
"X-Ms-Correlation-Request-Id": [
"533d7cc8-73b5-460e-939b-480e9ca3e33a",
],
"X-Ms-Routing-Request-Id": [
"NORTHCENTRALUS:20260225T225205Z:533d7cc8-73b5-460e-939b-480e9ca3e33a",
],
"Strict-Transport-Security": [
"max-age=31536000; includeSubDomains",
],
"X-Cache": ["CONFIG_NOCACHE"],
"X-Msedge-Ref": [
"Ref A: 12D16754A04C47B4BBE045D037B9B762 Ref B: CHI261140502023 Ref C: 2026-02-25T22:52:05Z",
],
"Pragma": ["no-cache"],
"Content-Type": [
"application/json; charset=utf-8",
],
},
Body: <*exported.nopClosingBytesReader | 0xc0012c85e0>{
s: "{\"error\":{\"code\":\"NoRegisteredProviderFound\",\"message\":\"No registered resource provider found for location 'eastus2euap' and API version '2024-06-10-preview' for type 'locations/hcpOpenShiftVersions'. The supported api-versions are '2024-06-10-preview'. The supported locations are 'uksouth, switzerlandnorth, canadacentral, australiaeast, westeurope, centralindia, brazilsouth, eastus2'.\"}}",
i: 0,
},
ContentLength: 391,
TransferEncoding: nil,
Close: false,
Uncompressed: false,
Trailer: nil,
Request: {
Method: "GET",
URL: {
Scheme: "https",
Opaque: "",
User: nil,
Host: "management.azure.com",
Path: "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOpenShiftVersions",
RawPath: "",
OmitHost: false,
ForceQuery: false,
RawQuery: "api-version=2024-06-10-preview",
Fragment: "",
RawFragment: "",
},
Proto: "HTTP/1.1",
ProtoMajor: 1,
ProtoMinor: 1,
Header: {
"Accept": ["application/json"],
"User-Agent": [
"azsdk-go-armredhatopenshifthcp/v0.1.0 (go1.25.7; linux)",
],
"Authorization": ["redacted"],
},
Body: nil,
GetBody: nil,
ContentLength: 0,
TransferEncoding: nil,
Close: false,
Host: "management.azure.com",
Form: nil,
PostForm: nil,
MultipartForm: nil,
Trailer: nil,
RemoteAddr: "",
RequestURI: "",
TLS: nil,
Cancel: nil,
Response: nil,
Pattern: "",
ctx: <suppressed context>,
pat: nil,
matches: nil,
otherValues: nil,
},
TLS: {
Version: 772,
HandshakeComplete: true,
DidResume: fals...
Gomega truncated this representation as it exceeds 'format.MaxLength'.
Consider having the object provide a custom 'GomegaStringer' representation
or adjust the parameters in Gomega's 'format' package.
Learn more here: https://onsi.github.io/gomega/#adjusting-output
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_versions.go:42]: Unexpected error:
<*exported.ResponseError | 0xc001480ed0>:
GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOpenShiftVersions
--------------------------------------------------------------------------------
RESPONSE 400: 400 Bad Request
ERROR CODE: NoRegisteredProviderFound
--------------------------------------------------------------------------------
{
"error": {
"code": "NoRegisteredProviderFound",
"message": "No registered resource provider found for location 'eastus2euap' and API version '2024-06-10-preview' for type 'locations/hcpOpenShiftVersions'. The supported api-versions are '2024-06-10-preview'. The supported locations are 'uksouth, switzerlandnorth, canadacentral, australiaeast, westeurope, centralindia, brazilsouth, eastus2'."
}
}
--------------------------------------------------------------------------------
{
ErrorCode: "NoRegisteredProviderFound",
StatusCode: 400,
RawResponse: {
Status: "400 Bad Request",
StatusCode: 400,
Proto: "HTTP/2.0",
ProtoMajor: 2,
ProtoMinor: 0,
Header: {
"Expires": ["-1"],
"X-Ms-Failure-Cause": ["gateway"],
"X-Content-Type-Options": ["nosniff"],
"Date": [
"Wed, 25 Feb 2026 22:52:04 GMT",
],
"Cache-Control": ["no-cache"],
"Content-Length": ["391"],
"X-Ms-Request-Id": [
"533d7cc8-73b5-460e-939b-480e9ca3e33a",
],
"X-Ms-Correlation-Request-Id": [
"533d7cc8-73b5-460e-939b-480e9ca3e33a",
],
"X-Ms-Routing-Request-Id": [
"NORTHCENTRALUS:20260225T225205Z:533d7cc8-73b5-460e-939b-480e9ca3e33a",
],
"Strict-Transport-Security": [
"max-age=31536000; includeSubDomains",
],
"X-Cache": ["CONFIG_NOCACHE"],
"X-Msedge-Ref": [
"Ref A: 12D16754A04C47B4BBE045D037B9B762 Ref B: CHI261140502023 Ref C: 2026-02-25T22:52:05Z",
],
"Pragma": ["no-cache"],
"Content-Type": [
"application/json; charset=utf-8",
],
},
Body: <*exported.nopClosingBytesReader | 0xc0012c85e0>{
s: "{\"error\":{\"code\":\"NoRegisteredProviderFound\",\"message\":\"No registered resource provider found for location 'eastus2euap' and API version '2024-06-10-preview' for type 'locations/hcpOpenShiftVersions'. The supported api-versions are '2024-06-10-preview'. The supported locations are 'uksouth, switzerlandnorth, canadacentral, australiaeast, westeurope, centralindia, brazilsouth, eastus2'.\"}}",
i: 0,
},
ContentLength: 391,
TransferEncoding: nil,
Close: false,
Uncompressed: false,
Trailer: nil,
Request: {
Method: "GET",
URL: {
Scheme: "https",
Opaque: "",
User: nil,
Host: "management.azure.com",
Path: "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOpenShiftVersions",
RawPath: "",
OmitHost: false,
ForceQuery: false,
RawQuery: "api-version=2024-06-10-preview",
Fragment: "",
RawFragment: "",
},
Proto: "HTTP/1.1",
ProtoMajor: 1,
ProtoMinor: 1,
Header: {
"Accept": ["application/json"],
"User-Agent": [
"azsdk-go-armredhatopenshifthcp/v0.1.0 (go1.25.7; linux)",
],
"Authorization": ["redacted"],
},
Body: nil,
GetBody: nil,
ContentLength: 0,
TransferEncoding: nil,
Close: false,
Host: "management.azure.com",
Form: nil,
PostForm: nil,
MultipartForm: nil,
Trailer: nil,
RemoteAddr: "",
RequestURI: "",
TLS: nil,
Cancel: nil,
Response: nil,
Pattern: "",
ctx: <suppressed context>,
pat: nil,
matches: nil,
otherValues: nil,
},
TLS: {
Version: 772,
HandshakeComplete: true,
DidResume: fals...
Gomega truncated this representation as it exceeds 'format.MaxLength'.
Consider having the object provide a custom 'GomegaStringer' representation
or adjust the parameters in Gomega's 'format' package.
Learn more here: https://onsi.github.io/gomega/#adjusting-output
occurredContributing tests (1)
Affected runs (2)
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Interrupted by Userfull failure pattern: Interrupted by User Signal: IndeterminateSignal: Indeterminate — no prior history; active 2 day(s) — New failure pattern — no prior history | e2e | 2 | 22.22%2 of 9 job runs affected | IndeterminateSignal: Indeterminate — no prior history; active 2 day(s) — New failure pattern — no prior history | Show trend detailsFeb 16: 0 · Feb 17: 0 · Feb 18: 0 · Feb 19: 0 · Feb 20: 0 · Feb 21: 0 · Feb 22: 0 · Feb 23: 0 · Feb 24: 0 · Feb 25: 0 · Feb 26: 1 · Feb 27: 1 · Feb 28: 0 · Mar 1: 0 | DEV, INT | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Full failure examples (2)fail [github.com/Azure/ARO-HCP/test/e2e/external_auth_create.go:47]: Interrupted by User fail [github.com/Azure/ARO-HCP/test/e2e/external_auth_create.go:47]: Interrupted by User fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_autoscaling.go:36]: Interrupted by User fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_autoscaling.go:36]: Interrupted by User Contributing tests (2)
Affected runs (2)
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Version .* (doesn't exist|is disabled)full failure pattern: Version .* (doesn't exist|is disabled) Signal: IndeterminateSignal: Indeterminate — no prior history; active 1 day(s) — New failure pattern — no prior history | e2e | 2 | 22.22%2 of 9 job runs affected | IndeterminateSignal: Indeterminate — no prior history; active 1 day(s) — New failure pattern — no prior history | Show trend detailsFeb 16: 0 · Feb 17: 0 · Feb 18: 0 · Feb 19: 0 · Feb 20: 0 · Feb 21: 0 · Feb 22: 0 · Feb 23: 0 · Feb 24: 0 · Feb 25: 2 · Feb 26: 0 · Feb 27: 0 · Feb 28: 0 · Mar 1: 0 | DEV | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Full failure examples (2)fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_missing_info.go:82]: Expected
<*fmt.wrapError | 0xc0005302c0>:
failed to create HCP cluster illegal-hcp-cluster: failed starting cluster creation "illegal-hcp-cluster" in resourcegroup="illegal-ocp-version-67gxwd": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/illegal-ocp-version-67gxwd/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/illegal-hcp-cluster
--------------------------------------------------------------------------------
RESPONSE 400: 400 Bad Request
ERROR CODE: LocationNotAvailableForResourceType
--------------------------------------------------------------------------------
{
"error": {
"code": "LocationNotAvailableForResourceType",
"message": "The provided location 'eastus2euap' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'uksouth,switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2'."
}
}
--------------------------------------------------------------------------------
{
msg: "failed to create HCP cluster illegal-hcp-cluster: failed starting cluster creation \"illegal-hcp-cluster\" in resourcegroup=\"illegal-ocp-version-67gxwd\": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/illegal-ocp-version-67gxwd/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/illegal-hcp-cluster\n--------------------------------------------------------------------------------\nRESPONSE 400: 400 Bad Request\nERROR CODE: LocationNotAvailableForResourceType\n--------------------------------------------------------------------------------\n{\n \"error\": {\n \"code\": \"LocationNotAvailableForResourceType\",\n \"message\": \"The provided location 'eastus2euap' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'uksouth,switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2'.\"\n }\n}\n--------------------------------------------------------------------------------\n",
err: <*fmt.wrapError | 0xc0005302a0>{
msg: "failed starting cluster creation \"illegal-hcp-cluster\" in resourcegroup=\"illegal-ocp-version-67gxwd\": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/illegal-ocp-version-67gxwd/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/illegal-hcp-cluster\n--------------------------------------------------------------------------------\nRESPONSE 400: 400 Bad Request\nERROR CODE: LocationNotAvailableForResourceType\n--------------------------------------------------------------------------------\n{\n \"error\": {\n \"code\": \"LocationNotAvailableForResourceType\",\n \"message\": \"The provided location 'eastus2euap' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'uksouth,switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2'.\"\n }\n}\n--------------------------------------------------------------------------------\n",
err: <*exported.ResponseError | 0xc000d5ed80>{
ErrorCode: "LocationNotAvailableForResourceType",
StatusCode: 400,
RawResponse: {
Status: "400 Bad Request",
StatusCode: 400,
Proto: "HTTP/2.0",
ProtoMajor: 2,
ProtoMinor: 0,
Header: {
"Date": [
"Wed, 25 Feb 2026 23:28:40 GMT",
],
"Cache-Control": ["no-cache"],
"Content-Length": ["340"],
"Content-Type": [
"application/json; charset=utf-8",
],
"Expires": ["-1"],
"X-Ms-Failure-Cause": ["gateway"],
"X-Content-Type-Options": ["nosniff"],
"X-Msedge-Ref": [
"Ref A: 0DAC2D5643A7404DAE98A8F972EDD76A Ref B: CHI261140510042 Ref C: 2026-02-25T23:28:40Z",
],
"Pragma": ["no-cache"],
"X-Ms-Request-Id": [
"ca92703c-eee1-425f-bf36-91286519e05e",
],
"X-Ms-Correlation-Request-Id": [
"ca92703c-eee1-425f-bf36-91286519e05e",
],
"X-Ms-Routing-Request-Id": [
"EASTUS2EUAP:20260225T232840Z:ca92703c-eee1-425f-bf36-91286519e05e",
],
"Strict-Transport-Security": [
"max-age=31536000; includeSubDomains",
],
"X-Cache": ["CONFIG_...
Gomega truncated this representation as it exceeds 'format.MaxLength'.
Consider having the object provide a custom 'GomegaStringer' representation
or adjust the parameters in Gomega's 'format' package.
Learn more here: https://onsi.github.io/gomega/#adjusting-output
to match error
<*matchers.MatchRegexpMatcher | 0xc000d5ee10>: {
Regexp: "Version .* (doesn't exist|is disabled)",
Args: nil,
}
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_missing_info.go:82]: Expected
<*fmt.wrapError | 0xc0005302c0>:
failed to create HCP cluster illegal-hcp-cluster: failed starting cluster creation "illegal-hcp-cluster" in resourcegroup="illegal-ocp-version-67gxwd": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/illegal-ocp-version-67gxwd/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/illegal-hcp-cluster
--------------------------------------------------------------------------------
RESPONSE 400: 400 Bad Request
ERROR CODE: LocationNotAvailableForResourceType
--------------------------------------------------------------------------------
{
"error": {
"code": "LocationNotAvailableForResourceType",
"message": "The provided location 'eastus2euap' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'uksouth,switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2'."
}
}
--------------------------------------------------------------------------------
{
msg: "failed to create HCP cluster illegal-hcp-cluster: failed starting cluster creation \"illegal-hcp-cluster\" in resourcegroup=\"illegal-ocp-version-67gxwd\": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/illegal-ocp-version-67gxwd/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/illegal-hcp-cluster\n--------------------------------------------------------------------------------\nRESPONSE 400: 400 Bad Request\nERROR CODE: LocationNotAvailableForResourceType\n--------------------------------------------------------------------------------\n{\n \"error\": {\n \"code\": \"LocationNotAvailableForResourceType\",\n \"message\": \"The provided location 'eastus2euap' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'uksouth,switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2'.\"\n }\n}\n--------------------------------------------------------------------------------\n",
err: <*fmt.wrapError | 0xc0005302a0>{
msg: "failed starting cluster creation \"illegal-hcp-cluster\" in resourcegroup=\"illegal-ocp-version-67gxwd\": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/illegal-ocp-version-67gxwd/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/illegal-hcp-cluster\n--------------------------------------------------------------------------------\nRESPONSE 400: 400 Bad Request\nERROR CODE: LocationNotAvailableForResourceType\n--------------------------------------------------------------------------------\n{\n \"error\": {\n \"code\": \"LocationNotAvailableForResourceType\",\n \"message\": \"The provided location 'eastus2euap' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'uksouth,switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2'.\"\n }\n}\n--------------------------------------------------------------------------------\n",
err: <*exported.ResponseError | 0xc000d5ed80>{
ErrorCode: "LocationNotAvailableForResourceType",
StatusCode: 400,
RawResponse: {
Status: "400 Bad Request",
StatusCode: 400,
Proto: "HTTP/2.0",
ProtoMajor: 2,
ProtoMinor: 0,
Header: {
"Date": [
"Wed, 25 Feb 2026 23:28:40 GMT",
],
"Cache-Control": ["no-cache"],
"Content-Length": ["340"],
"Content-Type": [
"application/json; charset=utf-8",
],
"Expires": ["-1"],
"X-Ms-Failure-Cause": ["gateway"],
"X-Content-Type-Options": ["nosniff"],
"X-Msedge-Ref": [
"Ref A: 0DAC2D5643A7404DAE98A8F972EDD76A Ref B: CHI261140510042 Ref C: 2026-02-25T23:28:40Z",
],
"Pragma": ["no-cache"],
"X-Ms-Request-Id": [
"ca92703c-eee1-425f-bf36-91286519e05e",
],
"X-Ms-Correlation-Request-Id": [
"ca92703c-eee1-425f-bf36-91286519e05e",
],
"X-Ms-Routing-Request-Id": [
"EASTUS2EUAP:20260225T232840Z:ca92703c-eee1-425f-bf36-91286519e05e",
],
"Strict-Transport-Security": [
"max-age=31536000; includeSubDomains",
],
"X-Cache": ["CONFIG_...
Gomega truncated this representation as it exceeds 'format.MaxLength'.
Consider having the object provide a custom 'GomegaStringer' representation
or adjust the parameters in Gomega's 'format' package.
Learn more here: https://onsi.github.io/gomega/#adjusting-output
to match error
<*matchers.MatchRegexpMatcher | 0xc000d5ee10>: {
Regexp: "Version .* (doesn't exist|is disabled)",
Args: nil,
}fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_missing_info.go:82]: Expected
<*fmt.wrapError | 0xc00085b2e0>:
failed to create HCP cluster illegal-hcp-cluster: failed starting cluster creation "illegal-hcp-cluster" in resourcegroup="illegal-ocp-version-dlwl2b": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/illegal-ocp-version-dlwl2b/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/illegal-hcp-cluster
--------------------------------------------------------------------------------
RESPONSE 400: 400 Bad Request
ERROR CODE: LocationNotAvailableForResourceType
--------------------------------------------------------------------------------
{
"error": {
"code": "LocationNotAvailableForResourceType",
"message": "The provided location 'eastus2euap' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'uksouth,switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2'."
}
}
--------------------------------------------------------------------------------
{
msg: "failed to create HCP cluster illegal-hcp-cluster: failed starting cluster creation \"illegal-hcp-cluster\" in resourcegroup=\"illegal-ocp-version-dlwl2b\": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/illegal-ocp-version-dlwl2b/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/illegal-hcp-cluster\n--------------------------------------------------------------------------------\nRESPONSE 400: 400 Bad Request\nERROR CODE: LocationNotAvailableForResourceType\n--------------------------------------------------------------------------------\n{\n \"error\": {\n \"code\": \"LocationNotAvailableForResourceType\",\n \"message\": \"The provided location 'eastus2euap' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'uksouth,switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2'.\"\n }\n}\n--------------------------------------------------------------------------------\n",
err: <*fmt.wrapError | 0xc00085b2c0>{
msg: "failed starting cluster creation \"illegal-hcp-cluster\" in resourcegroup=\"illegal-ocp-version-dlwl2b\": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/illegal-ocp-version-dlwl2b/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/illegal-hcp-cluster\n--------------------------------------------------------------------------------\nRESPONSE 400: 400 Bad Request\nERROR CODE: LocationNotAvailableForResourceType\n--------------------------------------------------------------------------------\n{\n \"error\": {\n \"code\": \"LocationNotAvailableForResourceType\",\n \"message\": \"The provided location 'eastus2euap' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'uksouth,switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2'.\"\n }\n}\n--------------------------------------------------------------------------------\n",
err: <*exported.ResponseError | 0xc00104d080>{
ErrorCode: "LocationNotAvailableForResourceType",
StatusCode: 400,
RawResponse: {
Status: "400 Bad Request",
StatusCode: 400,
Proto: "HTTP/2.0",
ProtoMajor: 2,
ProtoMinor: 0,
Header: {
"Content-Length": ["340"],
"Content-Type": [
"application/json; charset=utf-8",
],
"X-Ms-Request-Id": [
"6dd460a5-e068-463f-a891-4a01a9e430de",
],
"X-Ms-Correlation-Request-Id": [
"6dd460a5-e068-463f-a891-4a01a9e430de",
],
"X-Content-Type-Options": ["nosniff"],
"X-Msedge-Ref": [
"Ref A: BDE668315FDE4ABFABDE511D62D6CB12 Ref B: CHI261140504054 Ref C: 2026-02-25T22:56:35Z",
],
"Pragma": ["no-cache"],
"Expires": ["-1"],
"X-Ms-Failure-Cause": ["gateway"],
"X-Ms-Routing-Request-Id": [
"EASTUS2EUAP:20260225T225635Z:6dd460a5-e068-463f-a891-4a01a9e430de",
],
"Strict-Transport-Security": [
"max-age=31536000; includeSubDomains",
],
"X-Cache": ["CONFIG_NOCACHE"],
"Date": [
"Wed, 25 Feb 2026 22:56:35 GMT",
],
"Cache-Control": ["n...
Gomega truncated this representation as it exceeds 'format.MaxLength'.
Consider having the object provide a custom 'GomegaStringer' representation
or adjust the parameters in Gomega's 'format' package.
Learn more here: https://onsi.github.io/gomega/#adjusting-output
to match error
<*matchers.MatchRegexpMatcher | 0xc00104d110>: {
Regexp: "Version .* (doesn't exist|is disabled)",
Args: nil,
}
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_missing_info.go:82]: Expected
<*fmt.wrapError | 0xc00085b2e0>:
failed to create HCP cluster illegal-hcp-cluster: failed starting cluster creation "illegal-hcp-cluster" in resourcegroup="illegal-ocp-version-dlwl2b": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/illegal-ocp-version-dlwl2b/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/illegal-hcp-cluster
--------------------------------------------------------------------------------
RESPONSE 400: 400 Bad Request
ERROR CODE: LocationNotAvailableForResourceType
--------------------------------------------------------------------------------
{
"error": {
"code": "LocationNotAvailableForResourceType",
"message": "The provided location 'eastus2euap' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'uksouth,switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2'."
}
}
--------------------------------------------------------------------------------
{
msg: "failed to create HCP cluster illegal-hcp-cluster: failed starting cluster creation \"illegal-hcp-cluster\" in resourcegroup=\"illegal-ocp-version-dlwl2b\": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/illegal-ocp-version-dlwl2b/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/illegal-hcp-cluster\n--------------------------------------------------------------------------------\nRESPONSE 400: 400 Bad Request\nERROR CODE: LocationNotAvailableForResourceType\n--------------------------------------------------------------------------------\n{\n \"error\": {\n \"code\": \"LocationNotAvailableForResourceType\",\n \"message\": \"The provided location 'eastus2euap' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'uksouth,switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2'.\"\n }\n}\n--------------------------------------------------------------------------------\n",
err: <*fmt.wrapError | 0xc00085b2c0>{
msg: "failed starting cluster creation \"illegal-hcp-cluster\" in resourcegroup=\"illegal-ocp-version-dlwl2b\": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/illegal-ocp-version-dlwl2b/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/illegal-hcp-cluster\n--------------------------------------------------------------------------------\nRESPONSE 400: 400 Bad Request\nERROR CODE: LocationNotAvailableForResourceType\n--------------------------------------------------------------------------------\n{\n \"error\": {\n \"code\": \"LocationNotAvailableForResourceType\",\n \"message\": \"The provided location 'eastus2euap' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'uksouth,switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2'.\"\n }\n}\n--------------------------------------------------------------------------------\n",
err: <*exported.ResponseError | 0xc00104d080>{
ErrorCode: "LocationNotAvailableForResourceType",
StatusCode: 400,
RawResponse: {
Status: "400 Bad Request",
StatusCode: 400,
Proto: "HTTP/2.0",
ProtoMajor: 2,
ProtoMinor: 0,
Header: {
"Content-Length": ["340"],
"Content-Type": [
"application/json; charset=utf-8",
],
"X-Ms-Request-Id": [
"6dd460a5-e068-463f-a891-4a01a9e430de",
],
"X-Ms-Correlation-Request-Id": [
"6dd460a5-e068-463f-a891-4a01a9e430de",
],
"X-Content-Type-Options": ["nosniff"],
"X-Msedge-Ref": [
"Ref A: BDE668315FDE4ABFABDE511D62D6CB12 Ref B: CHI261140504054 Ref C: 2026-02-25T22:56:35Z",
],
"Pragma": ["no-cache"],
"Expires": ["-1"],
"X-Ms-Failure-Cause": ["gateway"],
"X-Ms-Routing-Request-Id": [
"EASTUS2EUAP:20260225T225635Z:6dd460a5-e068-463f-a891-4a01a9e430de",
],
"Strict-Transport-Security": [
"max-age=31536000; includeSubDomains",
],
"X-Cache": ["CONFIG_NOCACHE"],
"Date": [
"Wed, 25 Feb 2026 22:56:35 GMT",
],
"Cache-Control": ["n...
Gomega truncated this representation as it exceeds 'format.MaxLength'.
Consider having the object provide a custom 'GomegaStringer' representation
or adjust the parameters in Gomega's 'format' package.
Learn more here: https://onsi.github.io/gomega/#adjusting-output
to match error
<*matchers.MatchRegexpMatcher | 0xc00104d110>: {
Regexp: "Version .* (doesn't exist|is disabled)",
Args: nil,
}Contributing tests (1)
Affected runs (2)
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
failed waiting for nodepool="<nodepool>" for cluster "<cluster>" in resourcegroup="<resource-group>" to finish creating: context deadline exceededfull failure pattern: failed waiting for nodepool="<nodepool>" for cluster "<cluster>" in resourcegroup="<resource-group>" to finish creating: context deadline exceeded Signal: IndeterminateSignal: Indeterminate — no prior history; active 2 day(s) — New failure pattern — no prior history | e2e | 2 | 22.22%2 of 9 job runs affected | IndeterminateSignal: Indeterminate — no prior history; active 2 day(s) — New failure pattern — no prior history | Show trend detailsFeb 16: 0 · Feb 17: 0 · Feb 18: 0 · Feb 19: 0 · Feb 20: 0 · Feb 21: 0 · Feb 22: 0 · Feb 23: 0 · Feb 24: 0 · Feb 25: 0 · Feb 26: 1 · Feb 27: 0 · Feb 28: 0 · Mar 1: 1 | none | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Full failure examples (2)fail [github.com/Azure/ARO-HCP/test/e2e/cluster_version_backlevel.go:190]: Unexpected error:
<*fmt.wrapError | 0xc0008101a0>:
failed waiting for nodepool="np-ver-4-19-7" for cluster "cluster-ver-4-19" in resourcegroup="rg-cluster-back-version-w8wfrk" to finish creating: context deadline exceeded
{
msg: "failed waiting for nodepool=\"np-ver-4-19-7\" for cluster \"cluster-ver-4-19\" in resourcegroup=\"rg-cluster-back-version-w8wfrk\" to finish creating: context deadline exceeded",
err: <context.deadlineExceededError>{},
}
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_version_backlevel.go:190]: Unexpected error:
<*fmt.wrapError | 0xc0008101a0>:
failed waiting for nodepool="np-ver-4-19-7" for cluster "cluster-ver-4-19" in resourcegroup="rg-cluster-back-version-w8wfrk" to finish creating: context deadline exceeded
{
msg: "failed waiting for nodepool=\"np-ver-4-19-7\" for cluster \"cluster-ver-4-19\" in resourcegroup=\"rg-cluster-back-version-w8wfrk\" to finish creating: context deadline exceeded",
err: <context.deadlineExceededError>{},
}
occurredfail [github.com/Azure/ARO-HCP/test/e2e/cluster_version_backlevel.go:190]: Unexpected error:
<*fmt.wrapError | 0xc000bee820>:
failed waiting for nodepool="np-ver-4-19-7" for cluster "cluster-ver-4-19" in resourcegroup="rg-cluster-back-version-ck5wpf" to finish creating: context deadline exceeded
{
msg: "failed waiting for nodepool=\"np-ver-4-19-7\" for cluster \"cluster-ver-4-19\" in resourcegroup=\"rg-cluster-back-version-ck5wpf\" to finish creating: context deadline exceeded",
err: <context.deadlineExceededError>{},
}
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_version_backlevel.go:190]: Unexpected error:
<*fmt.wrapError | 0xc000bee820>:
failed waiting for nodepool="np-ver-4-19-7" for cluster "cluster-ver-4-19" in resourcegroup="rg-cluster-back-version-ck5wpf" to finish creating: context deadline exceeded
{
msg: "failed waiting for nodepool=\"np-ver-4-19-7\" for cluster \"cluster-ver-4-19\" in resourcegroup=\"rg-cluster-back-version-ck5wpf\" to finish creating: context deadline exceeded",
err: <context.deadlineExceededError>{},
}
occurredContributing tests (1)
Affected runs (2)
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
UpdateHCPCluster timed out after <minutes> minutes; context deadline exceededfull failure pattern: UpdateHCPCluster timed out after <minutes> minutes; context deadline exceeded Signal: IndeterminateSignal: Indeterminate — no prior history; active 1 day(s) — New failure pattern — no prior history | e2e | 1 | 11.11%1 of 9 job runs affected | IndeterminateSignal: Indeterminate — no prior history; active 1 day(s) — New failure pattern — no prior history | Show trend detailsFeb 16: 0 · Feb 17: 0 · Feb 18: 0 · Feb 19: 0 · Feb 20: 0 · Feb 21: 0 · Feb 22: 0 · Feb 23: 0 · Feb 24: 0 · Feb 25: 0 · Feb 26: 0 · Feb 27: 0 · Feb 28: 1 · Mar 1: 0 | none | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Full failure examples (1)fail [github.com/Azure/ARO-HCP/test/e2e/cluster_update.go:192]: Unexpected error:
<*fmt.wrapErrors | 0xc000cec090>:
failed waiting for hcpCluster="patch-tags-cluster" in resourcegroup="patch-tags-pczlh6" to finish updating, caused by: timeout '10.000000' minutes exceeded during UpdateHCPCluster for cluster patch-tags-cluster in resource group patch-tags-pczlh6, error: context deadline exceeded
{
msg: "failed waiting for hcpCluster=\"patch-tags-cluster\" in resourcegroup=\"patch-tags-pczlh6\" to finish updating, caused by: timeout '10.000000' minutes exceeded during UpdateHCPCluster for cluster patch-tags-cluster in resource group patch-tags-pczlh6, error: context deadline exceeded",
errs: [
<*errors.errorString | 0xc00027e1b0>{
s: "timeout '10.000000' minutes exceeded during UpdateHCPCluster for cluster patch-tags-cluster in resource group patch-tags-pczlh6",
},
<context.deadlineExceededError>{},
],
}
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_update.go:192]: Unexpected error:
<*fmt.wrapErrors | 0xc000cec090>:
failed waiting for hcpCluster="patch-tags-cluster" in resourcegroup="patch-tags-pczlh6" to finish updating, caused by: timeout '10.000000' minutes exceeded during UpdateHCPCluster for cluster patch-tags-cluster in resource group patch-tags-pczlh6, error: context deadline exceeded
{
msg: "failed waiting for hcpCluster=\"patch-tags-cluster\" in resourcegroup=\"patch-tags-pczlh6\" to finish updating, caused by: timeout '10.000000' minutes exceeded during UpdateHCPCluster for cluster patch-tags-cluster in resource group patch-tags-pczlh6, error: context deadline exceeded",
errs: [
<*errors.errorString | 0xc00027e1b0>{
s: "timeout '10.000000' minutes exceeded during UpdateHCPCluster for cluster patch-tags-cluster in resource group patch-tags-pczlh6",
},
<context.deadlineExceededError>{},
],
}
occurredContributing tests (1)
Affected runs (1)
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||