<
Single Day: Oct 22▾
Pick a day
>
View JSON API
↗ Open Failure patterns for this day

Environment: INT

Time (UTC)RuntimeJobRegionFailed atResultPRFailed testsDetails
30m 0s branch-ci-Azure-ARO-HCP-main-e2e-integration-e2e-parallel-all
post-goodmerged PR
No additional run metadata captured
n/a unknown failed n/a 1 non-artifact-backed failure (no junit artifacts)
30m 0s branch-ci-Azure-ARO-HCP-main-e2e-integration-e2e-parallel-all
post-goodmerged PR
No additional run metadata captured
n/a unknown failed n/a 1 non-artifact-backed failure (no junit artifacts)
30m 1s branch-ci-Azure-ARO-HCP-main-e2e-integration-e2e-parallel-all
post-goodmerged PR
No additional run metadata captured
n/a unknown failed n/a 1 non-artifact-backed failure (no junit artifacts)
1h 20m 36s branch-ci-Azure-ARO-HCP-main-e2e-integration-e2e-parallel-all
post-goodmerged PR
No additional run metadata captured
westus3 unknown failed n/a 2
unknown (2)
ERROR CODE: DeadlineExceeded; detail message node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is <count>, want <count>; node pool condition AllNodesHealthy is False: <count> of <count> machines are not healthy Machine <node>: InspectionFailed: Waiting for a Node with spec.providerID azure:///subscriptions/<subscription>/resourcegroups/<resource-group>/providers/Microsoft.Compute/virtualMachines/<node> to exist Machine <node>: InspectionFailed: Waiting for a Node with spec.providerID azure:///subscriptions/<subscription>/resourcegroups/<resource-group>/providers/Microsoft.Compute/virtualMachines/<node> to exist; node pool condition AllMachinesReady is False: <count> of <count> machines are not ready Machine <node>: ReadyUnknown: * NodeHealthy: Waiting for a Node with spec.providerID azure:///subscriptions/<subscription>/resourcegroups/<resource-group>/providers/Microsoft.Compute/virtualMachines/<node> to exist Machine <node>: ReadyUnknown: * NodeHealthy: Waiting for a Node with spec.providerID azure:///subscriptions/<subscription>/resourcegroups/<resource-group>/providers/Microsoft.Compute/virtualMachines/<node> to exist; provider Microsoft.RedHatOpenShift
14:13:25 UTC · failed at unknown · test Customer should not be able to perform various invalid operations on cluster resources · suite integration/parallel/all
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/simple_negative_cases.go:159]: failed to create node pool np-1
Unexpected error:
    <*fmt.wrapError | 0x1e6e8804d3c0>: 
    failed to create NodePool np-1: failed waiting for nodepool="np-1" for cluster "negative-tests-cluster" in resourcegroup="rg-negative-tests-m5vrcvff2ctf" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westus3/hcpOperationStatuses/75b8b1f9-f718-45b2-b1ee-a3009f824cf8
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: DeadlineExceeded
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westus3/hcpOperationStatuses/75b8b1f9-f718-45b2-b1ee-a3009f824cf8",
      "name": "75b8b1f9-f718-45b2-b1ee-a3009f824cf8",
      "status": "Failed",
      "startTime": "2026-10-22T14:53:42.149827787Z",
      "endTime": "2026-10-22T15:12:50.02702561Z",
      "error": {
        "code": "DeadlineExceeded",
        "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine c2g2u9m5x1z4p0m-np-1-bfdzc-5kdm2: InspectionFailed: Waiting for a Node with spec.providerID azure:///subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-negative-tests-m5vrcvff2ctf--managed/providers/Microsoft.Compute/virtualMachines/c2g2u9m5x1z4p0m-np-1-bfdzc-5kdm2 to exist\nMachine c2g2u9m5x1z4p0m-np-1-bfdzc-kbcws: InspectionFailed: Waiting for a Node with spec.providerID azure:///subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-negative-tests-m5vrcvff2ctf--managed/providers/Microsoft.Compute/virtualMachines/c2g2u9m5x1z4p0m-np-1-bfdzc-kbcws to exist\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine c2g2u9m5x1z4p0m-np-1-bfdzc-5kdm2: ReadyUnknown: * NodeHealthy: Waiting for a Node with spec.providerID azure:///subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-negative-tests-m5vrcvff2ctf--managed/providers/Microsoft.Compute/virtualMachines/c2g2u9m5x1z4p0m-np-1-bfdzc-5kdm2 to exist\nMachine c2g2u9m5x1z4p0m-np-1-bfdzc-kbcws: ReadyUnknown: * NodeHealthy: Waiting for a Node with spec.providerID azure:///subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-negative-tests-m5vrcvff2ctf--managed/providers/Microsoft.Compute/virtualMachines/c2g2u9m5x1z4p0m-np-1-bfdzc-kbcws to exist\n"
      }
    }
    --------------------------------------------------------------------------------
    
    <Azure error internals omitted>
        Azure request IDs:
          GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westus3/hcpOperationStatuses/75b8b1f9-f718-45b2-b1ee-a3009f824cf8
            correlation ID: e990e88e-110c-474c-9878-aa84f73234da
            client request ID: unavailable
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/simple_negative_cases.go:159]: failed to create node pool np-1
Unexpected error:
    <*fmt.wrapError | 0x1e6e8804d3c0>: 
    failed to create NodePool np-1: failed waiting for nodepool="np-1" for cluster "negative-tests-cluster" in resourcegroup="rg-negative-tests-m5vrcvff2ctf" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westus3/hcpOperationStatuses/75b8b1f9-f718-45b2-b1ee-a3009f824cf8
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: DeadlineExceeded
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westus3/hcpOperationStatuses/75b8b1f9-f718-45b2-b1ee-a3009f824cf8",
      "name": "75b8b1f9-f718-45b2-b1ee-a3009f824cf8",
      "status": "Failed",
      "startTime": "2026-10-22T14:53:42.149827787Z",
      "endTime": "2026-10-22T15:12:50.02702561Z",
      "error": {
        "code": "DeadlineExceeded",
        "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine c2g2u9m5x1z4p0m-np-1-bfdzc-5kdm2: InspectionFailed: Waiting for a Node with spec.providerID azure:///subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-negative-tests-m5vrcvff2ctf--managed/providers/Microsoft.Compute/virtualMachines/c2g2u9m5x1z4p0m-np-1-bfdzc-5kdm2 to exist\nMachine c2g2u9m5x1z4p0m-np-1-bfdzc-kbcws: InspectionFailed: Waiting for a Node with spec.providerID azure:///subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-negative-tests-m5vrcvff2ctf--managed/providers/Microsoft.Compute/virtualMachines/c2g2u9m5x1z4p0m-np-1-bfdzc-kbcws to exist\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine c2g2u9m5x1z4p0m-np-1-bfdzc-5kdm2: ReadyUnknown: * NodeHealthy: Waiting for a Node with spec.providerID azure:///subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-negative-tests-m5vrcvff2ctf--managed/providers/Microsoft.Compute/virtualMachines/c2g2u9m5x1z4p0m-np-1-bfdzc-5kdm2 to exist\nMachine c2g2u9m5x1z4p0m-np-1-bfdzc-kbcws: ReadyUnknown: * NodeHealthy: Waiting for a Node with spec.providerID azure:///subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-negative-tests-m5vrcvff2ctf--managed/providers/Microsoft.Compute/virtualMachines/c2g2u9m5x1z4p0m-np-1-bfdzc-kbcws to exist\n"
      }
    }
    --------------------------------------------------------------------------------
    
    <Azure error internals omitted>
        Azure request IDs:
          GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westus3/hcpOperationStatuses/75b8b1f9-f718-45b2-b1ee-a3009f824cf8
            correlation ID: e990e88e-110c-474c-9878-aa84f73234da
            client request ID: unavailable
occurred
ERROR CODE: DeploymentFailed; detail code DeploymentScriptACIProvisioningTimeout; detail message The supporting resource: 'Azure Container Instance' did not provision during the expected time. Correlation Id: <uuid>. Please try again later, if the issue persists contact technical support for further investigation.
14:13:25 UTC · failed at unknown · test Cluster Etcd Managed HSM Encryption should create a cluster with Managed HSM keyVaultType and rotate the etcd KMS key · suite integration/parallel/all
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_etcd_mhsm.go:106]: failed to deploy and activate Managed HSM
Unexpected error:
    <*fmt.wrapError | 0x563a226e760>: 
    failed waiting for deployment "mhsm-deploy-etcd-mhsm" in resourcegroup="etcd-mhsm-blgzqwztp2gh" to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourcegroups/etcd-mhsm-blgzqwztp2gh/providers/Microsoft.Resources/deployments/mhsm-deploy-etcd-mhsm/operationStatuses/08584089270308456271
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: DeploymentFailed
    --------------------------------------------------------------------------------
    {
      "status": "Failed",
      "error": {
        "code": "DeploymentFailed",
        "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.",
        "details": [
          {
            "code": "Conflict",
            "message": "{\r\n  \"status\": \"failed\",\r\n  \"error\": {\r\n    \"code\": \"ResourceDeploymentFailure\",\r\n    \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'failed'.\",\r\n    \"details\": [\r\n      {\r\n        \"code\": \"DeploymentScriptACIProvisioningTimeout\",\r\n        \"message\": \"The supporting resource: 'Azure Container Instance' did not provision during the expected time. Correlation Id: c1661afd-ea85-487b-8e90-65a3f8664855. Please try again later, if the issue persists contact technical support for further investigation.\"\r\n      }\r\n    ]\r\n  }\r\n}"
          }
        ]
      }
    }
    --------------------------------------------------------------------------------
    
    <Azure error internals omitted>
        Azure request IDs:
          GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourcegroups/etcd-mhsm-blgzqwztp2gh/providers/Microsoft.Resources/deployments/mhsm-deploy-etcd-mhsm/operationStatuses/08584089270308456271
            correlation ID: 989b1e9c-002b-494d-ae3b-825cc5f4adf2
            client request ID: unavailable
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_etcd_mhsm.go:106]: failed to deploy and activate Managed HSM
Unexpected error:
    <*fmt.wrapError | 0x563a226e760>: 
    failed waiting for deployment "mhsm-deploy-etcd-mhsm" in resourcegroup="etcd-mhsm-blgzqwztp2gh" to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourcegroups/etcd-mhsm-blgzqwztp2gh/providers/Microsoft.Resources/deployments/mhsm-deploy-etcd-mhsm/operationStatuses/08584089270308456271
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: DeploymentFailed
    --------------------------------------------------------------------------------
    {
      "status": "Failed",
      "error": {
        "code": "DeploymentFailed",
        "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.",
        "details": [
          {
            "code": "Conflict",
            "message": "{\r\n  \"status\": \"failed\",\r\n  \"error\": {\r\n    \"code\": \"ResourceDeploymentFailure\",\r\n    \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'failed'.\",\r\n    \"details\": [\r\n      {\r\n        \"code\": \"DeploymentScriptACIProvisioningTimeout\",\r\n        \"message\": \"The supporting resource: 'Azure Container Instance' did not provision during the expected time. Correlation Id: c1661afd-ea85-487b-8e90-65a3f8664855. Please try again later, if the issue persists contact technical support for further investigation.\"\r\n      }\r\n    ]\r\n  }\r\n}"
          }
        ]
      }
    }
    --------------------------------------------------------------------------------
    
    <Azure error internals omitted>
        Azure request IDs:
          GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourcegroups/etcd-mhsm-blgzqwztp2gh/providers/Microsoft.Resources/deployments/mhsm-deploy-etcd-mhsm/operationStatuses/08584089270308456271
            correlation ID: 989b1e9c-002b-494d-ae3b-825cc5f4adf2
            client request ID: unavailable
occurred
1h 33m 43s branch-ci-Azure-ARO-HCP-main-e2e-integration-e2e-parallel-all
post-goodmerged PR
No additional run metadata captured
westus3 unknown failed n/a 2
unknown (2)
CreateOrUpdateExternalAuthAndWait timed out after <minutes> minutes; context deadline exceeded
10:51:53 UTC · failed at unknown · test Authorized CIDRs Connectivity should allow API access only from authorized VM IP · suite integration/parallel/all
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_authorized_cidrs_connectivity.go:276]: failed to create external auth config "ext-auth-cidr"
Unexpected error:
    <*fmt.wrapErrors | 0x2cc075309380>: 
    failed waiting for external auth "ext-auth-cidr" in resourcegroup="e2e-cidr-connectivity-rhgbbsmxx2nr" for cluster="cidr-connectivity-test" to finish creating or updating, caused by: timeout '15.000000' minutes exceeded during CreateOrUpdateExternalAuthAndWait for external auth ext-auth-cidr in cluster cidr-connectivity-test in resource group e2e-cidr-connectivity-rhgbbsmxx2nr, error: context deadline exceeded
    {
        msg: "failed waiting for external auth \"ext-auth-cidr\" in resourcegroup=\"e2e-cidr-connectivity-rhgbbsmxx2nr\" for cluster=\"cidr-connectivity-test\" to finish creating or updating, caused by: timeout '15.000000' minutes exceeded during CreateOrUpdateExternalAuthAndWait for external auth ext-auth-cidr in cluster cidr-connectivity-test in resource group e2e-cidr-connectivity-rhgbbsmxx2nr, error: context deadline exceeded",
        errs: [
            <*errors.errorString | 0x2cc074e74090>{
                s: "timeout '15.000000' minutes exceeded during CreateOrUpdateExternalAuthAndWait for external auth ext-auth-cidr in cluster cidr-connectivity-test in resource group e2e-cidr-connectivity-rhgbbsmxx2nr",
            },
            <context.deadlineExceededError>{},
        ],
    }
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_authorized_cidrs_connectivity.go:276]: failed to create external auth config "ext-auth-cidr"
Unexpected error:
    <*fmt.wrapErrors | 0x2cc075309380>: 
    failed waiting for external auth "ext-auth-cidr" in resourcegroup="e2e-cidr-connectivity-rhgbbsmxx2nr" for cluster="cidr-connectivity-test" to finish creating or updating, caused by: timeout '15.000000' minutes exceeded during CreateOrUpdateExternalAuthAndWait for external auth ext-auth-cidr in cluster cidr-connectivity-test in resource group e2e-cidr-connectivity-rhgbbsmxx2nr, error: context deadline exceeded
    {
        msg: "failed waiting for external auth \"ext-auth-cidr\" in resourcegroup=\"e2e-cidr-connectivity-rhgbbsmxx2nr\" for cluster=\"cidr-connectivity-test\" to finish creating or updating, caused by: timeout '15.000000' minutes exceeded during CreateOrUpdateExternalAuthAndWait for external auth ext-auth-cidr in cluster cidr-connectivity-test in resource group e2e-cidr-connectivity-rhgbbsmxx2nr, error: context deadline exceeded",
        errs: [
            <*errors.errorString | 0x2cc074e74090>{
                s: "timeout '15.000000' minutes exceeded during CreateOrUpdateExternalAuthAndWait for external auth ext-auth-cidr in cluster cidr-connectivity-test in resource group e2e-cidr-connectivity-rhgbbsmxx2nr",
            },
            <context.deadlineExceededError>{},
        ],
    }
occurred
console fixture namespace "<namespace>": configure Console ExternalAuth through RP: context deadline exceeded; Dex namespace="console-xzp4sq4"; Deployment/dex condition="Available" status="True" reason="MinimumReplicasAvailable"; Deployment/dex
10:51:53 UTC · failed at unknown · test ARO-HCP should be able to perform a control plane and node pool install with OCP candidate channel for 4.20 · suite integration/parallel/all
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:194]: failed to set up Console login fixture for cluster "cluster-candidate-4-20-8dwfln"
Unexpected error:
    <*fmt.wrapError | 0x22821099b6a0>: 
    console fixture namespace "console-xzp4sq4": configure Console ExternalAuth through RP: context deadline exceeded; Dex namespace="console-xzp4sq4"; Deployment/dex condition="Available" status="True" reason="MinimumReplicasAvailable"; Deployment/dex condition="Progressing" status="True" reason="NewReplicaSetAvailable"; Deployment/dex generation=1 observed=1 replicas=1 ready=1 available=1; Dex Pods=1; Dex ReplicaSets=1; Pod/dex-78f4dcc67d-swzhr condition="ContainersReady" status="True" reason=""; Pod/dex-78f4dcc67d-swzhr condition="Initialized" status="True" reason=""; Pod/dex-78f4dcc67d-swzhr condition="PodReadyToStartContainers" status="True" reason=""; Pod/dex-78f4dcc67d-swzhr condition="PodScheduled" status="True" reason=""; Pod/dex-78f4dcc67d-swzhr condition="Ready" status="True" reason=""; Pod/dex-78f4dcc67d-swzhr container="dex" ready=true restarts=0; Pod/dex-78f4dcc67d-swzhr phase="Running" reason=""
    {
        msg: "console fixture namespace \"console-xzp4sq4\": configure Console ExternalAuth through RP: context deadline exceeded; Dex namespace=\"console-xzp4sq4\"; Deployment/dex condition=\"Available\" status=\"True\" reason=\"MinimumReplicasAvailable\"; Deployment/dex condition=\"Progressing\" status=\"True\" reason=\"NewReplicaSetAvailable\"; Deployment/dex generation=1 observed=1 replicas=1 ready=1 available=1; Dex Pods=1; Dex ReplicaSets=1; Pod/dex-78f4dcc67d-swzhr condition=\"ContainersReady\" status=\"True\" reason=\"\"; Pod/dex-78f4dcc67d-swzhr condition=\"Initialized\" status=\"True\" reason=\"\"; Pod/dex-78f4dcc67d-swzhr condition=\"PodReadyToStartContainers\" status=\"True\" reason=\"\"; Pod/dex-78f4dcc67d-swzhr condition=\"PodScheduled\" status=\"True\" reason=\"\"; Pod/dex-78f4dcc67d-swzhr condition=\"Ready\" status=\"True\" reason=\"\"; Pod/dex-78f4dcc67d-swzhr container=\"dex\" ready=true restarts=0; Pod/dex-78f4dcc67d-swzhr phase=\"Running\" reason=\"\"",
        err: <*fmt.wrapError | 0x228210e20900>{
            msg: "configure Console ExternalAuth through RP: context deadline exceeded",
            err: <context.deadlineExceededError>{},
        },
    }
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:194]: failed to set up Console login fixture for cluster "cluster-candidate-4-20-8dwfln"
Unexpected error:
    <*fmt.wrapError | 0x22821099b6a0>: 
    console fixture namespace "console-xzp4sq4": configure Console ExternalAuth through RP: context deadline exceeded; Dex namespace="console-xzp4sq4"; Deployment/dex condition="Available" status="True" reason="MinimumReplicasAvailable"; Deployment/dex condition="Progressing" status="True" reason="NewReplicaSetAvailable"; Deployment/dex generation=1 observed=1 replicas=1 ready=1 available=1; Dex Pods=1; Dex ReplicaSets=1; Pod/dex-78f4dcc67d-swzhr condition="ContainersReady" status="True" reason=""; Pod/dex-78f4dcc67d-swzhr condition="Initialized" status="True" reason=""; Pod/dex-78f4dcc67d-swzhr condition="PodReadyToStartContainers" status="True" reason=""; Pod/dex-78f4dcc67d-swzhr condition="PodScheduled" status="True" reason=""; Pod/dex-78f4dcc67d-swzhr condition="Ready" status="True" reason=""; Pod/dex-78f4dcc67d-swzhr container="dex" ready=true restarts=0; Pod/dex-78f4dcc67d-swzhr phase="Running" reason=""
    {
        msg: "console fixture namespace \"console-xzp4sq4\": configure Console ExternalAuth through RP: context deadline exceeded; Dex namespace=\"console-xzp4sq4\"; Deployment/dex condition=\"Available\" status=\"True\" reason=\"MinimumReplicasAvailable\"; Deployment/dex condition=\"Progressing\" status=\"True\" reason=\"NewReplicaSetAvailable\"; Deployment/dex generation=1 observed=1 replicas=1 ready=1 available=1; Dex Pods=1; Dex ReplicaSets=1; Pod/dex-78f4dcc67d-swzhr condition=\"ContainersReady\" status=\"True\" reason=\"\"; Pod/dex-78f4dcc67d-swzhr condition=\"Initialized\" status=\"True\" reason=\"\"; Pod/dex-78f4dcc67d-swzhr condition=\"PodReadyToStartContainers\" status=\"True\" reason=\"\"; Pod/dex-78f4dcc67d-swzhr condition=\"PodScheduled\" status=\"True\" reason=\"\"; Pod/dex-78f4dcc67d-swzhr condition=\"Ready\" status=\"True\" reason=\"\"; Pod/dex-78f4dcc67d-swzhr container=\"dex\" ready=true restarts=0; Pod/dex-78f4dcc67d-swzhr phase=\"Running\" reason=\"\"",
        err: <*fmt.wrapError | 0x228210e20900>{
            msg: "configure Console ExternalAuth through RP: context deadline exceeded",
            err: <context.deadlineExceededError>{},
        },
    }
occurred
1h 12m 13s branch-ci-Azure-ARO-HCP-main-e2e-integration-e2e-parallel-all
post-goodmerged PR
No additional run metadata captured
westus3 unknown failed n/a 2
unknown (2)
CreateOrUpdateExternalAuthAndWait timed out after <minutes> minutes; context deadline exceeded
08:38:55 UTC · failed at unknown · test Customer should be able to create a cluster with an external auth config and get the external auth config · suite integration/parallel/all
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/external_auth_create.go:181]: failed to create external auth config external-auth
Unexpected error:
    <*fmt.wrapErrors | 0x32cbbfa08480>: 
    failed waiting for external auth "external-auth" in resourcegroup="external-auth-cluster-s8rx2m7kc8lm" for cluster="ea-cluster" to finish creating or updating, caused by: timeout '15.000000' minutes exceeded during CreateOrUpdateExternalAuthAndWait for external auth external-auth in cluster ea-cluster in resource group external-auth-cluster-s8rx2m7kc8lm, error: context deadline exceeded
    {
        msg: "failed waiting for external auth \"external-auth\" in resourcegroup=\"external-auth-cluster-s8rx2m7kc8lm\" for cluster=\"ea-cluster\" to finish creating or updating, caused by: timeout '15.000000' minutes exceeded during CreateOrUpdateExternalAuthAndWait for external auth external-auth in cluster ea-cluster in resource group external-auth-cluster-s8rx2m7kc8lm, error: context deadline exceeded",
        errs: [
            <*errors.errorString | 0x32cbbf682f10>{
                s: "timeout '15.000000' minutes exceeded during CreateOrUpdateExternalAuthAndWait for external auth external-auth in cluster ea-cluster in resource group external-auth-cluster-s8rx2m7kc8lm",
            },
            <context.deadlineExceededError>{},
        ],
    }
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/external_auth_create.go:181]: failed to create external auth config external-auth
Unexpected error:
    <*fmt.wrapErrors | 0x32cbbfa08480>: 
    failed waiting for external auth "external-auth" in resourcegroup="external-auth-cluster-s8rx2m7kc8lm" for cluster="ea-cluster" to finish creating or updating, caused by: timeout '15.000000' minutes exceeded during CreateOrUpdateExternalAuthAndWait for external auth external-auth in cluster ea-cluster in resource group external-auth-cluster-s8rx2m7kc8lm, error: context deadline exceeded
    {
        msg: "failed waiting for external auth \"external-auth\" in resourcegroup=\"external-auth-cluster-s8rx2m7kc8lm\" for cluster=\"ea-cluster\" to finish creating or updating, caused by: timeout '15.000000' minutes exceeded during CreateOrUpdateExternalAuthAndWait for external auth external-auth in cluster ea-cluster in resource group external-auth-cluster-s8rx2m7kc8lm, error: context deadline exceeded",
        errs: [
            <*errors.errorString | 0x32cbbf682f10>{
                s: "timeout '15.000000' minutes exceeded during CreateOrUpdateExternalAuthAndWait for external auth external-auth in cluster ea-cluster in resource group external-auth-cluster-s8rx2m7kc8lm",
            },
            <context.deadlineExceededError>{},
        ],
    }
occurred
console fixture namespace "<namespace>": configure Console ExternalAuth through RP: context deadline exceeded; Dex namespace="console-mr8rvbh"; Deployment/dex condition="Available" status="True" reason="MinimumReplicasAvailable"; Deployment/dex
08:38:55 UTC · failed at unknown · test ARO-HCP should be able to perform a control plane and node pool install with OCP candidate channel for 5.0 · suite integration/parallel/all
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:194]: failed to set up Console login fixture for cluster "cluster-candidate-5-0-6rxg4b"
Unexpected error:
    <*fmt.wrapError | 0x12de498cec20>: 
    console fixture namespace "console-mr8rvbh": configure Console ExternalAuth through RP: context deadline exceeded; Dex namespace="console-mr8rvbh"; Deployment/dex condition="Available" status="True" reason="MinimumReplicasAvailable"; Deployment/dex condition="Progressing" status="True" reason="NewReplicaSetAvailable"; Deployment/dex generation=1 observed=1 replicas=1 ready=1 available=1; Dex Pods=1; Dex ReplicaSets=1; Pod/dex-77657bcfc5-zhr7j condition="ContainersReady" status="True" reason=""; Pod/dex-77657bcfc5-zhr7j condition="Initialized" status="True" reason=""; Pod/dex-77657bcfc5-zhr7j condition="PodReadyToStartContainers" status="True" reason=""; Pod/dex-77657bcfc5-zhr7j condition="PodScheduled" status="True" reason=""; Pod/dex-77657bcfc5-zhr7j condition="Ready" status="True" reason=""; Pod/dex-77657bcfc5-zhr7j container="dex" ready=true restarts=0; Pod/dex-77657bcfc5-zhr7j phase="Running" reason=""
    {
        msg: "console fixture namespace \"console-mr8rvbh\": configure Console ExternalAuth through RP: context deadline exceeded; Dex namespace=\"console-mr8rvbh\"; Deployment/dex condition=\"Available\" status=\"True\" reason=\"MinimumReplicasAvailable\"; Deployment/dex condition=\"Progressing\" status=\"True\" reason=\"NewReplicaSetAvailable\"; Deployment/dex generation=1 observed=1 replicas=1 ready=1 available=1; Dex Pods=1; Dex ReplicaSets=1; Pod/dex-77657bcfc5-zhr7j condition=\"ContainersReady\" status=\"True\" reason=\"\"; Pod/dex-77657bcfc5-zhr7j condition=\"Initialized\" status=\"True\" reason=\"\"; Pod/dex-77657bcfc5-zhr7j condition=\"PodReadyToStartContainers\" status=\"True\" reason=\"\"; Pod/dex-77657bcfc5-zhr7j condition=\"PodScheduled\" status=\"True\" reason=\"\"; Pod/dex-77657bcfc5-zhr7j condition=\"Ready\" status=\"True\" reason=\"\"; Pod/dex-77657bcfc5-zhr7j container=\"dex\" ready=true restarts=0; Pod/dex-77657bcfc5-zhr7j phase=\"Running\" reason=\"\"",
        err: <*fmt.wrapError | 0x12de4a14f120>{
            msg: "configure Console ExternalAuth through RP: context deadline exceeded",
            err: <context.deadlineExceededError>{},
        },
    }
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:194]: failed to set up Console login fixture for cluster "cluster-candidate-5-0-6rxg4b"
Unexpected error:
    <*fmt.wrapError | 0x12de498cec20>: 
    console fixture namespace "console-mr8rvbh": configure Console ExternalAuth through RP: context deadline exceeded; Dex namespace="console-mr8rvbh"; Deployment/dex condition="Available" status="True" reason="MinimumReplicasAvailable"; Deployment/dex condition="Progressing" status="True" reason="NewReplicaSetAvailable"; Deployment/dex generation=1 observed=1 replicas=1 ready=1 available=1; Dex Pods=1; Dex ReplicaSets=1; Pod/dex-77657bcfc5-zhr7j condition="ContainersReady" status="True" reason=""; Pod/dex-77657bcfc5-zhr7j condition="Initialized" status="True" reason=""; Pod/dex-77657bcfc5-zhr7j condition="PodReadyToStartContainers" status="True" reason=""; Pod/dex-77657bcfc5-zhr7j condition="PodScheduled" status="True" reason=""; Pod/dex-77657bcfc5-zhr7j condition="Ready" status="True" reason=""; Pod/dex-77657bcfc5-zhr7j container="dex" ready=true restarts=0; Pod/dex-77657bcfc5-zhr7j phase="Running" reason=""
    {
        msg: "console fixture namespace \"console-mr8rvbh\": configure Console ExternalAuth through RP: context deadline exceeded; Dex namespace=\"console-mr8rvbh\"; Deployment/dex condition=\"Available\" status=\"True\" reason=\"MinimumReplicasAvailable\"; Deployment/dex condition=\"Progressing\" status=\"True\" reason=\"NewReplicaSetAvailable\"; Deployment/dex generation=1 observed=1 replicas=1 ready=1 available=1; Dex Pods=1; Dex ReplicaSets=1; Pod/dex-77657bcfc5-zhr7j condition=\"ContainersReady\" status=\"True\" reason=\"\"; Pod/dex-77657bcfc5-zhr7j condition=\"Initialized\" status=\"True\" reason=\"\"; Pod/dex-77657bcfc5-zhr7j condition=\"PodReadyToStartContainers\" status=\"True\" reason=\"\"; Pod/dex-77657bcfc5-zhr7j condition=\"PodScheduled\" status=\"True\" reason=\"\"; Pod/dex-77657bcfc5-zhr7j condition=\"Ready\" status=\"True\" reason=\"\"; Pod/dex-77657bcfc5-zhr7j container=\"dex\" ready=true restarts=0; Pod/dex-77657bcfc5-zhr7j phase=\"Running\" reason=\"\"",
        err: <*fmt.wrapError | 0x12de4a14f120>{
            msg: "configure Console ExternalAuth through RP: context deadline exceeded",
            err: <context.deadlineExceededError>{},
        },
    }
occurred
1h 32m 8s branch-ci-Azure-ARO-HCP-main-e2e-integration-e2e-parallel-all
post-goodmerged PR
No additional run metadata captured
westus3 n/a passed n/a 0 n/a
1h 29m 1s branch-ci-Azure-ARO-HCP-main-e2e-integration-e2e-parallel-all
post-goodmerged PR
No additional run metadata captured
westus3 unknown failed n/a 2
unknown (2)
DNS for route host <route-host> did not resolve: DNS for <route-host> did not resolve within <duration> (last error: lookup <host> on <dns-server>: dial udp <ip>: i/o timeout): context deadline exceeded
03:46:05 UTC · failed at unknown · test Customer should be able to retrieve cluster and nodepool status active versions · suite integration/parallel/all
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_nodepool_active_versions.go:149]: failed to verify simple web app runs on v20261001 cluster "v20261001"
Unexpected error:
    <*fmt.wrapError | 0x1342e25441a0>: 
    DNS for route host agnhost-e2e-sample-app-6g5k9.apps.aro.v20261001.1f89.westus3.aroapp-hcp.azure-test.net did not resolve: DNS for agnhost-e2e-sample-app-6g5k9.apps.aro.v20261001.1f89.westus3.aroapp-hcp.azure-test.net did not resolve within 10m0s (last error: lookup agnhost-e2e-sample-app-6g5k9.apps.aro.v20261001.1f89.westus3.aroapp-hcp.azure-test.net on 172.30.0.10:53: dial udp 172.30.0.10:53: i/o timeout): context deadline exceeded
    {
        msg: "DNS for route host agnhost-e2e-sample-app-6g5k9.apps.aro.v20261001.1f89.westus3.aroapp-hcp.azure-test.net did not resolve: DNS for agnhost-e2e-sample-app-6g5k9.apps.aro.v20261001.1f89.westus3.aroapp-hcp.azure-test.net did not resolve within 10m0s (last error: lookup agnhost-e2e-sample-app-6g5k9.apps.aro.v20261001.1f89.westus3.aroapp-hcp.azure-test.net on 172.30.0.10:53: dial udp 172.30.0.10:53: i/o timeout): context deadline exceeded",
        err: <*fmt.wrapError | 0x1342e251c060>{
            msg: "DNS for agnhost-e2e-sample-app-6g5k9.apps.aro.v20261001.1f89.westus3.aroapp-hcp.azure-test.net did not resolve within 10m0s (last error: lookup agnhost-e2e-sample-app-6g5k9.apps.aro.v20261001.1f89.westus3.aroapp-hcp.azure-test.net on 172.30.0.10:53: dial udp 172.30.0.10:53: i/o timeout): context deadline exceeded",
            err: <context.deadlineExceededError>{},
        },
    }
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_nodepool_active_versions.go:149]: failed to verify simple web app runs on v20261001 cluster "v20261001"
Unexpected error:
    <*fmt.wrapError | 0x1342e25441a0>: 
    DNS for route host agnhost-e2e-sample-app-6g5k9.apps.aro.v20261001.1f89.westus3.aroapp-hcp.azure-test.net did not resolve: DNS for agnhost-e2e-sample-app-6g5k9.apps.aro.v20261001.1f89.westus3.aroapp-hcp.azure-test.net did not resolve within 10m0s (last error: lookup agnhost-e2e-sample-app-6g5k9.apps.aro.v20261001.1f89.westus3.aroapp-hcp.azure-test.net on 172.30.0.10:53: dial udp 172.30.0.10:53: i/o timeout): context deadline exceeded
    {
        msg: "DNS for route host agnhost-e2e-sample-app-6g5k9.apps.aro.v20261001.1f89.westus3.aroapp-hcp.azure-test.net did not resolve: DNS for agnhost-e2e-sample-app-6g5k9.apps.aro.v20261001.1f89.westus3.aroapp-hcp.azure-test.net did not resolve within 10m0s (last error: lookup agnhost-e2e-sample-app-6g5k9.apps.aro.v20261001.1f89.westus3.aroapp-hcp.azure-test.net on 172.30.0.10:53: dial udp 172.30.0.10:53: i/o timeout): context deadline exceeded",
        err: <*fmt.wrapError | 0x1342e251c060>{
            msg: "DNS for agnhost-e2e-sample-app-6g5k9.apps.aro.v20261001.1f89.westus3.aroapp-hcp.azure-test.net did not resolve within 10m0s (last error: lookup agnhost-e2e-sample-app-6g5k9.apps.aro.v20261001.1f89.westus3.aroapp-hcp.azure-test.net on 172.30.0.10:53: dial udp 172.30.0.10:53: i/o timeout): context deadline exceeded",
            err: <context.deadlineExceededError>{},
        },
    }
occurred
console fixture namespace "<namespace>": configure Console ExternalAuth through RP: context deadline exceeded; Dex namespace="console-z2wqbfw"; Deployment/dex condition="Available" status="True" reason="MinimumReplicasAvailable"; Deployment/dex
03:46:05 UTC · failed at unknown · test ARO-HCP should be able to perform a control plane and node pool install with OCP candidate channel for 4.20 · suite integration/parallel/all
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:194]: failed to set up Console login fixture for cluster "cluster-candidate-4-20-9js6f6"
Unexpected error:
    <*fmt.wrapError | 0x10354e044ea0>: 
    console fixture namespace "console-z2wqbfw": configure Console ExternalAuth through RP: context deadline exceeded; Dex namespace="console-z2wqbfw"; Deployment/dex condition="Available" status="True" reason="MinimumReplicasAvailable"; Deployment/dex condition="Progressing" status="True" reason="NewReplicaSetAvailable"; Deployment/dex generation=1 observed=1 replicas=1 ready=1 available=1; Dex Pods=1; Dex ReplicaSets=1; Pod/dex-78f4dcc67d-6mptx condition="ContainersReady" status="True" reason=""; Pod/dex-78f4dcc67d-6mptx condition="Initialized" status="True" reason=""; Pod/dex-78f4dcc67d-6mptx condition="PodReadyToStartContainers" status="True" reason=""; Pod/dex-78f4dcc67d-6mptx condition="PodScheduled" status="True" reason=""; Pod/dex-78f4dcc67d-6mptx condition="Ready" status="True" reason=""; Pod/dex-78f4dcc67d-6mptx container="dex" ready=true restarts=0; Pod/dex-78f4dcc67d-6mptx phase="Running" reason=""
    {
        msg: "console fixture namespace \"console-z2wqbfw\": configure Console ExternalAuth through RP: context deadline exceeded; Dex namespace=\"console-z2wqbfw\"; Deployment/dex condition=\"Available\" status=\"True\" reason=\"MinimumReplicasAvailable\"; Deployment/dex condition=\"Progressing\" status=\"True\" reason=\"NewReplicaSetAvailable\"; Deployment/dex generation=1 observed=1 replicas=1 ready=1 available=1; Dex Pods=1; Dex ReplicaSets=1; Pod/dex-78f4dcc67d-6mptx condition=\"ContainersReady\" status=\"True\" reason=\"\"; Pod/dex-78f4dcc67d-6mptx condition=\"Initialized\" status=\"True\" reason=\"\"; Pod/dex-78f4dcc67d-6mptx condition=\"PodReadyToStartContainers\" status=\"True\" reason=\"\"; Pod/dex-78f4dcc67d-6mptx condition=\"PodScheduled\" status=\"True\" reason=\"\"; Pod/dex-78f4dcc67d-6mptx condition=\"Ready\" status=\"True\" reason=\"\"; Pod/dex-78f4dcc67d-6mptx container=\"dex\" ready=true restarts=0; Pod/dex-78f4dcc67d-6mptx phase=\"Running\" reason=\"\"",
        err: <*fmt.wrapError | 0x10354ed78880>{
            msg: "configure Console ExternalAuth through RP: context deadline exceeded",
            err: <context.deadlineExceededError>{},
        },
    }
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:194]: failed to set up Console login fixture for cluster "cluster-candidate-4-20-9js6f6"
Unexpected error:
    <*fmt.wrapError | 0x10354e044ea0>: 
    console fixture namespace "console-z2wqbfw": configure Console ExternalAuth through RP: context deadline exceeded; Dex namespace="console-z2wqbfw"; Deployment/dex condition="Available" status="True" reason="MinimumReplicasAvailable"; Deployment/dex condition="Progressing" status="True" reason="NewReplicaSetAvailable"; Deployment/dex generation=1 observed=1 replicas=1 ready=1 available=1; Dex Pods=1; Dex ReplicaSets=1; Pod/dex-78f4dcc67d-6mptx condition="ContainersReady" status="True" reason=""; Pod/dex-78f4dcc67d-6mptx condition="Initialized" status="True" reason=""; Pod/dex-78f4dcc67d-6mptx condition="PodReadyToStartContainers" status="True" reason=""; Pod/dex-78f4dcc67d-6mptx condition="PodScheduled" status="True" reason=""; Pod/dex-78f4dcc67d-6mptx condition="Ready" status="True" reason=""; Pod/dex-78f4dcc67d-6mptx container="dex" ready=true restarts=0; Pod/dex-78f4dcc67d-6mptx phase="Running" reason=""
    {
        msg: "console fixture namespace \"console-z2wqbfw\": configure Console ExternalAuth through RP: context deadline exceeded; Dex namespace=\"console-z2wqbfw\"; Deployment/dex condition=\"Available\" status=\"True\" reason=\"MinimumReplicasAvailable\"; Deployment/dex condition=\"Progressing\" status=\"True\" reason=\"NewReplicaSetAvailable\"; Deployment/dex generation=1 observed=1 replicas=1 ready=1 available=1; Dex Pods=1; Dex ReplicaSets=1; Pod/dex-78f4dcc67d-6mptx condition=\"ContainersReady\" status=\"True\" reason=\"\"; Pod/dex-78f4dcc67d-6mptx condition=\"Initialized\" status=\"True\" reason=\"\"; Pod/dex-78f4dcc67d-6mptx condition=\"PodReadyToStartContainers\" status=\"True\" reason=\"\"; Pod/dex-78f4dcc67d-6mptx condition=\"PodScheduled\" status=\"True\" reason=\"\"; Pod/dex-78f4dcc67d-6mptx condition=\"Ready\" status=\"True\" reason=\"\"; Pod/dex-78f4dcc67d-6mptx container=\"dex\" ready=true restarts=0; Pod/dex-78f4dcc67d-6mptx phase=\"Running\" reason=\"\"",
        err: <*fmt.wrapError | 0x10354ed78880>{
            msg: "configure Console ExternalAuth through RP: context deadline exceeded",
            err: <context.deadlineExceededError>{},
        },
    }
occurred
1h 30m 49s branch-ci-Azure-ARO-HCP-main-e2e-integration-e2e-parallel-all
post-goodmerged PR
No additional run metadata captured
westus3 n/a passed n/a 0 n/a