CIHealth
<
Single Day: Mar 12▾
Pick a day
>
View JSON API
↗ Open Failure patterns for this day

Environment: PROD

Time (UTC)JobFailed atResultPRFailed testsDetails
periodic-ci-Azure-ARO-HCP-main-periodic-prod-e2e-parallel
post-goodmerged PR
No additional run metadata captured
 n/a passed n/a 0 n/a
periodic-ci-Azure-ARO-HCP-main-periodic-prod-e2e-parallel
post-goodmerged PR
No additional run metadata captured
 e2e failed n/a 1
e2e (1)
ERROR CODE: ResourceGroupDeletionFailed
02:54:14 UTC · failed at e2e · test Customer should be able to create a cluster with an external auth config and get the external auth config · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:243]: Unexpected error:
    <*errors.joinError | 0xc001716168>: 
    failed to cleanup resource group: failed waiting for resourcegroup="external-auth-cluster-b67ng8" to finish deleting: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/operationresults/eyJqb2JJZCI6IlJFU09VUkNFR1JPVVBERUxFVElPTkpPQi1FWFRFUk5BTDoyREFVVEg6MkRDTFVTVEVSOjJEQjY3Tkc4LUVBU1RVUzIiLCJqb2JMb2NhdGlvbiI6ImVhc3R1czIifQ
    --------------------------------------------------------------------------------
    RESPONSE 409: 409 Conflict
    ERROR CODE: ResourceGroupDeletionFailed
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "ResourceGroupDeletionFailed",
        "message": "The deletion of resource group 'external-auth-cluster-b67ng8' did not finish within allowed time. Resource group provisioning state rolled back. Please check audit logs for more details."
      }
    }
    --------------------------------------------------------------------------------
    
    {
        errs: [
            <*fmt.wrapError | 0xc000f5e3a0>{
                msg: "failed to cleanup resource group: failed waiting for resourcegroup=\"external-auth-cluster-b67ng8\" to finish deleting: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/operationresults/eyJqb2JJZCI6IlJFU09VUkNFR1JPVVBERUxFVElPTkpPQi1FWFRFUk5BTDoyREFVVEg6MkRDTFVTVEVSOjJEQjY3Tkc4LUVBU1RVUzIiLCJqb2JMb2NhdGlvbiI6ImVhc3R1czIifQ\n--------------------------------------------------------------------------------\nRESPONSE 409: 409 Conflict\nERROR CODE: ResourceGroupDeletionFailed\n--------------------------------------------------------------------------------\n{\n  \"error\": {\n    \"code\": \"ResourceGroupDeletionFailed\",\n    \"message\": \"The deletion of resource group 'external-auth-cluster-b67ng8' did not finish within allowed time. Resource group provisioning state rolled back. Please check audit logs for more details.\"\n  }\n}\n--------------------------------------------------------------------------------\n",
                err: <*fmt.wrapError | 0xc000f5e380>{
                    msg: "failed waiting for resourcegroup=\"external-auth-cluster-b67ng8\" to finish deleting: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/operationresults/eyJqb2JJZCI6IlJFU09VUkNFR1JPVVBERUxFVElPTkpPQi1FWFRFUk5BTDoyREFVVEg6MkRDTFVTVEVSOjJEQjY3Tkc4LUVBU1RVUzIiLCJqb2JMb2NhdGlvbiI6ImVhc3R1czIifQ\n--------------------------------------------------------------------------------\nRESPONSE 409: 409 Conflict\nERROR CODE: ResourceGroupDeletionFailed\n--------------------------------------------------------------------------------\n{\n  \"error\": {\n    \"code\": \"ResourceGroupDeletionFailed\",\n    \"message\": \"The deletion of resource group 'external-auth-cluster-b67ng8' did not finish within allowed time. Resource group provisioning state rolled back. Please check audit logs for more details.\"\n  }\n}\n--------------------------------------------------------------------------------\n",
                    err: <*exported.ResponseError | 0xc001332a50>{
                        ErrorCode: "ResourceGroupDeletionFailed",
                        StatusCode: 409,
                        RawResponse: {
                            Status: "409 Conflict",
                            StatusCode: 409,
                            Proto: "HTTP/2.0",
                            ProtoMajor: 2,
                            ProtoMinor: 0,
                            Header: {
                                "Cache-Control": ["no-cache"],
                                "Pragma": ["no-cache"],
                                "Content-Type": [
                                    "application/json; charset=utf-8",
                                ],
                                "X-Ms-Correlation-Request-Id": [
                                    "8a7a4e2c-0c5c-420e-a471-80fa19b53da3",
                                ],
                                "X-Cache": ["CONFIG_NOCACHE"],
                                "Date": [
                                    "Thu, 12 Mar 2026 04:04:21 GMT",
                                ],
                                "Content-Length": ["247"],
                                "X-Ms-Ratelimit-Remaining-Subscription-Global-Reads": ["3749"],
                                "X-Ms-Routing-Request-Id": [
                                    "CANADACENTRAL:20260312T040421Z:8a7a4e2c-0c5c-420e-a471-80fa19b53da3",
                                ],
                                "X-Ms-Request-Id": [
                                    "8a7a4e2c-0c5c-420e-a471-80fa19b53da3",
                                ],
                                "Strict-Transport-Security": [
                                    "max-age=31536000; includeSubDomains",
                                ],
                                "X-Msedge-Ref": [
                                    "Ref A: D8B73840D3704...

Gomega truncated this representation as it exceeds 'format.MaxLength'.
Consider having the object provide a custom 'GomegaStringer' representation
or adjust the parameters in Gomega's 'format' package.

Learn more here: https://onsi.github.io/gomega/#adjusting-output

occurred
fail [github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:243]: Unexpected error:
    <*errors.joinError | 0xc001716168>: 
    failed to cleanup resource group: failed waiting for resourcegroup="external-auth-cluster-b67ng8" to finish deleting: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/operationresults/eyJqb2JJZCI6IlJFU09VUkNFR1JPVVBERUxFVElPTkpPQi1FWFRFUk5BTDoyREFVVEg6MkRDTFVTVEVSOjJEQjY3Tkc4LUVBU1RVUzIiLCJqb2JMb2NhdGlvbiI6ImVhc3R1czIifQ
    --------------------------------------------------------------------------------
    RESPONSE 409: 409 Conflict
    ERROR CODE: ResourceGroupDeletionFailed
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "ResourceGroupDeletionFailed",
        "message": "The deletion of resource group 'external-auth-cluster-b67ng8' did not finish within allowed time. Resource group provisioning state rolled back. Please check audit logs for more details."
      }
    }
    --------------------------------------------------------------------------------
    
    {
        errs: [
            <*fmt.wrapError | 0xc000f5e3a0>{
                msg: "failed to cleanup resource group: failed waiting for resourcegroup=\"external-auth-cluster-b67ng8\" to finish deleting: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/operationresults/eyJqb2JJZCI6IlJFU09VUkNFR1JPVVBERUxFVElPTkpPQi1FWFRFUk5BTDoyREFVVEg6MkRDTFVTVEVSOjJEQjY3Tkc4LUVBU1RVUzIiLCJqb2JMb2NhdGlvbiI6ImVhc3R1czIifQ\n--------------------------------------------------------------------------------\nRESPONSE 409: 409 Conflict\nERROR CODE: ResourceGroupDeletionFailed\n--------------------------------------------------------------------------------\n{\n  \"error\": {\n    \"code\": \"ResourceGroupDeletionFailed\",\n    \"message\": \"The deletion of resource group 'external-auth-cluster-b67ng8' did not finish within allowed time. Resource group provisioning state rolled back. Please check audit logs for more details.\"\n  }\n}\n--------------------------------------------------------------------------------\n",
                err: <*fmt.wrapError | 0xc000f5e380>{
                    msg: "failed waiting for resourcegroup=\"external-auth-cluster-b67ng8\" to finish deleting: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/operationresults/eyJqb2JJZCI6IlJFU09VUkNFR1JPVVBERUxFVElPTkpPQi1FWFRFUk5BTDoyREFVVEg6MkRDTFVTVEVSOjJEQjY3Tkc4LUVBU1RVUzIiLCJqb2JMb2NhdGlvbiI6ImVhc3R1czIifQ\n--------------------------------------------------------------------------------\nRESPONSE 409: 409 Conflict\nERROR CODE: ResourceGroupDeletionFailed\n--------------------------------------------------------------------------------\n{\n  \"error\": {\n    \"code\": \"ResourceGroupDeletionFailed\",\n    \"message\": \"The deletion of resource group 'external-auth-cluster-b67ng8' did not finish within allowed time. Resource group provisioning state rolled back. Please check audit logs for more details.\"\n  }\n}\n--------------------------------------------------------------------------------\n",
                    err: <*exported.ResponseError | 0xc001332a50>{
                        ErrorCode: "ResourceGroupDeletionFailed",
                        StatusCode: 409,
                        RawResponse: {
                            Status: "409 Conflict",
                            StatusCode: 409,
                            Proto: "HTTP/2.0",
                            ProtoMajor: 2,
                            ProtoMinor: 0,
                            Header: {
                                "Cache-Control": ["no-cache"],
                                "Pragma": ["no-cache"],
                                "Content-Type": [
                                    "application/json; charset=utf-8",
                                ],
                                "X-Ms-Correlation-Request-Id": [
                                    "8a7a4e2c-0c5c-420e-a471-80fa19b53da3",
                                ],
                                "X-Cache": ["CONFIG_NOCACHE"],
                                "Date": [
                                    "Thu, 12 Mar 2026 04:04:21 GMT",
                                ],
                                "Content-Length": ["247"],
                                "X-Ms-Ratelimit-Remaining-Subscription-Global-Reads": ["3749"],
                                "X-Ms-Routing-Request-Id": [
                                    "CANADACENTRAL:20260312T040421Z:8a7a4e2c-0c5c-420e-a471-80fa19b53da3",
                                ],
                                "X-Ms-Request-Id": [
                                    "8a7a4e2c-0c5c-420e-a471-80fa19b53da3",
                                ],
                                "Strict-Transport-Security": [
                                    "max-age=31536000; includeSubDomains",
                                ],
                                "X-Msedge-Ref": [
                                    "Ref A: D8B73840D3704...

Gomega truncated this representation as it exceeds 'format.MaxLength'.
Consider having the object provide a custom 'GomegaStringer' representation
or adjust the parameters in Gomega's 'format' package.

Learn more here: https://onsi.github.io/gomega/#adjusting-output

occurred
periodic-ci-Azure-ARO-HCP-main-periodic-prod-e2e-parallel
post-goodmerged PR
No additional run metadata captured
 e2e failed n/a 2
e2e (2)
ERROR CODE: ResourceGroupDeletionFailed
00:28:00 UTC · failed at e2e · test Customer should be able to list HCP clusters without node pools at both subscription and resource group levels · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:243]: Unexpected error:
    <*errors.joinError | 0xc000d743d8>: 
    failed to cleanup resource group: failed waiting for resourcegroup="cluster-listing-76k4gt" to finish deleting: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/operationresults/eyJqb2JJZCI6IlJFU09VUkNFR1JPVVBERUxFVElPTkpPQi1DTFVTVEVSOjJETElTVElORzoyRDc2SzRHVC1FQVNUVVMyIiwiam9iTG9jYXRpb24iOiJlYXN0dXMyIn0
    --------------------------------------------------------------------------------
    RESPONSE 409: 409 Conflict
    ERROR CODE: ResourceGroupDeletionFailed
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "ResourceGroupDeletionFailed",
        "message": "The deletion of resource group 'cluster-listing-76k4gt' did not finish within allowed time. Resource group provisioning state rolled back. Please check audit logs for more details."
      }
    }
    --------------------------------------------------------------------------------
    
    {
        errs: [
            <*fmt.wrapError | 0xc0004aaa40>{
                msg: "failed to cleanup resource group: failed waiting for resourcegroup=\"cluster-listing-76k4gt\" to finish deleting: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/operationresults/eyJqb2JJZCI6IlJFU09VUkNFR1JPVVBERUxFVElPTkpPQi1DTFVTVEVSOjJETElTVElORzoyRDc2SzRHVC1FQVNUVVMyIiwiam9iTG9jYXRpb24iOiJlYXN0dXMyIn0\n--------------------------------------------------------------------------------\nRESPONSE 409: 409 Conflict\nERROR CODE: ResourceGroupDeletionFailed\n--------------------------------------------------------------------------------\n{\n  \"error\": {\n    \"code\": \"ResourceGroupDeletionFailed\",\n    \"message\": \"The deletion of resource group 'cluster-listing-76k4gt' did not finish within allowed time. Resource group provisioning state rolled back. Please check audit logs for more details.\"\n  }\n}\n--------------------------------------------------------------------------------\n",
                err: <*fmt.wrapError | 0xc0004aaa20>{
                    msg: "failed waiting for resourcegroup=\"cluster-listing-76k4gt\" to finish deleting: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/operationresults/eyJqb2JJZCI6IlJFU09VUkNFR1JPVVBERUxFVElPTkpPQi1DTFVTVEVSOjJETElTVElORzoyRDc2SzRHVC1FQVNUVVMyIiwiam9iTG9jYXRpb24iOiJlYXN0dXMyIn0\n--------------------------------------------------------------------------------\nRESPONSE 409: 409 Conflict\nERROR CODE: ResourceGroupDeletionFailed\n--------------------------------------------------------------------------------\n{\n  \"error\": {\n    \"code\": \"ResourceGroupDeletionFailed\",\n    \"message\": \"The deletion of resource group 'cluster-listing-76k4gt' did not finish within allowed time. Resource group provisioning state rolled back. Please check audit logs for more details.\"\n  }\n}\n--------------------------------------------------------------------------------\n",
                    err: <*exported.ResponseError | 0xc001422fc0>{
                        ErrorCode: "ResourceGroupDeletionFailed",
                        StatusCode: 409,
                        RawResponse: {
                            Status: "409 Conflict",
                            StatusCode: 409,
                            Proto: "HTTP/2.0",
                            ProtoMajor: 2,
                            ProtoMinor: 0,
                            Header: {
                                "X-Ms-Request-Id": [
                                    "c656b4eb-b02d-4689-9e61-ae252c2c4c34",
                                ],
                                "X-Ms-Correlation-Request-Id": [
                                    "c656b4eb-b02d-4689-9e61-ae252c2c4c34",
                                ],
                                "X-Ms-Routing-Request-Id": [
                                    "CANADAEAST:20260312T011924Z:c656b4eb-b02d-4689-9e61-ae252c2c4c34",
                                ],
                                "Strict-Transport-Security": [
                                    "max-age=31536000; includeSubDomains",
                                ],
                                "X-Cache": ["CONFIG_NOCACHE"],
                                "Cache-Control": ["no-cache"],
                                "Pragma": ["no-cache"],
                                "Expires": ["-1"],
                                "X-Msedge-Ref": [
                                    "Ref A: AAC5210B403E49FB999F61B4009EE151 Ref B: CHI261140501034 Ref C: 2026-03-12T01:19:24Z",
                                ],
                                "Content-Length": ["241"],
                                "Content-Type": [
                                    "application/json; charset=utf-8",
                                ],
                                "X-Ms-Ratelimit-Remaining-Subscription-Reads": ["249"],
                                "X-Ms-Rateli...

Gomega truncated this representation as it exceeds 'format.MaxLength'.
Consider having the object provide a custom 'GomegaStringer' representation
or adjust the parameters in Gomega's 'format' package.

Learn more here: https://onsi.github.io/gomega/#adjusting-output

occurred
fail [github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:243]: Unexpected error:
    <*errors.joinError | 0xc000d743d8>: 
    failed to cleanup resource group: failed waiting for resourcegroup="cluster-listing-76k4gt" to finish deleting: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/operationresults/eyJqb2JJZCI6IlJFU09VUkNFR1JPVVBERUxFVElPTkpPQi1DTFVTVEVSOjJETElTVElORzoyRDc2SzRHVC1FQVNUVVMyIiwiam9iTG9jYXRpb24iOiJlYXN0dXMyIn0
    --------------------------------------------------------------------------------
    RESPONSE 409: 409 Conflict
    ERROR CODE: ResourceGroupDeletionFailed
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "ResourceGroupDeletionFailed",
        "message": "The deletion of resource group 'cluster-listing-76k4gt' did not finish within allowed time. Resource group provisioning state rolled back. Please check audit logs for more details."
      }
    }
    --------------------------------------------------------------------------------
    
    {
        errs: [
            <*fmt.wrapError | 0xc0004aaa40>{
                msg: "failed to cleanup resource group: failed waiting for resourcegroup=\"cluster-listing-76k4gt\" to finish deleting: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/operationresults/eyJqb2JJZCI6IlJFU09VUkNFR1JPVVBERUxFVElPTkpPQi1DTFVTVEVSOjJETElTVElORzoyRDc2SzRHVC1FQVNUVVMyIiwiam9iTG9jYXRpb24iOiJlYXN0dXMyIn0\n--------------------------------------------------------------------------------\nRESPONSE 409: 409 Conflict\nERROR CODE: ResourceGroupDeletionFailed\n--------------------------------------------------------------------------------\n{\n  \"error\": {\n    \"code\": \"ResourceGroupDeletionFailed\",\n    \"message\": \"The deletion of resource group 'cluster-listing-76k4gt' did not finish within allowed time. Resource group provisioning state rolled back. Please check audit logs for more details.\"\n  }\n}\n--------------------------------------------------------------------------------\n",
                err: <*fmt.wrapError | 0xc0004aaa20>{
                    msg: "failed waiting for resourcegroup=\"cluster-listing-76k4gt\" to finish deleting: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/operationresults/eyJqb2JJZCI6IlJFU09VUkNFR1JPVVBERUxFVElPTkpPQi1DTFVTVEVSOjJETElTVElORzoyRDc2SzRHVC1FQVNUVVMyIiwiam9iTG9jYXRpb24iOiJlYXN0dXMyIn0\n--------------------------------------------------------------------------------\nRESPONSE 409: 409 Conflict\nERROR CODE: ResourceGroupDeletionFailed\n--------------------------------------------------------------------------------\n{\n  \"error\": {\n    \"code\": \"ResourceGroupDeletionFailed\",\n    \"message\": \"The deletion of resource group 'cluster-listing-76k4gt' did not finish within allowed time. Resource group provisioning state rolled back. Please check audit logs for more details.\"\n  }\n}\n--------------------------------------------------------------------------------\n",
                    err: <*exported.ResponseError | 0xc001422fc0>{
                        ErrorCode: "ResourceGroupDeletionFailed",
                        StatusCode: 409,
                        RawResponse: {
                            Status: "409 Conflict",
                            StatusCode: 409,
                            Proto: "HTTP/2.0",
                            ProtoMajor: 2,
                            ProtoMinor: 0,
                            Header: {
                                "X-Ms-Request-Id": [
                                    "c656b4eb-b02d-4689-9e61-ae252c2c4c34",
                                ],
                                "X-Ms-Correlation-Request-Id": [
                                    "c656b4eb-b02d-4689-9e61-ae252c2c4c34",
                                ],
                                "X-Ms-Routing-Request-Id": [
                                    "CANADAEAST:20260312T011924Z:c656b4eb-b02d-4689-9e61-ae252c2c4c34",
                                ],
                                "Strict-Transport-Security": [
                                    "max-age=31536000; includeSubDomains",
                                ],
                                "X-Cache": ["CONFIG_NOCACHE"],
                                "Cache-Control": ["no-cache"],
                                "Pragma": ["no-cache"],
                                "Expires": ["-1"],
                                "X-Msedge-Ref": [
                                    "Ref A: AAC5210B403E49FB999F61B4009EE151 Ref B: CHI261140501034 Ref C: 2026-03-12T01:19:24Z",
                                ],
                                "Content-Length": ["241"],
                                "Content-Type": [
                                    "application/json; charset=utf-8",
                                ],
                                "X-Ms-Ratelimit-Remaining-Subscription-Reads": ["249"],
                                "X-Ms-Rateli...

Gomega truncated this representation as it exceeds 'format.MaxLength'.
Consider having the object provide a custom 'GomegaStringer' representation
or adjust the parameters in Gomega's 'format' package.

Learn more here: https://onsi.github.io/gomega/#adjusting-output

occurred
Interrupted by User
00:28:00 UTC · failed at e2e · test Authorized CIDRs Connectivity should allow API access only from authorized VM IP · suite prod/parallel
Show raw failure
fail [:0]: Interrupted by User
fail [:0]: Interrupted by User
periodic-ci-Azure-ARO-HCP-main-periodic-prod-e2e-parallel
post-goodmerged PR
No additional run metadata captured
 n/a passed n/a 0 n/a