CIHealth Run Log

Environment: PROD

Time (UTC)	Runtime	Job	Region	Failed at	Result	PR	Failed tests	Details
23:00:45	1h 38m 59s	periodic-ci-Azure-ARO-HCP-main-periodic-prod-e2e-parallel-ocp-nightly post-goodmerged PR No additional run metadata captured	canadacentral	e2e	failed	n/a	12	e2e (12) ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-update-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift 23:00:45 UTC · failed at e2e · test Customer should be able to successfully upgrade control plane minor version from 5.0 minor to 5.1 minor · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:167]: failed to trigger y-stream upgrade of cluster "cp-ystream-upgrade-5-1-h26qrg" to 5.1.0-0.nightly-multi-2026-09-30-181926 Unexpected error: <exported.ResponseError \| 0x34bdd7d932c0>: PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-cp-ystream-upgrade-5-1-5fxfbtp77wdj/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cp-ystream-upgrade-5-1-h26qrg -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: InvalidRequestContent -------------------------------------------------------------------------------- { "error": { "code": "InvalidRequestContent", "message": "Invalid value: \"aro-hcp.experimental.cluster.max-update-duration\": unrecognized experimental tag", "target": "tags[aro-hcp.experimental.cluster.max-update-duration]" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:167]: failed to trigger y-stream upgrade of cluster "cp-ystream-upgrade-5-1-h26qrg" to 5.1.0-0.nightly-multi-2026-09-30-181926 Unexpected error: <exported.ResponseError \| 0x34bdd7d932c0>: PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-cp-ystream-upgrade-5-1-5fxfbtp77wdj/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cp-ystream-upgrade-5-1-h26qrg -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: InvalidRequestContent -------------------------------------------------------------------------------- { "error": { "code": "InvalidRequestContent", "message": "Invalid value: \"aro-hcp.experimental.cluster.max-update-duration\": unrecognized experimental tag", "target": "tags[aro-hcp.experimental.cluster.max-update-duration]" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-update-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift 23:00:45 UTC · failed at e2e · test Update HCPOpenShiftCluster Positive creates a cluster and updates tags with a PATCH request · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_update.go:193]: failed to update HCP cluster tags via PATCH Unexpected error: <exported.ResponseError \| 0x156bbec7cb0>: PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/patch-tags-48mh6dg9zlr8/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/patch-tags-cluster -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: InvalidRequestContent -------------------------------------------------------------------------------- { "error": { "code": "InvalidRequestContent", "message": "Invalid value: \"aro-hcp.experimental.cluster.max-update-duration\": unrecognized experimental tag", "target": "tags[aro-hcp.experimental.cluster.max-update-duration]" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_update.go:193]: failed to update HCP cluster tags via PATCH Unexpected error: <exported.ResponseError \| 0x156bbec7cb0>: PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/patch-tags-48mh6dg9zlr8/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/patch-tags-cluster -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: InvalidRequestContent -------------------------------------------------------------------------------- { "error": { "code": "InvalidRequestContent", "message": "Invalid value: \"aro-hcp.experimental.cluster.max-update-duration\": unrecognized experimental tag", "target": "tags[aro-hcp.experimental.cluster.max-update-duration]" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-update-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift 23:00:45 UTC · failed at e2e · test Customer should be able to create an HCP cluster and manage ImageDigestMirrors · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/idms_lifecycle.go:175]: failed to update cluster with second ImageDigestMirror set Unexpected error: <exported.ResponseError \| 0x210c33ec90b0>: PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/idms-8fsbbrnx57fx/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/idms-e2e-hcp-cluster -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: InvalidRequestContent -------------------------------------------------------------------------------- { "error": { "code": "InvalidRequestContent", "message": "Invalid value: \"aro-hcp.experimental.cluster.max-update-duration\": unrecognized experimental tag", "target": "tags[aro-hcp.experimental.cluster.max-update-duration]" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/idms_lifecycle.go:175]: failed to update cluster with second ImageDigestMirror set Unexpected error: <exported.ResponseError \| 0x210c33ec90b0>: PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/idms-8fsbbrnx57fx/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/idms-e2e-hcp-cluster -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: InvalidRequestContent -------------------------------------------------------------------------------- { "error": { "code": "InvalidRequestContent", "message": "Invalid value: \"aro-hcp.experimental.cluster.max-update-duration\": unrecognized experimental tag", "target": "tags[aro-hcp.experimental.cluster.max-update-duration]" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InvalidRequestContent; detail message Invalid value: "4.21": must be specified as MAJOR.MINOR.PATCH (optionally with a pre-release, e.g. a nightly build suffix) when channelGroup is "nightly"; provider Microsoft.RedHatOpenShift 23:00:45 UTC · failed at e2e · test Customer should be able to create a cluster and node pools with aggregated advanced features · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_feature_aggregation.go:275]: failed to begin cluster "agg-cluster" authorized CIDR update Unexpected error: <exported.ResponseError \| 0xd97df9bad80>: PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/feature-aggregation-cskpwn77dkwd/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/agg-cluster -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: InvalidRequestContent -------------------------------------------------------------------------------- { "error": { "code": "InvalidRequestContent", "message": "Invalid value: \"4.21\": must be specified as MAJOR.MINOR.PATCH (optionally with a pre-release, e.g. a nightly build suffix) when channelGroup is \"nightly\"", "target": "properties.version.id" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_feature_aggregation.go:275]: failed to begin cluster "agg-cluster" authorized CIDR update Unexpected error: <exported.ResponseError \| 0xd97df9bad80>: PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/feature-aggregation-cskpwn77dkwd/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/agg-cluster -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: InvalidRequestContent -------------------------------------------------------------------------------- { "error": { "code": "InvalidRequestContent", "message": "Invalid value: \"4.21\": must be specified as MAJOR.MINOR.PATCH (optionally with a pre-release, e.g. a nightly build suffix) when channelGroup is \"nightly\"", "target": "properties.version.id" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InvalidRequestContent; detail message Invalid value: "<ocp-nightly>": must be at least 4.20.8; provider Microsoft.RedHatOpenShift 23:00:45 UTC · failed at e2e · test ARO-HCP should be able to perform a control plane and node pool install with OCP nightly channel for 4.20 · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:176]: failed to create node pool "np-4kgdcw" for cluster "cluster-nightly-4-20-4kgdcw" Unexpected error: <fmt.wrapError \| 0x781abe80140>: failed to create NodePool np-4kgdcw: failed starting nodepool creation "np-4kgdcw" for cluster "cluster-nightly-4-20-4kgdcw" in resourcegroup="rg-nightly-4-20-4t8l44cr8nbr": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-nightly-4-20-4t8l44cr8nbr/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cluster-nightly-4-20-4kgdcw/nodePools/np-4kgdcw -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: InvalidRequestContent -------------------------------------------------------------------------------- { "error": { "code": "InvalidRequestContent", "message": "Invalid value: \"4.20.0-0.nightly-multi-2026-10-01-125400\": must be at least 4.20.8", "target": "properties.version.id" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:176]: failed to create node pool "np-4kgdcw" for cluster "cluster-nightly-4-20-4kgdcw" Unexpected error: <fmt.wrapError \| 0x781abe80140>: failed to create NodePool np-4kgdcw: failed starting nodepool creation "np-4kgdcw" for cluster "cluster-nightly-4-20-4kgdcw" in resourcegroup="rg-nightly-4-20-4t8l44cr8nbr": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-nightly-4-20-4t8l44cr8nbr/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cluster-nightly-4-20-4kgdcw/nodePools/np-4kgdcw -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: InvalidRequestContent -------------------------------------------------------------------------------- { "error": { "code": "InvalidRequestContent", "message": "Invalid value: \"4.20.0-0.nightly-multi-2026-10-01-125400\": must be at least 4.20.8", "target": "properties.version.id" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-update-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift 23:00:45 UTC · failed at e2e · test Customer should be able to create an HCP cluster with custom autoscaling and update the autoscaling configuration · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_autoscaling.go:173]: failed to update cluster "autoscaling-hcp-cluster" autoscaling configuration Unexpected error: <exported.ResponseError \| 0x2d7c17881b00>: PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/autoscaling-cluster-hn22fz5qv7wv/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/autoscaling-hcp-cluster -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: InvalidRequestContent -------------------------------------------------------------------------------- { "error": { "code": "InvalidRequestContent", "message": "Invalid value: \"aro-hcp.experimental.cluster.max-update-duration\": unrecognized experimental tag", "target": "tags[aro-hcp.experimental.cluster.max-update-duration]" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_autoscaling.go:173]: failed to update cluster "autoscaling-hcp-cluster" autoscaling configuration Unexpected error: <exported.ResponseError \| 0x2d7c17881b00>: PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/autoscaling-cluster-hn22fz5qv7wv/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/autoscaling-hcp-cluster -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: InvalidRequestContent -------------------------------------------------------------------------------- { "error": { "code": "InvalidRequestContent", "message": "Invalid value: \"aro-hcp.experimental.cluster.max-update-duration\": unrecognized experimental tag", "target": "tags[aro-hcp.experimental.cluster.max-update-duration]" } } -------------------------------------------------------------------------------- ... occurred VerifyAllAPIServicesAvailable failed: failed to list all APIServices: Unauthorized 23:00:45 UTC · failed at e2e · test ARO HCP Service should successfully provision cluster when MI role assignments are added after cluster creation · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_delayed_role_assignments.go:241]: failed to verify HCP cluster viability Unexpected error: <errors.joinError \| 0x29a4ab6ac030>: VerifyBasicAccess failed: failed to list services: Unauthorized VerifyAllAPIServicesAvailable failed: failed to list all APIServices: Unauthorized ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_delayed_role_assignments.go:241]: failed to verify HCP cluster viability Unexpected error: <errors.joinError \| 0x29a4ab6ac030>: VerifyBasicAccess failed: failed to list services: Unauthorized VerifyAllAPIServicesAvailable failed: failed to list all APIServices: Unauthorized ... occurred ERROR CODE: InvalidRequestContent; detail message Invalid value: "<ocp-nightly>": must be at least 4.20.8; provider Microsoft.RedHatOpenShift 23:00:45 UTC · failed at e2e · test Customer should upgrade and update a nodepool from 4.20.z to 4.21.zLatest · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:165]: failed to create node pool npupgrade-4-20 with version 4.20.0-0.nightly-multi-2026-10-01-125400 Unexpected error: <fmt.wrapError \| 0x31d5efc20c00>: failed to create NodePool npupgrade-4-20: failed starting nodepool creation "npupgrade-4-20" for cluster "np-version-upgrade-cluster-b4wh48" in resourcegroup="rg-np-version-upgrade-b4wh48-bzk7j574q45w": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-version-upgrade-b4wh48-bzk7j574q45w/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/np-version-upgrade-cluster-b4wh48/nodePools/npupgrade-4-20 -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: InvalidRequestContent -------------------------------------------------------------------------------- { "error": { "code": "InvalidRequestContent", "message": "Invalid value: \"4.20.0-0.nightly-multi-2026-10-01-125400\": must be at least 4.20.8", "target": "properties.version.id" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:165]: failed to create node pool npupgrade-4-20 with version 4.20.0-0.nightly-multi-2026-10-01-125400 Unexpected error: <fmt.wrapError \| 0x31d5efc20c00>: failed to create NodePool npupgrade-4-20: failed starting nodepool creation "npupgrade-4-20" for cluster "np-version-upgrade-cluster-b4wh48" in resourcegroup="rg-np-version-upgrade-b4wh48-bzk7j574q45w": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-version-upgrade-b4wh48-bzk7j574q45w/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/np-version-upgrade-cluster-b4wh48/nodePools/npupgrade-4-20 -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: InvalidRequestContent -------------------------------------------------------------------------------- { "error": { "code": "InvalidRequestContent", "message": "Invalid value: \"4.20.0-0.nightly-multi-2026-10-01-125400\": must be at least 4.20.8", "target": "properties.version.id" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-update-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift 23:00:45 UTC · failed at e2e · test Customer should be able to successfully upgrade control plane minor version from 4.20 minor to 4.21 minor · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:167]: failed to trigger y-stream upgrade of cluster "cp-ystream-upgrade-4-21-54csls" to 4.21.0-0.nightly-multi-2026-10-02-115050 Unexpected error: <exported.ResponseError \| 0x2086109860c0>: PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-cp-ystream-upgrade-4-21-hx8x6fq8plbx/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cp-ystream-upgrade-4-21-54csls -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: InvalidRequestContent -------------------------------------------------------------------------------- { "error": { "code": "InvalidRequestContent", "message": "Invalid value: \"aro-hcp.experimental.cluster.max-update-duration\": unrecognized experimental tag", "target": "tags[aro-hcp.experimental.cluster.max-update-duration]" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:167]: failed to trigger y-stream upgrade of cluster "cp-ystream-upgrade-4-21-54csls" to 4.21.0-0.nightly-multi-2026-10-02-115050 Unexpected error: <exported.ResponseError \| 0x2086109860c0>: PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-cp-ystream-upgrade-4-21-hx8x6fq8plbx/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cp-ystream-upgrade-4-21-54csls -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: InvalidRequestContent -------------------------------------------------------------------------------- { "error": { "code": "InvalidRequestContent", "message": "Invalid value: \"aro-hcp.experimental.cluster.max-update-duration\": unrecognized experimental tag", "target": "tags[aro-hcp.experimental.cluster.max-update-duration]" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.nodepool.max-update-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift 23:00:45 UTC · failed at e2e · test Customer should be able to update node pool labels and taints · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_labels_taints.go:186]: failed to update node pool np-1 with new taints and scale up Unexpected error: <fmt.wrapError \| 0x3cb7bb3ff960>: failed to start nodepool "np-1" update in cluster "cluster-np-labels-taints" resourcegroup="rg-np-labels-taints-pf8r4pxpb5hl": PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-labels-taints-pf8r4pxpb5hl/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cluster-np-labels-taints/nodePools/np-1 -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: InvalidRequestContent -------------------------------------------------------------------------------- { "error": { "code": "InvalidRequestContent", "message": "Invalid value: \"aro-hcp.experimental.nodepool.max-update-duration\": unrecognized experimental tag", "target": "tags[aro-hcp.experimental.nodepool.max-update-duration]" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_labels_taints.go:186]: failed to update node pool np-1 with new taints and scale up Unexpected error: <fmt.wrapError \| 0x3cb7bb3ff960>: failed to start nodepool "np-1" update in cluster "cluster-np-labels-taints" resourcegroup="rg-np-labels-taints-pf8r4pxpb5hl": PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-labels-taints-pf8r4pxpb5hl/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cluster-np-labels-taints/nodePools/np-1 -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: InvalidRequestContent -------------------------------------------------------------------------------- { "error": { "code": "InvalidRequestContent", "message": "Invalid value: \"aro-hcp.experimental.nodepool.max-update-duration\": unrecognized experimental tag", "target": "tags[aro-hcp.experimental.nodepool.max-update-duration]" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InvalidRequestContent; detail message Invalid value: "4.21": must be specified as MAJOR.MINOR.PATCH (optionally with a pre-release, e.g. a nightly build suffix) when channelGroup is "nightly"; provider Microsoft.RedHatOpenShift 23:00:45 UTC · failed at e2e · test Authorized CIDRs Connectivity should allow API access only from authorized VM IP · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_authorized_cidrs_connectivity.go:408]: failed to begin updating cluster "cidr-connectivity-test" authorized CIDRs Unexpected error: <exported.ResponseError \| 0xcf8828ab170>: PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/e2e-cidr-connectivity-crv2w79vmlxc/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cidr-connectivity-test -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: InvalidRequestContent -------------------------------------------------------------------------------- { "error": { "code": "InvalidRequestContent", "message": "Invalid value: \"4.21\": must be specified as MAJOR.MINOR.PATCH (optionally with a pre-release, e.g. a nightly build suffix) when channelGroup is \"nightly\"", "target": "properties.version.id" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_authorized_cidrs_connectivity.go:408]: failed to begin updating cluster "cidr-connectivity-test" authorized CIDRs Unexpected error: <exported.ResponseError \| 0xcf8828ab170>: PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/e2e-cidr-connectivity-crv2w79vmlxc/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cidr-connectivity-test -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: InvalidRequestContent -------------------------------------------------------------------------------- { "error": { "code": "InvalidRequestContent", "message": "Invalid value: \"4.21\": must be specified as MAJOR.MINOR.PATCH (optionally with a pre-release, e.g. a nightly build suffix) when channelGroup is \"nightly\"", "target": "properties.version.id" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.nodepool.max-update-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift 23:00:45 UTC · failed at e2e · test Customer should be able to create a cluster with default autoscaling and a nodepool with autoscaling enabled up to replica limits · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_autoscaling.go:189]: failed to update AZ nodepool autoscale-az max replicas to 4 Unexpected error: <fmt.wrapError \| 0x325c79dc0060>: failed to start nodepool "autoscale-az" update in cluster "np-autoscale-cluster" resourcegroup="np-autoscaling-987nc9dd4vhq": PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/np-autoscaling-987nc9dd4vhq/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/np-autoscale-cluster/nodePools/autoscale-az -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: InvalidRequestContent -------------------------------------------------------------------------------- { "error": { "code": "InvalidRequestContent", "message": "Invalid value: \"aro-hcp.experimental.nodepool.max-update-duration\": unrecognized experimental tag", "target": "tags[aro-hcp.experimental.nodepool.max-update-duration]" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_autoscaling.go:189]: failed to update AZ nodepool autoscale-az max replicas to 4 Unexpected error: <fmt.wrapError \| 0x325c79dc0060>: failed to start nodepool "autoscale-az" update in cluster "np-autoscale-cluster" resourcegroup="np-autoscaling-987nc9dd4vhq": PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/np-autoscaling-987nc9dd4vhq/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/np-autoscale-cluster/nodePools/autoscale-az -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: InvalidRequestContent -------------------------------------------------------------------------------- { "error": { "code": "InvalidRequestContent", "message": "Invalid value: \"aro-hcp.experimental.nodepool.max-update-duration\": unrecognized experimental tag", "target": "tags[aro-hcp.experimental.nodepool.max-update-duration]" } } -------------------------------------------------------------------------------- ... occurred
09:10:07	2h 49m 54s	branch-ci-Azure-ARO-HCP-main-e2e-prod-e2e-parallel post-goodmerged PR No additional run metadata captured	brazilsouth	n/a	passed	n/a	0	n/a
09:09:36	2h 40m 20s	branch-ci-Azure-ARO-HCP-main-e2e-prod-e2e-parallel post-goodmerged PR No additional run metadata captured	westeurope	e2e	failed	n/a	2	e2e (2) ERROR CODE: DeadlineExceeded; provider Microsoft.Compute 09:09:36 UTC · failed at e2e · test Customer should be able to create a cluster and node pools with aggregated advanced features · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_feature_aggregation.go:404]: failed to verify cilium is running and nodes are Ready for cluster "agg-cluster" Unexpected error: <errors.joinError \| 0x2861f94db110>: failed to create NodePool agg-np-a: failed waiting for nodepool="agg-np-a" for cluster "agg-cluster" in resourcegroup="feature-aggregation-s8tjmf2mg47r" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westeurope/hcpOperationStatuses/a70c1239-194d-44ad-b66c-18f11327f9a4 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westeurope/hcpOperationStatuses/a70c1239-194d-44ad-b66c-18f11327f9a4", "name": "a70c1239-194d-44ad-b66c-18f11327f9a4", "status": "Failed", "startTime": "2026-10-03T09:45:52.709942431Z", "endTime": "2026-10-03T10:04:55.390706618Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 1, want \u003e= 2 (autoscaling min); node pool condition AllNodesHealthy is False: 1 of 2 machines are not healthy\nMachine agg-cluster-agg-np-a-mf948-nwnch: InspectionFailed: Waiting for a Node with spec.providerID azure:///subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/feature-aggregation-s8tjmf2mg47r--managed/providers/Microsoft.Compute/virtualMachines/agg-cluster-agg-np-a-mf948-nwnch to exist\n; node pool condition AllMachinesReady is False: 1 of 2 machines are not ready\nMachine agg-cluster-agg-np-a-mf948-nwnch: ReadyUnknown: NodeHealthy: Waiting for a Node with spec.providerID azure:///subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/feature-aggregation-s8tjmf2mg47r--managed/providers/Microsoft.Compute/virtualMachines/agg-cluster-agg-np-a-mf948-nwnch to exist\n" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_feature_aggregation.go:404]: failed to verify cilium is running and nodes are Ready for cluster "agg-cluster" Unexpected error: <errors.joinError \| 0x2861f94db110>: failed to create NodePool agg-np-a: failed waiting for nodepool="agg-np-a" for cluster "agg-cluster" in resourcegroup="feature-aggregation-s8tjmf2mg47r" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westeurope/hcpOperationStatuses/a70c1239-194d-44ad-b66c-18f11327f9a4 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westeurope/hcpOperationStatuses/a70c1239-194d-44ad-b66c-18f11327f9a4", "name": "a70c1239-194d-44ad-b66c-18f11327f9a4", "status": "Failed", "startTime": "2026-10-03T09:45:52.709942431Z", "endTime": "2026-10-03T10:04:55.390706618Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 1, want \u003e= 2 (autoscaling min); node pool condition AllNodesHealthy is False: 1 of 2 machines are not healthy\nMachine agg-cluster-agg-np-a-mf948-nwnch: InspectionFailed: Waiting for a Node with spec.providerID azure:///subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/feature-aggregation-s8tjmf2mg47r--managed/providers/Microsoft.Compute/virtualMachines/agg-cluster-agg-np-a-mf948-nwnch to exist\n; node pool condition AllMachinesReady is False: 1 of 2 machines are not ready\nMachine agg-cluster-agg-np-a-mf948-nwnch: ReadyUnknown: NodeHealthy: Waiting for a Node with spec.providerID azure:///subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/feature-aggregation-s8tjmf2mg47r--managed/providers/Microsoft.Compute/virtualMachines/agg-cluster-agg-np-a-mf948-nwnch to exist\n" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeadlineExceeded; detail message node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is <count>, want <count>; node pool condition AllNodesHealthy is False: <count> of <count> machines are not healthy Machine <node>: InspectionFailed: Waiting for AzureMachine to report spec.providerID; node pool condition AllMachinesReady is False: <count> of <count> machines are not ready Machine <node>: Creating: virtualmachine creating or updating; provider Microsoft.RedHatOpenShift 09:09:36 UTC · failed at e2e · test FIPS Mode Support with v2026-09-01 API should create an HCP cluster with FIPS mode enabled via cryptoRestrictions property · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_fips_mode.go:103]: failed to create node pool "np-1" for fips-enabled cluster "fips-enabled-cluster" Unexpected error: <fmt.wrapError \| 0x33e8cc763a20>: failed to create NodePool np-1: failed waiting for nodepool="np-1" for cluster "fips-enabled-cluster" in resourcegroup="fips-enabled-wrvrktrj9pn6" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westeurope/hcpOperationStatuses/a021811c-f25e-4fdd-b88c-cc64a68c8188 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westeurope/hcpOperationStatuses/a021811c-f25e-4fdd-b88c-cc64a68c8188", "name": "a021811c-f25e-4fdd-b88c-cc64a68c8188", "status": "Failed", "startTime": "2026-10-03T09:41:16.318835994Z", "endTime": "2026-10-03T10:00:22.618172935Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 1, want 2; node pool condition AllNodesHealthy is False: 1 of 2 machines are not healthy\nMachine k3u6n5v9b5f7j2v-np-1-9khfh-nvt5s: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 1 of 2 machines are not ready\nMachine k3u6n5v9b5f7j2v-np-1-9khfh-nvt5s: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_fips_mode.go:103]: failed to create node pool "np-1" for fips-enabled cluster "fips-enabled-cluster" Unexpected error: <fmt.wrapError \| 0x33e8cc763a20>: failed to create NodePool np-1: failed waiting for nodepool="np-1" for cluster "fips-enabled-cluster" in resourcegroup="fips-enabled-wrvrktrj9pn6" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westeurope/hcpOperationStatuses/a021811c-f25e-4fdd-b88c-cc64a68c8188 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westeurope/hcpOperationStatuses/a021811c-f25e-4fdd-b88c-cc64a68c8188", "name": "a021811c-f25e-4fdd-b88c-cc64a68c8188", "status": "Failed", "startTime": "2026-10-03T09:41:16.318835994Z", "endTime": "2026-10-03T10:00:22.618172935Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 1, want 2; node pool condition AllNodesHealthy is False: 1 of 2 machines are not healthy\nMachine k3u6n5v9b5f7j2v-np-1-9khfh-nvt5s: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 1 of 2 machines are not ready\nMachine k3u6n5v9b5f7j2v-np-1-9khfh-nvt5s: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred
09:04:21	2h 34m 34s	branch-ci-Azure-ARO-HCP-main-e2e-prod-e2e-parallel post-goodmerged PR No additional run metadata captured	switzerlandnorth	n/a	passed	n/a	0	n/a
08:38:53	2h 42m 34s	branch-ci-Azure-ARO-HCP-main-e2e-prod-e2e-parallel post-goodmerged PR No additional run metadata captured	centralindia	e2e	failed	n/a	1	e2e (1) failed to create nodepool <nodepool>: failed waiting for nodepool="<nodepool>" for cluster "<cluster>" in resourcegroup="<resource-group>" to finish creating: GET <url> 08:38:53 UTC · failed at e2e · test Customer should be able to delete a nodepool · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_delete.go:128]: failed to create node pools Expected success, but got an error: <fmt.wrapError \| 0x3a380b9ff820>: failed to create NodePool np-deletable: failed waiting for nodepool="np-deletable" for cluster "np-delete-hcp-cluster" in resourcegroup="nodepool-delete-jdf825hs8fn9" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/centralindia/hcpOperationStatuses/1ffbe111-707b-4771-8ddd-d413913d8bde -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/centralindia/hcpOperationStatuses/1ffbe111-707b-4771-8ddd-d413913d8bde", "name": "1ffbe111-707b-4771-8ddd-d413913d8bde", "status": "Failed", "startTime": "2026-10-03T10:22:50.222769152Z", "endTime": "2026-10-03T10:41:50.376552801Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 1, want 2; node pool condition AllNodesHealthy is False: 1 of 2 machines are not healthy\nMachine f0m1e1q3s5q9h4n-np-deletable-cb4h4-4ccd6: InspectionFailed: Waiting for a Node with spec.providerID azure:///subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/nodepool-delete-jdf825hs8fn9--managed/providers/Microsoft.Compute/virtualMachines/f0m1e1q3s5q9h4n-np-deletable-cb4h4-4ccd6 to exist\n; node pool condition AllMachinesReady is False: 1 of 2 machines are not ready\nMachine f0m1e1q3s5q9h4n-np-deletable-cb4h4-4ccd6: ReadyUnknown: NodeHealthy: Waiting for a Node with spec.providerID azure:///subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/nodepool-delete-jdf825hs8fn9--managed/providers/Microsoft.Compute/virtualMachines/f0m1e1q3s5q9h4n-np-deletable-cb4h4-4ccd6 to exist\n" } } -------------------------------------------------------------------------------- ... fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_delete.go:128]: failed to create node pools Expected success, but got an error: <fmt.wrapError \| 0x3a380b9ff820>: failed to create NodePool np-deletable: failed waiting for nodepool="np-deletable" for cluster "np-delete-hcp-cluster" in resourcegroup="nodepool-delete-jdf825hs8fn9" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/centralindia/hcpOperationStatuses/1ffbe111-707b-4771-8ddd-d413913d8bde -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/centralindia/hcpOperationStatuses/1ffbe111-707b-4771-8ddd-d413913d8bde", "name": "1ffbe111-707b-4771-8ddd-d413913d8bde", "status": "Failed", "startTime": "2026-10-03T10:22:50.222769152Z", "endTime": "2026-10-03T10:41:50.376552801Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 1, want 2; node pool condition AllNodesHealthy is False: 1 of 2 machines are not healthy\nMachine f0m1e1q3s5q9h4n-np-deletable-cb4h4-4ccd6: InspectionFailed: Waiting for a Node with spec.providerID azure:///subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/nodepool-delete-jdf825hs8fn9--managed/providers/Microsoft.Compute/virtualMachines/f0m1e1q3s5q9h4n-np-deletable-cb4h4-4ccd6 to exist\n; node pool condition AllMachinesReady is False: 1 of 2 machines are not ready\nMachine f0m1e1q3s5q9h4n-np-deletable-cb4h4-4ccd6: ReadyUnknown: NodeHealthy: Waiting for a Node with spec.providerID azure:///subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/nodepool-delete-jdf825hs8fn9--managed/providers/Microsoft.Compute/virtualMachines/f0m1e1q3s5q9h4n-np-deletable-cb4h4-4ccd6 to exist\n" } } -------------------------------------------------------------------------------- ...
02:00:44	2h 4m 43s	periodic-ci-Azure-ARO-HCP-main-periodic-prod-e2e-parallel post-goodmerged PR No additional run metadata captured	australiaeast	e2e	failed	n/a	22	e2e (22) ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-update-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift 02:00:44 UTC · failed at e2e · test Customer should be able to successfully upgrade control plane minor version from 4.22 minor to 5.0 minor · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:167]: failed to trigger y-stream upgrade of cluster "cp-ystream-upgrade-5-0-v6zlk7" to 5.0 Unexpected error: <exported.ResponseError \| 0x3aa50a408f30>: PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-cp-ystream-upgrade-5-0-7fls5z997c4n/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cp-ystream-upgrade-5-0-v6zlk7 -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: InvalidRequestContent -------------------------------------------------------------------------------- { "error": { "code": "InvalidRequestContent", "message": "Invalid value: \"aro-hcp.experimental.cluster.max-update-duration\": unrecognized experimental tag", "target": "tags[aro-hcp.experimental.cluster.max-update-duration]" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:167]: failed to trigger y-stream upgrade of cluster "cp-ystream-upgrade-5-0-v6zlk7" to 5.0 Unexpected error: <exported.ResponseError \| 0x3aa50a408f30>: PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-cp-ystream-upgrade-5-0-7fls5z997c4n/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cp-ystream-upgrade-5-0-v6zlk7 -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: InvalidRequestContent -------------------------------------------------------------------------------- { "error": { "code": "InvalidRequestContent", "message": "Invalid value: \"aro-hcp.experimental.cluster.max-update-duration\": unrecognized experimental tag", "target": "tags[aro-hcp.experimental.cluster.max-update-duration]" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-update-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift 02:00:44 UTC · failed at e2e · test Service Provider should upgrade the control plane z-stream automatically on behalf of the customer for 4.21 · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:157]: failed to enable immediate z-stream updates for cluster "cluster-zstream-4-21-msl8ps" on minor version 4.21 Unexpected error: <exported.ResponseError \| 0x21a7a658f5c0>: PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-zstream-upgrade-4-21-l7wmbxqck8gf/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cluster-zstream-4-21-msl8ps -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: InvalidRequestContent -------------------------------------------------------------------------------- { "error": { "code": "InvalidRequestContent", "message": "Invalid value: \"aro-hcp.experimental.cluster.max-update-duration\": unrecognized experimental tag", "target": "tags[aro-hcp.experimental.cluster.max-update-duration]" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:157]: failed to enable immediate z-stream updates for cluster "cluster-zstream-4-21-msl8ps" on minor version 4.21 Unexpected error: <exported.ResponseError \| 0x21a7a658f5c0>: PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-zstream-upgrade-4-21-l7wmbxqck8gf/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cluster-zstream-4-21-msl8ps -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: InvalidRequestContent -------------------------------------------------------------------------------- { "error": { "code": "InvalidRequestContent", "message": "Invalid value: \"aro-hcp.experimental.cluster.max-update-duration\": unrecognized experimental tag", "target": "tags[aro-hcp.experimental.cluster.max-update-duration]" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-update-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift 02:00:44 UTC · failed at e2e · test Service Provider should upgrade the control plane z-stream automatically on behalf of the customer for 4.20 · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:157]: failed to enable immediate z-stream updates for cluster "cluster-zstream-4-20-9hrm6z" on minor version 4.20 Unexpected error: <exported.ResponseError \| 0x1415c20a5500>: PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-zstream-upgrade-4-20-cck68gj6qfsp/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cluster-zstream-4-20-9hrm6z -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: InvalidRequestContent -------------------------------------------------------------------------------- { "error": { "code": "InvalidRequestContent", "message": "Invalid value: \"aro-hcp.experimental.cluster.max-update-duration\": unrecognized experimental tag", "target": "tags[aro-hcp.experimental.cluster.max-update-duration]" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:157]: failed to enable immediate z-stream updates for cluster "cluster-zstream-4-20-9hrm6z" on minor version 4.20 Unexpected error: <exported.ResponseError \| 0x1415c20a5500>: PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-zstream-upgrade-4-20-cck68gj6qfsp/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cluster-zstream-4-20-9hrm6z -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: InvalidRequestContent -------------------------------------------------------------------------------- { "error": { "code": "InvalidRequestContent", "message": "Invalid value: \"aro-hcp.experimental.cluster.max-update-duration\": unrecognized experimental tag", "target": "tags[aro-hcp.experimental.cluster.max-update-duration]" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-update-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift 02:00:44 UTC · failed at e2e · test Customer should be able to create an HCP cluster with custom autoscaling and update the autoscaling configuration · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_autoscaling.go:173]: failed to update cluster "autoscaling-hcp-cluster" autoscaling configuration Unexpected error: <exported.ResponseError \| 0x61296253c20>: PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/autoscaling-cluster-96mcz9ktg57h/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/autoscaling-hcp-cluster -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: InvalidRequestContent -------------------------------------------------------------------------------- { "error": { "code": "InvalidRequestContent", "message": "Invalid value: \"aro-hcp.experimental.cluster.max-update-duration\": unrecognized experimental tag", "target": "tags[aro-hcp.experimental.cluster.max-update-duration]" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_autoscaling.go:173]: failed to update cluster "autoscaling-hcp-cluster" autoscaling configuration Unexpected error: <exported.ResponseError \| 0x61296253c20>: PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/autoscaling-cluster-96mcz9ktg57h/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/autoscaling-hcp-cluster -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: InvalidRequestContent -------------------------------------------------------------------------------- { "error": { "code": "InvalidRequestContent", "message": "Invalid value: \"aro-hcp.experimental.cluster.max-update-duration\": unrecognized experimental tag", "target": "tags[aro-hcp.experimental.cluster.max-update-duration]" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.z-stream-update-policy": unrecognized experimental tag; provider Microsoft.RedHatOpenShift 02:00:44 UTC · failed at e2e · test Service Provider should upgrade the control plane z-stream automatically on behalf of the customer for 5.0 · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:157]: failed to enable immediate z-stream updates for cluster "cluster-zstream-5-0-x4v9ss" on minor version 5.0 Unexpected error: <exported.ResponseError \| 0x5a0752a7c50>: PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-zstream-upgrade-5-0-9j7fg4k8csln/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cluster-zstream-5-0-x4v9ss -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: InvalidRequestContent -------------------------------------------------------------------------------- { "error": { "code": "InvalidRequestContent", "message": "Invalid value: \"aro-hcp.experimental.cluster.z-stream-update-policy\": unrecognized experimental tag", "target": "tags[aro-hcp.experimental.cluster.z-stream-update-policy]" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:157]: failed to enable immediate z-stream updates for cluster "cluster-zstream-5-0-x4v9ss" on minor version 5.0 Unexpected error: <exported.ResponseError \| 0x5a0752a7c50>: PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-zstream-upgrade-5-0-9j7fg4k8csln/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cluster-zstream-5-0-x4v9ss -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: InvalidRequestContent -------------------------------------------------------------------------------- { "error": { "code": "InvalidRequestContent", "message": "Invalid value: \"aro-hcp.experimental.cluster.z-stream-update-policy\": unrecognized experimental tag", "target": "tags[aro-hcp.experimental.cluster.z-stream-update-policy]" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.nodepool.max-update-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift 02:00:44 UTC · failed at e2e · test Customer should downgrade a nodepool to a lower minor version from 4.22.zLatest to 4.20.zLatest · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:768]: failed to update nodepool npdg-422-420 to downgrade target 4.20.41 Unexpected error: <fmt.wrapError \| 0x1ed03c0ba1c0>: failed to start nodepool "npdg-422-420" update in cluster "np-dg-minor-nrr89q" resourcegroup="rg-np-dg-minor-nrr89q-d5btp9lwtkwk": PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-dg-minor-nrr89q-d5btp9lwtkwk/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/np-dg-minor-nrr89q/nodePools/npdg-422-420 -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: InvalidRequestContent -------------------------------------------------------------------------------- { "error": { "code": "InvalidRequestContent", "message": "Invalid value: \"aro-hcp.experimental.nodepool.max-update-duration\": unrecognized experimental tag", "target": "tags[aro-hcp.experimental.nodepool.max-update-duration]" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:768]: failed to update nodepool npdg-422-420 to downgrade target 4.20.41 Unexpected error: <fmt.wrapError \| 0x1ed03c0ba1c0>: failed to start nodepool "npdg-422-420" update in cluster "np-dg-minor-nrr89q" resourcegroup="rg-np-dg-minor-nrr89q-d5btp9lwtkwk": PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-dg-minor-nrr89q-d5btp9lwtkwk/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/np-dg-minor-nrr89q/nodePools/npdg-422-420 -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: InvalidRequestContent -------------------------------------------------------------------------------- { "error": { "code": "InvalidRequestContent", "message": "Invalid value: \"aro-hcp.experimental.nodepool.max-update-duration\": unrecognized experimental tag", "target": "tags[aro-hcp.experimental.nodepool.max-update-duration]" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-update-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift 02:00:44 UTC · failed at e2e · test Customer should be able to successfully upgrade control plane minor version from 4.20 minor to 4.21 minor · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:167]: failed to trigger y-stream upgrade of cluster "cp-ystream-upgrade-4-21-w78qcb" to 4.21 Unexpected error: <exported.ResponseError \| 0x39bf51e7ba70>: PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-cp-ystream-upgrade-4-21-l8ksqxqxvhll/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cp-ystream-upgrade-4-21-w78qcb -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: InvalidRequestContent -------------------------------------------------------------------------------- { "error": { "code": "InvalidRequestContent", "message": "Invalid value: \"aro-hcp.experimental.cluster.max-update-duration\": unrecognized experimental tag", "target": "tags[aro-hcp.experimental.cluster.max-update-duration]" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:167]: failed to trigger y-stream upgrade of cluster "cp-ystream-upgrade-4-21-w78qcb" to 4.21 Unexpected error: <exported.ResponseError \| 0x39bf51e7ba70>: PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-cp-ystream-upgrade-4-21-l8ksqxqxvhll/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cp-ystream-upgrade-4-21-w78qcb -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: InvalidRequestContent -------------------------------------------------------------------------------- { "error": { "code": "InvalidRequestContent", "message": "Invalid value: \"aro-hcp.experimental.cluster.max-update-duration\": unrecognized experimental tag", "target": "tags[aro-hcp.experimental.cluster.max-update-duration]" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.nodepool.max-update-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift 02:00:44 UTC · failed at e2e · test Customer should upgrade a nodepool skipping one minor version (+2) from 4.20.z to 4.22.zLatest · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:489]: failed to upgrade nodepool nps-420-422 from 4.20.41 to 4.22.17 Unexpected error: <fmt.wrapError \| 0x6bee01cdac0>: failed to start nodepool "nps-420-422" update in cluster "np-skip-minor-6td8ds" resourcegroup="rg-np-skip-minor-6td8ds-d2dvtglcn2l2": PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-skip-minor-6td8ds-d2dvtglcn2l2/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/np-skip-minor-6td8ds/nodePools/nps-420-422 -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: InvalidRequestContent -------------------------------------------------------------------------------- { "error": { "code": "InvalidRequestContent", "message": "Invalid value: \"aro-hcp.experimental.nodepool.max-update-duration\": unrecognized experimental tag", "target": "tags[aro-hcp.experimental.nodepool.max-update-duration]" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:489]: failed to upgrade nodepool nps-420-422 from 4.20.41 to 4.22.17 Unexpected error: <fmt.wrapError \| 0x6bee01cdac0>: failed to start nodepool "nps-420-422" update in cluster "np-skip-minor-6td8ds" resourcegroup="rg-np-skip-minor-6td8ds-d2dvtglcn2l2": PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-skip-minor-6td8ds-d2dvtglcn2l2/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/np-skip-minor-6td8ds/nodePools/nps-420-422 -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: InvalidRequestContent -------------------------------------------------------------------------------- { "error": { "code": "InvalidRequestContent", "message": "Invalid value: \"aro-hcp.experimental.nodepool.max-update-duration\": unrecognized experimental tag", "target": "tags[aro-hcp.experimental.nodepool.max-update-duration]" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.nodepool.max-update-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift 02:00:44 UTC · failed at e2e · test Customer should upgrade and update a nodepool from 4.20.z to 4.20.zLatest · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:195]: failed to upgrade node pool npupgrade-4-20 to version 4.20.41 Unexpected error: <fmt.wrapError \| 0x177ce192fa20>: failed to start nodepool "npupgrade-4-20" update in cluster "np-version-upgrade-cluster-z4pwtb" resourcegroup="rg-np-version-upgrade-z4pwtb-4qn5bp5csjzb": PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-version-upgrade-z4pwtb-4qn5bp5csjzb/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/np-version-upgrade-cluster-z4pwtb/nodePools/npupgrade-4-20 -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: InvalidRequestContent -------------------------------------------------------------------------------- { "error": { "code": "InvalidRequestContent", "message": "Invalid value: \"aro-hcp.experimental.nodepool.max-update-duration\": unrecognized experimental tag", "target": "tags[aro-hcp.experimental.nodepool.max-update-duration]" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:195]: failed to upgrade node pool npupgrade-4-20 to version 4.20.41 Unexpected error: <fmt.wrapError \| 0x177ce192fa20>: failed to start nodepool "npupgrade-4-20" update in cluster "np-version-upgrade-cluster-z4pwtb" resourcegroup="rg-np-version-upgrade-z4pwtb-4qn5bp5csjzb": PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-version-upgrade-z4pwtb-4qn5bp5csjzb/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/np-version-upgrade-cluster-z4pwtb/nodePools/npupgrade-4-20 -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: InvalidRequestContent -------------------------------------------------------------------------------- { "error": { "code": "InvalidRequestContent", "message": "Invalid value: \"aro-hcp.experimental.nodepool.max-update-duration\": unrecognized experimental tag", "target": "tags[aro-hcp.experimental.nodepool.max-update-duration]" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.nodepool.max-update-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift 02:00:44 UTC · failed at e2e · test Customer should upgrade and update a nodepool from 4.20.z to 4.21.zLatest · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:195]: failed to upgrade node pool npupgrade-4-20 to version 4.21.36 Unexpected error: <fmt.wrapError \| 0x143a1310b240>: failed to start nodepool "npupgrade-4-20" update in cluster "np-version-upgrade-cluster-rc8d5d" resourcegroup="rg-np-version-upgrade-rc8d5d-w2t72x2zshww": PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-version-upgrade-rc8d5d-w2t72x2zshww/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/np-version-upgrade-cluster-rc8d5d/nodePools/npupgrade-4-20 -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: InvalidRequestContent -------------------------------------------------------------------------------- { "error": { "code": "InvalidRequestContent", "message": "Invalid value: \"aro-hcp.experimental.nodepool.max-update-duration\": unrecognized experimental tag", "target": "tags[aro-hcp.experimental.nodepool.max-update-duration]" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:195]: failed to upgrade node pool npupgrade-4-20 to version 4.21.36 Unexpected error: <fmt.wrapError \| 0x143a1310b240>: failed to start nodepool "npupgrade-4-20" update in cluster "np-version-upgrade-cluster-rc8d5d" resourcegroup="rg-np-version-upgrade-rc8d5d-w2t72x2zshww": PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-version-upgrade-rc8d5d-w2t72x2zshww/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/np-version-upgrade-cluster-rc8d5d/nodePools/npupgrade-4-20 -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: InvalidRequestContent -------------------------------------------------------------------------------- { "error": { "code": "InvalidRequestContent", "message": "Invalid value: \"aro-hcp.experimental.nodepool.max-update-duration\": unrecognized experimental tag", "target": "tags[aro-hcp.experimental.nodepool.max-update-duration]" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-update-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift 02:00:44 UTC · failed at e2e · test Customer should be able to successfully upgrade control plane minor version from 4.22 minor to 4.23 minor · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:167]: failed to trigger y-stream upgrade of cluster "cp-ystream-upgrade-4-23-w2grzz" to 4.23 Unexpected error: <exported.ResponseError \| 0x30efed875560>: PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-cp-ystream-upgrade-4-23-52rk6bpxrc9w/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cp-ystream-upgrade-4-23-w2grzz -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: InvalidRequestContent -------------------------------------------------------------------------------- { "error": { "code": "InvalidRequestContent", "message": "Invalid value: \"aro-hcp.experimental.cluster.max-update-duration\": unrecognized experimental tag", "target": "tags[aro-hcp.experimental.cluster.max-update-duration]" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:167]: failed to trigger y-stream upgrade of cluster "cp-ystream-upgrade-4-23-w2grzz" to 4.23 Unexpected error: <exported.ResponseError \| 0x30efed875560>: PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-cp-ystream-upgrade-4-23-52rk6bpxrc9w/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cp-ystream-upgrade-4-23-w2grzz -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: InvalidRequestContent -------------------------------------------------------------------------------- { "error": { "code": "InvalidRequestContent", "message": "Invalid value: \"aro-hcp.experimental.cluster.max-update-duration\": unrecognized experimental tag", "target": "tags[aro-hcp.experimental.cluster.max-update-duration]" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-update-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift 02:00:44 UTC · failed at e2e · test Customer should be able to create an HCP cluster and manage ImageDigestMirrors · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/idms_lifecycle.go:175]: failed to update cluster with second ImageDigestMirror set Unexpected error: <exported.ResponseError \| 0x148c3b6e47b0>: PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/idms-tvg824p96xln/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/idms-e2e-hcp-cluster -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: InvalidRequestContent -------------------------------------------------------------------------------- { "error": { "code": "InvalidRequestContent", "message": "Invalid value: \"aro-hcp.experimental.cluster.max-update-duration\": unrecognized experimental tag", "target": "tags[aro-hcp.experimental.cluster.max-update-duration]" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/idms_lifecycle.go:175]: failed to update cluster with second ImageDigestMirror set Unexpected error: <exported.ResponseError \| 0x148c3b6e47b0>: PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/idms-tvg824p96xln/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/idms-e2e-hcp-cluster -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: InvalidRequestContent -------------------------------------------------------------------------------- { "error": { "code": "InvalidRequestContent", "message": "Invalid value: \"aro-hcp.experimental.cluster.max-update-duration\": unrecognized experimental tag", "target": "tags[aro-hcp.experimental.cluster.max-update-duration]" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-update-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift 02:00:44 UTC · failed at e2e · test Update HCPOpenShiftCluster Positive creates a cluster and updates tags with a PATCH request · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_update.go:193]: failed to update HCP cluster tags via PATCH Unexpected error: <exported.ResponseError \| 0x95d711c6660>: PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/patch-tags-fqfljmzwnn7j/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/patch-tags-cluster -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: InvalidRequestContent -------------------------------------------------------------------------------- { "error": { "code": "InvalidRequestContent", "message": "Invalid value: \"aro-hcp.experimental.cluster.max-update-duration\": unrecognized experimental tag", "target": "tags[aro-hcp.experimental.cluster.max-update-duration]" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_update.go:193]: failed to update HCP cluster tags via PATCH Unexpected error: <exported.ResponseError \| 0x95d711c6660>: PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/patch-tags-fqfljmzwnn7j/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/patch-tags-cluster -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: InvalidRequestContent -------------------------------------------------------------------------------- { "error": { "code": "InvalidRequestContent", "message": "Invalid value: \"aro-hcp.experimental.cluster.max-update-duration\": unrecognized experimental tag", "target": "tags[aro-hcp.experimental.cluster.max-update-duration]" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.nodepool.max-update-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift 02:00:44 UTC · failed at e2e · test Customer should be able to create a cluster with default autoscaling and a nodepool with autoscaling enabled up to replica limits · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_autoscaling.go:189]: failed to update AZ nodepool autoscale-az max replicas to 4 Unexpected error: <fmt.wrapError \| 0x16c8ce6032a0>: failed to start nodepool "autoscale-az" update in cluster "np-autoscale-cluster" resourcegroup="np-autoscaling-4rvxbgx64wn7": PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/np-autoscaling-4rvxbgx64wn7/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/np-autoscale-cluster/nodePools/autoscale-az -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: InvalidRequestContent -------------------------------------------------------------------------------- { "error": { "code": "InvalidRequestContent", "message": "Invalid value: \"aro-hcp.experimental.nodepool.max-update-duration\": unrecognized experimental tag", "target": "tags[aro-hcp.experimental.nodepool.max-update-duration]" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_autoscaling.go:189]: failed to update AZ nodepool autoscale-az max replicas to 4 Unexpected error: <fmt.wrapError \| 0x16c8ce6032a0>: failed to start nodepool "autoscale-az" update in cluster "np-autoscale-cluster" resourcegroup="np-autoscaling-4rvxbgx64wn7": PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/np-autoscaling-4rvxbgx64wn7/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/np-autoscale-cluster/nodePools/autoscale-az -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: InvalidRequestContent -------------------------------------------------------------------------------- { "error": { "code": "InvalidRequestContent", "message": "Invalid value: \"aro-hcp.experimental.nodepool.max-update-duration\": unrecognized experimental tag", "target": "tags[aro-hcp.experimental.nodepool.max-update-duration]" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-update-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift 02:00:44 UTC · failed at e2e · test Customer should be able to successfully upgrade control plane minor version from 5.0 minor to 5.1 minor · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:167]: failed to trigger y-stream upgrade of cluster "cp-ystream-upgrade-5-1-2bw2nd" to 5.1 Unexpected error: <exported.ResponseError \| 0x3e7c72aaef90>: PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-cp-ystream-upgrade-5-1-h5wrz4q9tk28/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cp-ystream-upgrade-5-1-2bw2nd -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: InvalidRequestContent -------------------------------------------------------------------------------- { "error": { "code": "InvalidRequestContent", "message": "Invalid value: \"aro-hcp.experimental.cluster.max-update-duration\": unrecognized experimental tag", "target": "tags[aro-hcp.experimental.cluster.max-update-duration]" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:167]: failed to trigger y-stream upgrade of cluster "cp-ystream-upgrade-5-1-2bw2nd" to 5.1 Unexpected error: <exported.ResponseError \| 0x3e7c72aaef90>: PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-cp-ystream-upgrade-5-1-h5wrz4q9tk28/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cp-ystream-upgrade-5-1-2bw2nd -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: InvalidRequestContent -------------------------------------------------------------------------------- { "error": { "code": "InvalidRequestContent", "message": "Invalid value: \"aro-hcp.experimental.cluster.max-update-duration\": unrecognized experimental tag", "target": "tags[aro-hcp.experimental.cluster.max-update-duration]" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-update-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift 02:00:44 UTC · failed at e2e · test Customer should be able to successfully upgrade control plane minor version from 4.21 minor to 4.22 minor · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:167]: failed to trigger y-stream upgrade of cluster "cp-ystream-upgrade-4-22-4c45hm" to 4.22 Unexpected error: <exported.ResponseError \| 0x1c31b53c8540>: PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-cp-ystream-upgrade-4-22-7jn7lrmbjhwn/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cp-ystream-upgrade-4-22-4c45hm -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: InvalidRequestContent -------------------------------------------------------------------------------- { "error": { "code": "InvalidRequestContent", "message": "Invalid value: \"aro-hcp.experimental.cluster.max-update-duration\": unrecognized experimental tag", "target": "tags[aro-hcp.experimental.cluster.max-update-duration]" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:167]: failed to trigger y-stream upgrade of cluster "cp-ystream-upgrade-4-22-4c45hm" to 4.22 Unexpected error: <exported.ResponseError \| 0x1c31b53c8540>: PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-cp-ystream-upgrade-4-22-7jn7lrmbjhwn/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cp-ystream-upgrade-4-22-4c45hm -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: InvalidRequestContent -------------------------------------------------------------------------------- { "error": { "code": "InvalidRequestContent", "message": "Invalid value: \"aro-hcp.experimental.cluster.max-update-duration\": unrecognized experimental tag", "target": "tags[aro-hcp.experimental.cluster.max-update-duration]" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.nodepool.max-update-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift 02:00:44 UTC · failed at e2e · test Customer should upgrade a nodepool to a version without Cincinnati upgrade edge z-stream upgrade without Cincinnati edge in 4.20 · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:349]: failed to upgrade nodepool npnoedge-4-20 from 4.20.40 to 4.20.41 Unexpected error: <fmt.wrapError \| 0x18d8f5386060>: failed to start nodepool "npnoedge-4-20" update in cluster "np-noedge-xvr7qd" resourcegroup="rg-np-noedge-xvr7qd-fdgmzp5mlmp2": PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-noedge-xvr7qd-fdgmzp5mlmp2/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/np-noedge-xvr7qd/nodePools/npnoedge-4-20 -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: InvalidRequestContent -------------------------------------------------------------------------------- { "error": { "code": "InvalidRequestContent", "message": "Invalid value: \"aro-hcp.experimental.nodepool.max-update-duration\": unrecognized experimental tag", "target": "tags[aro-hcp.experimental.nodepool.max-update-duration]" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:349]: failed to upgrade nodepool npnoedge-4-20 from 4.20.40 to 4.20.41 Unexpected error: <fmt.wrapError \| 0x18d8f5386060>: failed to start nodepool "npnoedge-4-20" update in cluster "np-noedge-xvr7qd" resourcegroup="rg-np-noedge-xvr7qd-fdgmzp5mlmp2": PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-noedge-xvr7qd-fdgmzp5mlmp2/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/np-noedge-xvr7qd/nodePools/npnoedge-4-20 -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: InvalidRequestContent -------------------------------------------------------------------------------- { "error": { "code": "InvalidRequestContent", "message": "Invalid value: \"aro-hcp.experimental.nodepool.max-update-duration\": unrecognized experimental tag", "target": "tags[aro-hcp.experimental.nodepool.max-update-duration]" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.nodepool.max-update-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift 02:00:44 UTC · failed at e2e · test Customer should downgrade a nodepool version z-stream downgrade from 4.21.zLatest to 4.21.zPrevious · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:628]: failed to update nodepool npdg-4-21 to downgrade target 4.21.35 Unexpected error: <fmt.wrapError \| 0xa952f808ee0>: failed to start nodepool "npdg-4-21" update in cluster "np-downgrade-x47s9n" resourcegroup="rg-np-downgrade-x47s9n-djktngm9cgxt": PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-downgrade-x47s9n-djktngm9cgxt/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/np-downgrade-x47s9n/nodePools/npdg-4-21 -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: InvalidRequestContent -------------------------------------------------------------------------------- { "error": { "code": "InvalidRequestContent", "message": "Invalid value: \"aro-hcp.experimental.nodepool.max-update-duration\": unrecognized experimental tag", "target": "tags[aro-hcp.experimental.nodepool.max-update-duration]" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:628]: failed to update nodepool npdg-4-21 to downgrade target 4.21.35 Unexpected error: <fmt.wrapError \| 0xa952f808ee0>: failed to start nodepool "npdg-4-21" update in cluster "np-downgrade-x47s9n" resourcegroup="rg-np-downgrade-x47s9n-djktngm9cgxt": PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-downgrade-x47s9n-djktngm9cgxt/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/np-downgrade-x47s9n/nodePools/npdg-4-21 -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: InvalidRequestContent -------------------------------------------------------------------------------- { "error": { "code": "InvalidRequestContent", "message": "Invalid value: \"aro-hcp.experimental.nodepool.max-update-duration\": unrecognized experimental tag", "target": "tags[aro-hcp.experimental.nodepool.max-update-duration]" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.z-stream-update-policy": unrecognized experimental tag; provider Microsoft.RedHatOpenShift 02:00:44 UTC · failed at e2e · test Service Provider should upgrade the control plane z-stream automatically on behalf of the customer for 4.22 · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:157]: failed to enable immediate z-stream updates for cluster "cluster-zstream-4-22-95bqxj" on minor version 4.22 Unexpected error: <exported.ResponseError \| 0x1aa712d69710>: PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-zstream-upgrade-4-22-b5g72gdsstzw/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cluster-zstream-4-22-95bqxj -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: InvalidRequestContent -------------------------------------------------------------------------------- { "error": { "code": "InvalidRequestContent", "message": "Invalid value: \"aro-hcp.experimental.cluster.z-stream-update-policy\": unrecognized experimental tag", "target": "tags[aro-hcp.experimental.cluster.z-stream-update-policy]" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:157]: failed to enable immediate z-stream updates for cluster "cluster-zstream-4-22-95bqxj" on minor version 4.22 Unexpected error: <exported.ResponseError \| 0x1aa712d69710>: PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-zstream-upgrade-4-22-b5g72gdsstzw/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cluster-zstream-4-22-95bqxj -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: InvalidRequestContent -------------------------------------------------------------------------------- { "error": { "code": "InvalidRequestContent", "message": "Invalid value: \"aro-hcp.experimental.cluster.z-stream-update-policy\": unrecognized experimental tag", "target": "tags[aro-hcp.experimental.cluster.z-stream-update-policy]" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.nodepool.max-update-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift 02:00:44 UTC · failed at e2e · test Customer should upgrade and update a nodepool from 4.21.z to 4.21.zLatest · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:195]: failed to upgrade node pool npupgrade-4-21 to version 4.21.36 Unexpected error: <fmt.wrapError \| 0xc0cb099f2c0>: failed to start nodepool "npupgrade-4-21" update in cluster "np-version-upgrade-cluster-cvfvg5" resourcegroup="rg-np-version-upgrade-cvfvg5-r74zmmc476nw": PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-version-upgrade-cvfvg5-r74zmmc476nw/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/np-version-upgrade-cluster-cvfvg5/nodePools/npupgrade-4-21 -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: InvalidRequestContent -------------------------------------------------------------------------------- { "error": { "code": "InvalidRequestContent", "message": "Invalid value: \"aro-hcp.experimental.nodepool.max-update-duration\": unrecognized experimental tag", "target": "tags[aro-hcp.experimental.nodepool.max-update-duration]" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:195]: failed to upgrade node pool npupgrade-4-21 to version 4.21.36 Unexpected error: <fmt.wrapError \| 0xc0cb099f2c0>: failed to start nodepool "npupgrade-4-21" update in cluster "np-version-upgrade-cluster-cvfvg5" resourcegroup="rg-np-version-upgrade-cvfvg5-r74zmmc476nw": PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-version-upgrade-cvfvg5-r74zmmc476nw/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/np-version-upgrade-cluster-cvfvg5/nodePools/npupgrade-4-21 -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: InvalidRequestContent -------------------------------------------------------------------------------- { "error": { "code": "InvalidRequestContent", "message": "Invalid value: \"aro-hcp.experimental.nodepool.max-update-duration\": unrecognized experimental tag", "target": "tags[aro-hcp.experimental.nodepool.max-update-duration]" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.nodepool.max-update-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift 02:00:44 UTC · failed at e2e · test Customer should be able to update node pool labels and taints · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_labels_taints.go:186]: failed to update node pool np-1 with new taints and scale up Unexpected error: <fmt.wrapError \| 0x110fe0e8c640>: failed to start nodepool "np-1" update in cluster "cluster-np-labels-taints" resourcegroup="rg-np-labels-taints-djddxfxjp2wr": PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-labels-taints-djddxfxjp2wr/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cluster-np-labels-taints/nodePools/np-1 -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: InvalidRequestContent -------------------------------------------------------------------------------- { "error": { "code": "InvalidRequestContent", "message": "Invalid value: \"aro-hcp.experimental.nodepool.max-update-duration\": unrecognized experimental tag", "target": "tags[aro-hcp.experimental.nodepool.max-update-duration]" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_labels_taints.go:186]: failed to update node pool np-1 with new taints and scale up Unexpected error: <fmt.wrapError \| 0x110fe0e8c640>: failed to start nodepool "np-1" update in cluster "cluster-np-labels-taints" resourcegroup="rg-np-labels-taints-djddxfxjp2wr": PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-labels-taints-djddxfxjp2wr/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cluster-np-labels-taints/nodePools/np-1 -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: InvalidRequestContent -------------------------------------------------------------------------------- { "error": { "code": "InvalidRequestContent", "message": "Invalid value: \"aro-hcp.experimental.nodepool.max-update-duration\": unrecognized experimental tag", "target": "tags[aro-hcp.experimental.nodepool.max-update-duration]" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.z-stream-update-policy": unrecognized experimental tag; provider Microsoft.RedHatOpenShift 02:00:44 UTC · failed at e2e · test Service Provider should upgrade the control plane z-stream automatically on behalf of the customer for 4.23 · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:157]: failed to enable immediate z-stream updates for cluster "cluster-zstream-4-23-79cvw2" on minor version 4.23 Unexpected error: <exported.ResponseError \| 0x23c456aecc90>: PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-zstream-upgrade-4-23-8lpbkjqgxtmb/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cluster-zstream-4-23-79cvw2 -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: InvalidRequestContent -------------------------------------------------------------------------------- { "error": { "code": "InvalidRequestContent", "message": "Invalid value: \"aro-hcp.experimental.cluster.z-stream-update-policy\": unrecognized experimental tag", "target": "tags[aro-hcp.experimental.cluster.z-stream-update-policy]" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:157]: failed to enable immediate z-stream updates for cluster "cluster-zstream-4-23-79cvw2" on minor version 4.23 Unexpected error: <exported.ResponseError \| 0x23c456aecc90>: PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-zstream-upgrade-4-23-8lpbkjqgxtmb/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cluster-zstream-4-23-79cvw2 -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: InvalidRequestContent -------------------------------------------------------------------------------- { "error": { "code": "InvalidRequestContent", "message": "Invalid value: \"aro-hcp.experimental.cluster.z-stream-update-policy\": unrecognized experimental tag", "target": "tags[aro-hcp.experimental.cluster.z-stream-update-policy]" } } -------------------------------------------------------------------------------- ... occurred