Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix the OSBMS scaling KUTTL test #1111

Merged
merged 1 commit into from
Nov 19, 2024
+73 −93
Merged

Fix the OSBMS scaling KUTTL test #1111

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion config/samples/osp-director_v1beta1_openstackbaremetalset.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ spec:
# How many nodes to provision
count: 1
# The image to install on the provisioned nodes
baseImageUrl: http://192.168.111.1/images/rhel-guest-image-8.4-992.x86_64.qcow2
baseImageUrl: http://192.168.111.1/images/rhel-guest-image-9.2-20230414.17.x86_64.qcow2
# use this provisionServerName instead of creating a new one use baseImageUrl
provisionServerName: openstack
# The secret containing the SSH pub key to place on the provisioned nodes
Expand Down
2 changes: 1 addition & 1 deletion config/samples/osp-director_v1beta1_openstackprovisionserver.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,5 +5,5 @@ metadata:
namespace: openstack
spec:
port: 6190
baseImageUrl: http://192.168.111.1/images/rhel-guest-image-8.4-992.x86_64.qcow2
baseImageUrl: http://192.168.111.1/images/rhel-guest-image-9.2-20230414.17.x86_64.qcow2
interface: br-ctlplane
9 changes: 5 additions & 4 deletions tests/kuttl/common/scripts/delete_bmh.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,11 +4,12 @@
# Dumps a BMH and its credentials secret to designated locations and then deletes the BMH
#

bmhname=$(oc get -n openstack osbms compute -o json | jq -r '.status.baremetalHosts["compute-0"].hostRef')
bmhsecretname=$(oc get bmh -n openshift-machine-api ${bmhname} -o json | jq -r '.spec.bmc.credentialsName')
oc get -n openshift-machine-api bmh ${bmhname} -o json |\
bmhname=$(oc get bmh -n openshift-machine-api -l osp-director.openstack.org/name=compute -l osp-director.openstack.org/namespace=openstack -o name | head -1)
bmhsecretname=$(oc get -n openshift-machine-api ${bmhname} -o json | jq -r '.spec.bmc.credentialsName')
Comment on lines -7 to +8
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't need to pick a specific host/BMH. We just need to delete one of them, and so we now just choose the first one in the list.

oc get -n openshift-machine-api ${bmhname} -o json |\
jq 'del(.metadata.labels,.metadata.annotations,.metadata.generation,.metadata.resourceVersion,.metadata.uid,.spec.consumerRef,.spec.userData,.spec.networkData,.spec.image)' |\
jq '.metadata.annotations={"inspect.metal3.io": "disabled"}' |\
jq '.spec.online=false' > /tmp/kuttl_bmh1.json
oc get -n openshift-machine-api secret ${bmhsecretname} -o json |\
jq 'del(.metadata.labels,.metadata.annotations,.metadata.generation,.metadata.resourceVersion,.metadata.uid,.metadata.ownerReferences)' > /tmp/kuttl_bmh1_secret.json
oc delete -n openshift-machine-api bmh ${bmhname}
oc delete -n openshift-machine-api ${bmhname}
2 changes: 1 addition & 1 deletion tests/kuttl/tests/openstackbackuprequest_save_restore/04-assert.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,7 @@ spec:
kind: OpenStackBaremetalSet
metadata:
finalizers:
- baremetalset.osp-director.openstack.org-compute
- baremetalset.osp-director.openstack.org
name: compute
namespace: openstack
openStackClients:
Expand Down
6 changes: 3 additions & 3 deletions tests/kuttl/tests/openstackbaremetalset_scale/03-assert.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ apiVersion: osp-director.openstack.org/v1beta1
kind: OpenStackBaremetalSet
metadata:
finalizers:
- baremetalset.osp-director.openstack.org-compute
- baremetalset.osp-director.openstack.org
labels:
ooo-subnetname/ctlplane: "true"
ooo-subnetname/internal_api: "true"
Expand All @@ -19,7 +19,7 @@ metadata:
name: compute
namespace: openstack
spec:
baseImageUrl: http://192.168.111.1/images/rhel-guest-image-8.4-992.x86_64.qcow2
baseImageUrl: http://192.168.111.1/images/rhel-guest-image-9.2-20230414.17.x86_64.qcow2
count: 0
ctlplaneInterface: enp1s0
deploymentSSHSecret: osp-controlplane-ssh-keys
Expand All @@ -41,7 +41,7 @@ metadata:
name: openstack
namespace: openstack
spec:
baseImageUrl: http://192.168.111.1/images/rhel-guest-image-8.4-992.x86_64.qcow2
baseImageUrl: http://192.168.111.1/images/rhel-guest-image-9.2-20230414.17.x86_64.qcow2
interface: br-ctlplane
port: 6190
status:
Expand Down
10 changes: 8 additions & 2 deletions tests/kuttl/tests/openstackbaremetalset_scale/03-create_openstackbaremetalset.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,8 +11,14 @@ commands:
- command: |
oc patch provisioning provisioning-configuration --type='json' -p='[{"op": "replace", "path": "/spec/provisioningInterface", "value": "br-ctlplane"}]'
namespaced: true
- command: oc delete pods -n openshift-machine-api -l baremetal.openshift.io/cluster-baremetal-operator=metal3-state
namespaced: true
# Delete the Metal3 pod so that it is recreated and works with the br-ctlplane bridge that was
# created in the previous step. The scale command below will cause the existing Metal3 pod to
# terminate, but then the baremetal cluster operator will reset the replicas to 1 and create a
# new pod
- script: |
oc scale deployment metal3 -n openshift-machine-api --replicas=0
sleep 10
oc wait deployment metal3 -n openshift-machine-api --for condition=Available --timeout=300s
# Create the required userpassword Secret
- command: oc apply -f ../../common/manifests/userpassword.yaml
namespaced: true
Expand Down
12 changes: 6 additions & 6 deletions tests/kuttl/tests/openstackbaremetalset_scale/04-assert.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,11 +15,11 @@ apiVersion: osp-director.openstack.org/v1beta1
kind: OpenStackBaremetalSet
metadata:
finalizers:
- baremetalset.osp-director.openstack.org-compute
- baremetalset.osp-director.openstack.org
name: compute
namespace: openstack
spec:
baseImageUrl: http://192.168.111.1/images/rhel-guest-image-8.4-992.x86_64.qcow2
baseImageUrl: http://192.168.111.1/images/rhel-guest-image-9.2-20230414.17.x86_64.qcow2
count: 2
ctlplaneInterface: enp1s0
deploymentSSHSecret: osp-controlplane-ssh-keys
Expand Down Expand Up @@ -70,7 +70,7 @@ spec:
namespace: openshift-machine-api
online: true
rootDeviceHints:
deviceName: /dev/sda
hctl: "0:0:0:0"
userData:
namespace: openshift-machine-api
status:
Expand All @@ -82,7 +82,7 @@ status:
provisioning:
bootMode: legacy
rootDeviceHints:
deviceName: /dev/sda
hctl: "0:0:0:0"
state: provisioned
---
apiVersion: metal3.io/v1alpha1
Expand All @@ -107,7 +107,7 @@ spec:
namespace: openshift-machine-api
online: true
rootDeviceHints:
deviceName: /dev/sda
hctl: "0:0:0:0"
userData:
namespace: openshift-machine-api
status:
Expand All @@ -119,7 +119,7 @@ status:
provisioning:
bootMode: legacy
rootDeviceHints:
deviceName: /dev/sda
hctl: "0:0:0:0"
state: provisioned
---
apiVersion: osp-director.openstack.org/v1beta1
Expand Down
4 changes: 2 additions & 2 deletions tests/kuttl/tests/openstackbaremetalset_scale/04-scale_up_openstackbaremetalset.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
#
# - Scale OpenStackBaremetalSet to 2 BaremetalHosts
# - Also try to then immediately scale to 3, which should fail due to webhook validation (because we only
# have 2 BMHs available) and thus do nothing
# have 2 BMHs available in a standard osp-director-dev-tools cluster) and thus do nothing
#

apiVersion: kuttl.dev/v1beta1
Expand All @@ -12,7 +12,7 @@ commands:
namespaced: true
- script: |
RES=$(oc patch -n openstack openstackbaremetalset compute --type='json' -p='[{"op": "replace", "path": "/spec/count", "value":3}]' 2>&1)
if grep -q "Unable to find 3 requested BaremetalHost count" <<< "$RES"; then
if grep -qi "unable to find 3 requested BaremetalHost count" <<< "$RES"; then
exit 0
else
exit 1
Expand Down
8 changes: 4 additions & 4 deletions tests/kuttl/tests/openstackbaremetalset_scale/06-assert.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,11 +11,11 @@ apiVersion: osp-director.openstack.org/v1beta1
kind: OpenStackBaremetalSet
metadata:
finalizers:
- baremetalset.osp-director.openstack.org-compute
- baremetalset.osp-director.openstack.org
name: compute
namespace: openstack
spec:
baseImageUrl: http://192.168.111.1/images/rhel-guest-image-8.4-992.x86_64.qcow2
baseImageUrl: http://192.168.111.1/images/rhel-guest-image-9.2-20230414.17.x86_64.qcow2
count: 1
ctlplaneInterface: enp1s0
deploymentSSHSecret: osp-controlplane-ssh-keys
Expand Down Expand Up @@ -76,7 +76,7 @@ spec:
namespace: openshift-machine-api
online: true
rootDeviceHints:
deviceName: /dev/sda
hctl: "0:0:0:0"
userData:
namespace: openshift-machine-api
status:
Expand All @@ -88,7 +88,7 @@ status:
provisioning:
bootMode: legacy
rootDeviceHints:
deviceName: /dev/sda
hctl: "0:0:0:0"
state: provisioned
---
apiVersion: osp-director.openstack.org/v1beta1
Expand Down
4 changes: 2 additions & 2 deletions tests/kuttl/tests/openstackbaremetalset_scale/07-assert.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ spec:
namespace: openshift-machine-api
online: true
rootDeviceHints:
deviceName: /dev/sda
hctl: "0:0:0:0"
userData:
namespace: openshift-machine-api
status:
Expand All @@ -38,7 +38,7 @@ status:
provisioning:
bootMode: legacy
rootDeviceHints:
deviceName: /dev/sda
hctl: "0:0:0:0"
state: provisioned
---
# We don't know which BMH was chosen as compute-0 originally, so we just look
Expand Down
22 changes: 2 additions & 20 deletions tests/kuttl/tests/openstackbaremetalset_scale/09-assert.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ apiVersion: osp-director.openstack.org/v1beta1
kind: OpenStackBaremetalSet
metadata:
finalizers:
- baremetalset.osp-director.openstack.org-compute
- baremetalset.osp-director.openstack.org
labels:
ooo-subnetname/ctlplane: "true"
ooo-subnetname/internal_api: "true"
Expand All @@ -17,7 +17,7 @@ metadata:
name: compute
namespace: openstack
spec:
baseImageUrl: http://192.168.111.1/images/rhel-guest-image-8.4-992.x86_64.qcow2
baseImageUrl: http://192.168.111.1/images/rhel-guest-image-9.2-20230414.17.x86_64.qcow2
count: 2
ctlplaneInterface: enp1s0
roleName: Compute
Expand All @@ -32,21 +32,3 @@ status:
provisioningStatus:
readyCount: 1
state: Error
---
# We don't know which BMH was chosen as compute-0 originally, so we use a script to
# check for the "BaremetalHostNotFound" error (as the actual error message will vary
# based on which BMH was actually deleted)
apiVersion: kuttl.dev/v1beta1
kind: TestAssert
namespaced: true
timeout: 600
commands:
- script: |
set -x
sleep 5
osbmsstate=$(oc get osbms -n openstack compute -o json | jq -r '.status.provisioningStatus.reason')
if grep -q "Please check BaremetalHost resources and re-add" <<< "$osbmsstate"; then
exit 0
else
exit 1
fi
Comment on lines -35 to -52
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was unnecessary. Just checking the OSBMS status.provisioningStatus.readyCount and status.provisioningStatus.state is enough to detect the problem we are expecting.

11 changes: 0 additions & 11 deletions tests/kuttl/tests/openstackbaremetalset_scale/09-errors.yaml
View file
Open in desktop

This file was deleted.

7 changes: 2 additions & 5 deletions tests/kuttl/tests/openstackbaremetalset_scale/10-assert.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,11 +11,11 @@ apiVersion: osp-director.openstack.org/v1beta1
kind: OpenStackBaremetalSet
metadata:
finalizers:
- baremetalset.osp-director.openstack.org-compute
- baremetalset.osp-director.openstack.org
name: compute
namespace: openstack
spec:
baseImageUrl: http://192.168.111.1/images/rhel-guest-image-8.4-992.x86_64.qcow2
baseImageUrl: http://192.168.111.1/images/rhel-guest-image-9.2-20230414.17.x86_64.qcow2
count: 2
ctlplaneInterface: enp1s0
deploymentSSHSecret: osp-controlplane-ssh-keys
Expand All @@ -33,7 +33,4 @@ status:
hostname: compute-1
provisioningStatus:
readyCount: 1
reason: Existing BaremetalHost "openshift-worker-0" not found for OpenStackBaremetalSet
compute. Please check BaremetalHost resources and re-add "openshift-worker-0"
to continue
Comment on lines -36 to -38
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was unnecessary. Just checking the OSBMS status.provisioningStatus.readyCount and status.provisioningStatus.state is enough to detect the problem we are expecting.

state: Error
4 changes: 2 additions & 2 deletions tests/kuttl/tests/openstackbaremetalset_scale/10-errors.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,8 @@ apiVersion: osp-director.openstack.org/v1beta1
kind: OpenStackBaremetalSet
metadata:
finalizers:
- baremetalset.osp-director.openstack.org-compute
- baremetalset.osp-director.openstack.org
name: compute
namespace: openstack
spec:
coubt: 1
count: 1
2 changes: 1 addition & 1 deletion tests/kuttl/tests/openstackbaremetalset_scale/10-scale_down_openstackbaremetalset.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
apiVersion: kuttl.dev/v1beta1
kind: TestStep
commands:
- command: oc annotate bmh openshift-worker-1 osp-director.openstack.org/delete-host=true -n openshift-machine-api
- command: oc annotate bmh -l osp-director.openstack.org/name=compute -l osp-director.openstack.org/namespace=openstack osp-director.openstack.org/delete-host=true -n openshift-machine-api
namespaced: true
- script: |
RES=$(oc patch -n openstack openstackbaremetalset compute --type='json' -p='[{"op": "replace", "path": "/spec/count", "value":1}]' 2>&1)
Expand Down
7 changes: 2 additions & 5 deletions tests/kuttl/tests/openstackbaremetalset_scale/11-assert.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,11 +11,11 @@ apiVersion: osp-director.openstack.org/v1beta1
kind: OpenStackBaremetalSet
metadata:
finalizers:
- baremetalset.osp-director.openstack.org-compute
- baremetalset.osp-director.openstack.org
name: compute
namespace: openstack
spec:
baseImageUrl: http://192.168.111.1/images/rhel-guest-image-8.4-992.x86_64.qcow2
baseImageUrl: http://192.168.111.1/images/rhel-guest-image-9.2-20230414.17.x86_64.qcow2
count: 2
ctlplaneInterface: enp1s0
deploymentSSHSecret: osp-controlplane-ssh-keys
Expand All @@ -33,7 +33,4 @@ status:
hostname: compute-1
provisioningStatus:
readyCount: 1
reason: Existing BaremetalHost "openshift-worker-0" not found for OpenStackBaremetalSet
compute. Please check BaremetalHost resources and re-add "openshift-worker-0"
to continue
Comment on lines -36 to -38
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was unnecessary. Just checking the OSBMS status.provisioningStatus.readyCount and status.provisioningStatus.state is enough to detect the problem we are expecting.

state: Error
4 changes: 2 additions & 2 deletions tests/kuttl/tests/openstackbaremetalset_scale/11-errors.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,8 @@ apiVersion: osp-director.openstack.org/v1beta1
kind: OpenStackBaremetalSet
metadata:
finalizers:
- baremetalset.osp-director.openstack.org-compute
- baremetalset.osp-director.openstack.org
name: compute
namespace: openstack
spec:
coubt: 1
count: 3
2 changes: 1 addition & 1 deletion tests/kuttl/tests/openstackbaremetalset_scale/11-scale_up_openstackbaremetalset.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
apiVersion: kuttl.dev/v1beta1
kind: TestStep
commands:
- command: oc annotate bmh openshift-worker-1 osp-director.openstack.org/delete-host- -n openshift-machine-api --overwrite
- command: oc annotate bmh -l osp-director.openstack.org/name=compute -l osp-director.openstack.org/namespace=openstack osp-director.openstack.org/delete-host- -n openshift-machine-api
namespaced: true
- script: |
RES=$(oc patch -n openstack openstackbaremetalset compute --type='json' -p='[{"op": "replace", "path": "/spec/count", "value":3}]' 2>&1)
Expand Down
Loading
Loading