Enable Downstream trigger job in the architecture repo #374

raukadah · 2024-08-23T00:56:38Z

In order to test VA HCI related changes, we used to run testproject in the downstream.

downstream-va-hci-trigger-job drops the need of running testproject for VA HCI related changes.

An developer can trigger the job by adding trigger github-experimental in the pull request comment section. The trigger job will create a review on
https://review.rdoproject.org/r/q/project:tripleo-downstream-trigger-nested-virt repo with proper depends-on with the origin pr and dependendent change.

From tripleo-downstream-trigger-nested-virt gerrit review, it will start VA HCI job downstream in rdo-check pipeline. Once the job finishes, It will post the results on the gerrit review. Devs can verify the results before merging it.

Note: It modifies create-zuul-jobs.py to include experimental jobs.

softwarefactory-project-zuul · 2024-08-23T00:56:44Z

Zuul encountered a syntax error while parsing its
configuration in the repo openstack-k8s-operators/architecture on branch main. The
problem was:

while parsing a block mapping
in "openstack-k8s-operators/architecture/zuul.d/projects.yaml@main", line 1, column 3
did not find expected key
in "openstack-k8s-operators/architecture/zuul.d/projects.yaml@main", line 12, column 5

raukadah · 2024-08-23T01:02:21Z

Here is an example of trigger job in action with results.

raukadah · 2024-08-23T01:02:44Z

/assign @abays

fultonj · 2024-08-23T13:22:57Z

I see the logs from the ceph deploy from the new trigger and it looks good. Thank you @raukadah this will be helpful. I assume we'll keep gating with podified-multinode-hci-deployment-crc but add this additional command when necessary to see downstream result when necessary. Do you agree?

I'm +2 to add this feature and don't see anything wrong with the patch myself.

https://sf.hosted.upshift.rdu2.redhat.com/logs/54/54154/5/check-rdo/ci-framework-baremetal-static-node-rhel9-va-hci/a774960/logs/controller-0/ci-framework-data/logs/ci_script_001_run_deploy.log

fultonj · 2024-08-23T13:23:50Z

I do see a red result here test-zuul.

https://github.com/openstack-k8s-operators/architecture/actions/runs/10518027471/job/29143170869?pr=374

softwarefactory-project-zuul · 2024-08-26T09:04:09Z

Zuul encountered a syntax error while parsing its
configuration in the repo openstack-k8s-operators/architecture on branch main. The
problem was:

expected a list for dictionary value @ data['github-experimental-trigger']['jobs']

The problem appears in the the following project stanza:

project:
github-check:
jobs: &id001
- noop
- rhoso-architecture-validate-bgp
- rhoso-architecture-validate-bgp_dt01
- rhoso-architecture-validate-hci
- rhoso-architecture-validate-nfv-ovs-dpdk-sriov-hci
- rhoso-architecture-validate-osasinfra
...

in "openstack-k8s-operators/architecture/zuul.d/projects.yaml@main", line 1

raukadah · 2024-08-26T09:08:36Z

I see the logs from the ceph deploy from the new trigger and it looks good. Thank you @raukadah this will be helpful. I assume we'll keep gating with podified-multinode-hci-deployment-crc but add this additional command when necessary to see downstream result when necessary. Do you agree?

I'm +2 to add this feature and don't see anything wrong with the patch myself.

https://sf.hosted.upshift.rdu2.redhat.com/logs/54/54154/5/check-rdo/ci-framework-baremetal-static-node-rhel9-va-hci/a774960/logs/controller-0/ci-framework-data/logs/ci_script_001_run_deploy.log

@fultonj yup! The goal for this job is to trigger when we need downstream results. In order to avoid the misuse, We are also working on github-action pr to restrict triggering to approvers only.

cjeanner · 2024-08-26T09:16:51Z

zuul.d/trigger-jobs.yaml

+      `\trigger github-experimental` in the github pull request comment.
+      These jobs runs in github-experimental-trigger pipeline.
+    files:
+      - ^lib


we should probably start having one file per DT/VA, instead of the main "defaults.yml" nested in the automation tree. That would allow to trigger the job upon automation file modification as well.

@fultonj @abays IIRC the first proposal for the automation description file was, indeed, to get one file per VA/DT. IMHO, having everything in one single file was a mistake - we can probably correct that without the need to break all of the existing jobs, by ensuring the filename matches the scenario name.

We'd then "just" need a simple change in the framework to consume the file, and voilà.

Any thoughts?

Otherwise, the file listed here looks OK, with the current automation layout.

Some VA/DTs already have dedicated files, so I think we can move the ones to a dedicated file for each VA/DT without any issue.

I have no objections to having a separate file per VA/DT.

If we get a PR which proposes separate file per VA/DT, then I'd be happy to help merge it.

As a way forward can we do this?

A ci-framework patch and architecture patch to split default into multiple files

then we merge this patch

Or is there a need to merge this patch sooner?

Above approach looks good to me. I will work on splitting the defaults changes and then merge this patch.
Below are the list of patches to achieve the same:

Move automation default vars into seperate scenario files #375

Use cifmw_architecture_scenario to set proper automation file ci-framework#2267

openstack-k8s-operators#374 adds the trigger job which will run different Baremetal VA jobs downstream on different architecture file changes. Currently automation/vars/default.yaml contains different multiple scenarios, stored in a single file. Trigger job may run unwanted jobs downstream without testing the proper architecture changes. By moving automations vars into different scenario files allow us to run selective trigger job and test the proper prs. Signed-off-by: Chandan Kumar (raukadah) <raukadah@gmail.com>

openstack-k8s-operators#374 adds the trigger job which will run different Baremetal VA jobs downstream on different architecture file changes. Currently automation/vars/default.yaml contains different multiple scenarios, stored in a single file. Trigger job may run unwanted jobs downstream without testing the proper architecture changes. By moving automations vars into different scenario files allow us to run selective trigger job and test the proper prs. Let's keep the defaults.yaml file till we finish the migration. Signed-off-by: Chandan Kumar (raukadah) <raukadah@gmail.com>

Move automation default vars into seperate scenario files #374 adds the trigger job which will run different Baremetal VA jobs downstream on different architecture file changes. Currently automation/vars/default.yaml contains different multiple scenarios, stored in a single file. Trigger job may run unwanted jobs downstream without testing the proper architecture changes. By moving automations vars into different scenario files allow us to run selective trigger job and test the proper prs. Reviewed-by: John Fulton <johfulto@redhat.com>

In order to test VA HCI related changes, we used to run testproject in the downstream. downstream-va-hci-trigger-job drops the need of running testproject for VA HCI related changes. An developer can trigger the job by adding `trigger github-experimental` in the pull request comment section. The trigger job will create a review on https://review.rdoproject.org/r/q/project:tripleo-downstream-trigger-nested-virt repo with proper depends-on with the origin pr and dependendent change. From tripleo-downstream-trigger-nested-virt gerrit review, it will start VA HCI job downstream in rdo-check pipeline. Once the job finishes, It will post the results on the gerrit review. Devs can verify the results before merging it. Note: It modifies create-zuul-jobs.py to include experimental jobs. Signed-off-by: Chandan Kumar (raukadah) <raukadah@gmail.com>

cjeanner · 2024-09-05T13:23:18Z

/lgtm

fultonj

/approve
/lgtm

openshift-ci · 2024-09-05T13:24:02Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: fultonj, raukadah

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [fultonj,raukadah]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

abays · 2024-09-05T13:55:12Z

recheck-gate

softwarefactory-project-zuul · 2024-09-05T13:55:27Z

Build succeeded (gate pipeline).
https://softwarefactory-project.io/zuul/t/rdoproject.org/buildset/70b7f9a80dec43f59ba2b41a50cfffc1

✔️ noop SUCCESS in 0s

openstack-k8s-operators#374 adds the trigger job which will run different Baremetal VA jobs downstream on different architecture file changes. Currently automation/vars/default.yaml contains different multiple scenarios, stored in a single file. Trigger job may run unwanted jobs downstream without testing the proper architecture changes. By moving automations vars into different scenario files allow us to run selective trigger job and test the proper prs. Let's keep the defaults.yaml file till we finish the migration. Signed-off-by: Chandan Kumar (raukadah) <raukadah@gmail.com>

openshift-ci bot requested review from cjeanner and fultonj August 23, 2024 00:56

openshift-ci bot added the approved label Aug 23, 2024

raukadah force-pushed the trigger_job branch from aba28fa to c122398 Compare August 23, 2024 01:00

openshift-ci bot assigned abays Aug 23, 2024

raukadah force-pushed the trigger_job branch from c122398 to 68f3243 Compare August 26, 2024 09:04

raukadah force-pushed the trigger_job branch from 68f3243 to b744513 Compare August 26, 2024 09:06

cjeanner reviewed Aug 26, 2024

View reviewed changes

raukadah mentioned this pull request Aug 27, 2024

Move automation default vars into seperate scenario files #375

Merged

raukadah force-pushed the trigger_job branch from b744513 to 9643736 Compare September 5, 2024 13:20

openshift-ci bot assigned cjeanner Sep 5, 2024

openshift-ci bot added the lgtm label Sep 5, 2024

fultonj approved these changes Sep 5, 2024

View reviewed changes

openshift-ci bot assigned fultonj Sep 5, 2024

softwarefactory-project-zuul bot merged commit 4b43496 into openstack-k8s-operators:main Sep 5, 2024
7 checks passed

rdiazcam mentioned this pull request Sep 15, 2024

Move automation default vars into seperate scenario files #396

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Enable Downstream trigger job in the architecture repo #374

Enable Downstream trigger job in the architecture repo #374

raukadah commented Aug 23, 2024 •

edited

Loading

softwarefactory-project-zuul bot commented Aug 23, 2024

raukadah commented Aug 23, 2024

raukadah commented Aug 23, 2024

fultonj commented Aug 23, 2024

fultonj commented Aug 23, 2024

softwarefactory-project-zuul bot commented Aug 26, 2024

raukadah commented Aug 26, 2024 •

edited

Loading

cjeanner Aug 26, 2024

pablintino Aug 26, 2024

abays Aug 26, 2024

fultonj Aug 26, 2024

fultonj Aug 26, 2024 •

edited

Loading

raukadah Aug 27, 2024 •

edited

Loading

cjeanner commented Sep 5, 2024

fultonj left a comment

openshift-ci bot commented Sep 5, 2024

abays commented Sep 5, 2024

softwarefactory-project-zuul bot commented Sep 5, 2024

Enable Downstream trigger job in the architecture repo #374

Enable Downstream trigger job in the architecture repo #374

Conversation

raukadah commented Aug 23, 2024 • edited Loading

softwarefactory-project-zuul bot commented Aug 23, 2024

raukadah commented Aug 23, 2024

raukadah commented Aug 23, 2024

fultonj commented Aug 23, 2024

fultonj commented Aug 23, 2024

softwarefactory-project-zuul bot commented Aug 26, 2024

raukadah commented Aug 26, 2024 • edited Loading

cjeanner Aug 26, 2024

Choose a reason for hiding this comment

pablintino Aug 26, 2024

Choose a reason for hiding this comment

abays Aug 26, 2024

Choose a reason for hiding this comment

fultonj Aug 26, 2024

Choose a reason for hiding this comment

fultonj Aug 26, 2024 • edited Loading

Choose a reason for hiding this comment

raukadah Aug 27, 2024 • edited Loading

Choose a reason for hiding this comment

cjeanner commented Sep 5, 2024

fultonj left a comment

Choose a reason for hiding this comment

openshift-ci bot commented Sep 5, 2024

abays commented Sep 5, 2024

softwarefactory-project-zuul bot commented Sep 5, 2024

raukadah commented Aug 23, 2024 •

edited

Loading

raukadah commented Aug 26, 2024 •

edited

Loading

fultonj Aug 26, 2024 •

edited

Loading

raukadah Aug 27, 2024 •

edited

Loading