Add KinD test verifying the source deploys and becomes ready

[Cali0707]

Overview

We recently found that even though build and unit tests had been passing, the receive adapter panicked when it started when you actually deployed the gitlab source. To address this, we should add a KinD test job that will:

1. Install eventing and the gitlab source
2. Create a gitlab source
3. Verify that the source becomes ready

More info

These resources will likely be helpful while working on this:

• https://github.com/knative/eventing/blob/main/.github/workflows/kind-e2e.yaml
• https://github.com/knative-extensions/reconciler-test
• https://github.com/knative/eventing/tree/main/test/rekt

[Cali0707]

/help

[knative-prow]

@Cali0707:
This request has been marked as needing help from a contributor.

Please ensure the request meets the requirements listed here.

If this request no longer meets these requirements, the label can be removed
by commenting with the /remove-help command.

In response to this:

/help

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[github-actions]

This issue is stale because it has been open for 90 days with no
activity. It will automatically close after 30 more days of
inactivity. Reopen the issue with /reopen. Mark the issue as
fresh by adding the comment /remove-lifecycle stale.

[Cali0707]

/remove-lifecycle stale
/triage accepted

[harsh098]

Hi @Cali0707
I am trying to take a knack on this issue. I am trying to implement this small test on my own repo.
I went through

• Knative End to End Testing Framework

Is there a design document on how the eventing extensions are structured in general ?
That would really speed up the process.
I am having a little friction in understanding the moving parts of the repo. My only source of understanding things as of now is going through the knative.dev/pkg docs.
Looking forward.

[Cali0707]

cc @pierDipi @creydr

[harsh098]

For the test should I deploy gitlab on the cluster on the test KiND cluster as the example shown in https://github.com/knative/eventing/blob/main/.github/workflows/kind-e2e.yaml or use an external repo ?

[creydr]

Hey @harsh098,
thanks for checking on this. Yes, you need to deploy the needed knative-gitlab artifacts on top of an eventing installation. Therefor I see the following to do:

1. Add the eventing nightly builds into this repo (e.g. into a third_party folder). In the eventing-kafka-broker repo we do the same. Therefor you need to update the update-nightlies job, which will take the nightlies and push them into this repo. Check for example, how this is done for eventing-kafka-broker here
2. Add an e2e test which installs a Gitlab source and checks if it becomes ready (this can be verified locally already). Check out for example some e2e tests in the eventing repo, which installs a pingsourcesource and checks if it becomes ready: https://github.com/knative/eventing/blob/36e0721b385204978e4eeb9e2321613090a82346/test/rekt/features/pingsource/readyness.go#L28.
3. Create a new Github workflow for the end to end tests which installs KinD, deploys the eventing-core artifacts (from the third_party directory), then checks out the eventing-gitlab sources and installs it into the cluster and finally runs the e2e tests. You can check for example

I hope this helps to get started, otherwise let us know

[harsh098]

Thanks a lot progressing in this direction now.

[harsh098]

/assign

[creydr]

One more addition: Many knative and knative-extension repos have some kind-e2e.yaml Github workflow defined, which runs some e2e tests on KinD. Maybe those help you too (e.g. for reconciler-test repo)

[harsh098]

Yes I did look into it. Will be reporting soon on this.

[harsh098]

Hey @harsh098, thanks for checking on this. Yes, you need to deploy the needed knative-gitlab artifacts on top of an eventing installation. Therefor I see the following to do:

1. Add the eventing nightly builds into this repo (e.g. into a third_party folder). In the eventing-kafka-broker repo we do the same. Therefor you need to update the update-nightlies job, which will take the nightlies and push them into this repo. Check for example, how this is done for eventing-kafka-broker here
2. Add an e2e test which installs a Gitlab source and checks if it becomes ready (this can be verified locally already). Check out for example some e2e tests in the eventing repo, which installs a pingsourcesource and checks if it becomes ready: https://github.com/knative/eventing/blob/36e0721b385204978e4eeb9e2321613090a82346/test/rekt/features/pingsource/readyness.go#L28.
3. Create a new Github workflow for the end to end tests which installs KinD, deploys the eventing-core artifacts (from the third_party directory), then checks out the eventing-gitlab sources and installs it into the cluster and finally runs the e2e tests. You can check for example

I hope this helps to get started, otherwise let us know

Can you please elaborate more on pushing the eventing nighly builds ? Whose nightly builds are these knative-eventing ? or the eventing-gitlab charts ?

[creydr]

Can you please elaborate more on pushing the eventing nighly builds ? Whose nightly builds are these knative-eventing ? or the eventing-gitlab charts ?

The knative-eventing nightly builds come from eventing-core (knative/eventing) and represents the latest (nightly) builds. The yamls for those are in a Google Cloud Storage Bucket. As we often need the nightly builds in other repositories for tests, we have automated this and a job, which pushes the yamls into the different repos (check the config for this for example [here](https://github.com/knative-extensions/knobots/blob/6863df9c3a5504ea5522e21d307d2486c5c0ef8d/.github/workflows/update-nightlies.yaml#L83-L91. This results in PRs like this).
You need those yamls, as you need to install first eventing-core on your KinD cluster, before you install the eventing-gitlab manifests.
So I would do the following:

1. Add a config like this for eventing-gitlab to get the latest eventing-core manifests pushed daily into a folder like third_party/eventing-latest
2. Wait until you have at least one of those automated PRs merged (like this): [Automated] Update eventing-kafka-broker-eventing nightly eventing-kafka-broker#4081
3. Setup you KinD cluster and install those manifests from third_party/eventing-latest before you install the eventing-gitlab manifests

[harsh098]

Got it thanks !

[harsh098]

Just made a PR here
knative-extensions/knobots#402

[creydr]

@harsh098 you can add the "initial" version for the manifests then via

for x in eventing-crds.yaml eventing-core.yaml; do
  curl https://storage.googleapis.com/knative-nightly/eventing/latest/$x --create-dirs -o ./third_party/eventing-latest/$x
done

[harsh098]

Sure. Let me take a look into it

[harsh098]

I'm waiting for the PR to be merged. Any updates ?

[creydr]

knative-extensions/knobots#402 will update them, after an "initial version" of eventing-crds.yaml eventing-core.yaml exists (I didn't know they need to exist before). So you can add them in your PR too / need to add an initial version of the files. You can do this for example as said in #496 (comment).
Does this help?