Skip to content

Latest commit

 

History

History
106 lines (69 loc) · 6.33 KB

molecule.adoc

File metadata and controls

106 lines (69 loc) · 6.33 KB
Raw

Molecule testing

Docs:

Scenarios:

  • default: Run tests on the currently logged in Kubernetes cluster. The operator image must be pushed into a registry. We focus in this scenario.

  • kind: Run tests in a one node vanilla Kubernetes started by kind. The operator image is loaded from local docker.

Simple run molecule test runs tests in a recreated namespace ($TEST_OPERATOR_NAMESPACE default to osdk-test). Namespace is destroyed before and after the test (which can be annoying). Molecule has certain stages for execution that can be run individually (similary to Maven). The current stage is saved (Reset → Created → Converged) in temporary folders, so we don’t have to start from scratch every time. See molecule list for current state.

Command test executes a whole cycle, but during development you rather want to execute one-by-one (a whole molecule test takes a lot of time).

  • molecule create: Reach Created state. Plays create.yml and prepare.yml, that updates config/testing/kustomization.yaml with our test namespace set in molecule.yml.

  • molecule converge: Reach Converged state. Plays destroy.yml, converge.yml and kustoize.yml. (Re)Creates test namespace and deploys the operator.

  • molecule verify: Plays verify.yml. Runs all test files from tasks/*_test.yml.

  • molecule destroy: Remove deployment and test namespace.

Verify runs the tests that are regular Ansible scripts usually with k8s steps creating the CR and assert steps to verify certain conditions. Check wait_condition that waits until the CR reaches a Running state which means the operator is done with the reconciliation loop.

Note

Molecule seems to require yml extension (instead of yaml), which is confusing if we use yaml everywhere else in the projects.
Note

The whole molecule directory is practically a bunch of Ansible scripts. Make sure to understand how they work and customize them to your own project’s needs. See some ideas below.

Requirements

For the generated default molecule test:

  • Use kustomize version 3.5.4. It’s expected on the path: cp bin/kustomize /usr/local/bin/kustomize or set export KUSTOMIZE_PATH=/absolute_path/bin/kustomize

  • Install yamllint: brew install yamllint

  • Error ModuleNotFoundError: No module named 'openshift': pip install openshift

  • Install required pacakges: ansible-galaxy collection install -r requirements.yml

  • Set export OPERATOR_IMAGE=quay.io/bszeti/memcached-operator:v0.0.1 (this sets newName: quay.io/bszeti/memcached-operator and newTag: v0.0.1 in config/testing/kustomization.yaml)

Run test:

mulecule converge
molecule verify
molecule destroy
# or
molecule test

Select tests to run

Running all the tests can take long. During developing a new feature we usually only want to run specific tests (then all at the end). Edit included task files in molecule/default/molecule.yml or introduce an env var like:

- name: Import all test files
  include_tasks: '{{ item }}'
  with_fileglob:
    - "{{ lookup('env','TEST_TASKS') | default('tasks/*_test.yml',True) }}"

yamlint

Any error detected by yamlint stops test execution. Also it generates a lot of unnecessary warnings about Jinja templates. We can skip cheking either by naming files to *.yaml.j2 or excluding path by adding ignore: '/templates/' in molecule/default/molecule.yml. Or remove lint completely in molecule/default/molecule.yml

idempotence

By default molecule test destroys and recreates the test namespace. Also it has an idempotence step when the converge steps are replayed. This is unnecessary because we only deploy the operator at this point, so it’s not related to the idempotency of the code we develop. It’s possible to remove idempotence step from the test sequence.

Also if any steps ends with changed status during the idempotence step - which is unaccepted by molecule, we can add changed_when: False to the task.

Multi-tenant testing

Developers should use their own test namespace and built operator image to run tests. Fortunately these can easily be set by the TEST_OPERATOR_NAMESPACE and OPERATOR_IMAGE env vars. (Note: molecule is not run through make so the IMG env var is not used by default - but the molecule.yml file can easily be updated accordingly).

If we used a shared cluster, it’s important not to delete the CRD during molecule destroy, because it deletes all the CRs on the cluster. Add a line in molecule/default/kustomize.yml to skip the CRD during delete phase:

when: not (item.kind == 'CustomResourceDefinition' and state == 'absent')

Verbose output

The verify.yml prints a lot of output in case of an error.

  • JSON of resources (Deployment, Pod, Secret, ConfigMap) in the namespace. It’s suggested to limit this list, especially Secrets can be long becaus of all the ServiceAccount secrets. Or simply remove "Output gathered resources" step.

  • The operator Pod’s log. Can be disabled by removing "Retrieve Pod logs" and "Output gathered logs" steps.

This information can be useful in a CI/CD process, but it’s too much during manual execution.

Test CRs

By default the generated molecule/default/tasks/mykind_test.yml creates a CR during test execution using the sample in config/samples/cache_v1alpha1_mykind.yaml. Samples are useful for developers and users, but it’s better to have dedicated test CR files instead. Especially that these files may need some simple templating. For example using {{namespace}} in the hostname of a Route or Ingress.

Prepare test namespace

Sometimes the test namespace needs some preparation (for example a pull secret linked to a ServiceAccount) before the operator can be deployed. The best place is probably in molecule/default/converge.yml after creating the namespace.

Run with kind

Local testing can be performed with a temporary Kubernets cluster using kind:

  • Install kind: brew install kind

  • Install docker python module: pip install docker

  • Run test: molecule test -s kind