diff --git a/src/data/nav.yaml b/src/data/nav.yaml index 7a758d98..204bbd4c 100644 --- a/src/data/nav.yaml +++ b/src/data/nav.yaml @@ -95,25 +95,28 @@ - title: Installing an Ecosystem using Helm path: /docs/ecosystem/ecosystem-installing-k8s - title: Configuring authentication - path: /docs/ecosystem/ecosystem-authentication - - title: Test streams - path: /docs/writing-own-tests/test-streams - - title: Managing integrated test runs - path: /docs/ecosystem/ecosystem-manage-cps + path: /docs/ecosystem/ecosystem-authentication - title: Configuring an Ecosystem using resource files path: /docs/ecosystem/cps-yaml + - title: Updating credentials in an Ecosystem + path: /docs/ecosystem/ecosystem-update-creds + - title: Managing configuration properties + path: /docs/ecosystem/ecosystem-manage-cps + - title: Managing tests in a Galasa Ecosystem + path: /docs/manage-ecosystem + items: + - title: Test streams + path: /docs/manage-ecosystem/test-streams - title: Selecting tests to run in an Ecosystem - path: /docs/cli-command-reference/ecosystem-cli-runs-prepare + path: /docs/manage-ecosystem/ecosystem-cli-runs-prepare - title: Running tests in an Ecosystem - path: /docs/cli-command-reference/ecosystem-cli-runs-submit + path: /docs/manage-ecosystem/ecosystem-cli-runs-submit - title: Retrying and cancelling tests - path: /docs/cli-command-reference/runs-reset-cancel + path: /docs/manage-ecosystem/runs-reset-cancel - title: Viewing test run results - path: /docs/cli-command-reference/cli-runs-get + path: /docs/manage-ecosystem/cli-runs-get - title: Downloading test artifacts - path: /docs/cli-command-reference/ecosystem-cli-runs-download - - title: Updating credentials in an Ecosystem - path: /docs/ecosystem/ecosystem-update-creds + path: /docs/manage-ecosystem/ecosystem-cli-runs-download - title: Upgrading path: /docs/upgrading - title: Managers diff --git a/src/markdown-pages/docs/ecosystem/ecosystem-manage-cps.md b/src/markdown-pages/docs/ecosystem/ecosystem-manage-cps.md index fdfb44ef..cafb4355 100644 --- a/src/markdown-pages/docs/ecosystem/ecosystem-manage-cps.md +++ b/src/markdown-pages/docs/ecosystem/ecosystem-manage-cps.md @@ -1,6 +1,6 @@ --- path: "/docs/ecosystem/ecosystem-manage-cps" -title: "Managing integrated test runs" +title: "Managing configuration properties" --- [Retrieving namespaces](#retrievingnamespaces)
diff --git a/src/markdown-pages/docs/manage-ecosystem/manage-ecosystem.md b/src/markdown-pages/docs/manage-ecosystem/manage-ecosystem.md new file mode 100644 index 00000000..012cd386 --- /dev/null +++ b/src/markdown-pages/docs/manage-ecosystem/manage-ecosystem.md @@ -0,0 +1,10 @@ +--- +path: "/docs/manage-ecosystem" +title: "Managing tests in a Galasa Ecosystem" +--- + +Read the topics contained in this section to understand how to simply and effectively manage the scheduling and running of large numbers of Galasa tests in automation in a Galasa Ecosystem. You can group your tests using the test stream function, select groups of tests to run, retry and cancel failed tests, view results in a number of formats, and download test artifacts quickly and easily by using the Galasa CLI. + + + + diff --git a/src/markdown-pages/docs/cli-command-reference/runs-download.md b/src/markdown-pages/docs/manage-ecosystem/runs-download.md similarity index 98% rename from src/markdown-pages/docs/cli-command-reference/runs-download.md rename to src/markdown-pages/docs/manage-ecosystem/runs-download.md index 0a650d73..596b46c9 100644 --- a/src/markdown-pages/docs/cli-command-reference/runs-download.md +++ b/src/markdown-pages/docs/manage-ecosystem/runs-download.md @@ -1,5 +1,5 @@ --- -path: "/docs/cli-command-reference/ecosystem-cli-runs-download" +path: "/docs/manage-ecosystem/ecosystem-cli-runs-download" title: "Downloading test artifacts" --- diff --git a/src/markdown-pages/docs/cli-command-reference/runs-get.md b/src/markdown-pages/docs/manage-ecosystem/runs-get.md similarity index 99% rename from src/markdown-pages/docs/cli-command-reference/runs-get.md rename to src/markdown-pages/docs/manage-ecosystem/runs-get.md index 257fccdd..53ca149e 100644 --- a/src/markdown-pages/docs/cli-command-reference/runs-get.md +++ b/src/markdown-pages/docs/manage-ecosystem/runs-get.md @@ -1,5 +1,5 @@ --- -path: "/docs/cli-command-reference/cli-runs-get" +path: "/docs/manage-ecosystem/cli-runs-get" title: "Viewing test run results" --- diff --git a/src/markdown-pages/docs/cli-command-reference/runs-prepare.md b/src/markdown-pages/docs/manage-ecosystem/runs-prepare.md similarity index 97% rename from src/markdown-pages/docs/cli-command-reference/runs-prepare.md rename to src/markdown-pages/docs/manage-ecosystem/runs-prepare.md index 3585f1a1..02e63459 100644 --- a/src/markdown-pages/docs/cli-command-reference/runs-prepare.md +++ b/src/markdown-pages/docs/manage-ecosystem/runs-prepare.md @@ -1,11 +1,11 @@ --- -path: "/docs/cli-command-reference/ecosystem-cli-runs-prepare" +path: "/docs/manage-ecosystem/ecosystem-cli-runs-prepare" title: "Selecting tests to run in an Ecosystem" --- If you have a group of tests that you want to run in batch, the simplest way to do this is by defining these tests inside a portfolio. You can create a portfolio of tests by using the `runs prepare` command. You can build the portfolio from single or multiple [test streams](https://galasa.dev/docs/writing-own-tests/test-streams). -This portfolio of tests can then be run by using the [runs submit command](/docs/cli-command-reference/ecosystem-cli-runs-submit). When you submit the tests to run, you need specify only the portfolio name on the command, rather than listing all of the classes. +This portfolio of tests can then be run by using the [runs submit command](/docs/manage-ecosystem/ecosystem-cli-runs-submit). When you submit the tests to run, you need specify only the portfolio name on the command, rather than listing all of the classes. ## Working with the `runs prepare` command diff --git a/src/markdown-pages/docs/cli-command-reference/runs-reset-cancel.md b/src/markdown-pages/docs/manage-ecosystem/runs-reset-cancel.md similarity index 98% rename from src/markdown-pages/docs/cli-command-reference/runs-reset-cancel.md rename to src/markdown-pages/docs/manage-ecosystem/runs-reset-cancel.md index 8182ea0d..325c1d96 100644 --- a/src/markdown-pages/docs/cli-command-reference/runs-reset-cancel.md +++ b/src/markdown-pages/docs/manage-ecosystem/runs-reset-cancel.md @@ -1,5 +1,5 @@ --- -path: "/docs/cli-command-reference/runs-reset-cancel" +path: "/docs/manage-ecosystem/runs-reset-cancel" title: "Retrying and cancelling tests" --- diff --git a/src/markdown-pages/docs/cli-command-reference/runs-submit.md b/src/markdown-pages/docs/manage-ecosystem/runs-submit.md similarity index 91% rename from src/markdown-pages/docs/cli-command-reference/runs-submit.md rename to src/markdown-pages/docs/manage-ecosystem/runs-submit.md index 801a3613..c5f89c73 100644 --- a/src/markdown-pages/docs/cli-command-reference/runs-submit.md +++ b/src/markdown-pages/docs/manage-ecosystem/runs-submit.md @@ -1,11 +1,11 @@ --- -path: "/docs/cli-command-reference/ecosystem-cli-runs-submit" +path: "/docs/manage-ecosystem/ecosystem-cli-runs-submit" title: "Running tests in an Ecosystem" --- The `runs submit` command submits and monitors tests in the Galasa Ecosystem. Tests can be input either from a portfolio or directly from a test package. -For information about creating a portfolio by using the Galasa CLI, see the documentation for the [runs prepare](/docs/cli-command-reference/ecosystem-cli-runs-prepare) command. +For information about creating a portfolio by using the Galasa CLI, see the documentation for the [runs prepare](/docs/manage-ecosystem/ecosystem-cli-runs-prepare) command. ## Working with the `runs submit` command @@ -13,7 +13,7 @@ The following section provides a subset of examples of how you can use the `runs ### Submitting tests to an ecosystem from a portfolio -The following example assumes that you have created a `my_portfolio.yaml` portfolio by using the [runs prepare](/docs/cli-command-reference/ecosystem-cli-runs-prepare) command. The command submits tests from the `my_portfolio.yaml` portfolio, and specifies the following settings. +The following example assumes that you have created a `my_portfolio.yaml` portfolio by using the [runs prepare](/docs/manage-ecosystem/ecosystem-cli-runs-prepare) command. The command submits tests from the `my_portfolio.yaml` portfolio, and specifies the following settings. On Mac or Unix: diff --git a/src/markdown-pages/docs/manage-ecosystem/test-streams-architecture.svg b/src/markdown-pages/docs/manage-ecosystem/test-streams-architecture.svg new file mode 100644 index 00000000..225825b6 --- /dev/null +++ b/src/markdown-pages/docs/manage-ecosystem/test-streams-architecture.svg @@ -0,0 +1,40 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/src/markdown-pages/docs/manage-ecosystem/test-streams.md b/src/markdown-pages/docs/manage-ecosystem/test-streams.md new file mode 100644 index 00000000..0534a7e5 --- /dev/null +++ b/src/markdown-pages/docs/manage-ecosystem/test-streams.md @@ -0,0 +1,58 @@ +--- +path: "/docs/manage-ecosystem/test-streams" +title: "Test streams" +--- + +Galasa's Ecosystem organises tests into _streams_. Test streams give you more options when it comes to organising your test projects, how often they are built and where they are deployed. + +A test stream is a group of tests that you want to run in automation, represented by a single OBR (OSGi Bundle Repository) and its equivalent test catalog. Galasa uses the OBR to locate your tests in a Maven repository, along with all of the Managers that a test project requires. A test catalog is generated directly from the test source, is always up to date, and is specified in the Galasa CLI to select tests to run in automation. Administrators can create few or many test streams, depending on testing requirements. Testers can contribute tests to one or more test streams. + +## Test stream components + +A test stream is made up of a name, description, OBR, location and repository components. + +- The OBR component is a URL that points to a list of Maven coodinates in the format `mvn:{grp}/{artifact id}/{version}/{obr}`. +- The location component is a URL that points to the location of the test catalog that is associated with a particular OBR in the format `http://points-to-my-testcatalog`. A test catalog is an index of tests. + +- The repository component provides the location of the Maven repository where the test code and test catalog are deployed + +The following diagram shows the relationship between the test code, test catalog, and test stream. + +![test stream architecture:](test-streams-architecture.svg) + + +## Creating and retrieving test stream components + +The components of the `test.stream` property are set by using `galasactl properties set` command, as shown in the following example: + +``` +galasactl properties set --namespace framework --name test.stream.mystream.description --value "My stream to use as an example" +galasactl properties set --namespace framework --name test.stream.mystream.repo --value http://points-to-my-maven-repo.example.org +galasactl properties set --namespace framework --name test.stream.mystream.location --value http://points-to-my-test-catalog.example.org +galasactl properties set --namespace framework --name test.stream.mystream.obr --value mvn:myorg/myartifact/0.0.1/obr +``` + +These four commands each set a part of the stream. Streams must always be created in the `framework` namespace and must always have all four components set. + +You can view all test streams in the `framework` namespace by using the `galasactl properties get` command, as shown in the following example: + +``` +galasactl properties get --namespace framework --name test.streams +namespace name value +framework test.streams integrationtests, regressiontests + +Total:2 +``` + +## Organising test streams + +There are many ways to organise test streams: + +- Single test stream: A single test stream can contain all of your tests. This system is of most use when you are learning about Galasa and experimenting with the automation system. At some point, it is much more beneficial for you to move to a dual stream setup. +- Dual test streams: Often, you will need to run a _production_ test stream and a _best so far_ test stream. The production stream is used by the CI/CD pipeline to test the application, while the best so far stream will include tests under development that you wish to observe under automation. + + The source code for tests in dual streams is usually from the same SCM (Source Code Management) area - in git terms, this would mean the same repository but in different branches. The best so far stream contains all of the production stream tests plus new or modified tests. When you are happy that a test has reached a good level of quality, you can promote it to the production stream and make it available to the CI/CD pipeline. Usually, you would organise things so that you have a Maven build per stream. + +- Many streams: You might organise your streams so that you have one per team, or dual test streams per team. If you have lots of tests per application area, having a stream per area is also possible. + +Build time and frequency usually dictates how streams are structured. If you are building your test projects frequently, and those build times are long, you might wish to split the test projects into streams. Alternatively, you could stick with a single stream, but split the projects into separate builds that all kick off the build of the test stream OBR when complete. diff --git a/src/markdown-pages/docs/writing-own-tests/setting-up-galasa-project.md b/src/markdown-pages/docs/writing-own-tests/setting-up-galasa-project.md index 39f0dbee..94d66d26 100644 --- a/src/markdown-pages/docs/writing-own-tests/setting-up-galasa-project.md +++ b/src/markdown-pages/docs/writing-own-tests/setting-up-galasa-project.md @@ -226,7 +226,7 @@ The following extract from the generated parent pom.xml shows some of the key el pom ``` -- The `` is used to group related Maven projects in a Maven repository. It is recommended (but not enforced) that all projects in a [test stream](../writing-own-tests/test-streams) share the same `groupId`. +- The `` is used to group related Maven projects in a Maven repository. It is recommended (but not enforced) that all projects in a [test stream](../manage-ecosystem/test-streams) share the same `groupId`. - The `` must be unique for each Maven project under a `groupId`. To prevent confusion, you could make it unique across `groupId`s. The `groupId` and `artifactId` can nominally be anything you choose, but if you were to ever consider publishing the project on Maven Central, you would have to ensure that they were unique across Maven Central. Because of this, and to avoid future name collisions, it is conventional to use (reversed) company domain names, which leads to patterns like `dev.galasa.example.banking`. - The `` in this project is set to `0.1.0-SNAPSHOT`. - `` indicates what type of Maven project this is - in this case, a `pom` project. diff --git a/src/markdown-pages/docs/writing-own-tests/test-streams.md b/src/markdown-pages/docs/writing-own-tests/test-streams.md deleted file mode 100644 index c6b8d9b1..00000000 --- a/src/markdown-pages/docs/writing-own-tests/test-streams.md +++ /dev/null @@ -1,21 +0,0 @@ ---- -path: "/docs/writing-own-tests/test-streams" -title: "Test streams" ---- - -Galasa's ecosystem organises tests into _streams_. Test streams give you more options when it comes to organising your test projects, how often they are built and where they are deployed. - -You can have as few or as many test streams as you wish. - -A stream is a group of tests that you wish to run in automation represented by a single OBR (OSGi Bundle Repository) and its equivalent test catalog. Galasa uses an OBR to locate your tests in a Maven repository, along with all of the Managers that a test project requires. A test catalog is generated directly from the test source, is always up to date, and is what you would specify in the Galasa CLI to select tests to run in automation. - -There are many ways to organise test streams: - -- Single test stream: A single test stream can contain all of your tests. This system is of most use when you are learning about Galasa and experimenting with the automation system. At some point, it is much more beneficial for you to move to a dual stream setup. -- Dual test streams: Often, you will need to run a _production_ test stream and a _best so far_ test stream. The production stream is used by the CI/CD pipeline to test the application, while the best so far stream will include tests under development that you wish to observe under automation. - - The source code for tests in dual streams is usually from the same SCM (Source Code Management) area - in git terms, this would mean the same repository but in different branches. The best so far stream contains all of the production stream tests plus new or modified tests. When you are happy that a test has reached a good level of quality, you can promote it to the production stream and make it available to the CI/CD pipeline. Usually, you would organise things so that you have a Maven build per stream. - -- Many streams: You might organise your streams so that you have one per team, or dual test streams per team. If you have lots of tests per application area, having a stream per area is also possible. - -Build time and frequency usually dictates how streams are structured. If you are building your test projects frequently, and those build times are long, you might wish to split the test projects into streams. Alternatively, you could stick with a single stream, but split the projects into separate builds that all kick off the build of the test stream OBR when complete.