diff --git a/docs/UsersGuide.asciidoc b/docs/UsersGuide.asciidoc index d71075104a4..76470ea5770 100644 --- a/docs/UsersGuide.asciidoc +++ b/docs/UsersGuide.asciidoc @@ -1082,19 +1082,23 @@ openqa-load-templates test.json == Asset handling == -Multiple parameters exist to reference "assets" to be used by tests. "Assets" are essentially -content that is stored by the openQA web-UI and provided to the workers; when sending jobs to -os-autoinst on the workers, openQA adjusts the parameter values to refer to an absolute path -where the worker will be able to access the content. Things that are typically assets include the -ISOs and other images that are tested, for example. +Multiple parameters exist to reference "assets" to be used by tests. "Assets" +are essentially content that is stored by the openQA web-UI and provided to the +workers. Things that are typically assets include the ISOs and other images that +are tested, for example. Some assets can also be produced by a job, sent back to the web-UI, and used by a later job (see explanation of 'storing' and 'publishing' assets, below). Assets can also be seen in the web-UI and downloaded directly (though there is a configuration option to hide some or all asset types from public view in the web-UI). -The parameters treated as assets are as follows. Where you see e.g. `ISO_n`, that means `ISO_1`, -`ISO_2` etc. will all be treated as assets. +Assets may be shared between the web-UI and the workers by having them literally use a shared +filesystem (this used to be the only option), or by having the workers download them from the +server when needed and cache them locally. Checkout the documentation about +<> for more on this. + +=== Specifying assets required by a job === +The following job settings are specifying that an asset is required by a job: * `ISO` (type `iso`) * `ISO_n` (type `iso`) @@ -1105,6 +1109,8 @@ The parameters treated as assets are as follows. Where you see e.g. `ISO_n`, tha * `KERNEL` (type `other`) * `INITRD` (type `other`) +Where you see e.g. `ISO_n`, that means `ISO_1`, `ISO_2` etc. will all be treated as assets. + The values of the above parameters are expected to be the name of a file - or, in the case of `REPO_n`, a directory - that exists under the path `/var/lib/openqa/share/factory` on the openQA web-UI. That path has subdirectories for each of the asset types, and the file or directory must @@ -1142,34 +1148,46 @@ uncompress it to `foo2.iso`, store it in `/var/lib/openqa/share/factory/iso` and `ISO_1=foo2.iso`. Again, you can also set `ISO_1` to change the name the file will be downloaded and uncompressed as. -Assets may be shared between the web-UI and the workers by having them literally use a shared -filesystem (this used to be the only option), or by having the workers download them from the -server when needed and cache them locally. Checkout the documentation about -<> for more on this. +=== Specifying assets created by a job === +Jobs can upload assets to the web-UI so other jobs can used them as `HDD_n` and +`UEFI_PFLASH_VARS` assets as described in the previous section. + +To declare an asset to be uploaded, you can use the job settings `PUBLISH_HDD_n` +and `PUBLISH_PFLASH_VARS`. For instance, if you specify +`PUBLISH_HDD_1=updated.qcow2`, the `HDD_1` disk image as it exists at the end of +the test will be uploaded back to the web-UI and stored under the name +`updated.qcow2`. Any other job can then specify `HDD_1=updated.qcow2` to use +this published image as its `HDD_1`. -`HDD_n` assets can be 'stored' or 'published' by a job, and `UEFI_PFLASH_VARS` assets can be -'published'. These both mean that if the job completes successfully, the resulting state of those -disk assets will be sent back to the web-UI and made available as an `hdd`-type asset. To 'store' -an asset, you can specify e.g. `STORE_HDD_1`. To 'publish' it, you can specify e.g. -`PUBLISH_HDD_1` or `PUBLISH_PFLASH_VARS`. If you specify `PUBLISH_HDD_1=updated.qcow2`, the -`HDD_1` disk image as it exists at the end of the test will be uploaded back to the web-UI and -stored under the name `updated.qcow2`; any other job can then specify `HDD_1=updated.qcow2` to use -this published image as its `HDD_1`. To force publishing assets even in case -of a failed job one can try the `FORCE_PUBLISH_HDD_` variable. - -The difference between 'storing' and 'publishing' is that when 'storing' an asset, it will be -altered in some way (currently, by prepending the job ID to the filename) to associate it with -the particular job that produced it. That means that many jobs can 'store' an asset under "the -same name" without conflicting. Of course, that would seem to make it hard for other jobs to use -the 'stored' image - but for "chained" jobs, the reverse operation is done transparently. This -all means that a 'parent' job template can specify `STORE_HDD_1=somename.qcow2` and its 'child' -job template(s) can specify `HDD_1=somename.qcow2`, and everything will work, without multiple -runs of the same jobs overwriting the asset. For more on "chained" jobs, see the documentation -of <>. - -When using this mechanism you will often also want to use the +IMPORTANT: Assets that are already existing will be overridden. If the same +asset is uploaded by multiple jobs concurrently this will lead to file +corruption. So be sure to use unique names or use private assets as explained +in the subsection below. + +NOTE: Note that assets are by default only uploaded if the job completes +successfully. To force publishing assets even in case of a failed job one can +specify the `FORCE_PUBLISH_HDD_` variable. + +NOTE: When using this mechanism you will often also want to use the <> mechanism. +==== Private assets ==== +There is a mechanism to alter an asset's file name automatically to associate +it with the particular job that produced it (currently, by prepending the job ID +to the filename). To make use of it, use `STORE_HDD_n` (instead of +`PUBLISH_HDD_n`). Those assets can then be consumed by chained jobs. For +instance, if a parent job uploads an asset via `STORE_HDD_1=somename.qcow2`, its +children can use it via `HDD_1=somename.qcow2` without having to worry about +naming conflicts. + +IMPORTANT: This only works if the jobs uploading and consuming jobs have a +chained dependency. For more on "chained" jobs, see the documentation of +<>. + +NOTE: Access to private assets is not protected. Theoretically, jobs outside the +chain can still access the asset by explicitly prepending the ID of the creating +job. + [id="asset_cleanup"] === Asset cleanup === For more information on assets, start reading from the beginning of the