From b8704e7cf1b5661e31c39dc3c8173644657e7cb0 Mon Sep 17 00:00:00 2001 From: John O'Hara Date: Tue, 19 Nov 2024 06:16:50 +0000 Subject: [PATCH] Cleanup references to Bearer tokens --- .../content/en/docs/Concepts/users/index.md | 7 ++-- .../content/en/docs/Deployment/bare-metal.md | 6 --- .../docs/Reference/APIFilterQueryReference.md | 4 +- .../en/docs/Tasks/import-export/index.md | 10 ++--- .../en/docs/Tasks/manage-reports/index.md | 6 +-- .../content/en/docs/Tasks/upload-run/index.md | 40 ++----------------- .../docs/Tutorials/create-test-run/index.md | 18 ++++++--- .../docs/Tutorials/query-collector/index.md | 2 +- .../Tutorials/query-elasticserach/index.md | 4 +- 9 files changed, 31 insertions(+), 66 deletions(-) diff --git a/docs/site/content/en/docs/Concepts/users/index.md b/docs/site/content/en/docs/Concepts/users/index.md index 67956e1ad..2ddf9e1a2 100644 --- a/docs/site/content/en/docs/Concepts/users/index.md +++ b/docs/site/content/en/docs/Concepts/users/index.md @@ -14,10 +14,6 @@ We define 3 levels of access to each item (test, run, dataset or schema): - public: available even to non-authenticated users (for reading) - protected: available to all authenticated users that have the `viewer` role (see below) - private: available only to users who 'own' this data - those who have the team role. - -In addition to these 3 levels, runs and schemas can have a 'token' (randomly generated string): everyone who knows this token can read the record. This token is reset any time the restriction level changes. - -Tests can have tokens, too: you can have an arbitrary number of tokens, each with a subset of read, modify and upload privileges. ## Users and roles @@ -32,6 +28,9 @@ There are few generic roles automatically created during initial realm import. The `admin` role is a system-wide role and is not restricted to a particular teams. +## API Keys + +Users can generate an API key, that will provide programatic access to the Horreum API with the same authorization permissions as the user who created the API Key. ## User authentication diff --git a/docs/site/content/en/docs/Deployment/bare-metal.md b/docs/site/content/en/docs/Deployment/bare-metal.md index 4e8bd660f..f83085239 100644 --- a/docs/site/content/en/docs/Deployment/bare-metal.md +++ b/docs/site/content/en/docs/Deployment/bare-metal.md @@ -135,12 +135,6 @@ QUARKUS_MAILER_LOGIN=disabled # By default webhook notifications that fail to verify TLS integrity fail; set this to ignore verification result. # HORREUM_HOOK_TLS_INSECURE=true -# This is an offline token for the __user_reader user. You can obtain that with -# curl -s -X POST https://keycloak.example.com/auth/realms/horreum/protocol/openid-connect/token \ -# -H 'content-type: application/x-www-form-urlencoded' \ -# -d 'username=__user_reader&password='$PASSWORD'&grant_type=password&client_id=horreum-ui&scope=offline_access' \ -# | jq -r .refresh_token -HORREUM_KEYCLOAK_USER_READER_TOKEN=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx ``` With all this in you can finally start Horreum as any other Java application (note: dependencies in `repo/target/lib/` must be present): diff --git a/docs/site/content/en/docs/Reference/APIFilterQueryReference.md b/docs/site/content/en/docs/Reference/APIFilterQueryReference.md index 7740a0c44..1badc1af2 100644 --- a/docs/site/content/en/docs/Reference/APIFilterQueryReference.md +++ b/docs/site/content/en/docs/Reference/APIFilterQueryReference.md @@ -12,7 +12,7 @@ API operations that support Filter Query provide an HTTP request parameter named To use the Filter Query with an operation that returns a [Run](/docs/concepts/core-concepts/#run) data can be done as follows. ```bash -curl -s 'http://localhost:8080/api/sql/1/queryrun?query=$.*&array=true' -H 'content-type: application/json' -H 'Authorization: Bearer '$TOKEN +curl -s 'http://localhost:8080/api/sql/1/queryrun?query=$.*&array=true' -H 'content-type: application/json' -H 'X-Horreum-API-Key: '$API_KEY { "valid": true, "jsonpath": "$.*", @@ -29,7 +29,7 @@ The response contains an enclosing JSON object that is common for Filter Query o To filter the above API operation to retrieve only the *results* property of the uploaded [Run](/docs/concepts/core-concepts/#run) object the *query* parameter is defined as `$.results` ```bash -curl -s 'http://localhost:8080/api/sql/1/queryrun?query=$.results&array=true' -H 'content-type: application/json' -H 'Authorization: Bearer '$TOKEN +curl -s 'http://localhost:8080/api/sql/1/queryrun?query=$.results&array=true' -H 'content-type: application/json' -H 'X-Horreum-API-Key: '$API_KEY { "valid": true, "jsonpath": "$.results", diff --git a/docs/site/content/en/docs/Tasks/import-export/index.md b/docs/site/content/en/docs/Tasks/import-export/index.md index 66d7f0b26..2ea9c32b5 100644 --- a/docs/site/content/en/docs/Tasks/import-export/index.md +++ b/docs/site/content/en/docs/Tasks/import-export/index.md @@ -44,18 +44,18 @@ Click on Schema/Test and there is a button where you can select either _Import S ```bash curl 'http://localhost:8080/api/schema/import/' \ -s -X POST -H 'content-type: application/json' \ - -H 'Authorization: Bearer '$TOKEN \ + -H 'X-Horreum-API-Key: '$API_KEY \ -d @/path/to/schema.json ``` -If you are unfamiliar with creating the auth token please see [Upload Run](/docs/tasks/upload-new-run/). +If you are unfamiliar with generating an API Key, please see [Upload Run](/docs/tasks/api-keys/). ## Import Tests ```bash curl 'http://localhost:8080/api/test/import/' \ -s -X POST -H 'content-type: application/json' \ - -H 'Authorization: Bearer '$TOKEN \ + -H 'X-Horreum-API-Key: '$API_KEY \ -d @/path/to/test.json ``` @@ -64,7 +64,7 @@ curl 'http://localhost:8080/api/test/import/' \ ```bash SCHEMAID='123' curl 'http://localhost:8080/api/schema/export/?id='$SCHEMAID \ - -H 'Authorization: Bearer '$TOKEN \ + -H 'X-Horreum-API-Key: '$API_KEY \ -O --output-dir /path/to/folder ``` @@ -74,6 +74,6 @@ curl 'http://localhost:8080/api/schema/export/?id='$SCHEMAID \ TESTID='123' curl 'http://localhost:8080/api/test/export/?id=$TESTID' \ -s -X POST -H 'content-type: application/json' \ - -H 'Authorization: Bearer '$TOKEN \ + -H 'X-Horreum-API-Key: '$API_KEY \ -O --output-dir /path/to/folder ``` diff --git a/docs/site/content/en/docs/Tasks/manage-reports/index.md b/docs/site/content/en/docs/Tasks/manage-reports/index.md index e2ec7a518..b0bf27ff9 100644 --- a/docs/site/content/en/docs/Tasks/manage-reports/index.md +++ b/docs/site/content/en/docs/Tasks/manage-reports/index.md @@ -25,10 +25,6 @@ Available Report configurations The same task can be repeated using the web API to delete a `Report`. Copy and paste this into your shell. Note, modify the REPORT_ID parameter. The response to expect is a 204 HTTP response code for a successful deletion. ```bash -TOKEN=$(curl -s http://localhost:8180/realms/horreum/protocol/openid-connect/token \ - -d 'username=user' -d 'password=secret' \ - -d 'grant_type=password' -d 'client_id=horreum-ui' \ - | jq -r .access_token) REPORT_ID= -curl 'http://localhost:8080/api/report/table/'$REPORT_ID -H 'content-type: application/json' -H 'Authorization: Bearer '$TOKEN --request DELETE -v +curl 'http://localhost:8080/api/report/table/'$REPORT_ID -H 'content-type: application/json' -H 'X-Horreum-API-Key: '$API_KEY --request DELETE -v ``` diff --git a/docs/site/content/en/docs/Tasks/upload-run/index.md b/docs/site/content/en/docs/Tasks/upload-run/index.md index 60661d07b..6509b734c 100644 --- a/docs/site/content/en/docs/Tasks/upload-run/index.md +++ b/docs/site/content/en/docs/Tasks/upload-run/index.md @@ -7,44 +7,11 @@ weight: 3 Horreum accepts any valid **JSON** as the input. To get maximum out of Horreum, though, it is recommended to categorize the input using [JSON schema](https://json-schema.org/). -There are two principal ways to authorize operations: - -- Authentication against OIDC provider (Keycloak): This is the standard way that you use when accessing Horreum UI - you use your credentials to get a JSON Web Token (JWT) and this is stored in the browser session. When accessing Horreum over the REST API you need to use this for [Bearer Authentication](https://datatracker.ietf.org/doc/html/rfc6750#section-2.1). The authorization is based on the teams and roles within those teams that you have. -- Horreum API Keys: See more in [API keys](/docs/tasks/api-keys). These replace the so called "Horreum Tokens" that were used in the past in operations that accepted a `token` parameter. +Operations are authorized via API Keys, for details on how to generate an API Key, please refer to [API keys](/docs/tasks/api-keys). If you're running your tests in Jenkins you can skip a lot of the complexity below using [Horreum Plugin](https://plugins.jenkins.io/horreum/). This plugin supports both Jenkins Pipeline and Freeform jobs. -## Getting JWT token - -New data can be uploaded into Horreum only by authorized users. This user must have the permission to upload for given team, e.g. if you'll use `dev-team` as the owner this role is called `dev-uploader` and it is a composition of the team role (`dev-team`) and `uploader` role. You can read more about user management [here](/docs/concepts/users). - -```bash -TOKEN=$(curl -s http://localhost:8180/realms/horreum/protocol/openid-connect/token \ - -d 'username=user' -d 'password=secret' \ - -d 'grant_type=password' -d 'client_id=horreum-ui' \ - | jq -r .access_token) -``` - -A note on JWT token issuer: OIDC-enabled applications usually validate the URL that issued the request vs. URL of the authentication server the application is configured to use - if those don't match you receive 403 Forbidden response without further information. Had you used `http://localhost:8180` as `KEYCLOAK_URL` in the example above you would get rejected in developer mode with the default infrastructure, even though `localhost` resolves to `127.0.0.1` - the URL has to match to what you have in `horreum-backend/.env` as `QUARKUS_OIDC_AUTH_SERVER_URL`. You can disable this check with `-Dquarkus.oidc.token.issuer=any` but that is definitely not recommended in production. - -## Using offline JWT token - -Access token has very limited lifespan; when you want to perform the upload from CI script and don't want to store the password inside you can keep an offline token. This token cannot be used directly as an access token; instead you can store it and use it to obtain a regular short-lived access token: - -```bash -OFFLINE_TOKEN=$(curl -s http://localhost:8180/realms/horreum/protocol/openid-connect/token \ - -d 'username=user' -d 'password=secret' \ - -d 'grant_type=password' -d 'client_id=horreum-ui' -d 'scope=offline_access' \ - | jq -r .refresh_token) -TOKEN=$(curl -s http://localhost:8180/realms/horreum/protocol/openid-connect/token \ - -d 'refresh_token='$OFFLINE_TOKEN \ - -d 'grant_type=refresh_token' -d 'client_id=horreum-ui' \ - | jq -r .access_token) -``` - -Note that the offline token also expires eventually, by default after 30 days. - -## Uploading the data +## Uploading Data There are several mandatory parameters for the upload: @@ -59,6 +26,7 @@ Optionally you can also set `schema` with URI of the JSON schema, overriding (or The upload itself can look like: ```bash +API_KEY='HUSR_00000000_0000_0000_0000_000000000000' TEST='$.info.benchmark' START='2021-08-01T10:35:22.00Z' STOP='2021-08-01T10:40:28.00Z' @@ -66,7 +34,7 @@ OWNER='dev-team' ACCESS='PUBLIC' curl 'http://localhost:8080/api/run/data?test='$TEST'&start='$START'&stop='$STOP'&owner='$OWNER'&access='$ACCESS \ -s -X POST -H 'content-type: application/json' \ - -H 'Authorization: Bearer '$TOKEN \ + -H 'X-Horreum-API-Key: '$API_KEY \ -d @/path/to/data.json ``` diff --git a/docs/site/content/en/docs/Tutorials/create-test-run/index.md b/docs/site/content/en/docs/Tutorials/create-test-run/index.md index a9039a642..21fff50c7 100644 --- a/docs/site/content/en/docs/Tutorials/create-test-run/index.md +++ b/docs/site/content/en/docs/Tutorials/create-test-run/index.md @@ -16,6 +16,7 @@ We assume that you already went through the [previous tutorial](/docs/tutorials/ User logged in {{% /imgproc %}} +### 1. Create Test Press the **New Test** button and fill the test name. Test names must be unique within Horreum. @@ -25,6 +26,8 @@ Create new test Click on **Save** button on left side at the bottom - a blue banner will confirm that the test was created. +### 2. Prepare test data + Now we can prepare the JSON document representing our benchmark results - open `/tmp/run.json` with your favorite text editor and paste in this: ```json @@ -34,23 +37,28 @@ Now we can prepare the JSON document representing our benchmark results - open ` } ``` +### 3. Generate API Key + +To interact with the REST API, you will need an API Key, please follow the instructions to [generate an API Key](/docs/tasks/api-keys/) + +### 4. Upload Data + Now copy and paste this into shell; the reply will be just `1` - the ID of the newly uploaded run. ```bash -TOKEN=$(curl -s http://localhost:8180/realms/horreum/protocol/openid-connect/token \ - -d 'username=user' -d 'password=secret' \ - -d 'grant_type=password' -d 'client_id=horreum-ui' \ - | jq -r .access_token) +API_KEY='HUSR_00000000_0000_0000_0000_000000000000' TEST='Foobar' START='2021-08-01T10:35:22.00Z' STOP='2021-08-01T10:40:28.00Z' OWNER='dev-team' ACCESS='PUBLIC' curl 'http://localhost:8080/api/run/data?test='$TEST'&start='$START'&stop='$STOP'&owner='$OWNER'&access='$ACCESS \ - -s -H 'content-type: application/json' -H 'Authorization: Bearer '$TOKEN \ + -s -H 'content-type: application/json' -H 'X-Horreum-API-Key: '$API_KEY \ -d @/tmp/run.json ``` +### 5. View Data in Horreum + Let's navigate into the tests overview (main page) by clicking the **Tests** link in the main menu on the top left of the page: {{% imgproc tests Fit "1200x300" %}} diff --git a/docs/site/content/en/docs/Tutorials/query-collector/index.md b/docs/site/content/en/docs/Tutorials/query-collector/index.md index 3297094cf..52909c4d5 100644 --- a/docs/site/content/en/docs/Tutorials/query-collector/index.md +++ b/docs/site/content/en/docs/Tutorials/query-collector/index.md @@ -49,7 +49,7 @@ The query can be executed by making a call to the Horreum API; ```bash $ curl 'http://localhost:8080/api/run/data?test='$TEST'&start='$START'&stop='$STOP'&owner='$OWNER'&access='$ACCESS \ - -s -H 'content-type: application/json' -H 'Authorization: Bearer '$TOKEN \ + -s -H 'content-type: application/json' -H 'X-Horreum-API-Key: '$API_KEY \ -d @/tmp/collector_query.json ``` diff --git a/docs/site/content/en/docs/Tutorials/query-elasticserach/index.md b/docs/site/content/en/docs/Tutorials/query-elasticserach/index.md index a48c75d7d..4e56af240 100644 --- a/docs/site/content/en/docs/Tutorials/query-elasticserach/index.md +++ b/docs/site/content/en/docs/Tutorials/query-elasticserach/index.md @@ -52,7 +52,7 @@ The document query can then be sumitted to the Horreum API; ```bash $ curl 'http://localhost:8080/api/run/data?test='$TEST'&start='$START'&stop='$STOP'&owner='$OWNER'&access='$ACCESS \ - -s -H 'content-type: application/json' -H 'Authorization: Bearer '$TOKEN \ + -s -H 'content-type: application/json' -H 'X-Horreum-API-Key: '$API_KEY \ -d @/tmp/elastic_payload.json ``` @@ -101,7 +101,7 @@ The query can be executed by making a call to the Horreum API; ```bash $ curl 'http://localhost:8080/api/run/data?test='$TEST'&start='$START'&stop='$STOP'&owner='$OWNER'&access='$ACCESS \ - -s -H 'content-type: application/json' -H 'Authorization: Bearer '$TOKEN \ + -s -H 'content-type: application/json' -H 'X-Horreum-API-Key: '$API_KEY \ -d @/tmp/elastic_query.json ```