Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixed Bare Metal Expiry #346

Merged
merged 6 commits into from
Sep 30, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .github/scripts/adhoc.sh
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,8 @@
set -o errexit
set -o pipefail

# Copyright (c) 2023-2024 Deephaven Data Labs and Patent Pending

# Provides what is needed to set up an adhoc benchmark run, including bare metal and labels
# ex. adhoc.sh make-labels "where" "0.36.0" "user123:branch-name-123"
# ex. adhoc.sh metal-deploy api-key project-id c3.small.x86 server-name "2 days"
Expand Down
2 changes: 2 additions & 0 deletions .github/scripts/build-server-distribution-remote.sh
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,8 @@
set -o errexit
set -o pipefail

# Copyright (c) 2023-2024 Deephaven Data Labs and Patent Pending

# Assemble the Deephaven server artifacts on the remote side if needed
# The supplied argument can be an image name or <owner>::<branch>
# Ensure that the artifacts and Deephaven version are available in standard directories
Expand Down
2 changes: 2 additions & 0 deletions .github/scripts/dh-jetty-start.sh
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,8 @@
set -o errexit
set -o pipefail

# Copyright (c) 2023-2024 Deephaven Data Labs and Patent Pending

# Build and start the python native Deephaven Community Server

MY_ID=DH_JETTY_BY_STAN
Expand Down
2 changes: 2 additions & 0 deletions .github/scripts/dh-jetty-stop.sh
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,8 @@
# set -o pipefail
# set -o nounset

# Copyright (c) 2023-2024 Deephaven Data Labs and Patent Pending

# Stop the native python Deephaven Community Server

MY_ID=DH_JETTY_BY_STAN
Expand Down
2 changes: 2 additions & 0 deletions .github/scripts/fetch-results-local.sh
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,8 @@ set -o errexit
set -o pipefail
set -o nounset

# Copyright (c) 2023-2024 Deephaven Data Labs and Patent Pending

# Fetches Benchmark results and logs from the remote test server and
# compresses the runs before upload. Writes an output file with the
# SET_LABEL that was used for the set directory name
Expand Down
2 changes: 2 additions & 0 deletions .github/scripts/manage-deephaven-remote.sh
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,8 @@
set -o errexit
set -o pipefail

# Copyright (c) 2023-2024 Deephaven Data Labs and Patent Pending

# Start or Stop a Deephaven image based on the given directive and image/branch name
# The directives argument can be start or stop
# The supplied image argument can be an image name or <owner>::<branch>
Expand Down
2 changes: 2 additions & 0 deletions .github/scripts/matrix_array.sh
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,8 @@
set -o errexit
set -o pipefail

# Copyright (c) 2023-2024 Deephaven Data Labs and Patent Pending

# For a given descriptor, return the appropriate array that can be read by a Github
# workflow with fromJSON(array)
# ex. matrix_array.sh matrix-arr adhoc 5
Expand Down
2 changes: 2 additions & 0 deletions .github/scripts/run-benchmarks-remote.sh
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,8 @@ set -o errexit
set -o pipefail
set -o nounset

# Copyright (c) 2023-2024 Deephaven Data Labs and Patent Pending

# Run benchmarks on the remote side doing one iteration according to the following contract:
# - If TAG_NAME is "Any", run all tests
# - If TAG_NAME starts with "!", run all tests except the named tag
Expand Down
2 changes: 2 additions & 0 deletions .github/scripts/run-publish-local.sh
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,8 @@ set -o errexit
set -o pipefail
set -o nounset

# Copyright (c) 2023-2024 Deephaven Data Labs and Patent Pending

# Run queries that publish a secret slack channel. Queries operation exclusively
# the deephaven-benchmark GCloud bucket

Expand Down
2 changes: 2 additions & 0 deletions .github/scripts/run-ssh-local.sh
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,8 @@ set -o errexit
set -o pipefail
set -o nounset

# Copyright (c) 2023-2024 Deephaven Data Labs and Patent Pending

# Executes a local script on a remote server while storing output relative to the
# local working directory. Also, this script wraps arguments provided for the
# remote scripts in single-quotes to avoid syntax errors.
Expand Down
2 changes: 2 additions & 0 deletions .github/scripts/setup-ssh-local.sh
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,8 @@ set -o errexit
set -o pipefail
set -o nounset

# Copyright (c) 2023-2024 Deephaven Data Labs and Patent Pending

# Setup SSH for connection between github node and remote host

HOST=$1
Expand Down
4 changes: 3 additions & 1 deletion .github/scripts/setup-test-server-remote.sh
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,9 @@ set -o errexit
set -o pipefail
set -o nounset

# Runs the remote side of test server setup
# Copyright (c) 2023-2024 Deephaven Data Labs and Patent Pending

# Runs the remote test server setup where the benchmarks will be run

if [ ! -d "/root" ]; then
echo "$0: Missing the Benchmark install directory"
Expand Down
1 change: 1 addition & 0 deletions .github/scripts/uninstall-docker.sh
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
# Copyright (c) 2023-2024 Deephaven Data Labs and Patent Pending

apt-get purge docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin docker-ce-rootless-extras
rm -rf /var/lib/docker
Expand Down
4 changes: 3 additions & 1 deletion .github/workflows/adhoc-auto-remote-benchmarks.yml
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
# Copyright (c) 2023-2024 Deephaven Data Labs and Patent Pending

# Provision a server and run benchmarks on that system, cleaning up after
# - Deploys a new server on the fly
# - Calls the reusable worflow remote-benchmarks.yml to run and upload benchmarks
Expand Down Expand Up @@ -93,7 +95,7 @@ jobs:
- name: Deploy Bare Metal
id: deploy-metal
run: |
${SD}/adhoc.sh deploy-metal ${METAL_API_KEY} ${METAL_PROJECT_ID} ${METAL_PLAN} ${METAL_ACTOR} ${METAL_EXPIRE}
${SD}/adhoc.sh deploy-metal "${METAL_API_KEY}" "${METAL_PROJECT_ID}" "${METAL_PLAN}" "${METAL_ACTOR}" "${METAL_EXPIRE}"
cat adhoc-deploy-metal.out >> "$GITHUB_OUTPUT"

- name: Archive Setup Logs
Expand Down
2 changes: 2 additions & 0 deletions .github/workflows/adhoc-exist-remote-benchmarks.yml
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
# Copyright (c) 2023-2024 Deephaven Data Labs and Patent Pending

# Run benchmarks on an already provisioned system
# - Calls the reusable worflow remote-benchmarks.yml to run and upload benchmarks
# - Runs the given options with the given image/branch
Expand Down
2 changes: 2 additions & 0 deletions .github/workflows/compare-remote-benchmarks.yml
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
# Copyright (c) 2023-2024 Deephaven Data Labs and Patent Pending

# Run comparison (competitive) benchmarks on a remote system
# according to the release defined in resources/release-benchmark-docker-compose.yml
# in resources/release-benchmark-docker-compose.yml
Expand Down
2 changes: 2 additions & 0 deletions .github/workflows/mvn-integration-test.yml
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
# Copyright (c) 2023-2024 Deephaven Data Labs and Patent Pending

# Run Integration Tests against a Docker install on Github

name: Integration Test on Docker Deephaven
Expand Down
2 changes: 2 additions & 0 deletions .github/workflows/mvn-package.yml
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
# Copyright (c) 2023-2024 Deephaven Data Labs and Patent Pending

# Build, Run Unit tests, and make the Benchmark artifacts; jar and test-jar

name: Unit Test and Package
Expand Down
2 changes: 2 additions & 0 deletions .github/workflows/nightly-remote-benchmarks.yml
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
# Copyright (c) 2023-2024 Deephaven Data Labs and Patent Pending

# Run benchmarks on a remote system for the nightly (edge) build
# - Calls the reusable worflow remote-benchmarks.yml

Expand Down
2 changes: 2 additions & 0 deletions .github/workflows/release-remote-benchmarks.yml
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
# Copyright (c) 2023-2024 Deephaven Data Labs and Patent Pending

# Run benchmarks on a remote system according to the release defined
# in resources/release-benchmark-docker-compose.yml
# - Calls the reusable worflow remote-benchmarks.yml
Expand Down
2 changes: 2 additions & 0 deletions .github/workflows/remote-benchmarks.yml
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
# Copyright (c) 2023-2024 Deephaven Data Labs and Patent Pending

# Run benchmarks on a remote system for the release the project is using
# - This workflow is reusable with run_type set to release or nightly
# - Scripts ending in "-local" run on the github runner
Expand Down