Skip to content

Latest commit

 

History

History
114 lines (83 loc) · 3.4 KB

env.md

File metadata and controls

114 lines (83 loc) · 3.4 KB
Raw

Development environment

As IDE, we use Visual Studio Code. For developing both Python and TypeScript code, use Docker image janjones/awe-gradient.

Docker image

The development Docker image can be built either

  • locally,

    cd gradient
docker build -t janjones/awe-gradient .

  • by executing GitHub Action Gradient Docker Image which also pushes the image to Docker Hub as janjones/awe-gradient.

Remote environments described below usually assume existence of janjones/awe-gradient on Docker Hub.

Locally

To develop locally, either reproduce your environment according to gradient/Dockerfile1, or develop inside a Docker image as described below.

  1. Install Docker Desktop.

  2. Download the Docker image (or build it locally as described above).

    docker pull janjones/awe-gradient

  3. Start the Docker image.

    # Go up to repository root.
cd ..
# (Optional) Remove persistent volume to start fresh.
docker volume rm awe
# Run the container with persistent storage in a named volume.
docker run --rm -it -v awe:/storage -p 22:22 janjones/awe-gradient

  4. Now SSH can be used as below. Password is displayed in Docker output.

    ssh root@localhost

  5. Continue with VS Code SSH steps.

CPU on Gitpod

If CPU is enough (e.g., to process data or experiment with a simple model), Gitpod is configured and can be used.

GPU on Gradient.run

When GPU is needed, Gradient.run cloud can be used. Since it does not support direct SSH connections, a little trick is needed2.

  1. Create and start Gradient notebook. In advanced options

    • ensure Git repository is not set,
    • select the development Docker image (janjones/awe-gradient:latest) as a custom container, and
    • set the command to /run.sh.

  2. Install huproxy locally and configure SSH as shown below. Replace <JUPYTER_LAB_ID> by ID from URL that opens in Gradient when clicking on "Open in Jupyter Lab" button. As SSH password, use token from that URL. Alternatively, look into terminal output in Gradient web environment after the notebook starts, both the token and public URL should appear there.

    Host 127.0.0.1
  User root
  ProxyCommand <FULL_PATH_TO_huproxyclient.exe> wss://<JUPYTER_LAB_ID>.paperspacegradient.com/proxy/%h/%p

  3. Continue with VS Code SSH steps.

VS Code SSH

  1. Connect VS Code via Remote SSH and open directory /storage/awe/src.

  2. If this is the first time, clone the repository, configure Git and initialize VSCode:

    git clone https://github.com/jjonescz/awe .
./sh/configure.sh
./gradient/install.sh

Footnotes

  1. Note that all Python package versions are frozen in awe/requirements.txt. ↩

  2. This trick is based on directing SSH over WebSockets which might stop working if Gradient.run infrastructure is changed in a non-trivial way. ↩