Skip to content

Commit

Permalink
Merge branch 'develop' into elizabeth/refactor-skeletons-serialization
Browse files Browse the repository at this point in the history
  • Loading branch information
eberrigan committed Sep 10, 2024
2 parents 3a7e65a + 91d6c96 commit 08b43ed
Show file tree
Hide file tree
Showing 13 changed files with 618 additions and 240 deletions.
4 changes: 2 additions & 2 deletions .conda/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ This folder defines the conda package build for Linux and Windows. There are run
To build, first go to the base repo directory and install the build environment:

```
mamba env create -f environment_build.yml -n sleap_build && conda activate sleap_build
conda env create -f environment_build.yml -n sleap_build && conda activate sleap_build
```

And finally, run the build command pointing to this directory:
Expand All @@ -15,7 +15,7 @@ conda build .conda --output-folder build -c conda-forge -c nvidia -c https://con
To install the local package:

```
mamba create -n sleap_0 -c conda-forge -c nvidia -c ./build -c https://conda.anaconda.org/sleap/ -c anaconda sleap=x.x.x
conda create -n sleap_0 -c conda-forge -c nvidia -c ./build -c https://conda.anaconda.org/sleap/ -c anaconda sleap=x.x.x
```

replacing x.x.x with the version of SLEAP that you just built.
2 changes: 2 additions & 0 deletions .conda/meta.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,7 @@ requirements:
- conda-forge::attrs ==21.4.0
- conda-forge::cattrs ==1.1.1
- conda-forge::h5py ==3.7.0
- conda-forge::imageio-ffmpeg # Required for imageio to read/write videos with ffmpeg
- conda-forge::jsmin
- conda-forge::jsonpickle ==1.2
- conda-forge::networkx
Expand Down Expand Up @@ -60,6 +61,7 @@ requirements:
- conda-forge::python ==3.7.12 # Run into _MAX_WINDOWS_WORKERS not found if <
- conda-forge::attrs ==21.4.0
- conda-forge::cattrs ==1.1.1
- conda-forge::imageio-ffmpeg # Required for imageio to read/write videos with ffmpeg
- conda-forge::cudatoolkit ==11.3.1
- conda-forge::cudnn=8.2.1
- nvidia::cuda-nvcc=11.3
Expand Down
2 changes: 2 additions & 0 deletions .conda_mac/meta.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,7 @@ requirements:
- conda-forge::attrs >=21.2.0
- conda-forge::cattrs ==1.1.1
- conda-forge::h5py
- conda-forge::imageio-ffmpeg # Required for imageio to read/write videos with ffmpeg
- conda-forge::jsmin
- conda-forge::jsonpickle ==1.2
- conda-forge::keras <2.10.0,>=2.9.0rc0 # Required by tensorflow-macos
Expand Down Expand Up @@ -63,6 +64,7 @@ requirements:
- conda-forge::attrs >=21.2.0
- conda-forge::cattrs ==1.1.1
- conda-forge::h5py
- conda-forge::imageio-ffmpeg # Required for imageio to read/write videos with ffmpeg
- conda-forge::jsmin
- conda-forge::jsonpickle ==1.2
- conda-forge::keras <2.10.0,>=2.9.0rc0 # Required by tensorflow-macos
Expand Down
179 changes: 179 additions & 0 deletions .github/workflows/build_conda_ci.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,179 @@
# Run tests using built conda packages.
name: Build Conda CI (no upload)

# Run when changes to pip wheel
on:
push:
paths:
- ".conda/meta.yaml"
- ".conda_mac/meta.yaml"
- "setup.py"
- "requirements.txt"
- "dev_requirements.txt"
- "environment_build.yml"
- ".github/workflows/build_conda_ci.yml"

# If RUN_BUILD_JOB is set to true, then RUN_ID will be overwritten to the current run id
env:
RUN_BUILD_JOB: true
RUN_ID: 10713717594 # Only used if RUN_BUILD_JOB is false (to dowload build artifact)

jobs:
build:
name: Build package from push (${{ matrix.os }})
runs-on: ${{ matrix.os }}
strategy:
fail-fast: false
matrix:
os: ["windows-2022", "ubuntu-22.04", "macos-14"]
# https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#jobsjob_idstrategymatrixinclude
include:
# Use these variables as defaults
- condarc: .conda/condarc.yaml
- conda-folder: .conda
- pyver: "3.10"
- build-prefix: win
- os: "ubuntu-22.04"
build-prefix: linux
# Use special condarc if macos
- os: "macos-14"
condarc: .conda_mac/condarc.yaml
conda-folder: .conda_mac
build-prefix: osx

steps:
# Setup
- name: Checkout
if: env.RUN_BUILD_JOB == 'true'
uses: actions/checkout@v4

- name: Setup Miniconda
if: env.RUN_BUILD_JOB == 'true'
uses: conda-incubator/setup-miniconda@v3.0.4
with:
miniforge-version: latest
condarc-file: ${{ matrix.condarc }}
python-version: ${{ matrix.pyver }}
environment-file: environment_build.yml
activate-environment: sleap_ci
conda-solver: "libmamba"

- name: Print build environment info
if: env.RUN_BUILD_JOB == 'true'
shell: bash -l {0}
run: |
which python
conda list
pip freeze
# Build conda package
- name: Build conda package
if: env.RUN_BUILD_JOB == 'true'
shell: bash -l {0}
run: |
conda build ${{ matrix.conda-folder }} --output-folder build
# Upload artifact "tests" can use it
- name: Upload conda package artifact
if: env.RUN_BUILD_JOB == 'true'
uses: actions/upload-artifact@v4
with:
name: sleap-build-${{ matrix.build-prefix }}
path: build # Upload entire build directory
retention-days: 1
overwrite: true

tests:
name: Run tests using package (${{ matrix.os }})
runs-on: ${{ matrix.os }}
needs: build # Ensure the build job has completed before starting this job.
strategy:
fail-fast: false
matrix:
os: ["windows-2022", "ubuntu-22.04", "macos-14"]
# https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#jobsjob_idstrategymatrixinclude
include:
# Default values
- build-prefix: win
- build-suffix: 64
- test_args: pytest --durations=-1 tests/
- condarc: .conda/condarc.yaml
- pyver: "3.10"
- conda-channels: -c conda-forge -c nvidia -c anaconda
# Ubuntu specific values
- os: ubuntu-22.04
build-prefix: linux
# Otherwise core dumped in github actions
test_args: |
sudo apt install xvfb libxkbcommon-x11-0 libxcb-icccm4 libxcb-image0 libxcb-keysyms1 libxcb-randr0 libxcb-render-util0 libxcb-xinerama0 libxcb-xfixes0
sudo Xvfb :1 -screen 0 1024x768x24 </dev/null &
export DISPLAY=":1"
pytest tests -k 'not exclude_from_linux_pip_test'
# Use special condarc if macos
- os: "macos-14"
build-prefix: osx
build-suffix: arm64
condarc: .conda_mac/condarc.yaml
conda-channels: -c conda-forge -c anaconda

steps:
- name: Checkout repo
uses: actions/checkout@v4

- name: Setup Python
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.pyver }}

- name: Use current run id for conda package download
shell: bash -l {0}
if: env.RUN_BUILD_JOB == 'true'
run: echo "RUN_ID=${{ github.run_id }}" >> $GITHUB_ENV

# https://github.com/actions/download-artifact?tab=readme-ov-file#usage
- name: Download conda package artifact
uses: actions/download-artifact@v4
id: download
with:
name: sleap-build-${{ matrix.build-prefix }}
path: build
run-id: ${{ env.RUN_ID }}
github-token: ${{ secrets.GITHUB_TOKEN }}

- name: List items in current directory
run: |
ls .
ls -R build
- name: Setup Miniconda
uses: conda-incubator/setup-miniconda@v3.0.4
with:
miniforge-version: latest
condarc-file: ${{ matrix.condarc }}
python-version: ${{ matrix.pyver }}
conda-solver: "libmamba"

- name: Create conda environment
shell: bash -l {0}
run: conda create sleap -y -n sleap_ci -c ./build ${{ matrix.conda-channels }}

- name: Install packages for testing
shell: bash -l {0}
run: |
conda activate sleap_ci
pip install -r "dev_requirements.txt"
# Note: "conda activate" does not persist across steps
- name: Print environment info
shell: bash -l {0}
run: |
conda activate sleap_ci
conda info
conda list
pip freeze
- name: Test package
shell: bash -l {0}
run: |
conda activate sleap_ci
${{ matrix.test_args}}
Original file line number Diff line number Diff line change
@@ -1,17 +1,17 @@
# Run tests using built wheels.
name: Build CI (no upload)
name: Build PyPI CI (no upload)

# Run when changes to pip wheel
on:
push:
paths:
- 'setup.py'
- 'requirements.txt'
- 'dev_requirements.txt'
- 'jupyter_requirements.txt'
- 'pypi_requirements.txt'
- 'environment_build.yml'
- '.github/workflows/build_ci.yml'
- "setup.py"
- "requirements.txt"
- "dev_requirements.txt"
- "jupyter_requirements.txt"
- "pypi_requirements.txt"
- "environment_build.yml"
- ".github/workflows/build_pypi_ci.yml"

jobs:
build:
Expand Down Expand Up @@ -54,7 +54,7 @@ jobs:
shell: bash -l {0}
run: |
python setup.py bdist_wheel
# Upload artifact "tests" can use it
- name: Upload wheel artifact
uses: actions/upload-artifact@v4
Expand All @@ -66,7 +66,7 @@ jobs:
tests:
name: Run tests using wheel (${{ matrix.os }})
runs-on: ${{ matrix.os }}
needs: build # Ensure the build job has completed before starting this job.
needs: build # Ensure the build job has completed before starting this job.
strategy:
fail-fast: false
matrix:
Expand Down Expand Up @@ -103,12 +103,12 @@ jobs:
steps:
- name: Checkout repo
uses: actions/checkout@v4

- name: Setup Python
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.pyver }}

# Download wheel
- name: Download wheel artifact
uses: actions/download-artifact@v4
Expand Down Expand Up @@ -144,8 +144,8 @@ jobs:
run: |
which python
pip freeze
# Install and test the wheel
- name: Test the built wheel
run: |
${{ matrix.test_args}}
${{ matrix.test_args}}
71 changes: 71 additions & 0 deletions .github/workflows/comment-template.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
name: Reusable Comment Workflow

on:
workflow_call:
inputs:
subject_id:
required: true
type: string
body_prefix:
required: true
type: string
comment_type:
required: true
type: string

jobs:
comment:
runs-on: ubuntu-latest
steps:
- name: Post a comment
uses: actions/github-script@v6
with:
script: |
const { owner, repo } = context.repo;
const subject_id = '${{ inputs.subject_id }}';
const comment_type = '${{ inputs.comment_type }}';
const baseBody = `
We appreciate your input and will review it soon.
> [!WARNING]
> A friendly reminder that this is a public forum. Please be cautious when clicking links, downloading files, or running scripts posted by others.
>
> - Always verify the credibility of links and code.
> - Avoid running scripts or installing files from untrusted sources.
> - If you're unsure, ask for clarification before proceeding.
Stay safe and happy SLEAPing!
Best regards,
The Team
`;
const body = `${{ inputs.body_prefix }}\n\n${baseBody}`;
const mutation = comment_type === 'discussion'
? `
mutation($discussionId: ID!, $body: String!) {
addDiscussionComment(input: {discussionId: $discussionId, body: $body}) {
comment {
id
}
}
}
`
: `
mutation($issueId: ID!, $body: String!) {
addComment(input: {subjectId: $issueId, body: $body}) {
commentEdge {
node {
id
body
}
}
}
}
`;
const variables = comment_type === 'discussion'
? { discussionId: subject_id, body: body.trim() }
: { issueId: subject_id, body: body.trim() };
await github.graphql(mutation, variables);
24 changes: 24 additions & 0 deletions .github/workflows/comment.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
name: Comment on New Discussions and Issues

on:
discussion:
types: [created]
issues:
types: [opened]

jobs:
comment_on_discussion:
if: github.event_name == 'discussion'
uses: ./.github/workflows/comment-template.yml
with:
subject_id: ${{ github.event.discussion.node_id }}
body_prefix: "Thank you for starting a new discussion!"
comment_type: "discussion"

comment_on_issue:
if: github.event_name == 'issues'
uses: ./.github/workflows/comment-template.yml
with:
subject_id: ${{ github.event.issue.node_id }}
body_prefix: "Thank you for opening a new issue!"
comment_type: "issue"
Loading

0 comments on commit 08b43ed

Please sign in to comment.