Bump to 1.4.1a3 (py310)

[roomrys]

Description

This PR skips the attempted previous prerelease of 1.4.1a3 (that does no bumpy the python version). But, we were running into trouble with conflicting h5py packages on the WIndows build manual test. So, here we are - releasing what was intended to be 1.4.1a4 as 1.4.1a3.

Types of changes

• Bugfix
• New feature
• Refactor / Code style update (no logical changes)
• Build / CI changes
• Documentation Update
• Other (pre-release)

Does this address any currently open issues?

[list open issues here]

Outside contributors checklist

• Review the guidelines for contributing to this repository
• Read and sign the CLA and add yourself to the authors list
• Make sure you are making a pull request against the develop branch (not main). Also you should start your branch off develop
• Add tests that prove your fix is effective or that your feature works
• Add necessary documentation (if appropriate)

Thank you for contributing to SLEAP!

❤️

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 75.55%. Comparing base (8252868) to head (7b77b7a).

Additional details and impacted files

@@                             Coverage Diff                              @@
##           elizabeth/update-python-and-dependencies    #1989      +/-   ##
============================================================================
- Coverage                                     75.62%   75.55%   -0.07%     
============================================================================
  Files                                           133      133              
  Lines                                         24628    24628              
============================================================================
- Hits                                          18625    18608      -17     
- Misses                                         6003     6020      +17     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[roomrys]

Windows (manual test)

Backwards compatibility

pip freeze

mamba list

Training/Inference via GUI

GPU Memory usage skyrockets when we start the training loop.

        },
        "instance_cropping": {
            "center_on_part": "abdomen",
            "crop_size": 576,
            "crop_size_detection_padding": 16
        }
    },
    "model": {
        "backbone": {
            "leap": null,
            "unet": {
                "stem_stride": null,
                "max_stride": 32,
                "output_stride": 2,
                "filters": 24,
                "filters_rate": 1.5,
                "middle_block": true,
                "up_interpolate": true,
                "stacks": 1
            },
            "hourglass": null,
            "resnet": null,
            "pretrained_encoder": null
        },
        "heads": {
            "single_instance": null,
            "centroid": {
                "anchor_part": "abdomen",
                "sigma": 2.5,
                "output_stride": 2,
                "loss_weight": 1.0,
                "offset_refinement": false
            },
            "centered_instance": null,
            "multi_instance": null,
            "multi_class_bottomup": null,
            "multi_class_topdown": null
        },
        "base_checkpoint": null
    },
    "optimization": {
        "preload_data": true,
        "augmentation_config": {
            "rotate": true,
            "rotation_min_angle": -180.0,
            "rotation_max_angle": 180.0,
            "translate": false,
            "translate_min": -5,
            "translate_max": 5,
            "scale": false,
            "scale_min": 0.9,
            "scale_max": 1.1,
            "uniform_noise": false,
            "uniform_noise_min_val": 0.0,
            "uniform_noise_max_val": 10.0,
            "gaussian_noise": false,
            "gaussian_noise_mean": 5.0,
            "gaussian_noise_stddev": 1.0,
            "contrast": false,
            "contrast_min_gamma": 0.5,
            "contrast_max_gamma": 2.0,
            "brightness": false,
            "brightness_min_val": 0.0,
            "brightness_max_val": 10.0,
            "random_crop": false,
            "random_crop_height": 256,
            "random_crop_width": 256,
            "random_flip": false,
            "flip_horizontal": true
        },
        "online_shuffling": true,
        "shuffle_buffer_size": 128,
        "prefetch": true,
        "batch_size": 4,
        "batches_per_epoch": 200,
        "min_batches_per_epoch": 200,
        "val_batches_per_epoch": 10,
        "min_val_batches_per_epoch": 10,
        "epochs": 2,
        "optimizer": "adam",
        "initial_learning_rate": 0.0001,
        "learning_rate_schedule": {
            "reduce_on_plateau": true,
            "reduction_factor": 0.5,
            "plateau_min_delta": 1e-06,
            "plateau_patience": 5,
            "plateau_cooldown": 3,
            "min_learning_rate": 1e-08
        },
        "hard_keypoint_mining": {
            "online_mining": false,
            "hard_to_easy_ratio": 2.0,
            "min_hard_keypoints": 2,
            "max_hard_keypoints": null,
            "loss_scale": 5.0
        },
        "early_stopping": {
            "stop_training_on_plateau": true,
            "plateau_min_delta": 1e-08,
            "plateau_patience": 20
        }
    },
    "outputs": {
        "save_outputs": true,
        "run_name": "241009_114324.centroid.n=3",
        "run_name_prefix": "",
        "run_name_suffix": "",
        "runs_folder": "D:/social-leap-estimates-animal-poses/datasets/drosophila-melanogaster-courtship/drosophila-melanogaster-courtship\\models",
        "tags": [
            ""
        ],
        "save_visualizations": true,
        "keep_viz_images": false,
        "zip_outputs": false,
        "log_to_csv": true,
        "checkpointing": {
            "initial_model": false,
            "best_model": true,
            "every_epoch": false,
            "latest_model": false,
            "final_model": false
        },
        "tensorboard": {
            "write_logs": false,
            "loss_frequency": "epoch",
            "architecture_graph": false,
            "profile_graph": false,
            "visualizations": true
        },
        "zmq": {
            "subscribe_to_controller": true,
            "controller_address": "tcp://127.0.0.1:8998",
            "controller_polling_timeout": 10,
            "publish_updates": true,
            "publish_address": "tcp://127.0.0.1:9001"
        }
    },
    "name": "",
    "description": "",
    "sleap_version": "1.4.1a2",
    "filename": "C:\\Users\\TalmoLab\\AppData\\Local\\Temp\\tmp4fv5srb5\\241009_114325_training_job.json"
}
INFO:sleap.nn.training:
INFO:sleap.nn.training:Auto-selected GPU 0 with 22982 MiB of free memory.
INFO:sleap.nn.training:Using GPU 0 for acceleration.
INFO:sleap.nn.training:Disabled GPU memory pre-allocation.
INFO:sleap.nn.training:System:
GPUs: 1/1 available
  Device: /physical_device:GPU:0
         Available: True
       Initialized: False
     Memory growth: True
INFO:sleap.nn.training:
INFO:sleap.nn.training:Initializing trainer...
INFO:sleap.nn.training:Loading training labels from: D:/social-leap-estimates-animal-poses/datasets/drosophila-melanogaster-courtship/drosophila-melanogaster-courtship/labels.v001.slp
INFO:sleap.nn.training:Creating training and validation splits from validation fraction: 0.1
INFO:sleap.nn.training:  Splits: Training = 2 / Validation = 1.
INFO:sleap.nn.training:Setting up for training...
INFO:sleap.nn.training:Setting up pipeline builders...
INFO:sleap.nn.training:Setting up model...
INFO:sleap.nn.training:Building test pipeline...
2024-10-09 11:43:37.689755: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX AVX2
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2024-10-09 11:43:38.141710: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1532] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 21631 MB memory:  -> device: 0, name: NVIDIA RTX A5000, pci bus id: 0000:01:00.0, compute capability: 8.6
INFO:sleap.nn.training:Loaded test example. [1.944s]
INFO:sleap.nn.training:  Input shape: (1024, 1024, 1)
INFO:sleap.nn.training:Created Keras model.
INFO:sleap.nn.training:  Backbone: UNet(stacks=1, filters=24, filters_rate=1.5, kernel_size=3, stem_kernel_size=7, convs_per_block=2, stem_blocks=0, down_blocks=5, middle_block=True, up_blocks=4, up_interpolate=True, block_contraction=False)
INFO:sleap.nn.training:  Max stride: 32
INFO:sleap.nn.training:  Parameters: 1,685,625
INFO:sleap.nn.training:  Heads:
INFO:sleap.nn.training:    [0] = CentroidConfmapsHead(anchor_part='abdomen', sigma=2.5, output_stride=2, loss_weight=1.0)
INFO:sleap.nn.training:  Outputs:
INFO:sleap.nn.training:    [0] = KerasTensor(type_spec=TensorSpec(shape=(None, 512, 512, 1), dtype=tf.float32, name=None), name='CentroidConfmapsHead/BiasAdd:0', description="created by layer 'CentroidConfmapsHead'")
INFO:sleap.nn.training:Training from scratch
INFO:sleap.nn.training:Setting up data pipelines...
INFO:sleap.nn.training:Training set: n = 2
INFO:sleap.nn.training:Validation set: n = 1
INFO:sleap.nn.training:Setting up optimization...
INFO:sleap.nn.training:  Learning rate schedule: LearningRateScheduleConfig(reduce_on_plateau=True, reduction_factor=0.5, plateau_min_delta=1e-06, plateau_patience=5, plateau_cooldown=3, min_learning_rate=1e-08)
INFO:sleap.nn.training:  Early stopping: EarlyStoppingConfig(stop_training_on_plateau=True, plateau_min_delta=1e-08, plateau_patience=20)
INFO:sleap.nn.training:Setting up outputs...
INFO:sleap.nn.callbacks:Training controller subscribed to: tcp://127.0.0.1:8998 (topic: )
INFO:sleap.nn.training:  ZMQ controller subcribed to: tcp://127.0.0.1:8998
INFO:sleap.nn.callbacks:Progress reporter publishing on: tcp://127.0.0.1:9001 for: not_set
INFO:sleap.nn.training:  ZMQ progress reporter publish on: tcp://127.0.0.1:9001
INFO:sleap.nn.training:Created run path: D:/social-leap-estimates-animal-poses/datasets/drosophila-melanogaster-courtship/drosophila-melanogaster-courtship\models\241009_114324.centroid.n=3
INFO:sleap.nn.training:Setting up visualization...
2024-10-09 11:43:41.557756: W tensorflow/core/grappler/costs/op_level_cost_estimator.cc:690] Error in PredictCost() for the op: op: "CropAndResize" attr { key: "T" value { type: DT_FLOAT } } attr { key: "extrapolation_value" value { f: 0 } } attr { key: "method" value { s: "bilinear" } } inputs { dtype: DT_FLOAT shape { dim { size: -34 } dim { size: -35 } dim { size: -36 } dim { size: 1 } } } inputs { dtype: DT_FLOAT shape { dim { size: -2 } dim { size: 4 } } } inputs { dtype: DT_INT32 shape { dim { size: -2 } } } inputs { dtype: DT_INT32 shape { dim { size: 2 } } } device { type: "GPU" vendor: "NVIDIA" model: "NVIDIA RTX A5000" frequency: 1695 num_cores: 64 environment { key: "architecture" value: "8.6" } environment { key: "cuda" value: "11020" } environment { key: "cudnn" value: "8100" } num_registers: 65536 l1_cache_size: 24576 l2_cache_size: 6291456 shared_memory_size_per_multiprocessor: 102400 memory_size: 22681944064 bandwidth: 768096000 } outputs { dtype: DT_FLOAT shape { dim { size: -2 } dim { size: -37 } dim { size: -38 } dim { size: 1 } } }
2024-10-09 11:43:42.437965: W tensorflow/core/grappler/costs/op_level_cost_estimator.cc:690] Error in PredictCost() for the op: op: "CropAndResize" attr { key: "T" value { type: DT_FLOAT } } attr { key: "extrapolation_value" value { f: 0 } } attr { key: "method" value { s: "bilinear" } } inputs { dtype: DT_FLOAT shape { dim { size: -34 } dim { size: -35 } dim { size: -36 } dim { size: 1 } } } inputs { dtype: DT_FLOAT shape { dim { size: -2 } dim { size: 4 } } } inputs { dtype: DT_INT32 shape { dim { size: -2 } } } inputs { dtype: DT_INT32 shape { dim { size: 2 } } } device { type: "GPU" vendor: "NVIDIA" model: "NVIDIA RTX A5000" frequency: 1695 num_cores: 64 environment { key: "architecture" value: "8.6" } environment { key: "cuda" value: "11020" } environment { key: "cudnn" value: "8100" } num_registers: 65536 l1_cache_size: 24576 l2_cache_size: 6291456 shared_memory_size_per_multiprocessor: 102400 memory_size: 22681944064 bandwidth: 768096000 } outputs { dtype: DT_FLOAT shape { dim { size: -2 } dim { size: -37 } dim { size: -38 } dim { size: 1 } } }
INFO:sleap.nn.training:Finished trainer set up. [4.8s]
INFO:sleap.nn.training:Creating tf.data.Datasets for training data generation...
INFO:sleap.nn.training:Finished creating training datasets. [1.7s]
INFO:sleap.nn.training:Starting training loop...
Epoch 1/2
2024-10-09 11:43:47.494736: I tensorflow/stream_executor/cuda/cuda_dnn.cc:384] Loaded cuDNN version 8201
WARNING:tensorflow:Callback method `on_train_batch_end` is slow compared to the batch time (batch time: 0.0106s vs `on_train_batch_end` time: 0.1389s). Check your callbacks.
2024-10-09 11:44:42.094369: W tensorflow/core/common_runtime/bfc_allocator.cc:290] Allocator (GPU_0_bfc) ran out of memory trying to allocate 3.28GiB with freed_by_count=0. The caller indicates that this is not a failure, but this may mean that there could be performance gains if more memory were available.
2024-10-09 11:44:42.095361: W tensorflow/core/common_runtime/bfc_allocator.cc:290] Allocator (GPU_0_bfc) ran out of memory trying to allocate 3.28GiB with freed_by_count=0. The caller indicates that this is not a failure, but this may mean that there could be performance gains if more memory were available.
2024-10-09 11:44:42.096274: W tensorflow/core/common_runtime/bfc_allocator.cc:290] Allocator (GPU_0_bfc) ran out of memory trying to allocate 3.28GiB with freed_by_count=0. The caller indicates that this is not a failure, but this may mean that there could be performance gains if more memory were available.
2024-10-09 11:44:42.097218: W tensorflow/core/common_runtime/bfc_allocator.cc:290] Allocator (GPU_0_bfc) ran out of memory trying to allocate 3.28GiB with freed_by_count=0. The caller indicates that this is not a failure, but this may mean that there could be performance gains if more memory were available.
2024-10-09 11:44:45.250313: I tensorflow/stream_executor/cuda/cuda_blas.cc:1786] TensorFloat-32 will be used for the matrix multiplication. This will only be logged once.
200/200 - 62s - loss: 1.3333e-04 - val_loss: 6.3918e-05 - lr: 1.0000e-04 - 62s/epoch - 312ms/step
Epoch 2/2
Polling: D:/social-leap-estimates-animal-poses/datasets/drosophila-melanogaster-courtship/drosophila-melanogaster-courtship\models\241009_114324.centroid.n=3\viz\validation.*.png
200/200 - 40s - loss: 4.7547e-05 - val_loss: 7.7009e-05 - lr: 1.0000e-04 - 40s/epoch - 202ms/step
Polling: D:/social-leap-estimates-animal-poses/datasets/drosophila-melanogaster-courtship/drosophila-melanogaster-courtship\models\241009_114324.centroid.n=3\viz\validation.*.png
INFO:sleap.nn.training:Finished training loop. [1.7 min]
INFO:sleap.nn.training:Deleting visualization directory: D:/social-leap-estimates-animal-poses/datasets/drosophila-melanogaster-courtship/drosophila-melanogaster-courtship\models\241009_114324.centroid.n=3\viz
INFO:sleap.nn.training:Saving evaluation metrics to model folder...
Predicting... ----------------------------------------   0% ETA: -:--:-- ?2024-10-09 11:45:30.200022: W tensorflow/core/grappler/costs/op_level_cost_estimator.cc:690] Error in PredictCost() for the op: op: "CropAndResize" attr { key: "T" value { type: DT_FLOAT } } attr { key: "extrapolation_value" value { f: 0 } } attr { key: "method" value { s: "bilinear" } } inputs { dtype: DT_FLOAT shape { dim { size: -65 } dim { size: -66 } dim { size: -67 } dim { size: 1 } } } inputs { dtype: DT_FLOAT shape { dim { size: -2 } dim { size: 4 } } } inputs { dtype: DT_INT32 shape { dim { size: -2 } } } inputs { dtype: DT_INT32 shape { dim { size: 2 } } } device { type: "GPU" vendor: "NVIDIA" model: "NVIDIA RTX A5000" frequency: 1695 num_cores: 64 environment { key: "architecture" value: "8.6" } environment { key: "cuda" value: "11020" } environment { key: "cudnn" value: "8100" } num_registers: 65536 l1_cache_size: 24576 l2_cache_size: 6291456 shared_memory_size_per_multiprocessor: 102400 memory_size: 22681944064 bandwidth: 768096000 } outputs { dtype: DT_FLOAT shape { dim { size: -2 } dim { size: -68 } dim { size: -69 } dim { size: 1 } } }
2024-10-09 11:45:30.201453: W tensorflow/core/grappler/costs/op_level_cost_estimator.cc:690] Error in PredictCost() for the op: op: "CropAndResize" attr { key: "T" value { type: DT_UINT8 } } attr { key: "extrapolation_value" value { f: 0 } } attr { key: "method" value { s: "bilinear" } } inputs { dtype: DT_UINT8 shape { dim { size: 2 } dim { size: 1024 } dim { size: 1024 } dim { size: 1 } } } inputs { dtype: DT_FLOAT shape { dim { size: -2 } dim { size: 4 } } } inputs { dtype: DT_INT32 shape { dim { size: -2 } } } inputs { dtype: DT_INT32 shape { dim { size: 2 } } } device { type: "CPU" vendor: "GenuineIntel" model: "103" frequency: 3600 num_cores: 16 environment { key: "cpu_instruction_set" value: "SSE, SSE2" } environment { key: "eigen" value: "3.4.90" } l1_cache_size: 49152 l2_cache_size: 524288 l3_cache_size: 16777216 memory_size: 268435456 } outputs { dtype: DT_FLOAT shape { dim { size: -2 } dim { size: -77 } dim { size: -78 } dim { size: 1 } } }
Predicting... ---------------------------------------- 100% ETA: 0:00:00 ?
C:\Users\TalmoLab\mambaforge\envs\sleap_1.4.1a3_py310\lib\site-packages\sleap\nn\evals.py:539: RuntimeWarning: Mean of empty slice
  "dist.avg": np.nanmean(dists),
C:\Users\TalmoLab\mambaforge\envs\sleap_1.4.1a3_py310\lib\site-packages\sleap\nn\evals.py:572: RuntimeWarning: Mean of empty slice.
  mPCK = mPCK_parts.mean()
C:\Users\TalmoLab\mambaforge\envs\sleap_1.4.1a3_py310\lib\site-packages\numpy\core\_methods.py:129: RuntimeWarning: invalid value encountered in scalar divide
  ret = ret.dtype.type(ret / rcount)
C:\Users\TalmoLab\mambaforge\envs\sleap_1.4.1a3_py310\lib\site-packages\sleap\nn\evals.py:666: RuntimeWarning: Mean of empty slice.
  pair_pck = metrics["pck.pcks"].mean(axis=-1).mean(axis=-1)
C:\Users\TalmoLab\mambaforge\envs\sleap_1.4.1a3_py310\lib\site-packages\numpy\core\_methods.py:121: RuntimeWarning: invalid value encountered in divide

  ret = um.true_divide(
C:\Users\TalmoLab\mambaforge\envs\sleap_1.4.1a3_py310\lib\site-packages\sleap\nn\evals.py:668: RuntimeWarning: Mean of empty slice.
  metrics["oks.mOKS"] = pair_oks.mean()
WARNING:sleap.nn.evals:Failed to compute metrics.
INFO:sleap.nn.evals:Saved predictions: D:/social-leap-estimates-animal-poses/datasets/drosophila-melanogaster-courtship/drosophila-melanogaster-courtship\models\241009_114324.centroid.n=3\labels_pr.train.slp
Predicting... ----------------------------------------   0% ETA: -:--:-- ?2024-10-09 11:45:33.354849: W tensorflow/core/grappler/costs/op_level_cost_estimator.cc:690] Error in PredictCost() for the op: op: "CropAndResize" attr { key: "T" value { type: DT_FLOAT } } attr { key: "extrapolation_value" value { f: 0 } } attr { key: "method" value { s: "bilinear" } } inputs { dtype: DT_FLOAT shape { dim { size: -65 } dim { size: -66 } dim { size: -67 } dim { size: 1 } } } inputs { dtype: DT_FLOAT shape { dim { size: -2 } dim { size: 4 } } } inputs { dtype: DT_INT32 shape { dim { size: -2 } } } inputs { dtype: DT_INT32 shape { dim { size: 2 } } } device { type: "GPU" vendor: "NVIDIA" model: "NVIDIA RTX A5000" frequency: 1695 num_cores: 64 environment { key: "architecture" value: "8.6" } environment { key: "cuda" value: "11020" } environment { key: "cudnn" value: "8100" } num_registers: 65536 l1_cache_size: 24576 l2_cache_size: 6291456 shared_memory_size_per_multiprocessor: 102400 memory_size: 22681944064 bandwidth: 768096000 } outputs { dtype: DT_FLOAT shape { dim { size: -2 } dim { size: -68 } dim { size: -69 } dim { size: 1 } } }
2024-10-09 11:45:33.356694: W tensorflow/core/grappler/costs/op_level_cost_estimator.cc:690] Error in PredictCost() for the op: op: "CropAndResize" attr { key: "T" value { type: DT_UINT8 } } attr { key: "extrapolation_value" value { f: 0 } } attr { key: "method" value { s: "bilinear" } } inputs { dtype: DT_UINT8 shape { dim { size: 1 } dim { size: 1024 } dim { size: 1024 } dim { size: 1 } } } inputs { dtype: DT_FLOAT shape { dim { size: -2 } dim { size: 4 } } } inputs { dtype: DT_INT32 shape { dim { size: -2 } } } inputs { dtype: DT_INT32 shape { dim { size: 2 } } } device { type: "CPU" vendor: "GenuineIntel" model: "103" frequency: 3600 num_cores: 16 environment { key: "cpu_instruction_set" value: "SSE, SSE2" } environment { key: "eigen" value: "3.4.90" } l1_cache_size: 49152 l2_cache_size: 524288 l3_cache_size: 16777216 memory_size: 268435456 } outputs { dtype: DT_FLOAT shape { dim { size: -2 } dim { size: -77 } dim { size: -78 } dim { size: 1 } } }
Predicting... ---------------------------------------- 100% ETA: 0:00:00 ?
WARNING:sleap.nn.evals:Failed to compute metrics.
INFO:sleap.nn.evals:Saved predictions: D:/social-leap-estimates-animal-poses/datasets/drosophila-melanogaster-courtship/drosophila-melanogaster-courtship\models\241009_114324.centroid.n=3\labels_pr.val.slp
INFO:sleap.nn.callbacks:Closing the reporter controller/context.
INFO:sleap.nn.callbacks:Closing the training controller socket/context.
Run Path: D:/social-leap-estimates-animal-poses/datasets/drosophila-melanogaster-courtship/drosophila-melanogaster-courtship\models\241009_114324.centroid.n=3
Finished training centroid.
Resetting monitor window.
Polling: D:/social-leap-estimates-animal-poses/datasets/drosophila-melanogaster-courtship/drosophila-melanogaster-courtship\models\241009_114536.centered_instance.n=3\viz\validation.*.png
Start training centered_instance...
['sleap-train', 'C:\\Users\\TalmoLab\\AppData\\Local\\Temp\\tmp1y6bl0lp\\241009_114536_training_job.json', 'D:/social-leap-estimates-animal-poses/datasets/drosophila-melanogaster-courtship/drosophila-melanogaster-courtship/labels.v001.slp', '--zmq', '--controller_port', '8998', '--publish_port', '9001', '--save_viz']
C:\Users\TalmoLab\mambaforge\envs\sleap_1.4.1a3_py310\lib\site-packages\albumentations\__init__.py:13: UserWarning: A new version of Albumentations is available: 1.4.18 (you have 1.4.15). Upgrade using: pip install -U albumentations. To disable automatic update checks, set the environment variable NO_ALBUMENTATIONS_UPDATE to 1.
  check_for_updates()
INFO:sleap.nn.training:Versions:
SLEAP: 1.4.1a3
TensorFlow: 2.9.2
Numpy: 1.26.4
Python: 3.10.15
OS: Windows-10-10.0.19044-SP0
INFO:sleap.nn.training:Training labels file: D:/social-leap-estimates-animal-poses/datasets/drosophila-melanogaster-courtship/drosophila-melanogaster-courtship/labels.v001.slp
INFO:sleap.nn.training:Training profile: C:\Users\TalmoLab\AppData\Local\Temp\tmp1y6bl0lp\241009_114536_training_job.json
INFO:sleap.nn.training:
INFO:sleap.nn.training:Arguments:
INFO:sleap.nn.training:{
    "training_job_path": "C:\\Users\\TalmoLab\\AppData\\Local\\Temp\\tmp1y6bl0lp\\241009_114536_training_job.json",
    "labels_path": "D:/social-leap-estimates-animal-poses/datasets/drosophila-melanogaster-courtship/drosophila-melanogaster-courtship/labels.v001.slp",
    "video_paths": [
        ""
    ],
    "val_labels": null,
    "test_labels": null,
    "base_checkpoint": null,
    "tensorboard": false,
    "save_viz": true,
    "keep_viz": false,
    "zmq": true,
    "publish_port": 9001,
    "controller_port": 8998,
    "run_name": "",
    "prefix": "",
    "suffix": "",
    "cpu": false,
    "first_gpu": false,
    "last_gpu": false,
    "gpu": "auto"
}
INFO:sleap.nn.training:
INFO:sleap.nn.training:Training job:
INFO:sleap.nn.training:{
    "data": {
        "labels": {
            "training_labels": "D:\\social-leap-estimates-animal-poses\\datasets\\drosophila-melanogaster-courtship\\drosophila-melanogaster-courtship\\courtship_labels.slp",
            "validation_labels": null,
            "validation_fraction": 0.1,
            "test_labels": null,
            "split_by_inds": false,
            "training_inds": [
                38,
                35,
                63,
                11,
                13,
                42,
                78,
                25,
                61,
                57,
                79,
                24,
                85,
                12,
                89,
                64,
                18,
                96,
                32,
                72,
                26,
                20,
                46,
                68,
                84,
                6,
                59,
                73,
                17,
                75,
                29,
                66,
                56,
                7,
                9,
                77,
                31,
                41,
                80,
                94,
                76,
                27,
                15,
                60,
                39,
                45,
                49,
                69,
                92,
                65,
                55,
                34,
                48,
                16,
                33,
                14,
                2,
                8,
                44,
                28,
                47,
                21,
                54,
                87,
                3,
                37,
                99,
                98,
                58,
                4,
                10,
                0,
                95,
                91,
                50,
                22,
                67,
                74,
                40,
                82,
                62,
                19,
                86,
                36,
                88,
                51,
                30,
                71,
                23,
                83,
                52
            ],
            "validation_inds": [
                53,
                43,
                100,
                90,
                1,
                70,
                97,
                5,
                93,
                81
            ],
            "test_inds": null,
            "search_path_hints": [
                "",
                "",
                "",
                "",
                "",
                "",
                "",
                "",
                "",
                "",
                "",
                "",
                "",
                "",
                "",
                "",
                "",
                "",
                ""
            ],
            "skeletons": []
        },
        "preprocessing": {
            "ensure_rgb": false,
            "ensure_grayscale": false,
            "imagenet_mode": null,
            "input_scaling": 1.0,
            "pad_to_stride": 1,
            "resize_and_pad_to_target": true,
            "target_height": 1280,
            "target_width": 1280
        },
        "instance_cropping": {
            "center_on_part": "abdomen",
            "crop_size": 256,
            "crop_size_detection_padding": 16
        }
    },
    "model": {
        "backbone": {
            "leap": null,
            "unet": {
                "stem_stride": null,
                "max_stride": 128,
                "output_stride": 4,
                "filters": 24,
                "filters_rate": 2.0,
                "middle_block": true,
                "up_interpolate": true,
                "stacks": 1
            },
            "hourglass": null,
            "resnet": null,
            "pretrained_encoder": null
        },
        "heads": {
            "single_instance": null,
            "centroid": null,
            "centered_instance": {
                "anchor_part": "abdomen",
                "part_names": [
                    "head",
                    "thorax",
                    "abdomen",
                    "wingL",
                    "wingR",
                    "forelegL4",
                    "forelegR4",
                    "midlegL4",
                    "midlegR4",
                    "hindlegL4",
                    "hindlegR4",
                    "eyeL",
                    "eyeR"
                ],
                "sigma": 2.5,
                "output_stride": 4,
                "loss_weight": 1.0,
                "offset_refinement": false
            },
            "multi_instance": null,
            "multi_class_bottomup": null,
            "multi_class_topdown": null
        },
        "base_checkpoint": null
    },
    "optimization": {
        "preload_data": true,
        "augmentation_config": {
            "rotate": true,
            "rotation_min_angle": -180.0,
            "rotation_max_angle": 180.0,
            "translate": false,
            "translate_min": -5,
            "translate_max": 5,
            "scale": false,
            "scale_min": 0.9,
            "scale_max": 1.1,
            "uniform_noise": false,
            "uniform_noise_min_val": 0.0,
            "uniform_noise_max_val": 10.0,
            "gaussian_noise": false,
            "gaussian_noise_mean": 5.0,
            "gaussian_noise_stddev": 1.0,
            "contrast": false,
            "contrast_min_gamma": 0.5,
            "contrast_max_gamma": 2.0,
            "brightness": false,
            "brightness_min_val": 0.0,
            "brightness_max_val": 10.0,
            "random_crop": false,
            "random_crop_height": 256,
            "random_crop_width": 256,
            "random_flip": false,
            "flip_horizontal": false
        },
        "online_shuffling": true,
        "shuffle_buffer_size": 128,
        "prefetch": true,
        "batch_size": 4,
        "batches_per_epoch": 200,
        "min_batches_per_epoch": 200,
        "val_batches_per_epoch": 10,
        "min_val_batches_per_epoch": 10,
        "epochs": 2,
        "optimizer": "adam",
        "initial_learning_rate": 0.0001,
        "learning_rate_schedule": {
            "reduce_on_plateau": true,
            "reduction_factor": 0.5,
            "plateau_min_delta": 1e-06,
            "plateau_patience": 5,
            "plateau_cooldown": 3,
            "min_learning_rate": 1e-08
        },
        "hard_keypoint_mining": {
            "online_mining": false,
            "hard_to_easy_ratio": 2.0,
            "min_hard_keypoints": 2,
            "max_hard_keypoints": null,
            "loss_scale": 5.0
        },
        "early_stopping": {
            "stop_training_on_plateau": true,
            "plateau_min_delta": 1e-08,
            "plateau_patience": 10
        }
    },
    "outputs": {
        "save_outputs": true,
        "run_name": "241009_114536.centered_instance.n=3",
        "run_name_prefix": "",
        "run_name_suffix": "",
        "runs_folder": "D:/social-leap-estimates-animal-poses/datasets/drosophila-melanogaster-courtship/drosophila-melanogaster-courtship\\models",
        "tags": [
            ""
        ],
        "save_visualizations": true,
        "keep_viz_images": false,
        "zip_outputs": false,
        "log_to_csv": true,
        "checkpointing": {
            "initial_model": false,
            "best_model": true,
            "every_epoch": false,
            "latest_model": false,
            "final_model": false
        },
        "tensorboard": {
            "write_logs": false,
            "loss_frequency": "epoch",
            "architecture_graph": false,
            "profile_graph": false,
            "visualizations": true
        },
        "zmq": {
            "subscribe_to_controller": true,
            "controller_address": "tcp://127.0.0.1:8998",
            "controller_polling_timeout": 10,
            "publish_updates": true,
            "publish_address": "tcp://127.0.0.1:9001"
        }
    },
    "name": "",
    "description": "",
    "sleap_version": "1.4.1a2",
    "filename": "C:\\Users\\TalmoLab\\AppData\\Local\\Temp\\tmp1y6bl0lp\\241009_114536_training_job.json"
}
INFO:sleap.nn.training:
INFO:sleap.nn.training:Auto-selected GPU 0 with 23459 MiB of free memory.
INFO:sleap.nn.training:Using GPU 0 for acceleration.
INFO:sleap.nn.training:Disabled GPU memory pre-allocation.
INFO:sleap.nn.training:System:
GPUs: 1/1 available
  Device: /physical_device:GPU:0
         Available: True
       Initialized: False
     Memory growth: True
INFO:sleap.nn.training:
INFO:sleap.nn.training:Initializing trainer...
INFO:sleap.nn.training:Loading training labels from: D:/social-leap-estimates-animal-poses/datasets/drosophila-melanogaster-courtship/drosophila-melanogaster-courtship/labels.v001.slp
INFO:sleap.nn.training:Creating training and validation splits from validation fraction: 0.1
INFO:sleap.nn.training:  Splits: Training = 2 / Validation = 1.
INFO:sleap.nn.training:Setting up for training...
INFO:sleap.nn.training:Setting up pipeline builders...
INFO:sleap.nn.training:Setting up model...
INFO:sleap.nn.training:Building test pipeline...
2024-10-09 11:45:42.918245: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX AVX2
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2024-10-09 11:45:43.267306: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1532] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 21631 MB memory:  -> device: 0, name: NVIDIA RTX A5000, pci bus id: 0000:01:00.0, compute capability: 8.6
2024-10-09 11:45:45.138164: W tensorflow/core/grappler/costs/op_level_cost_estimator.cc:690] Error in PredictCost() for the op: op: "CropAndResize" attr { key: "T" value { type: DT_FLOAT } } attr { key: "extrapolation_value" value { f: 0 } } attr { key: "method" value { s: "bilinear" } } inputs { dtype: DT_FLOAT shape { dim { size: 1 } dim { size: 1280 } dim { size: 1280 } dim { size: 1 } } } inputs { dtype: DT_FLOAT shape { dim { size: -2 } dim { size: 4 } } } inputs { dtype: DT_INT32 shape { dim { size: -2 } } } inputs { dtype: DT_INT32 shape { dim { size: 2 } } } device { type: "CPU" vendor: "GenuineIntel" model: "103" frequency: 3600 num_cores: 16 environment { key: "cpu_instruction_set" value: "SSE, SSE2" } environment { key: "eigen" value: "3.4.90" } l1_cache_size: 49152 l2_cache_size: 524288 l3_cache_size: 16777216 memory_size: 268435456 } outputs { dtype: DT_FLOAT shape { dim { size: -2 } dim { size: 256 } dim { size: 256 } dim { size: 1 } } }
INFO:sleap.nn.training:Loaded test example. [2.358s]
INFO:sleap.nn.training:  Input shape: (256, 256, 1)
INFO:sleap.nn.training:Created Keras model.
INFO:sleap.nn.training:  Backbone: UNet(stacks=1, filters=24, filters_rate=2.0, kernel_size=3, stem_kernel_size=7, convs_per_block=2, stem_blocks=0, down_blocks=7, middle_block=True, up_blocks=5, up_interpolate=True, block_contraction=False)
INFO:sleap.nn.training:  Max stride: 128
INFO:sleap.nn.training:  Parameters: 283,019,413
INFO:sleap.nn.training:  Heads:
INFO:sleap.nn.training:    [0] = CenteredInstanceConfmapsHead(part_names=['head', 'thorax', 'abdomen', 'wingL', 'wingR', 'forelegL4', 'forelegR4', 'midlegL4', 'midlegR4', 'hindlegL4', 'hindlegR4', 'eyeL', 'eyeR'], anchor_part='abdomen', sigma=2.5, output_stride=4, loss_weight=1.0)
INFO:sleap.nn.training:  Outputs:
INFO:sleap.nn.training:    [0] = KerasTensor(type_spec=TensorSpec(shape=(None, 64, 64, 13), dtype=tf.float32, name=None), name='CenteredInstanceConfmapsHead/BiasAdd:0', description="created by layer 'CenteredInstanceConfmapsHead'")
INFO:sleap.nn.training:Training from scratch
INFO:sleap.nn.training:Setting up data pipelines...
INFO:sleap.nn.training:Training set: n = 2
INFO:sleap.nn.training:Validation set: n = 1
INFO:sleap.nn.training:Setting up optimization...
INFO:sleap.nn.training:  Learning rate schedule: LearningRateScheduleConfig(reduce_on_plateau=True, reduction_factor=0.5, plateau_min_delta=1e-06, plateau_patience=5, plateau_cooldown=3, min_learning_rate=1e-08)
INFO:sleap.nn.training:  Early stopping: EarlyStoppingConfig(stop_training_on_plateau=True, plateau_min_delta=1e-08, plateau_patience=10)
INFO:sleap.nn.training:Setting up outputs...
INFO:sleap.nn.callbacks:Training controller subscribed to: tcp://127.0.0.1:8998 (topic: )
INFO:sleap.nn.training:  ZMQ controller subcribed to: tcp://127.0.0.1:8998
INFO:sleap.nn.callbacks:Progress reporter publishing on: tcp://127.0.0.1:9001 for: not_set
INFO:sleap.nn.training:  ZMQ progress reporter publish on: tcp://127.0.0.1:9001
INFO:sleap.nn.training:Created run path: D:/social-leap-estimates-animal-poses/datasets/drosophila-melanogaster-courtship/drosophila-melanogaster-courtship\models\241009_114536.centered_instance.n=3
INFO:sleap.nn.training:Setting up visualization...
2024-10-09 11:45:46.311081: W tensorflow/core/grappler/costs/op_level_cost_estimator.cc:690] Error in PredictCost() for the op: op: "CropAndResize" attr { key: "T" value { type: DT_FLOAT } } attr { key: "extrapolation_value" value { f: 0 } } attr { key: "method" value { s: "bilinear" } } inputs { dtype: DT_FLOAT shape { dim { size: 1 } dim { size: 1280 } dim { size: 1280 } dim { size: 1 } } } inputs { dtype: DT_FLOAT shape { dim { size: -2 } dim { size: 4 } } } inputs { dtype: DT_INT32 shape { dim { size: -2 } } } inputs { dtype: DT_INT32 shape { dim { size: 2 } } } device { type: "CPU" vendor: "GenuineIntel" model: "103" frequency: 3600 num_cores: 16 environment { key: "cpu_instruction_set" value: "SSE, SSE2" } environment { key: "eigen" value: "3.4.90" } l1_cache_size: 49152 l2_cache_size: 524288 l3_cache_size: 16777216 memory_size: 268435456 } outputs { dtype: DT_FLOAT shape { dim { size: -2 } dim { size: 256 } dim { size: 256 } dim { size: 1 } } }
2024-10-09 11:45:47.120562: W tensorflow/core/grappler/costs/op_level_cost_estimator.cc:690] Error in PredictCost() for the op: op: "CropAndResize" attr { key: "T" value { type: DT_FLOAT } } attr { key: "extrapolation_value" value { f: 0 } } attr { key: "method" value { s: "bilinear" } } inputs { dtype: DT_FLOAT shape { dim { size: 1 } dim { size: 1280 } dim { size: 1280 } dim { size: 1 } } } inputs { dtype: DT_FLOAT shape { dim { size: -2 } dim { size: 4 } } } inputs { dtype: DT_INT32 shape { dim { size: -2 } } } inputs { dtype: DT_INT32 shape { dim { size: 2 } } } device { type: "CPU" vendor: "GenuineIntel" model: "103" frequency: 3600 num_cores: 16 environment { key: "cpu_instruction_set" value: "SSE, SSE2" } environment { key: "eigen" value: "3.4.90" } l1_cache_size: 49152 l2_cache_size: 524288 l3_cache_size: 16777216 memory_size: 268435456 } outputs { dtype: DT_FLOAT shape { dim { size: -2 } dim { size: 256 } dim { size: 256 } dim { size: 1 } } }
INFO:sleap.nn.training:Finished trainer set up. [4.3s]
INFO:sleap.nn.training:Creating tf.data.Datasets for training data generation...
2024-10-09 11:45:48.033762: W tensorflow/core/grappler/costs/op_level_cost_estimator.cc:690] Error in PredictCost() for the op: op: "CropAndResize" attr { key: "T" value { type: DT_FLOAT } } attr { key: "extrapolation_value" value { f: 0 } } attr { key: "method" value { s: "bilinear" } } inputs { dtype: DT_FLOAT shape { dim { size: 1 } dim { size: 1280 } dim { size: 1280 } dim { size: 1 } } } inputs { dtype: DT_FLOAT shape { dim { size: -2 } dim { size: 4 } } } inputs { dtype: DT_INT32 shape { dim { size: -2 } } } inputs { dtype: DT_INT32 shape { dim { size: 2 } } } device { type: "CPU" vendor: "GenuineIntel" model: "103" frequency: 3600 num_cores: 16 environment { key: "cpu_instruction_set" value: "SSE, SSE2" } environment { key: "eigen" value: "3.4.90" } l1_cache_size: 49152 l2_cache_size: 524288 l3_cache_size: 16777216 memory_size: 268435456 } outputs { dtype: DT_FLOAT shape { dim { size: -2 } dim { size: 256 } dim { size: 256 } dim { size: 1 } } }
2024-10-09 11:45:49.597663: W tensorflow/core/grappler/costs/op_level_cost_estimator.cc:690] Error in PredictCost() for the op: op: "CropAndResize" attr { key: "T" value { type: DT_FLOAT } } attr { key: "extrapolation_value" value { f: 0 } } attr { key: "method" value { s: "bilinear" } } inputs { dtype: DT_FLOAT shape { dim { size: 1 } dim { size: 1280 } dim { size: 1280 } dim { size: 1 } } } inputs { dtype: DT_FLOAT shape { dim { size: -2 } dim { size: 4 } } } inputs { dtype: DT_INT32 shape { dim { size: -2 } } } inputs { dtype: DT_INT32 shape { dim { size: 2 } } } device { type: "CPU" vendor: "GenuineIntel" model: "103" frequency: 3600 num_cores: 16 environment { key: "cpu_instruction_set" value: "SSE, SSE2" } environment { key: "eigen" value: "3.4.90" } l1_cache_size: 49152 l2_cache_size: 524288 l3_cache_size: 16777216 memory_size: 268435456 } outputs { dtype: DT_FLOAT shape { dim { size: -2 } dim { size: 256 } dim { size: 256 } dim { size: 1 } } }
INFO:sleap.nn.training:Finished creating training datasets. [2.7s]
INFO:sleap.nn.training:Starting training loop...
2024-10-09 11:45:50.090577: W tensorflow/core/grappler/costs/op_level_cost_estimator.cc:690] Error in PredictCost() for the op: op: "CropAndResize" attr { key: "T" value { type: DT_FLOAT } } attr { key: "extrapolation_value" value { f: 0 } } attr { key: "method" value { s: "bilinear" } } inputs { dtype: DT_FLOAT shape { dim { size: 1 } dim { size: 1280 } dim { size: 1280 } dim { size: 1 } } } inputs { dtype: DT_FLOAT shape { dim { size: -2 } dim { size: 4 } } } inputs { dtype: DT_INT32 shape { dim { size: -2 } } } inputs { dtype: DT_INT32 shape { dim { size: 2 } } } device { type: "CPU" vendor: "GenuineIntel" model: "103" frequency: 3600 num_cores: 16 environment { key: "cpu_instruction_set" value: "SSE, SSE2" } environment { key: "eigen" value: "3.4.90" } l1_cache_size: 49152 l2_cache_size: 524288 l3_cache_size: 16777216 memory_size: 268435456 } outputs { dtype: DT_FLOAT shape { dim { size: -2 } dim { size: 256 } dim { size: 256 } dim { size: 1 } } }
Epoch 1/2
2024-10-09 11:45:52.328777: I tensorflow/stream_executor/cuda/cuda_dnn.cc:384] Loaded cuDNN version 8201
WARNING:tensorflow:Callback method `on_train_batch_end` is slow compared to the batch time (batch time: 0.0130s vs `on_train_batch_end` time: 0.1220s). Check your callbacks.
2024-10-09 11:46:29.998998: W tensorflow/core/grappler/costs/op_level_cost_estimator.cc:690] Error in PredictCost() for the op: op: "CropAndResize" attr { key: "T" value { type: DT_FLOAT } } attr { key: "extrapolation_value" value { f: 0 } } attr { key: "method" value { s: "bilinear" } } inputs { dtype: DT_FLOAT shape { dim { size: 1 } dim { size: 1280 } dim { size: 1280 } dim { size: 1 } } } inputs { dtype: DT_FLOAT shape { dim { size: -2 } dim { size: 4 } } } inputs { dtype: DT_INT32 shape { dim { size: -2 } } } inputs { dtype: DT_INT32 shape { dim { size: 2 } } } device { type: "CPU" vendor: "GenuineIntel" model: "103" frequency: 3600 num_cores: 16 environment { key: "cpu_instruction_set" value: "SSE, SSE2" } environment { key: "eigen" value: "3.4.90" } l1_cache_size: 49152 l2_cache_size: 524288 l3_cache_size: 16777216 memory_size: 268435456 } outputs { dtype: DT_FLOAT shape { dim { size: -2 } dim { size: 256 } dim { size: 256 } dim { size: 1 } } }
2024-10-09 11:46:31.824231: I tensorflow/stream_executor/cuda/cuda_blas.cc:1786] TensorFloat-32 will be used for the matrix multiplication. This will only be logged once.
2024-10-09 11:46:32.432660: W tensorflow/core/grappler/costs/op_level_cost_estimator.cc:690] Error in PredictCost() for the op: op: "CropAndResize" attr { key: "T" value { type: DT_FLOAT } } attr { key: "extrapolation_value" value { f: 0 } } attr { key: "method" value { s: "bilinear" } } inputs { dtype: DT_FLOAT shape { dim { size: 13 } dim { size: 64 } dim { size: 64 } dim { size: 1 } } } inputs { dtype: DT_FLOAT shape { dim { size: -2 } dim { size: 4 } } } inputs { dtype: DT_INT32 shape { dim { size: -2 } } } inputs { dtype: DT_INT32 shape { dim { size: 2 } } } device { type: "GPU" vendor: "NVIDIA" model: "NVIDIA RTX A5000" frequency: 1695 num_cores: 64 environment { key: "architecture" value: "8.6" } environment { key: "cuda" value: "11020" } environment { key: "cudnn" value: "8100" } num_registers: 65536 l1_cache_size: 24576 l2_cache_size: 6291456 shared_memory_size_per_multiprocessor: 102400 memory_size: 22681944064 bandwidth: 768096000 } outputs { dtype: DT_FLOAT shape { dim { size: -2 } dim { size: -5 } dim { size: -6 } dim { size: 1 } } }
200/200 - 48s - loss: 0.0024 - head: 0.0018 - thorax: 0.0021 - abdomen: 0.0027 - wingL: 0.0030 - wingR: 0.0031 - forelegL4: 0.0014 - forelegR4: 9.9911e-05 - midlegL4: 0.0027 - midlegR4: 0.0035 - hindlegL4: 0.0016 - hindlegR4: 0.0034 - eyeL: 0.0028 - eyeR: 0.0027 - val_loss: 0.0027 - val_head: 0.0036 - val_thorax: 0.0019 - val_abdomen: 0.0013 - val_wingL: 0.0028 - val_wingR: 0.0032 - val_forelegL4: 0.0013 - val_forelegR4: 2.5745e-04 - val_midlegL4: 0.0033 - val_midlegR4: 0.0033 - val_hindlegL4: 0.0023 - val_hindlegR4: 0.0037 - val_eyeL: 0.0039 - val_eyeR: 0.0037 - lr: 1.0000e-04 - 48s/epoch - 240ms/step
Epoch 2/2
Polling: D:/social-leap-estimates-animal-poses/datasets/drosophila-melanogaster-courtship/drosophila-melanogaster-courtship\models\241009_114324.centroid.n=3\viz\validation.*.png
Polling: D:/social-leap-estimates-animal-poses/datasets/drosophila-melanogaster-courtship/drosophila-melanogaster-courtship\models\241009_114536.centered_instance.n=3\viz\validation.*.png
2024-10-09 11:47:12.089059: W tensorflow/core/grappler/costs/op_level_cost_estimator.cc:690] Error in PredictCost() for the op: op: "CropAndResize" attr { key: "T" value { type: DT_FLOAT } } attr { key: "extrapolation_value" value { f: 0 } } attr { key: "method" value { s: "bilinear" } } inputs { dtype: DT_FLOAT shape { dim { size: 1 } dim { size: 1280 } dim { size: 1280 } dim { size: 1 } } } inputs { dtype: DT_FLOAT shape { dim { size: -2 } dim { size: 4 } } } inputs { dtype: DT_INT32 shape { dim { size: -2 } } } inputs { dtype: DT_INT32 shape { dim { size: 2 } } } device { type: "CPU" vendor: "GenuineIntel" model: "103" frequency: 3600 num_cores: 16 environment { key: "cpu_instruction_set" value: "SSE, SSE2" } environment { key: "eigen" value: "3.4.90" } l1_cache_size: 49152 l2_cache_size: 524288 l3_cache_size: 16777216 memory_size: 268435456 } outputs { dtype: DT_FLOAT shape { dim { size: -2 } dim { size: 256 } dim { size: 256 } dim { size: 1 } } }
200/200 - 40s - loss: 8.4480e-04 - head: 6.9070e-04 - thorax: 6.0177e-04 - abdomen: 7.6382e-04 - wingL: 7.1380e-04 - wingR: 7.3655e-04 - forelegL4: 0.0011 - forelegR4: 8.6102e-05 - midlegL4: 0.0012 - midlegR4: 9.9908e-04 - hindlegL4: 0.0013 - hindlegR4: 0.0014 - eyeL: 7.9498e-04 - eyeR: 7.0381e-04 - val_loss: 0.0025 - val_head: 0.0040 - val_thorax: 0.0015 - val_abdomen: 0.0012 - val_wingL: 0.0025 - val_wingR: 0.0025 - val_forelegL4: 0.0012 - val_forelegR4: 9.1834e-05 - val_midlegL4: 0.0028 - val_midlegR4: 0.0033 - val_hindlegL4: 0.0020 - val_hindlegR4: 0.0047 - val_eyeL: 0.0030 - val_eyeR: 0.0030 - lr: 1.0000e-04 - 40s/epoch - 202ms/step
INFO:sleap.nn.training:Finished training loop. [1.5 min]
INFO:sleap.nn.training:Deleting visualization directory: D:/social-leap-estimates-animal-poses/datasets/drosophila-melanogaster-courtship/drosophila-melanogaster-courtship\models\241009_114536.centered_instance.n=3\viz
INFO:sleap.nn.training:Saving evaluation metrics to model folder...
Predicting... ----------------------------------------   0% ETA: -:--:-- ?Polling: D:/social-leap-estimates-animal-poses/datasets/drosophila-melanogaster-courtship/drosophila-melanogaster-courtship\models\241009_114324.centroid.n=3\viz\validation.*.png
Polling: D:/social-leap-estimates-animal-poses/datasets/drosophila-melanogaster-courtship/drosophila-melanogaster-courtship\models\241009_114536.centered_instance.n=3\viz\validation.*.png
2024-10-09 11:47:20.567626: W tensorflow/core/grappler/costs/op_level_cost_estimator.cc:690] Error in PredictCost() for the op: op: "CropAndResize" attr { key: "T" value { type: DT_UINT8 } } attr { key: "extrapolation_value" value { f: 0 } } attr { key: "method" value { s: "bilinear" } } inputs { dtype: DT_UINT8 shape { dim { size: 2 } dim { size: 1280 } dim { size: 1280 } dim { size: 1 } } } inputs { dtype: DT_FLOAT shape { dim { size: -2 } dim { size: 4 } } } inputs { dtype: DT_INT32 shape { dim { size: -2 } } } inputs { dtype: DT_INT32 shape { dim { size: 2 } } } device { type: "CPU" vendor: "GenuineIntel" model: "103" frequency: 3600 num_cores: 16 environment { key: "cpu_instruction_set" value: "SSE, SSE2" } environment { key: "eigen" value: "3.4.90" } l1_cache_size: 49152 l2_cache_size: 524288 l3_cache_size: 16777216 memory_size: 268435456 } outputs { dtype: DT_FLOAT shape { dim { size: -2 } dim { size: -24 } dim { size: -25 } dim { size: 1 } } }
2024-10-09 11:47:20.579069: W tensorflow/core/grappler/costs/op_level_cost_estimator.cc:690] Error in PredictCost() for the op: op: "CropAndResize" attr { key: "T" value { type: DT_FLOAT } } attr { key: "extrapolation_value" value { f: 0 } } attr { key: "method" value { s: "bilinear" } } inputs { dtype: DT_FLOAT shape { dim { size: -65 } dim { size: -66 } dim { size: -67 } dim { size: 1 } } } inputs { dtype: DT_FLOAT shape { dim { size: -9 } dim { size: 4 } } } inputs { dtype: DT_INT32 shape { dim { size: -9 } } } inputs { dtype: DT_INT32 shape { dim { size: 2 } } } device { type: "GPU" vendor: "NVIDIA" model: "NVIDIA RTX A5000" frequency: 1695 num_cores: 64 environment { key: "architecture" value: "8.6" } environment { key: "cuda" value: "11020" } environment { key: "cudnn" value: "8100" } num_registers: 65536 l1_cache_size: 24576 l2_cache_size: 6291456 shared_memory_size_per_multiprocessor: 102400 memory_size: 22681944064 bandwidth: 768096000 } outputs { dtype: DT_FLOAT shape { dim { size: -9 } dim { size: -70 } dim { size: -71 } dim { size: 1 } } }
error: Can't find libdevice directory ${CUDA_DIR}/nvvm/libdevice
error: Can't find libdevice directory ${CUDA_DIR}/nvvm/libdevice
error: Can't find libdevice directory ${CUDA_DIR}/nvvm/libdevice
error: Can't find libdevice directory ${CUDA_DIR}/nvvm/libdevice
error: Can't find libdevice directory ${CUDA_DIR}/nvvm/libdevice
error: Can't find libdevice directory ${CUDA_DIR}/nvvm/libdevice
error: Can't find libdevice directory ${CUDA_DIR}/nvvm/libdevice
error: Can't find libdevice directory ${CUDA_DIR}/nvvm/libdevice
error: Can't find libdevice directory ${CUDA_DIR}/nvvm/libdevice
error: Can't find libdevice directory ${CUDA_DIR}/nvvm/libdevice
error: Can't find libdevice directory ${CUDA_DIR}/nvvm/libdevice
error: Can't find libdevice directory ${CUDA_DIR}/nvvm/libdevice
error: Can't find libdevice directory ${CUDA_DIR}/nvvm/libdevice
2024-10-09 11:47:21.848472: W tensorflow/core/framework/op_kernel.cc:1733] UNKNOWN: JIT compilation failed.
Predicting... ----------------------------------------   0% ETA: -:--:-- ?
Traceback (most recent call last):
  File "\\?\C:\Users\TalmoLab\mambaforge\envs\sleap_1.4.1a3_py310\Scripts\sleap-train-script.py", line 33, in <module>
    sys.exit(load_entry_point('sleap==1.4.1a3', 'console_scripts', 'sleap-train')())
  File "C:\Users\TalmoLab\mambaforge\envs\sleap_1.4.1a3_py310\lib\site-packages\sleap\nn\training.py", line 2039, in main
    trainer.train()
  File "C:\Users\TalmoLab\mambaforge\envs\sleap_1.4.1a3_py310\lib\site-packages\sleap\nn\training.py", line 953, in train
    self.evaluate()
  File "C:\Users\TalmoLab\mambaforge\envs\sleap_1.4.1a3_py310\lib\site-packages\sleap\nn\training.py", line 961, in evaluate
    sleap.nn.evals.evaluate_model(
  File "C:\Users\TalmoLab\mambaforge\envs\sleap_1.4.1a3_py310\lib\site-packages\sleap\nn\evals.py", line 744, in evaluate_model
    labels_pr: Labels = predictor.predict(labels_gt, make_labels=True)
  File "C:\Users\TalmoLab\mambaforge\envs\sleap_1.4.1a3_py310\lib\site-packages\sleap\nn\inference.py", line 527, in predict
    self._make_labeled_frames_from_generator(generator, data)
  File "C:\Users\TalmoLab\mambaforge\envs\sleap_1.4.1a3_py310\lib\site-packages\sleap\nn\inference.py", line 2645, in _make_labeled_frames_from_generator
    for ex in generator:
  File "C:\Users\TalmoLab\mambaforge\envs\sleap_1.4.1a3_py310\lib\site-packages\sleap\nn\inference.py", line 437, in _predict_generator
    ex = process_batch(ex)
  File "C:\Users\TalmoLab\mambaforge\envs\sleap_1.4.1a3_py310\lib\site-packages\sleap\nn\inference.py", line 400, in process_batch
    preds = self.inference_model.predict_on_batch(ex, numpy=True)
  File "C:\Users\TalmoLab\mambaforge\envs\sleap_1.4.1a3_py310\lib\site-packages\sleap\nn\inference.py", line 1070, in predict_on_batch
    outs = super().predict_on_batch(data, **kwargs)
  File "C:\Users\TalmoLab\mambaforge\envs\sleap_1.4.1a3_py310\lib\site-packages\keras\engine\training.py", line 2230, in predict_on_batch
    outputs = self.predict_function(iterator)
  File "C:\Users\TalmoLab\mambaforge\envs\sleap_1.4.1a3_py310\lib\site-packages\tensorflow\python\util\traceback_utils.py", line 153, in error_handler
    raise e.with_traceback(filtered_tb) from None
  File "C:\Users\TalmoLab\mambaforge\envs\sleap_1.4.1a3_py310\lib\site-packages\tensorflow\python\eager\execute.py", line 54, in quick_execute
    tensors = pywrap_tfe.TFE_Py_Execute(ctx._handle, device_name, op_name,
tensorflow.python.framework.errors_impl.UnknownError: Graph execution error:

Detected at node 'FloorMod' defined at (most recent call last):
    File "\\?\C:\Users\TalmoLab\mambaforge\envs\sleap_1.4.1a3_py310\Scripts\sleap-train-script.py", line 33, in <module>
      sys.exit(load_entry_point('sleap==1.4.1a3', 'console_scripts', 'sleap-train')())
    File "C:\Users\TalmoLab\mambaforge\envs\sleap_1.4.1a3_py310\lib\site-packages\sleap\nn\training.py", line 2039, in main
      trainer.train()
    File "C:\Users\TalmoLab\mambaforge\envs\sleap_1.4.1a3_py310\lib\site-packages\sleap\nn\training.py", line 953, in train
      self.evaluate()
    File "C:\Users\TalmoLab\mambaforge\envs\sleap_1.4.1a3_py310\lib\site-packages\sleap\nn\training.py", line 961, in evaluate
      sleap.nn.evals.evaluate_model(
    File "C:\Users\TalmoLab\mambaforge\envs\sleap_1.4.1a3_py310\lib\site-packages\sleap\nn\evals.py", line 744, in evaluate_model
      labels_pr: Labels = predictor.predict(labels_gt, make_labels=True)
    File "C:\Users\TalmoLab\mambaforge\envs\sleap_1.4.1a3_py310\lib\site-packages\sleap\nn\inference.py", line 527, in predict
      self._make_labeled_frames_from_generator(generator, data)
    File "C:\Users\TalmoLab\mambaforge\envs\sleap_1.4.1a3_py310\lib\site-packages\sleap\nn\inference.py", line 2645, in _make_labeled_frames_from_generator
      for ex in generator:
    File "C:\Users\TalmoLab\mambaforge\envs\sleap_1.4.1a3_py310\lib\site-packages\sleap\nn\inference.py", line 437, in _predict_generator
      ex = process_batch(ex)
    File "C:\Users\TalmoLab\mambaforge\envs\sleap_1.4.1a3_py310\lib\site-packages\sleap\nn\inference.py", line 400, in process_batch
      preds = self.inference_model.predict_on_batch(ex, numpy=True)
    File "C:\Users\TalmoLab\mambaforge\envs\sleap_1.4.1a3_py310\lib\site-packages\sleap\nn\inference.py", line 1070, in predict_on_batch
      outs = super().predict_on_batch(data, **kwargs)
    File "C:\Users\TalmoLab\mambaforge\envs\sleap_1.4.1a3_py310\lib\site-packages\keras\engine\training.py", line 2230, in predict_on_batch
      outputs = self.predict_function(iterator)
    File "C:\Users\TalmoLab\mambaforge\envs\sleap_1.4.1a3_py310\lib\site-packages\keras\engine\training.py", line 1845, in predict_function
      return step_function(self, iterator)
    File "C:\Users\TalmoLab\mambaforge\envs\sleap_1.4.1a3_py310\lib\site-packages\keras\engine\training.py", line 1834, in step_function
      outputs = model.distribute_strategy.run(run_step, args=(data,))
    File "C:\Users\TalmoLab\mambaforge\envs\sleap_1.4.1a3_py310\lib\site-packages\keras\engine\training.py", line 1823, in run_step
      outputs = model.predict_step(data)
    File "C:\Users\TalmoLab\mambaforge\envs\sleap_1.4.1a3_py310\lib\site-packages\keras\engine\training.py", line 1791, in predict_step
      return self(x, training=False)
    File "C:\Users\TalmoLab\mambaforge\envs\sleap_1.4.1a3_py310\lib\site-packages\keras\utils\traceback_utils.py", line 64, in error_handler
      return fn(*args, **kwargs)
    File "C:\Users\TalmoLab\mambaforge\envs\sleap_1.4.1a3_py310\lib\site-packages\keras\engine\training.py", line 490, in __call__
      return super().__call__(*args, **kwargs)
    File "C:\Users\TalmoLab\mambaforge\envs\sleap_1.4.1a3_py310\lib\site-packages\keras\utils\traceback_utils.py", line 64, in error_handler
      return fn(*args, **kwargs)
    File "C:\Users\TalmoLab\mambaforge\envs\sleap_1.4.1a3_py310\lib\site-packages\keras\engine\base_layer.py", line 1014, in __call__
      outputs = call_fn(inputs, *args, **kwargs)
    File "C:\Users\TalmoLab\mambaforge\envs\sleap_1.4.1a3_py310\lib\site-packages\keras\utils\traceback_utils.py", line 92, in error_handler
      return fn(*args, **kwargs)
    File "C:\Users\TalmoLab\mambaforge\envs\sleap_1.4.1a3_py310\lib\site-packages\sleap\nn\inference.py", line 2267, in call
      if isinstance(self.instance_peaks, FindInstancePeaksGroundTruth):
    File "C:\Users\TalmoLab\mambaforge\envs\sleap_1.4.1a3_py310\lib\site-packages\sleap\nn\inference.py", line 2276, in call
      peaks_output = self.instance_peaks(crop_output)
    File "C:\Users\TalmoLab\mambaforge\envs\sleap_1.4.1a3_py310\lib\site-packages\keras\utils\traceback_utils.py", line 64, in error_handler
      return fn(*args, **kwargs)
    File "C:\Users\TalmoLab\mambaforge\envs\sleap_1.4.1a3_py310\lib\site-packages\keras\engine\base_layer.py", line 1014, in __call__
      outputs = call_fn(inputs, *args, **kwargs)
    File "C:\Users\TalmoLab\mambaforge\envs\sleap_1.4.1a3_py310\lib\site-packages\keras\utils\traceback_utils.py", line 92, in error_handler
      return fn(*args, **kwargs)
    File "C:\Users\TalmoLab\mambaforge\envs\sleap_1.4.1a3_py310\lib\site-packages\sleap\nn\inference.py", line 2112, in call
      if self.offsets_ind is None:
    File "C:\Users\TalmoLab\mambaforge\envs\sleap_1.4.1a3_py310\lib\site-packages\sleap\nn\inference.py", line 2114, in call
      peak_points, peak_vals = sleap.nn.peak_finding.find_global_peaks(
    File "C:\Users\TalmoLab\mambaforge\envs\sleap_1.4.1a3_py310\lib\site-packages\sleap\nn\peak_finding.py", line 366, in find_global_peaks
      rough_peaks, peak_vals = find_global_peaks_rough(
    File "C:\Users\TalmoLab\mambaforge\envs\sleap_1.4.1a3_py310\lib\site-packages\sleap\nn\peak_finding.py", line 224, in find_global_peaks_rough
      channel_subs = tf.math.mod(tf.range(total_peaks, dtype=tf.int64), channels)
Node: 'FloorMod'
Detected at node 'FloorMod' defined at (most recent call last):
    File "\\?\C:\Users\TalmoLab\mambaforge\envs\sleap_1.4.1a3_py310\Scripts\sleap-train-script.py", line 33, in <module>
      sys.exit(load_entry_point('sleap==1.4.1a3', 'console_scripts', 'sleap-train')())
    File "C:\Users\TalmoLab\mambaforge\envs\sleap_1.4.1a3_py310\lib\site-packages\sleap\nn\training.py", line 2039, in main
      trainer.train()
    File "C:\Users\TalmoLab\mambaforge\envs\sleap_1.4.1a3_py310\lib\site-packages\sleap\nn\training.py", line 953, in train
      self.evaluate()
    File "C:\Users\TalmoLab\mambaforge\envs\sleap_1.4.1a3_py310\lib\site-packages\sleap\nn\training.py", line 961, in evaluate
      sleap.nn.evals.evaluate_model(
    File "C:\Users\TalmoLab\mambaforge\envs\sleap_1.4.1a3_py310\lib\site-packages\sleap\nn\evals.py", line 744, in evaluate_model
      labels_pr: Labels = predictor.predict(labels_gt, make_labels=True)
    File "C:\Users\TalmoLab\mambaforge\envs\sleap_1.4.1a3_py310\lib\site-packages\sleap\nn\inference.py", line 527, in predict
      self._make_labeled_frames_from_generator(generator, data)
    File "C:\Users\TalmoLab\mambaforge\envs\sleap_1.4.1a3_py310\lib\site-packages\sleap\nn\inference.py", line 2645, in _make_labeled_frames_from_generator
      for ex in generator:
    File "C:\Users\TalmoLab\mambaforge\envs\sleap_1.4.1a3_py310\lib\site-packages\sleap\nn\inference.py", line 437, in _predict_generator
      ex = process_batch(ex)
    File "C:\Users\TalmoLab\mambaforge\envs\sleap_1.4.1a3_py310\lib\site-packages\sleap\nn\inference.py", line 400, in process_batch
      preds = self.inference_model.predict_on_batch(ex, numpy=True)
    File "C:\Users\TalmoLab\mambaforge\envs\sleap_1.4.1a3_py310\lib\site-packages\sleap\nn\inference.py", line 1070, in predict_on_batch
      outs = super().predict_on_batch(data, **kwargs)
    File "C:\Users\TalmoLab\mambaforge\envs\sleap_1.4.1a3_py310\lib\site-packages\keras\engine\training.py", line 2230, in predict_on_batch
      outputs = self.predict_function(iterator)
    File "C:\Users\TalmoLab\mambaforge\envs\sleap_1.4.1a3_py310\lib\site-packages\keras\engine\training.py", line 1845, in predict_function
      return step_function(self, iterator)
    File "C:\Users\TalmoLab\mambaforge\envs\sleap_1.4.1a3_py310\lib\site-packages\keras\engine\training.py", line 1834, in step_function
      outputs = model.distribute_strategy.run(run_step, args=(data,))
    File "C:\Users\TalmoLab\mambaforge\envs\sleap_1.4.1a3_py310\lib\site-packages\keras\engine\training.py", line 1823, in run_step
      outputs = model.predict_step(data)
    File "C:\Users\TalmoLab\mambaforge\envs\sleap_1.4.1a3_py310\lib\site-packages\keras\engine\training.py", line 1791, in predict_step
      return self(x, training=False)
    File "C:\Users\TalmoLab\mambaforge\envs\sleap_1.4.1a3_py310\lib\site-packages\keras\utils\traceback_utils.py", line 64, in error_handler
      return fn(*args, **kwargs)
    File "C:\Users\TalmoLab\mambaforge\envs\sleap_1.4.1a3_py310\lib\site-packages\keras\engine\training.py", line 490, in __call__
      return super().__call__(*args, **kwargs)
    File "C:\Users\TalmoLab\mambaforge\envs\sleap_1.4.1a3_py310\lib\site-packages\keras\utils\traceback_utils.py", line 64, in error_handler
      return fn(*args, **kwargs)
    File "C:\Users\TalmoLab\mambaforge\envs\sleap_1.4.1a3_py310\lib\site-packages\keras\engine\base_layer.py", line 1014, in __call__
      outputs = call_fn(inputs, *args, **kwargs)
    File "C:\Users\TalmoLab\mambaforge\envs\sleap_1.4.1a3_py310\lib\site-packages\keras\utils\traceback_utils.py", line 92, in error_handler
      return fn(*args, **kwargs)
    File "C:\Users\TalmoLab\mambaforge\envs\sleap_1.4.1a3_py310\lib\site-packages\sleap\nn\inference.py", line 2267, in call
      if isinstance(self.instance_peaks, FindInstancePeaksGroundTruth):
    File "C:\Users\TalmoLab\mambaforge\envs\sleap_1.4.1a3_py310\lib\site-packages\sleap\nn\inference.py", line 2276, in call
      peaks_output = self.instance_peaks(crop_output)
    File "C:\Users\TalmoLab\mambaforge\envs\sleap_1.4.1a3_py310\lib\site-packages\keras\utils\traceback_utils.py", line 64, in error_handler
      return fn(*args, **kwargs)
    File "C:\Users\TalmoLab\mambaforge\envs\sleap_1.4.1a3_py310\lib\site-packages\keras\engine\base_layer.py", line 1014, in __call__
      outputs = call_fn(inputs, *args, **kwargs)
    File "C:\Users\TalmoLab\mambaforge\envs\sleap_1.4.1a3_py310\lib\site-packages\keras\utils\traceback_utils.py", line 92, in error_handler
      return fn(*args, **kwargs)
    File "C:\Users\TalmoLab\mambaforge\envs\sleap_1.4.1a3_py310\lib\site-packages\sleap\nn\inference.py", line 2112, in call
      if self.offsets_ind is None:
    File "C:\Users\TalmoLab\mambaforge\envs\sleap_1.4.1a3_py310\lib\site-packages\sleap\nn\inference.py", line 2114, in call
      peak_points, peak_vals = sleap.nn.peak_finding.find_global_peaks(
    File "C:\Users\TalmoLab\mambaforge\envs\sleap_1.4.1a3_py310\lib\site-packages\sleap\nn\peak_finding.py", line 366, in find_global_peaks
      rough_peaks, peak_vals = find_global_peaks_rough(
    File "C:\Users\TalmoLab\mambaforge\envs\sleap_1.4.1a3_py310\lib\site-packages\sleap\nn\peak_finding.py", line 224, in find_global_peaks_rough
      channel_subs = tf.math.mod(tf.range(total_peaks, dtype=tf.int64), channels)
Node: 'FloorMod'
2 root error(s) found.
  (0) UNKNOWN:  JIT compilation failed.
         [[{{node FloorMod}}]]
         [[top_down_inference_model/find_instance_peaks_1/RaggedFromValueRowIds_1/RowPartitionFromValueRowIds/assert_less/Assert/AssertGuard/pivot_f/_177/_415]]
  (1) UNKNOWN:  JIT compilation failed.
         [[{{node FloorMod}}]]
0 successful operations.
0 derived errors ignored. [Op:__inference_predict_function_31992]
INFO:sleap.nn.callbacks:Closing the reporter controller/context.
INFO:sleap.nn.callbacks:Closing the training controller socket/context.
Run Path: D:/social-leap-estimates-animal-poses/datasets/drosophila-melanogaster-courtship/drosophila-melanogaster-courtship\models\241009_114536.centered_instance.n=3

Training Dialog

While structuring TrainingJobConfig (1 sub-exception)
While structuring TrainingJobConfig (1 sub-exception)
While structuring TrainingJobConfig (1 sub-exception)
While structuring TrainingJobConfig (1 sub-exception)
While structuring TrainingJobConfig (1 sub-exception)
While structuring TrainingJobConfig (1 sub-exception)
While structuring TrainingJobConfig (1 sub-exception)
While structuring TrainingJobConfig (1 sub-exception)

sleap-label

(sleap_1.4.1a3_py310) λ sleap-label
C:\Users\TalmoLab\mambaforge\envs\sleap_1.4.1a3_py310\lib\site-packages\albumentations\__init__.py:13: UserWarning: A new version of Albumentations is available: 1.4.18 (you have 1.4.15). Upgrade using: pip install -U albumentations. To disable automatic update checks, set the environment variable NO_ALBUMENTATIONS_UPDATE to 1.
  check_for_updates()
Saving config: C:\Users\TalmoLab\.sleap\1.4.1a3\preferences.yaml

Software versions:
SLEAP: 1.4.1a3
TensorFlow: 2.9.2
Numpy: 1.26.4
Python: 3.10.15
OS: Windows-10-10.0.19044-SP0

Happy SLEAPing! :)

Installation

λ mamba create -n sleap_1.4.1a3_py310 -c sleap-deps -c conda-forge -c nvidia -c ./sleap-build-win -c anaconda sleap=1.4.1a3

                  __    __    __    __
                 /  \  /  \  /  \  /  \
                /    \/    \/    \/    \
███████████████/  /██/  /██/  /██/  /████████████████████████
              /  / \   / \   / \   / \  \____
             /  /   \_/   \_/   \_/   \    o \__,
            / _/                       \_____/  `
            |/
        ███╗   ███╗ █████╗ ███╗   ███╗██████╗  █████╗
        ████╗ ████║██╔══██╗████╗ ████║██╔══██╗██╔══██╗
        ██╔████╔██║███████║██╔████╔██║██████╔╝███████║
        ██║╚██╔╝██║██╔══██║██║╚██╔╝██║██╔══██╗██╔══██║
        ██║ ╚═╝ ██║██║  ██║██║ ╚═╝ ██║██████╔╝██║  ██║
        ╚═╝     ╚═╝╚═╝  ╚═╝╚═╝     ╚═╝╚═════╝ ╚═╝  ╚═╝

        mamba (1.4.1) supported by @QuantStack

        GitHub:  https://github.com/mamba-org/mamba
        Twitter: https://twitter.com/QuantStack

█████████████████████████████████████████████████████████████


Looking for: ['sleap=1.4.1a3']

sleap-deps/win-64                                           Using cache
sleap-deps/noarch                                           Using cache
nvidia/win-64                                               Using cache
nvidia/noarch                                               Using cache
anaconda/noarch                                             Using cache
file://C:/Users/TalmoLab/Downloads/sleap-build-w.. 927.0 B @   3.3MB/s  0.0s
file://C:/Users/TalmoLab/Downloads/sleap-build-w..  96.0 B @ 564.7kB/s  0.0s
anaconda/win-64                                      3.3MB @   5.7MB/s  0.7s
conda-forge/noarch                                  19.4MB @   7.5MB/s  3.3s
conda-forge/win-64                                  29.3MB @   6.8MB/s  5.4s
Transaction

  Prefix: C:\Users\TalmoLab\mambaforge\envs\sleap_1.4.1a3_py310

  Updating specs:

   - sleap=1.4.1a3


  Package                                            Version  Build                    Channel                                                  Size
------------------------------------------------------------------------------------------------------------------------------------------------------
  Install:
------------------------------------------------------------------------------------------------------------------------------------------------------

  + _libavif_api                                       1.1.1  h57928b3_1               conda-forge/win-64                                        9kB
  + albucore                                          0.0.16  pyhd8ed1ab_0             conda-forge/noarch                                       15kB
  + albumentations                                    1.4.15  pyhd8ed1ab_0             conda-forge/noarch                                      150kB
  + annotated-types                                    0.7.0  pyhd8ed1ab_0             conda-forge/noarch                                     Cached
  + aom                                                3.9.1  he0c23c2_0               conda-forge/win-64                                     Cached
  + attrs                                             24.2.0  pyh71513ae_0             conda-forge/noarch                                     Cached
  + blosc                                             1.21.6  h85f69ea_0               conda-forge/win-64                                     Cached
  + brotli                                             1.1.0  h2466b09_2               conda-forge/win-64                                     Cached
  + brotli-bin                                         1.1.0  h2466b09_2               conda-forge/win-64                                     Cached
  + bzip2                                              1.0.8  h2466b09_7               conda-forge/win-64                                     Cached
  + c-blosc2                                          2.15.1  hb461149_0               conda-forge/win-64                                     Cached
  + ca-certificates                                2024.8.30  h56e8100_0               conda-forge/win-64                                     Cached
  + cached-property                                    1.5.2  hd8ed1ab_1               conda-forge/noarch                                     Cached
  + cached_property                                    1.5.2  pyha770c72_1             conda-forge/noarch                                     Cached
  + cairo                                             1.18.0  h32b962e_3               conda-forge/win-64                                     Cached
  + cattrs                                            24.1.2  pyhd8ed1ab_0             conda-forge/noarch                                       52kB
  + certifi                                        2024.8.30  pyhd8ed1ab_0             conda-forge/noarch                                     Cached
  + charls                                             2.4.2  h1537add_0               conda-forge/win-64                                     Cached
  + contourpy                                          1.3.0  py310hc19bc0b_2          conda-forge/win-64                                      200kB
  + cuda-nvcc                                        11.3.58  hb8d16a4_0               nvidia/win-64                                          Cached
  + cudatoolkit                                       11.3.1  hf2f0253_13              conda-forge/win-64                                     Cached
  + cudnn                                           8.2.1.32  h754d62a_0               conda-forge/win-64                                     Cached
  + cycler                                            0.12.1  pyhd8ed1ab_0             conda-forge/noarch                                     Cached
  + dav1d                                              1.2.1  hcfcfb64_0               conda-forge/win-64                                     Cached
  + double-conversion                                  3.3.0  h63175ca_0               conda-forge/win-64                                     Cached
  + eval-type-backport                                 0.2.0  pyhd8ed1ab_0             conda-forge/noarch                                        7kB
  + eval_type_backport                                 0.2.0  pyha770c72_0             conda-forge/noarch                                     Cached
  + exceptiongroup                                     1.2.2  pyhd8ed1ab_0             conda-forge/noarch                                     Cached
  + expat                                              2.6.3  he0c23c2_0               conda-forge/win-64                                     Cached
  + ffmpeg                                             6.1.2  gpl_h0820249_105         conda-forge/win-64                                     Cached
  + font-ttf-dejavu-sans-mono                           2.37  hab24e00_0               conda-forge/noarch                                     Cached
  + font-ttf-inconsolata                               3.000  h77eed37_0               conda-forge/noarch                                     Cached
  + font-ttf-source-code-pro                           2.038  h77eed37_0               conda-forge/noarch                                     Cached
  + font-ttf-ubuntu                                     0.83  h77eed37_3               conda-forge/noarch                                     Cached
  + fontconfig                                        2.14.2  hbde0cde_0               conda-forge/win-64                                     Cached
  + fonts-conda-ecosystem                                  1  0                        conda-forge/noarch                                     Cached
  + fonts-conda-forge                                      1  0                        conda-forge/noarch                                     Cached
  + fonttools                                         4.54.1  py310ha8f682b_0          conda-forge/win-64                                        2MB
  + freeglut                                           3.2.2  he0c23c2_3               conda-forge/win-64                                     Cached
  + freetype                                          2.12.1  hdaf720e_2               conda-forge/win-64                                     Cached
  + giflib                                             5.2.2  h64bf75a_0               conda-forge/win-64                                     Cached
  + graphite2                                         1.3.13  h63175ca_1003            conda-forge/win-64                                     Cached
  + h5py                                              3.11.0  nompi_py310h2b0be38_102  conda-forge/win-64                                     Cached
  + harfbuzz                                           9.0.0  h2bedf89_1               conda-forge/win-64                                     Cached
  + hdf5                                              1.14.3  nompi_h2b43c12_105       conda-forge/win-64                                     Cached
  + hdmf                                              3.14.4  pyh2e8e312_0             conda-forge/noarch                                     Cached
  + icu                                                 75.1  he0c23c2_0               conda-forge/win-64                                     Cached
  + imagecodecs                                     2024.6.1  py310h0e2d205_5          conda-forge/win-64                                        2MB
  + imageio                                           2.35.1  pyh12aca89_0             conda-forge/noarch                                     Cached
  + imageio-ffmpeg                                     0.5.1  pyhd8ed1ab_0             conda-forge/noarch                                     Cached
  + imath                                             3.1.12  hbb528cf_0               conda-forge/win-64                                     Cached
  + importlib-metadata                                 8.5.0  pyha770c72_0             conda-forge/noarch                                     Cached
  + importlib_metadata                                 8.5.0  hd8ed1ab_0               conda-forge/noarch                                        9kB
  + importlib_resources                                6.4.5  pyhd8ed1ab_0             conda-forge/noarch                                     Cached
  + intel-openmp                                    2024.2.1  h57928b3_1083            conda-forge/win-64                                     Cached
  + jasper                                             4.2.4  hcb1a123_0               conda-forge/win-64                                     Cached
  + joblib                                             1.4.2  pyhd8ed1ab_0             conda-forge/noarch                                     Cached
  + jsmin                                              3.0.1  pyhd8ed1ab_0             conda-forge/noarch                                     Cached
  + jsonpickle                                         1.4.1  pyh9f0ad1d_0             conda-forge/noarch                                     Cached
  + jsonschema                                        4.23.0  pyhd8ed1ab_0             conda-forge/noarch                                     Cached
  + jsonschema-specifications                      2024.10.1  pyhd8ed1ab_0             conda-forge/noarch                                       16kB
  + jxrlib                                               1.1  hcfcfb64_3               conda-forge/win-64                                     Cached
  + khronos-opencl-icd-loader                     2024.05.08  hc70643c_0               conda-forge/win-64                                     Cached
  + kiwisolver                                         1.4.7  py310hc19bc0b_0          conda-forge/win-64                                     Cached
  + krb5                                              1.21.3  hdf4eb48_0               conda-forge/win-64                                     Cached
  + lazy-loader                                          0.4  pyhd8ed1ab_1             conda-forge/noarch                                     Cached
  + lazy_loader                                          0.4  pyhd8ed1ab_1             conda-forge/noarch                                        7kB
  + lcms2                                               2.16  h67d730c_0               conda-forge/win-64                                     Cached
  + lerc                                               4.0.0  h63175ca_0               conda-forge/win-64                                     Cached
  + libabseil                                     20240722.0  cxx17_he0c23c2_1         conda-forge/win-64                                     Cached
  + libaec                                             1.1.3  h63175ca_0               conda-forge/win-64                                     Cached
  + libasprintf                                       0.22.5  h5728263_3               conda-forge/win-64                                     Cached
  + libavif16                                          1.1.1  h4e96d62_1               conda-forge/win-64                                     Cached
  + libblas                                            3.9.0  24_win64_mkl             conda-forge/win-64                                     Cached
  + libbrotlicommon                                    1.1.0  h2466b09_2               conda-forge/win-64                                     Cached
  + libbrotlidec                                       1.1.0  h2466b09_2               conda-forge/win-64                                     Cached
  + libbrotlienc                                       1.1.0  h2466b09_2               conda-forge/win-64                                     Cached
  + libcblas                                           3.9.0  24_win64_mkl             conda-forge/win-64                                     Cached
  + libclang13                                        19.1.1  default_ha5278ca_0       conda-forge/win-64                                       27MB
  + libcurl                                           8.10.1  h1ee3ff0_0               conda-forge/win-64                                     Cached
  + libdeflate                                          1.22  h2466b09_0               conda-forge/win-64                                      156kB
  + libexpat                                           2.6.3  he0c23c2_0               conda-forge/win-64                                     Cached
  + libffi                                             3.4.2  h8ffe710_5               conda-forge/win-64                                     Cached
  + libgettextpo                                      0.22.5  h5728263_3               conda-forge/win-64                                     Cached
  + libglib                                           2.82.1  h7025463_0               conda-forge/win-64                                     Cached
  + libhwloc                                          2.11.1  default_h8125262_1000    conda-forge/win-64                                     Cached
  + libiconv                                            1.17  hcfcfb64_2               conda-forge/win-64                                     Cached
  + libintl                                           0.22.5  h5728263_3               conda-forge/win-64                                     Cached
  + libjpeg-turbo                                      3.0.0  hcfcfb64_1               conda-forge/win-64                                     Cached
  + liblapack                                          3.9.0  24_win64_mkl             conda-forge/win-64                                     Cached
  + liblapacke                                         3.9.0  24_win64_mkl             conda-forge/win-64                                     Cached
  + libopencv                                         4.10.0  qt6_py310h00b716a_607    conda-forge/win-64                                       33MB
  + libopenvino                                     2024.4.0  hfe1841e_1               conda-forge/win-64                                     Cached
  + libopenvino-auto-batch-plugin                   2024.4.0  h04f32e0_1               conda-forge/win-64                                     Cached
  + libopenvino-auto-plugin                         2024.4.0  h04f32e0_1               conda-forge/win-64                                     Cached
  + libopenvino-hetero-plugin                       2024.4.0  h372dad0_1               conda-forge/win-64                                     Cached
  + libopenvino-intel-cpu-plugin                    2024.4.0  hfe1841e_1               conda-forge/win-64                                     Cached
  + libopenvino-intel-gpu-plugin                    2024.4.0  hfe1841e_1               conda-forge/win-64                                     Cached
  + libopenvino-ir-frontend                         2024.4.0  h372dad0_1               conda-forge/win-64                                     Cached
  + libopenvino-onnx-frontend                       2024.4.0  h5707d70_1               conda-forge/win-64                                     Cached
  + libopenvino-paddle-frontend                     2024.4.0  h5707d70_1               conda-forge/win-64                                     Cached
  + libopenvino-pytorch-frontend                    2024.4.0  he0c23c2_1               conda-forge/win-64                                     Cached
  + libopenvino-tensorflow-frontend                 2024.4.0  hf4e5e90_1               conda-forge/win-64                                     Cached
  + libopenvino-tensorflow-lite-frontend            2024.4.0  he0c23c2_1               conda-forge/win-64                                     Cached
  + libopus                                            1.3.1  h8ffe710_1               conda-forge/win-64                                     Cached
  + libpng                                            1.6.44  h3ca93ac_0               conda-forge/win-64                                     Cached
  + libprotobuf                                       5.27.5  hcaed137_2               conda-forge/win-64                                     Cached
  + libsodium                                         1.0.20  hc70643c_0               conda-forge/win-64                                     Cached
  + libsqlite                                         3.46.1  h2466b09_0               conda-forge/win-64                                     Cached
  + libssh2                                           1.11.0  h7dfc565_0               conda-forge/win-64                                     Cached
  + libtiff                                            4.7.0  hfc51747_1               conda-forge/win-64                                      979kB
  + libwebp-base                                       1.4.0  hcfcfb64_0               conda-forge/win-64                                     Cached
  + libxcb                                              1.16  h013a479_1               conda-forge/win-64                                     Cached
  + libxml2                                           2.12.7  h0f24e4e_4               conda-forge/win-64                                     Cached
  + libxslt                                           1.1.39  h3df6e99_0               conda-forge/win-64                                     Cached
  + libzlib                                            1.3.1  h2466b09_2               conda-forge/win-64                                     Cached
  + libzopfli                                          1.0.3  h0e60522_0               conda-forge/win-64                                     Cached
  + lz4-c                                              1.9.4  hcfcfb64_0               conda-forge/win-64                                     Cached
  + m2w64-gcc-libgfortran                              5.3.0  6                        conda-forge/win-64                                     Cached
  + m2w64-gcc-libs                                     5.3.0  7                        conda-forge/win-64                                     Cached
  + m2w64-gcc-libs-core                                5.3.0  7                        conda-forge/win-64                                     Cached
  + m2w64-gmp                                          6.1.0  2                        conda-forge/win-64                                     Cached
  + m2w64-libwinpthread-git               5.0.0.4634.697f757  2                        conda-forge/win-64                                     Cached
  + markdown-it-py                                     3.0.0  pyhd8ed1ab_0             conda-forge/noarch                                     Cached
  + matplotlib-base                                    3.9.2  py310h37e0a56_1          conda-forge/win-64                                        7MB
  + mdurl                                              0.1.2  pyhd8ed1ab_0             conda-forge/noarch                                     Cached
  + mkl                                             2024.1.0  h66d3029_694             conda-forge/win-64                                     Cached
  + msys2-conda-epoch                               20160418  1                        conda-forge/win-64                                     Cached
  + munkres                                            1.1.4  pyh9f0ad1d_0             conda-forge/noarch                                     Cached
  + ndx-pose                                           0.1.1  pyhd8ed1ab_0             conda-forge/noarch                                     Cached
  + networkx                                             3.3  pyhd8ed1ab_1             conda-forge/noarch                                     Cached
  + numpy                                             1.26.4  py310hf667824_0          conda-forge/win-64                                     Cached
  + opencv                                            4.10.0  qt6_py310h896e0ad_607    conda-forge/win-64                                       27kB
  + openexr                                            3.2.2  h9aba623_2               conda-forge/win-64                                     Cached
  + openh264                                           2.4.1  h63175ca_0               conda-forge/win-64                                     Cached
  + openjpeg                                           2.5.2  h3d672ee_0               conda-forge/win-64                                     Cached
  + openssl                                            3.3.2  h2466b09_0               conda-forge/win-64                                     Cached
  + packaging                                           24.1  pyhd8ed1ab_0             conda-forge/noarch                                     Cached
  + pandas                                             2.2.3  py310hb4db72f_1          conda-forge/win-64                                       12MB
  + patsy                                              0.5.6  pyhd8ed1ab_0             conda-forge/noarch                                     Cached
  + pcre2                                              10.44  h3d7b363_2               conda-forge/win-64                                     Cached
  + pillow                                            10.4.0  py310h3e38d90_1          conda-forge/win-64                                     Cached
  + pip                                                 24.2  pyh8b19718_1             conda-forge/noarch                                     Cached
  + pixman                                            0.43.4  h63175ca_0               conda-forge/win-64                                     Cached
  + pkgutil-resolve-name                              1.3.10  pyhd8ed1ab_1             conda-forge/noarch                                     Cached
  + protobuf                                          5.27.5  py310h9e98ed7_0          conda-forge/win-64                                      373kB
  + psutil                                             6.0.0  py310ha8f682b_1          conda-forge/win-64                                     Cached
  + pthread-stubs                                        0.4  hcd874cb_1001            conda-forge/win-64                                     Cached
  + pthreads-win32                                     2.9.1  h2466b09_4               conda-forge/win-64                                     Cached
  + pugixml                                             1.14  h63175ca_0               conda-forge/win-64                                     Cached
  + py-opencv                                         4.10.0  qt6_py310h5f8bd55_607    conda-forge/win-64                                        1MB
  + pydantic                                           2.9.2  pyhd8ed1ab_0             conda-forge/noarch                                      301kB
  + pydantic-core                                     2.23.4  py310hc226416_0          conda-forge/win-64                                        2MB
  + pygments                                          2.18.0  pyhd8ed1ab_0             conda-forge/noarch                                     Cached
  + pykalman                                           0.9.7  pyhd8ed1ab_0             conda-forge/noarch                                     Cached
  + pynwb                                              2.8.2  pyh2e8e312_0             conda-forge/noarch                                     Cached
  + pyparsing                                          3.1.4  pyhd8ed1ab_0             conda-forge/noarch                                     Cached
  + pyside6                                            6.7.3  py310h60c6385_1          conda-forge/win-64                                        9MB
  + python                                           3.10.15  hfaddaf0_1_cpython       conda-forge/win-64                                     Cached
  + python-dateutil                                    2.9.0  pyhd8ed1ab_0             conda-forge/noarch                                     Cached
  + python-rapidjson                                    1.20  py310h9e98ed7_0          conda-forge/win-64                                     Cached
  + python-tzdata                                     2024.2  pyhd8ed1ab_0             conda-forge/noarch                                     Cached
  + python_abi                                          3.10  5_cp310                  conda-forge/win-64                                        7kB
  + pytz                                              2024.1  pyhd8ed1ab_0             conda-forge/noarch                                     Cached
  + pywavelets                                         1.7.0  py310hb0944cc_1          conda-forge/win-64                                        4MB
  + pyyaml                                             6.0.2  py310ha8f682b_1          conda-forge/win-64                                     Cached
  + pyzmq                                             26.2.0  py310h656833d_2          conda-forge/win-64                                     Cached
  + qhull                                             2020.2  hc790b64_5               conda-forge/win-64                                     Cached
  + qt6-main                                           6.7.3  hfb098fa_1               conda-forge/win-64                                       89MB
  + qtpy                                               2.4.1  pyhd8ed1ab_0             conda-forge/noarch                                     Cached
  + qudida                                             0.0.4  pyhd8ed1ab_0             conda-forge/noarch                                     Cached
  + rav1e                                              0.6.6  h975169c_2               conda-forge/win-64                                     Cached
  + referencing                                       0.35.1  pyhd8ed1ab_0             conda-forge/noarch                                     Cached
  + rich                                              13.9.2  pyhd8ed1ab_0             conda-forge/noarch                                      186kB
  + rpds-py                                           0.20.0  py310hc226416_1          conda-forge/win-64                                     Cached
  + ruamel.yaml                                       0.18.6  py310h8d17308_0          conda-forge/win-64                                     Cached
  + ruamel.yaml.clib                                   0.2.8  py310h8d17308_0          conda-forge/win-64                                     Cached
  + scikit-image                                      0.24.0  py310hb4db72f_2          conda-forge/win-64                                     Cached
  + scikit-learn                                       1.5.2  py310hf2a6c47_1          conda-forge/win-64                                        8MB
  + scipy                                             1.14.1  py310h46043a1_0          conda-forge/win-64                                     Cached
  + seaborn                                           0.13.2  hd8ed1ab_2               conda-forge/noarch                                        7kB
  + seaborn-base                                      0.13.2  pyhd8ed1ab_2             conda-forge/noarch                                     Cached
  + setuptools                                        75.1.0  pyhd8ed1ab_0             conda-forge/noarch                                     Cached
  + six                                               1.16.0  pyh6c4a22f_0             conda-forge/noarch                                     Cached
  + sleap                                            1.4.1a3  py310_0                  C:/Users/TalmoLab/Downloads/sleap-build-win/win-64        3MB
  + snappy                                             1.2.1  h23299a8_0               conda-forge/win-64                                     Cached
  + statsmodels                                       0.14.4  py310hb0944cc_0          conda-forge/win-64                                       10MB
  + svt-av1                                            2.2.1  he0c23c2_0               conda-forge/win-64                                     Cached
  + tbb                                            2021.13.0  hc790b64_0               conda-forge/win-64                                     Cached
  + tensorflow                                         2.9.2  py310h82bb817_1          sleap-deps/win-64                                       420MB
  + tensorflow-hub                                    0.12.0  pyhca92ed8_0             conda-forge/noarch                                     Cached
  + threadpoolctl                                      3.5.0  pyhc1e730c_0             conda-forge/noarch                                     Cached
  + tifffile                                       2024.9.20  pyhd8ed1ab_0             conda-forge/noarch                                      181kB
  + tk                                                8.6.13  h5226925_1               conda-forge/win-64                                     Cached
  + tomli                                              2.0.2  pyhd8ed1ab_0             conda-forge/noarch                                       18kB
  + typing-extensions                                 4.12.2  hd8ed1ab_0               conda-forge/noarch                                       10kB
  + typing_extensions                                 4.12.2  pyha770c72_0             conda-forge/noarch                                     Cached
  + tzdata                                             2024b  hc8b5060_0               conda-forge/noarch                                     Cached
  + ucrt                                        10.0.22621.0  h57928b3_1               conda-forge/win-64                                     Cached
  + unicodedata2                                      15.1.0  py310h8d17308_0          conda-forge/win-64                                     Cached
  + vc                                                  14.3  h8a93ad2_22              conda-forge/win-64                                     Cached
  + vc14_runtime                                 14.40.33810  hcc2c482_22              conda-forge/win-64                                     Cached
  + vs2015_runtime                               14.40.33810  h3bf8584_22              conda-forge/win-64                                     Cached
  + wheel                                             0.44.0  pyhd8ed1ab_0             conda-forge/noarch                                     Cached
  + x264                                          1!164.3095  h8ffe710_2               conda-forge/win-64                                     Cached
  + x265                                                 3.5  h2d74725_3               conda-forge/win-64                                     Cached
  + xorg-libxau                                       1.0.11  hcd874cb_0               conda-forge/win-64                                     Cached
  + xorg-libxdmcp                                      1.1.3  hcd874cb_0               conda-forge/win-64                                     Cached
  + xz                                                 5.2.6  h8d14728_0               conda-forge/win-64                                     Cached
  + yaml                                               0.2.5  h8ffe710_2               conda-forge/win-64                                     Cached
  + zeromq                                             4.3.5  ha9f60a1_6               conda-forge/win-64                                        3MB
  + zfp                                                1.0.1  he0c23c2_2               conda-forge/win-64                                      235kB
  + zipp                                              3.20.2  pyhd8ed1ab_0             conda-forge/noarch                                     Cached
  + zlib                                               1.3.1  h2466b09_2               conda-forge/win-64                                      107kB
  + zlib-ng                                            2.2.2  he0c23c2_0               conda-forge/win-64                                      109kB
  + zstd                                               1.5.6  h0ea2cb4_0               conda-forge/win-64                                     Cached

  Summary:

  Install: 216 packages

  Total download: 634MB

pip freeze

(sleap_1.4.1a3_py310) λ pip freeze
absl-py==2.1.0
albucore @ file:///home/conda/feedstock_root/build_artifacts/albucore_1727019653300/work
albumentations @ file:///home/conda/feedstock_root/build_artifacts/albumentations_1727025741991/work
annotated-types @ file:///home/conda/feedstock_root/build_artifacts/annotated-types_1716290248287/work
astunparse==1.6.3
attrs @ file:///home/conda/feedstock_root/build_artifacts/attrs_1722977137225/work
cached-property @ file:///home/conda/feedstock_root/build_artifacts/cached_property_1615209429212/work
cachetools==5.5.0
cattrs @ file:///home/conda/feedstock_root/build_artifacts/cattrs_1727088879957/work
certifi @ file:///home/conda/feedstock_root/build_artifacts/certifi_1725278078093/work/certifi
charset-normalizer==3.3.2
contourpy @ file:///D:/bld/contourpy_1727293672566/work
cycler @ file:///home/conda/feedstock_root/build_artifacts/cycler_1696677705766/work
debugpy==1.6.6
efficientnet==1.0.0
eval_type_backport @ file:///home/conda/feedstock_root/build_artifacts/eval_type_backport_1724150075856/work
exceptiongroup @ file:///home/conda/feedstock_root/build_artifacts/exceptiongroup_1720869315914/work
flatbuffers==1.12
fonttools @ file:///D:/bld/fonttools_1727206449729/work
gast==0.4.0
google-auth==2.35.0
google-auth-oauthlib==0.4.6
google-pasta==0.2.0
grpcio==1.66.2
h5py @ file:///D:/bld/h5py_1717664858971/work
hdmf @ file:///D:/bld/hdmf_1725567417856/work
idna==3.10
image-classifiers==1.0.0
imagecodecs @ file:///D:/bld/imagecodecs_1728267409164/work
imageio @ file:///home/conda/feedstock_root/build_artifacts/imageio_1724069053555/work
imageio-ffmpeg @ file:///home/conda/feedstock_root/build_artifacts/imageio-ffmpeg_1717461632069/work
imgstore==0.2.9
importlib_metadata @ file:///home/conda/feedstock_root/build_artifacts/importlib-metadata_1726082825846/work
importlib_resources @ file:///home/conda/feedstock_root/build_artifacts/importlib_resources_1725921340658/work
ipykernel==6.21.2
joblib @ file:///home/conda/feedstock_root/build_artifacts/joblib_1714665484399/work
jsmin @ file:///home/conda/feedstock_root/build_artifacts/jsmin_1642532731678/work
jsonpickle==1.4.1
jsonschema @ file:///home/conda/feedstock_root/build_artifacts/jsonschema_1720529478715/work
jsonschema-specifications @ file:///tmp/tmpvslgxhz5/src
jupyter_core==5.2.0
keras==2.9.0
Keras-Applications==1.0.8
Keras-Preprocessing==1.1.2
kiwisolver @ file:///D:/bld/kiwisolver_1725459382062/work
lazy_loader @ file:///home/conda/feedstock_root/build_artifacts/lazy-loader_1723774329602/work
libclang==18.1.1
Markdown==3.7
markdown-it-py @ file:///home/conda/feedstock_root/build_artifacts/markdown-it-py_1686175045316/work
MarkupSafe==2.1.5
matplotlib==3.9.2
mdurl @ file:///home/conda/feedstock_root/build_artifacts/mdurl_1704317613764/work
munkres==1.1.4
ndx-pose @ file:///home/conda/feedstock_root/build_artifacts/ndx-pose_1706810229855/work
nest-asyncio==1.5.6
networkx @ file:///home/conda/feedstock_root/build_artifacts/networkx_1712540363324/work
nixio==1.5.3
numpy @ file:///D:/bld/numpy_1707225570061/work/dist/numpy-1.26.4-cp310-cp310-win_amd64.whl#sha256=6761da75b1528684e6bf4dabdbdded9d1eb4d0e9b299482c7ce152cfb3155106
oauthlib==3.2.2
opencv-python==4.10.0
opencv-python-headless==4.10.0
opt_einsum==3.4.0
packaging @ file:///home/conda/feedstock_root/build_artifacts/packaging_1718189413536/work
pandas @ file:///D:/bld/pandas_1726878561601/work
patsy @ file:///home/conda/feedstock_root/build_artifacts/patsy_1704469236901/work
pillow @ file:///D:/bld/pillow_1726075253811/work
pkgutil_resolve_name @ file:///home/conda/feedstock_root/build_artifacts/pkgutil-resolve-name_1694617248815/work
protobuf==3.19.6
psutil==5.9.4
pyasn1==0.6.1
pyasn1_modules==0.4.1
pydantic @ file:///home/conda/feedstock_root/build_artifacts/pydantic_1726601062926/work
pydantic_core @ file:///D:/bld/pydantic-core_1726525117142/work
Pygments @ file:///home/conda/feedstock_root/build_artifacts/pygments_1714846767233/work
pykalman @ file:///home/conda/feedstock_root/build_artifacts/pykalman_1711547707628/work
pynwb @ file:///D:/bld/pynwb_1725927547197/work
pyparsing @ file:///home/conda/feedstock_root/build_artifacts/pyparsing_1724616129934/work
PySide6==6.7.3
python-dateutil @ file:///home/conda/feedstock_root/build_artifacts/python-dateutil_1709299778482/work
python-rapidjson @ file:///D:/bld/python-rapidjson_1722901949842/work
pytz @ file:///home/conda/feedstock_root/build_artifacts/pytz_1706886791323/work
PyWavelets==1.7.0
pywin32==305
PyYAML @ file:///D:/bld/pyyaml_1725456311802/work
pyzmq==25.0.0
qimage2ndarray==1.10.0
QtPy @ file:///home/conda/feedstock_root/build_artifacts/qtpy_1698112029416/work
qudida @ file:///home/conda/feedstock_root/build_artifacts/qudida_1651101164121/work
referencing @ file:///home/conda/feedstock_root/build_artifacts/referencing_1714619483868/work
requests==2.32.3
requests-oauthlib==2.0.0
rich @ file:///home/conda/feedstock_root/build_artifacts/rich_1728057819683/work/dist
rpds-py @ file:///D:/bld/rpds-py_1725327161963/work
rsa==4.9
ruamel.yaml @ file:///D:/bld/ruamel.yaml_1707298240950/work
ruamel.yaml.clib @ file:///D:/bld/ruamel.yaml.clib_1707314694548/work
scikit-image @ file:///D:/bld/scikit-image_1723842305941/work
scikit-learn @ file:///D:/bld/scikit-learn_1726082855864/work/dist/scikit_learn-1.5.2-cp310-cp310-win_amd64.whl#sha256=f8a2b0c9a97f54c5c064931b1b27e9443957ccd063bc8c437288596a61b2bd5d
scipy @ file:///C:/bld/scipy-split_1724327194933/work/dist/scipy-1.14.1-cp310-cp310-win_amd64.whl#sha256=e6bdc831fca55b320340d9d65555e680654f474b6605e599cfd60b4c00f7d5d6
seaborn @ file:///home/conda/feedstock_root/build_artifacts/seaborn-split_1714494649443/work
segmentation-models==1.0.1
shiboken6==6.7.3
six @ file:///home/conda/feedstock_root/build_artifacts/six_1620240208055/work
sleap==1.4.1a3
statsmodels @ file:///D:/bld/statsmodels_1727986825006/work
tensorboard==2.9.1
tensorboard-data-server==0.6.1
tensorboard-plugin-wit==1.8.1
tensorflow==2.9.2
tensorflow-estimator==2.9.0
tensorflow-hub @ file:///home/conda/feedstock_root/build_artifacts/tensorflow-hub_1618768305670/work/wheel_dir/tensorflow_hub-0.12.0-py2.py3-none-any.whl
tensorflow-io-gcs-filesystem==0.31.0
termcolor==2.4.0
threadpoolctl @ file:///home/conda/feedstock_root/build_artifacts/threadpoolctl_1714400101435/work
tifffile @ file:///home/conda/feedstock_root/build_artifacts/tifffile_1727250434425/work
tomli @ file:///home/conda/feedstock_root/build_artifacts/tomli_1727974628237/work
tornado==6.2
typing_extensions @ file:///home/conda/feedstock_root/build_artifacts/typing_extensions_1717802530399/work
tzdata @ file:///home/conda/feedstock_root/build_artifacts/python-tzdata_1727140567071/work
tzlocal==5.2
unicodedata2 @ file:///D:/bld/unicodedata2_1695848155043/work
urllib3==2.2.3
Werkzeug==3.0.4
wrapt==1.16.0
zipp @ file:///home/conda/feedstock_root/build_artifacts/zipp_1726248574750/work

mamba list

(sleap_1.4.1a3_py310) λ mamba list
# packages in environment at C:\Users\TalmoLab\mambaforge\envs\sleap_1.4.1a3_py310:
#
# Name                    Version                   Build  Channel
_libavif_api              1.1.1                h57928b3_1    conda-forge
absl-py                   2.1.0                    pypi_0    pypi
albucore                  0.0.16             pyhd8ed1ab_0    conda-forge
albumentations            1.4.15             pyhd8ed1ab_0    conda-forge
annotated-types           0.7.0              pyhd8ed1ab_0    conda-forge
aom                       3.9.1                he0c23c2_0    conda-forge
astunparse                1.6.3                    pypi_0    pypi
attrs                     24.2.0             pyh71513ae_0    conda-forge
blosc                     1.21.6               h85f69ea_0    conda-forge
brotli                    1.1.0                h2466b09_2    conda-forge
brotli-bin                1.1.0                h2466b09_2    conda-forge
bzip2                     1.0.8                h2466b09_7    conda-forge
c-blosc2                  2.15.1               hb461149_0    conda-forge
ca-certificates           2024.8.30            h56e8100_0    conda-forge
cached-property           1.5.2                hd8ed1ab_1    conda-forge
cached_property           1.5.2              pyha770c72_1    conda-forge
cachetools                5.5.0                    pypi_0    pypi
cairo                     1.18.0               h32b962e_3    conda-forge
cattrs                    24.1.2             pyhd8ed1ab_0    conda-forge
certifi                   2024.8.30          pyhd8ed1ab_0    conda-forge
charls                    2.4.2                h1537add_0    conda-forge
charset-normalizer        3.3.2                    pypi_0    pypi
contourpy                 1.3.0           py310hc19bc0b_2    conda-forge
cuda-nvcc                 11.3.58              hb8d16a4_0    nvidia
cudatoolkit               11.3.1              hf2f0253_13    conda-forge
cudnn                     8.2.1.32             h754d62a_0    conda-forge
cycler                    0.12.1             pyhd8ed1ab_0    conda-forge
dav1d                     1.2.1                hcfcfb64_0    conda-forge
double-conversion         3.3.0                h63175ca_0    conda-forge
efficientnet              1.0.0                    pypi_0    pypi
eval-type-backport        0.2.0              pyhd8ed1ab_0    conda-forge
eval_type_backport        0.2.0              pyha770c72_0    conda-forge
exceptiongroup            1.2.2              pyhd8ed1ab_0    conda-forge
expat                     2.6.3                he0c23c2_0    conda-forge
ffmpeg                    6.1.2           gpl_h0820249_105    conda-forge
flatbuffers               1.12                     pypi_0    pypi
font-ttf-dejavu-sans-mono 2.37                 hab24e00_0    conda-forge
font-ttf-inconsolata      3.000                h77eed37_0    conda-forge
font-ttf-source-code-pro  2.038                h77eed37_0    conda-forge
font-ttf-ubuntu           0.83                 h77eed37_3    conda-forge
fontconfig                2.14.2               hbde0cde_0    conda-forge
fonts-conda-ecosystem     1                             0    conda-forge
fonts-conda-forge         1                             0    conda-forge
fonttools                 4.54.1          py310ha8f682b_0    conda-forge
freeglut                  3.2.2                he0c23c2_3    conda-forge
freetype                  2.12.1               hdaf720e_2    conda-forge
gast                      0.4.0                    pypi_0    pypi
giflib                    5.2.2                h64bf75a_0    conda-forge
google-auth               2.35.0                   pypi_0    pypi
google-auth-oauthlib      0.4.6                    pypi_0    pypi
google-pasta              0.2.0                    pypi_0    pypi
graphite2                 1.3.13            h63175ca_1003    conda-forge
grpcio                    1.66.2                   pypi_0    pypi
h5py                      3.11.0          nompi_py310h2b0be38_102    conda-forge
harfbuzz                  9.0.0                h2bedf89_1    conda-forge
hdf5                      1.14.3          nompi_h2b43c12_105    conda-forge
hdmf                      3.14.4             pyh2e8e312_0    conda-forge
icu                       75.1                 he0c23c2_0    conda-forge
idna                      3.10                     pypi_0    pypi
image-classifiers         1.0.0                    pypi_0    pypi
imagecodecs               2024.6.1        py310h0e2d205_5    conda-forge
imageio                   2.35.1             pyh12aca89_0    conda-forge
imageio-ffmpeg            0.5.1              pyhd8ed1ab_0    conda-forge
imath                     3.1.12               hbb528cf_0    conda-forge
imgstore                  0.2.9                    pypi_0    pypi
importlib-metadata        8.5.0              pyha770c72_0    conda-forge
importlib_metadata        8.5.0                hd8ed1ab_0    conda-forge
importlib_resources       6.4.5              pyhd8ed1ab_0    conda-forge
intel-openmp              2024.2.1          h57928b3_1083    conda-forge
jasper                    4.2.4                hcb1a123_0    conda-forge
joblib                    1.4.2              pyhd8ed1ab_0    conda-forge
jsmin                     3.0.1              pyhd8ed1ab_0    conda-forge
jsonpickle                1.4.1              pyh9f0ad1d_0    conda-forge
jsonschema                4.23.0             pyhd8ed1ab_0    conda-forge
jsonschema-specifications 2024.10.1          pyhd8ed1ab_0    conda-forge
jxrlib                    1.1                  hcfcfb64_3    conda-forge
keras                     2.9.0                    pypi_0    pypi
keras-applications        1.0.8                    pypi_0    pypi
keras-preprocessing       1.1.2                    pypi_0    pypi
khronos-opencl-icd-loader 2024.05.08           hc70643c_0    conda-forge
kiwisolver                1.4.7           py310hc19bc0b_0    conda-forge
krb5                      1.21.3               hdf4eb48_0    conda-forge
lazy-loader               0.4                pyhd8ed1ab_1    conda-forge
lazy_loader               0.4                pyhd8ed1ab_1    conda-forge
lcms2                     2.16                 h67d730c_0    conda-forge
lerc                      4.0.0                h63175ca_0    conda-forge
libabseil                 20240722.0      cxx17_he0c23c2_1    conda-forge
libaec                    1.1.3                h63175ca_0    conda-forge
libasprintf               0.22.5               h5728263_3    conda-forge
libavif16                 1.1.1                h4e96d62_1    conda-forge
libblas                   3.9.0              24_win64_mkl    conda-forge
libbrotlicommon           1.1.0                h2466b09_2    conda-forge
libbrotlidec              1.1.0                h2466b09_2    conda-forge
libbrotlienc              1.1.0                h2466b09_2    conda-forge
libcblas                  3.9.0              24_win64_mkl    conda-forge
libclang                  18.1.1                   pypi_0    pypi
libclang13                19.1.1          default_ha5278ca_0    conda-forge
libcurl                   8.10.1               h1ee3ff0_0    conda-forge
libdeflate                1.22                 h2466b09_0    conda-forge
libexpat                  2.6.3                he0c23c2_0    conda-forge
libffi                    3.4.2                h8ffe710_5    conda-forge
libgettextpo              0.22.5               h5728263_3    conda-forge
libglib                   2.82.1               h7025463_0    conda-forge
libhwloc                  2.11.1          default_h8125262_1000    conda-forge
libiconv                  1.17                 hcfcfb64_2    conda-forge
libintl                   0.22.5               h5728263_3    conda-forge
libjpeg-turbo             3.0.0                hcfcfb64_1    conda-forge
liblapack                 3.9.0              24_win64_mkl    conda-forge
liblapacke                3.9.0              24_win64_mkl    conda-forge
libopencv                 4.10.0          qt6_py310h00b716a_607    conda-forge
libopenvino               2024.4.0             hfe1841e_1    conda-forge
libopenvino-auto-batch-plugin 2024.4.0             h04f32e0_1    conda-forge
libopenvino-auto-plugin   2024.4.0             h04f32e0_1    conda-forge
libopenvino-hetero-plugin 2024.4.0             h372dad0_1    conda-forge
libopenvino-intel-cpu-plugin 2024.4.0             hfe1841e_1    conda-forge
libopenvino-intel-gpu-plugin 2024.4.0             hfe1841e_1    conda-forge
libopenvino-ir-frontend   2024.4.0             h372dad0_1    conda-forge
libopenvino-onnx-frontend 2024.4.0             h5707d70_1    conda-forge
libopenvino-paddle-frontend 2024.4.0             h5707d70_1    conda-forge
libopenvino-pytorch-frontend 2024.4.0             he0c23c2_1    conda-forge
libopenvino-tensorflow-frontend 2024.4.0             hf4e5e90_1    conda-forge
libopenvino-tensorflow-lite-frontend 2024.4.0             he0c23c2_1    conda-forge
libopus                   1.3.1                h8ffe710_1    conda-forge
libpng                    1.6.44               h3ca93ac_0    conda-forge
libprotobuf               5.27.5               hcaed137_2    conda-forge
libsodium                 1.0.20               hc70643c_0    conda-forge
libsqlite                 3.46.1               h2466b09_0    conda-forge
libssh2                   1.11.0               h7dfc565_0    conda-forge
libtiff                   4.7.0                hfc51747_1    conda-forge
libwebp-base              1.4.0                hcfcfb64_0    conda-forge
libxcb                    1.16                 h013a479_1    conda-forge
libxml2                   2.12.7               h0f24e4e_4    conda-forge
libxslt                   1.1.39               h3df6e99_0    conda-forge
libzlib                   1.3.1                h2466b09_2    conda-forge
libzopfli                 1.0.3                h0e60522_0    conda-forge
lz4-c                     1.9.4                hcfcfb64_0    conda-forge
m2w64-gcc-libgfortran     5.3.0                         6    conda-forge
m2w64-gcc-libs            5.3.0                         7    conda-forge
m2w64-gcc-libs-core       5.3.0                         7    conda-forge
m2w64-gmp                 6.1.0                         2    conda-forge
m2w64-libwinpthread-git   5.0.0.4634.697f757               2    conda-forge
markdown                  3.7                      pypi_0    pypi
markdown-it-py            3.0.0              pyhd8ed1ab_0    conda-forge
markupsafe                2.1.5                    pypi_0    pypi
matplotlib-base           3.9.2           py310h37e0a56_1    conda-forge
mdurl                     0.1.2              pyhd8ed1ab_0    conda-forge
mkl                       2024.1.0           h66d3029_694    conda-forge
msys2-conda-epoch         20160418                      1    conda-forge
munkres                   1.1.4              pyh9f0ad1d_0    conda-forge
ndx-pose                  0.1.1              pyhd8ed1ab_0    conda-forge
networkx                  3.3                pyhd8ed1ab_1    conda-forge
nixio                     1.5.3                    pypi_0    pypi
numpy                     1.26.4          py310hf667824_0    conda-forge
oauthlib                  3.2.2                    pypi_0    pypi
opencv                    4.10.0          qt6_py310h896e0ad_607    conda-forge
openexr                   3.2.2                h9aba623_2    conda-forge
openh264                  2.4.1                h63175ca_0    conda-forge
openjpeg                  2.5.2                h3d672ee_0    conda-forge
openssl                   3.3.2                h2466b09_0    conda-forge
opt-einsum                3.4.0                    pypi_0    pypi
packaging                 24.1               pyhd8ed1ab_0    conda-forge
pandas                    2.2.3           py310hb4db72f_1    conda-forge
patsy                     0.5.6              pyhd8ed1ab_0    conda-forge
pcre2                     10.44                h3d7b363_2    conda-forge
pillow                    10.4.0          py310h3e38d90_1    conda-forge
pip                       24.2               pyh8b19718_1    conda-forge
pixman                    0.43.4               h63175ca_0    conda-forge
pkgutil-resolve-name      1.3.10             pyhd8ed1ab_1    conda-forge
protobuf                  3.19.6                   pypi_0    pypi
psutil                    6.0.0           py310ha8f682b_1    conda-forge
pthread-stubs             0.4               hcd874cb_1001    conda-forge
pthreads-win32            2.9.1                h2466b09_4    conda-forge
pugixml                   1.14                 h63175ca_0    conda-forge
py-opencv                 4.10.0          qt6_py310h5f8bd55_607    conda-forge
pyasn1                    0.6.1                    pypi_0    pypi
pyasn1-modules            0.4.1                    pypi_0    pypi
pydantic                  2.9.2              pyhd8ed1ab_0    conda-forge
pydantic-core             2.23.4          py310hc226416_0    conda-forge
pygments                  2.18.0             pyhd8ed1ab_0    conda-forge
pykalman                  0.9.7              pyhd8ed1ab_0    conda-forge
pynwb                     2.8.2              pyh2e8e312_0    conda-forge
pyparsing                 3.1.4              pyhd8ed1ab_0    conda-forge
pyside6                   6.7.3           py310h60c6385_1    conda-forge
python                    3.10.15         hfaddaf0_1_cpython    conda-forge
python-dateutil           2.9.0              pyhd8ed1ab_0    conda-forge
python-rapidjson          1.20            py310h9e98ed7_0    conda-forge
python-tzdata             2024.2             pyhd8ed1ab_0    conda-forge
python_abi                3.10                    5_cp310    conda-forge
pytz                      2024.1             pyhd8ed1ab_0    conda-forge
pywavelets                1.7.0           py310hb0944cc_1    conda-forge
pyyaml                    6.0.2           py310ha8f682b_1    conda-forge
pyzmq                     26.2.0          py310h656833d_2    conda-forge
qhull                     2020.2               hc790b64_5    conda-forge
qimage2ndarray            1.10.0                   pypi_0    pypi
qt6-main                  6.7.3                hfb098fa_1    conda-forge
qtpy                      2.4.1              pyhd8ed1ab_0    conda-forge
qudida                    0.0.4              pyhd8ed1ab_0    conda-forge
rav1e                     0.6.6                h975169c_2    conda-forge
referencing               0.35.1             pyhd8ed1ab_0    conda-forge
requests                  2.32.3                   pypi_0    pypi
requests-oauthlib         2.0.0                    pypi_0    pypi
rich                      13.9.2             pyhd8ed1ab_0    conda-forge
rpds-py                   0.20.0          py310hc226416_1    conda-forge
ruamel.yaml               0.18.6          py310h8d17308_0    conda-forge
ruamel.yaml.clib          0.2.8           py310h8d17308_0    conda-forge
scikit-image              0.24.0          py310hb4db72f_2    conda-forge
scikit-learn              1.5.2           py310hf2a6c47_1    conda-forge
scipy                     1.14.1          py310h46043a1_0    conda-forge
seaborn                   0.13.2               hd8ed1ab_2    conda-forge
seaborn-base              0.13.2             pyhd8ed1ab_2    conda-forge
segmentation-models       1.0.1                    pypi_0    pypi
setuptools                75.1.0             pyhd8ed1ab_0    conda-forge
six                       1.16.0             pyh6c4a22f_0    conda-forge
sleap                     1.4.1a3                  pypi_0    pypi
snappy                    1.2.1                h23299a8_0    conda-forge
statsmodels               0.14.4          py310hb0944cc_0    conda-forge
svt-av1                   2.2.1                he0c23c2_0    conda-forge
tbb                       2021.13.0            hc790b64_0    conda-forge
tensorboard               2.9.1                    pypi_0    pypi
tensorboard-data-server   0.6.1                    pypi_0    pypi
tensorboard-plugin-wit    1.8.1                    pypi_0    pypi
tensorflow                2.9.2                    pypi_0    pypi
tensorflow-estimator      2.9.0                    pypi_0    pypi
tensorflow-hub            0.12.0             pyhca92ed8_0    conda-forge
tensorflow-io-gcs-filesystem 0.31.0                   pypi_0    pypi
termcolor                 2.4.0                    pypi_0    pypi
threadpoolctl             3.5.0              pyhc1e730c_0    conda-forge
tifffile                  2024.9.20          pyhd8ed1ab_0    conda-forge
tk                        8.6.13               h5226925_1    conda-forge
tomli                     2.0.2              pyhd8ed1ab_0    conda-forge
typing-extensions         4.12.2               hd8ed1ab_0    conda-forge
typing_extensions         4.12.2             pyha770c72_0    conda-forge
tzdata                    2024b                hc8b5060_0    conda-forge
tzlocal                   5.2                      pypi_0    pypi
ucrt                      10.0.22621.0         h57928b3_1    conda-forge
unicodedata2              15.1.0          py310h8d17308_0    conda-forge
urllib3                   2.2.3                    pypi_0    pypi
vc                        14.3                h8a93ad2_22    conda-forge
vc14_runtime              14.40.33810         hcc2c482_22    conda-forge
vs2015_runtime            14.40.33810         h3bf8584_22    conda-forge
werkzeug                  3.0.4                    pypi_0    pypi
wheel                     0.44.0             pyhd8ed1ab_0    conda-forge
wrapt                     1.16.0                   pypi_0    pypi
x264                      1!164.3095           h8ffe710_2    conda-forge
x265                      3.5                  h2d74725_3    conda-forge
xorg-libxau               1.0.11               hcd874cb_0    conda-forge
xorg-libxdmcp             1.1.3                hcd874cb_0    conda-forge
xz                        5.2.6                h8d14728_0    conda-forge
yaml                      0.2.5                h8ffe710_2    conda-forge
zeromq                    4.3.5                ha9f60a1_6    conda-forge
zfp                       1.0.1                he0c23c2_2    conda-forge
zipp                      3.20.2             pyhd8ed1ab_0    conda-forge
zlib                      1.3.1                h2466b09_2    conda-forge
zlib-ng                   2.2.2                he0c23c2_0    conda-forge
zstd                      1.5.6                h0ea2cb4_0    conda-forge

[roomrys]

Template (manual test)

Backwards compatibility

pip freeze

mamba list

Training/Inference via GUI

Training Dialog

sleap-label

Installation

pip freeze

mamba list

[roomrys]

Mac (manual test)

Fails on training with leaked semaphore:

(sleap_1.4) liezlmaree:~$sleap-label
/Users/liezlmaree/micromamba/envs/sleap_1.4.1a3_py310/lib/python3.10/site-packages/albumentations/__init__.py:13: UserWarning: A new version of Albumentations is available: 1.4.18 (you have 1.4.15). Upgrade using: pip install -U albumentations. To disable automatic update checks, set the environment variable NO_ALBUMENTATIONS_UPDATE to 1.
  check_for_updates()
Saving config: /Users/liezlmaree/.sleap/1.4.1a3/preferences.yaml
qt.qpa.drawing: Layer-backing is always enabled.  QT_MAC_WANTS_LAYER/_q_mac_wantsLayer has no effect.

Software versions:
SLEAP: 1.4.1a3
TensorFlow: 2.12.0
Numpy: 1.26.4
Python: 3.10.15
OS: macOS-13.5-arm64-arm-64bit

Happy SLEAPing! :)
qt.qpa.drawing: Layer-backing is always enabled.  QT_MAC_WANTS_LAYER/_q_mac_wantsLayer has no effect.
2024-10-10 11:24:20.787 python[190:5585487] +[CATransaction synchronize] called within transaction
While structuring TrainingJobConfig (1 sub-exception)
While structuring TrainingJobConfig (1 sub-exception)
While structuring TrainingJobConfig (1 sub-exception)
While structuring TrainingJobConfig (1 sub-exception)
While structuring TrainingJobConfig (1 sub-exception)
While structuring TrainingJobConfig (1 sub-exception)
While structuring TrainingJobConfig (1 sub-exception)
While structuring TrainingJobConfig (1 sub-exception)
While structuring TrainingJobConfig (1 sub-exception)
While structuring TrainingJobConfig (1 sub-exception)
While structuring TrainingJobConfig (1 sub-exception)
While structuring TrainingJobConfig (1 sub-exception)
While structuring TrainingJobConfig (1 sub-exception)
While structuring TrainingJobConfig (1 sub-exception)
While structuring TrainingJobConfig (1 sub-exception)
While structuring TrainingJobConfig (1 sub-exception)
While structuring TrainingJobConfig (1 sub-exception)
While structuring TrainingJobConfig (1 sub-exception)
While structuring TrainingJobConfig (1 sub-exception)
While structuring TrainingJobConfig (1 sub-exception)
While structuring TrainingJobConfig (1 sub-exception)
While structuring TrainingJobConfig (1 sub-exception)
While structuring TrainingJobConfig (1 sub-exception)
While structuring TrainingJobConfig (1 sub-exception)
qt.qpa.drawing: Layer-backing is always enabled.  QT_MAC_WANTS_LAYER/_q_mac_wantsLayer has no effect.
qt.qpa.drawing: Layer-backing is always enabled.  QT_MAC_WANTS_LAYER/_q_mac_wantsLayer has no effect.
Resetting monitor window.
Polling: /Users/liezlmaree/Projects/sleap-datasets/drosophila-melanogaster-courtship/models/241010_112503.centroid.n=103/viz/validation.*.png
Start training centroid...
['sleap-train', '/var/folders/64/rjln6zpx7tlgwf8cqgvhm7fr0000gn/T/tmpdgvwrl6b/241010_112503_training_job.json', '/Users/liezlmaree/Projects/sleap-datasets/drosophila-melanogaster-courtship/courtship_labels.slp', '--zmq', '--controller_port', '9000', '--publish_port', '9001', '--save_viz']
/Users/liezlmaree/micromamba/envs/sleap_1.4.1a3_py310/lib/python3.10/site-packages/albumentations/__init__.py:13: UserWarning: A new version of Albumentations is available: 1.4.18 (you have 1.4.15). Upgrade using: pip install -U albumentations. To disable automatic update checks, set the environment variable NO_ALBUMENTATIONS_UPDATE to 1.
  check_for_updates()
INFO:sleap.nn.training:Versions:
SLEAP: 1.4.1a3
TensorFlow: 2.12.0
Numpy: 1.26.4
Python: 3.10.15
OS: macOS-13.5-arm64-arm-64bit
INFO:sleap.nn.training:Training labels file: /Users/liezlmaree/Projects/sleap-datasets/drosophila-melanogaster-courtship/courtship_labels.slp
INFO:sleap.nn.training:Training profile: /var/folders/64/rjln6zpx7tlgwf8cqgvhm7fr0000gn/T/tmpdgvwrl6b/241010_112503_training_job.json
INFO:sleap.nn.training:
INFO:sleap.nn.training:Arguments:
INFO:sleap.nn.training:{
    "training_job_path": "/var/folders/64/rjln6zpx7tlgwf8cqgvhm7fr0000gn/T/tmpdgvwrl6b/241010_112503_training_job.json",
    "labels_path": "/Users/liezlmaree/Projects/sleap-datasets/drosophila-melanogaster-courtship/courtship_labels.slp",
    "video_paths": [
        ""
    ],
    "val_labels": null,
    "test_labels": null,
    "base_checkpoint": null,
    "tensorboard": false,
    "save_viz": true,
    "keep_viz": false,
    "zmq": true,
    "publish_port": 9001,
    "controller_port": 9000,
    "run_name": "",
    "prefix": "",
    "suffix": "",
    "cpu": false,
    "first_gpu": false,
    "last_gpu": false,
    "gpu": "auto"
}
INFO:sleap.nn.training:
INFO:sleap.nn.training:Training job:
INFO:sleap.nn.training:{
    "data": {
        "labels": {
            "training_labels": "/Users/liezlmaree/Projects/sleap-datasets/drosophila-melanogaster-courtship/courtship_labels.slp",
            "validation_labels": null,
            "validation_fraction": 0.1,
            "test_labels": null,
            "split_by_inds": false,
            "training_inds": [
                50,
                95,
                44,
                90,
                20,
                4,
                47,
                6,
                58,
                13,
                33,
                40,
                57,
                8,
                56,
                7,
                80,
                12,
                28,
                91,
                16,
                36,
                11,
                52,
                100,
                76,
                23,
                84,
                5,
                75,
                81,
                87,
                2,
                19,
                51,
                27,
                43,
                39,
                72,
                24,
                89,
                35,
                34,
                25,
                29,
                30,
                102,
                32,
                98,
                26,
                60,
                55,
                65,
                77,
                73,
                97,
                59,
                17,
                15,
                83,
                64,
                62,
                86,
                82,
                74,
                21,
                9,
                61,
                54,
                1,
                31,
                10,
                99,
                22,
                70,
                48,
                14,
                101,
                18,
                92,
                88,
                3,
                42,
                46,
                94,
                67,
                93,
                78,
                37,
                68,
                79,
                45,
                49
            ],
            "validation_inds": [
                69,
                66,
                0,
                71,
                85,
                53,
                38,
                63,
                96,
                41
            ],
            "test_inds": null,
            "search_path_hints": [
                "",
                "",
                "",
                "",
                "",
                ""
            ],
            "skeletons": []
        },
        "preprocessing": {
            "ensure_rgb": false,
            "ensure_grayscale": false,
            "imagenet_mode": null,
            "input_scaling": 0.5,
            "pad_to_stride": 16,
            "resize_and_pad_to_target": true,
            "target_height": 1024,
            "target_width": 1024
        },
        "instance_cropping": {
            "center_on_part": "thorax",
            "crop_size": null,
            "crop_size_detection_padding": 16
        }
    },
    "model": {
        "backbone": {
            "leap": null,
            "unet": {
                "stem_stride": null,
                "max_stride": 16,
                "output_stride": 2,
                "filters": 16,
                "filters_rate": 2.0,
                "middle_block": true,
                "up_interpolate": true,
                "stacks": 1
            },
            "hourglass": null,
            "resnet": null,
            "pretrained_encoder": null
        },
        "heads": {
            "single_instance": null,
            "centroid": {
                "anchor_part": "thorax",
                "sigma": 2.5,
                "output_stride": 2,
                "loss_weight": 1.0,
                "offset_refinement": false
            },
            "centered_instance": null,
            "multi_instance": null,
            "multi_class_bottomup": null,
            "multi_class_topdown": null
        },
        "base_checkpoint": null
    },
    "optimization": {
        "preload_data": true,
        "augmentation_config": {
            "rotate": true,
            "rotation_min_angle": -180.0,
            "rotation_max_angle": 180.0,
            "translate": false,
            "translate_min": -5,
            "translate_max": 5,
            "scale": false,
            "scale_min": 0.9,
            "scale_max": 1.1,
            "uniform_noise": false,
            "uniform_noise_min_val": 0.0,
            "uniform_noise_max_val": 10.0,
            "gaussian_noise": false,
            "gaussian_noise_mean": 5.0,
            "gaussian_noise_stddev": 1.0,
            "contrast": false,
            "contrast_min_gamma": 0.5,
            "contrast_max_gamma": 2.0,
            "brightness": false,
            "brightness_min_val": 0.0,
            "brightness_max_val": 10.0,
            "random_crop": false,
            "random_crop_height": 256,
            "random_crop_width": 256,
            "random_flip": false,
            "flip_horizontal": false
        },
        "online_shuffling": true,
        "shuffle_buffer_size": 128,
        "prefetch": true,
        "batch_size": 4,
        "batches_per_epoch": 200,
        "min_batches_per_epoch": 200,
        "val_batches_per_epoch": 10,
        "min_val_batches_per_epoch": 10,
        "epochs": 2,
        "optimizer": "adam",
        "initial_learning_rate": 0.0001,
        "learning_rate_schedule": {
            "reduce_on_plateau": true,
            "reduction_factor": 0.5,
            "plateau_min_delta": 1e-06,
            "plateau_patience": 5,
            "plateau_cooldown": 3,
            "min_learning_rate": 1e-08
        },
        "hard_keypoint_mining": {
            "online_mining": false,
            "hard_to_easy_ratio": 2.0,
            "min_hard_keypoints": 2,
            "max_hard_keypoints": null,
            "loss_scale": 5.0
        },
        "early_stopping": {
            "stop_training_on_plateau": true,
            "plateau_min_delta": 1e-08,
            "plateau_patience": 20
        }
    },
    "outputs": {
        "save_outputs": true,
        "run_name": "241010_112503.centroid.n=103",
        "run_name_prefix": "",
        "run_name_suffix": "",
        "runs_folder": "/Users/liezlmaree/Projects/sleap-datasets/drosophila-melanogaster-courtship/models",
        "tags": [
            ""
        ],
        "save_visualizations": true,
        "keep_viz_images": false,
        "zip_outputs": false,
        "log_to_csv": true,
        "checkpointing": {
            "initial_model": false,
            "best_model": true,
            "every_epoch": false,
            "latest_model": false,
            "final_model": false
        },
        "tensorboard": {
            "write_logs": false,
            "loss_frequency": "epoch",
            "architecture_graph": false,
            "profile_graph": false,
            "visualizations": true
        },
        "zmq": {
            "subscribe_to_controller": true,
            "controller_address": "tcp://127.0.0.1:9000",
            "controller_polling_timeout": 10,
            "publish_updates": true,
            "publish_address": "tcp://127.0.0.1:9001"
        }
    },
    "name": "",
    "description": "",
    "sleap_version": "1.3.4",
    "filename": "/var/folders/64/rjln6zpx7tlgwf8cqgvhm7fr0000gn/T/tmpdgvwrl6b/241010_112503_training_job.json"
}
INFO:sleap.nn.training:
INFO:sleap.nn.training:Failed to query GPU memory from nvidia-smi. Defaulting to first GPU.
INFO:sleap.nn.training:Using GPU 0 for acceleration.
INFO:sleap.nn.training:Disabled GPU memory pre-allocation.
INFO:sleap.nn.training:System:
GPUs: 1/1 available
  Device: /physical_device:GPU:0
         Available: True
       Initialized: False
     Memory growth: True
INFO:sleap.nn.training:
INFO:sleap.nn.training:Initializing trainer...
INFO:sleap.nn.training:Loading training labels from: /Users/liezlmaree/Projects/sleap-datasets/drosophila-melanogaster-courtship/courtship_labels.slp
INFO:sleap.nn.training:Creating training and validation splits from validation fraction: 0.1
INFO:sleap.nn.training:  Splits: Training = 93 / Validation = 10.
INFO:sleap.nn.training:Setting up for training...
INFO:sleap.nn.training:Setting up pipeline builders...
INFO:sleap.nn.training:Setting up model...
INFO:sleap.nn.training:Building test pipeline...
2024-10-10 11:25:11.493519: W tensorflow/tsl/platform/profile_utils/cpu_utils.cc:128] Failed to get CPU frequency: 0 Hz
INFO:sleap.nn.training:Loaded test example. [0.964s]
INFO:sleap.nn.training:  Input shape: (512, 512, 1)
INFO:sleap.nn.training:Created Keras model.
INFO:sleap.nn.training:  Backbone: UNet(stacks=1, filters=16, filters_rate=2.0, kernel_size=3, stem_kernel_size=7, convs_per_block=2, stem_blocks=0, down_blocks=4, middle_block=True, up_blocks=3, up_interpolate=True, block_contraction=False)
INFO:sleap.nn.training:  Max stride: 16
INFO:sleap.nn.training:  Parameters: 1,953,105
INFO:sleap.nn.training:  Heads: 
INFO:sleap.nn.training:    [0] = CentroidConfmapsHead(anchor_part='thorax', sigma=2.5, output_stride=2, loss_weight=1.0)
INFO:sleap.nn.training:  Outputs: 
INFO:sleap.nn.training:    [0] = KerasTensor(type_spec=TensorSpec(shape=(None, 256, 256, 1), dtype=tf.float32, name=None), name='CentroidConfmapsHead/BiasAdd:0', description="created by layer 'CentroidConfmapsHead'")
INFO:sleap.nn.training:Training from scratch
INFO:sleap.nn.training:Setting up data pipelines...
INFO:sleap.nn.training:Training set: n = 93
INFO:sleap.nn.training:Validation set: n = 10
INFO:sleap.nn.training:Setting up optimization...
WARNING:absl:At this time, the v2.11+ optimizer `tf.keras.optimizers.Adam` runs slowly on M1/M2 Macs, please use the legacy Keras optimizer instead, located at `tf.keras.optimizers.legacy.Adam`.
INFO:sleap.nn.training:  Learning rate schedule: LearningRateScheduleConfig(reduce_on_plateau=True, reduction_factor=0.5, plateau_min_delta=1e-06, plateau_patience=5, plateau_cooldown=3, min_learning_rate=1e-08)
INFO:sleap.nn.training:  Early stopping: EarlyStoppingConfig(stop_training_on_plateau=True, plateau_min_delta=1e-08, plateau_patience=20)
WARNING:absl:There is a known slowdown when using v2.11+ Keras optimizers on M1/M2 Macs. Falling back to the legacy Keras optimizer, i.e., `tf.keras.optimizers.legacy.Adam`.
INFO:sleap.nn.training:Setting up outputs...
INFO:sleap.nn.callbacks:Training controller subscribed to: tcp://127.0.0.1:9000 (topic: )
INFO:sleap.nn.training:  ZMQ controller subcribed to: tcp://127.0.0.1:9000
INFO:sleap.nn.callbacks:Progress reporter publishing on: tcp://127.0.0.1:9001 for: not_set
INFO:sleap.nn.training:  ZMQ progress reporter publish on: tcp://127.0.0.1:9001
INFO:sleap.nn.training:Created run path: /Users/liezlmaree/Projects/sleap-datasets/drosophila-melanogaster-courtship/models/241010_112503.centroid.n=103
INFO:sleap.nn.training:Setting up visualization...
/Users/liezlmaree/micromamba/envs/sleap_1.4.1a3_py310/lib/python3.10/multiprocessing/resource_tracker.py:224: UserWarning: resource_tracker: There appear to be 2 leaked semaphore objects to clean up at shutdown
  warnings.warn('resource_tracker: There appear to be %d '
Run Path: /Users/liezlmaree/Projects/sleap-datasets/drosophila-melanogaster-courtship/models/241010_112503.centroid.n=103
qt.qpa.drawing: Layer-backing is always enabled.  QT_MAC_WANTS_LAYER/_q_mac_wantsLayer has no effect.
Resetting monitor window.