SpatialDataWrapper v2 (#373)

* (refactor): separate out schema creation

* (feat): initial pass at spatial data

* (feat): working notebook minus spots (maybe a vitessce bug?)

* (fix): try to get spots to appear

* (chore): comment out other stuff to make widget show more

* Fix visium config

* (feat): firstt pass at `from_spatialdata_object`

* (refactor): refactor `from_spatialdata_object` and util funcs from static methods

* (feat): add url capability

* (temp): spatial data fixes

* (chore): change variable names

* (chore): fix notebooks

* (refactor): `path`->`elem`

* (chore): more clean ups and fixes

* (refactor): small name changes

* (refactor): revert public changes

* (fix): revert `.ipynb` files

* (chore): fix last final args

* (fix): name

* (fix): remove `request_init` completely and fix `setup.cfg`

* (fix): adapt to api

* (chore): remove erroneous notebook

* (fix): add type support

* (chore): add test

* (fix): automatic lint

* (fix): add default for `obsEmbedding`

* (fix): redefinition

* (fix): `path` and `embeddingType` swapped

* (fix): add annotattions to `utils.py`

* (chore): add `spatialdata` dep

* (fix): oops, comma for `toml`

* (fix): no more python3.8 bc of spatialdata

* (fix): remove whitespace

* (api): small api cleanups

* (fix): args bug

* (fix): try spatialdata from master

* (chore): lint

* (chore): try `spatialdata` update

* (fix): `image_elem` usage + table key

* (chore): update SpatialData

* (fix): `shapes_elem`

* (fix): capitalize name of obs set

* (fix): use table for spots

* (fix): capitalization

* (fix): first letter only

* (fix): check `self._coordination_values`

* (chore): add `obsSets` check

* SpatialData changes into original SpatialData branch (#374)

* Try again

* Fix bugs

* Lint

* Update

* Omit

* Update pyproject.toml

* Keller mark/spatial data2 (#375)

* Try again

* Fix bugs

* Lint

* Update

* Omit

* Add test

* JS version

---------

Co-authored-by: ilan-gold <ilanbassgold@gmail.com>

.coveragerc_omit

@@ -2,6 +2,7 @@
 omit =
     vitessce/config.py
     vitessce/export.py
+    vitessce/file_def_utils.py
     vitessce/routes.py
     vitessce/widget.py
     vitessce/wrappers.py

.github/workflows/test.yml

@@ -7,7 +7,7 @@ jobs:
     runs-on: ubuntu-latest
     strategy:
       matrix:
-        version: ['3.8', '3.12']
+        version: ['3.9', '3.12']
     steps:
       - uses: actions/checkout@v2
       - uses: actions/setup-python@v2

docs/notebooks/spatial_data.ipynb

@@ -0,0 +1,267 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {
+    "nbsphinx": "hidden"
+   },
+   "source": [
+    "# Vitessce Widget Tutorial"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# Visualization of SpatialData Object"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 1. Import dependencies\n",
+    "\n",
+    "We need to import the classes and functions that we will be using from the corresponding packages."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "%load_ext autoreload\n",
+    "%autoreload 2\n",
+    "\n",
+    "from pathlib import Path\n",
+    "from urllib.request import urlretrieve\n",
+    "import dask\n",
+    "\n",
+    "dask.config.set({'dataframe.query-planning-warning': False})\n",
+    "\n",
+    "from spatialdata import read_zarr\n",
+    "import scanpy as sc\n",
+    "\n",
+    "from vitessce import (\n",
+    "    VitessceConfig,\n",
+    "    Component as cm,\n",
+    "    CoordinationType as ct,\n",
+    "    CoordinationLevel as CL,\n",
+    "    AbstractWrapper,\n",
+    "    SpatialDataWrapper,\n",
+    "    get_initial_coordination_scope_prefix\n",
+    ")\n",
+    "from vitessce.data_utils import (\n",
+    "    optimize_adata,\n",
+    "    VAR_CHUNK_SIZE,\n",
+    ")\n",
+    "import zipfile\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "zip_filepath = Path(\"data/visium.spatialdata.zarr.zip\")\n",
+    "spatialdata_filepath = zip_filepath.with_suffix('')\n",
+    "if not zip_filepath.exists():\n",
+    "    spatialdata_filepath.parent.mkdir(exist_ok=True)\n",
+    "    urlretrieve('https://s3.embl.de/spatialdata/spatialdata-sandbox/visium_associated_xenium_io.zip', zip_filepath)\n",
+    "if not spatialdata_filepath.exists():\n",
+    "    with zipfile.ZipFile(zip_filepath,\"r\") as zip_ref:\n",
+    "        zip_ref.extractall(spatialdata_filepath.parent)\n",
+    "        (spatialdata_filepath.parent / \"data.zarr\").rename(spatialdata_filepath)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 3. Load the data\n",
+    "\n",
+    "Note: this function may print a `FutureWarning`"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "spatialdata = read_zarr(spatialdata_filepath)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "spatialdata"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 4. Create the Vitessce widget configuration\n",
+    "\n",
+    "Vitessce needs to know which pieces of data we are interested in visualizing, the visualization types we would like to use, and how we want to coordinate (or link) the views."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### 4.1. Instantiate a `VitessceConfig` object\n",
+    "\n",
+    "Use the `VitessceConfig` constructor to create an instance."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "vc = VitessceConfig(schema_version=\"1.0.16\", name='Visium SpatialData Demo (visium_associated_xenium_io)', description='From https://spatialdata.scverse.org/en/latest/tutorials/notebooks/datasets/README.html')"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### 4.2. Add a dataset to the `VitessceConfig` instance\n",
+    "\n",
+    "In Vitessce, a dataset is a container for one file per data type. The `.add_dataset(name)` method on the `vc` instance sets up and returns a new dataset instance.\n",
+    "\n",
+    "Then, we can call the dataset's `.add_object(wrapper_object)` method to attach a \"data wrapper\" instance to our new dataset. For example, the `AnnDataWrapper` helps to configure AnnData Zarr stores for use in the Vitessce configuration.\n",
+    "\n",
+    "Dataset wrapper classes may require additional parameters to resolve ambiguities. For instance, `AnnData` objects may store multiple clusterings or cell type annotation columns in the `adata.obs` dataframe. We can use the parameter `obs_set_paths` to tell Vitessce that certain columns of the `obs` dataframe correspond to cell type annotations or cell clusterings."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "[wrapper] = SpatialDataWrapper.from_object(\n",
+    "    spatialdata,\n",
+    "    table_keys_to_image_elems={\"table\": \"images/CytAssist_FFPE_Human_Breast_Cancer_full_image\"},\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "dataset = vc.add_dataset(name='Breast Cancer Visium').add_object(wrapper)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "spatial = vc.add_view(\"spatialBeta\", dataset=dataset)\n",
+    "feature_list = vc.add_view(cm.FEATURE_LIST, dataset=dataset)\n",
+    "layer_controller = vc.add_view(\"layerControllerBeta\", dataset=dataset)\n",
+    "vc.link_views_by_dict([spatial, layer_controller], {\n",
+    "    'imageLayer': CL([{\n",
+    "        'photometricInterpretation': 'RGB',\n",
+    "    }]),\n",
+    "}, scope_prefix=get_initial_coordination_scope_prefix(\"A\", \"image\"))\n",
+    "vc.link_views_by_dict([spatial, layer_controller], {\n",
+    "    'spotLayer': CL([{\n",
+    "        'obsType': 'spot',\n",
+    "    }]),\n",
+    "}, scope_prefix=get_initial_coordination_scope_prefix(\"A\", \"obsSpots\"))\n",
+    "obs_sets = vc.add_view(cm.OBS_SETS, dataset=dataset)\n",
+    "vc.link_views([spatial, layer_controller, feature_list, obs_sets], ['obsType'], [wrapper.obs_type_label])"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### 4.4. Define the visualization layout\n",
+    "\n",
+    "The `vc.layout(view_concat)` method allows us to specify how our views will be arranged in the layout grid in the widget. The `|` and `/` characters are magic syntax for `hconcat(v1, v2)` and `vconcat(v1, v2)`, respectively."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "vc.layout(spatial | (feature_list / layer_controller / obs_sets));"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "scrolled": true
+   },
+   "outputs": [],
+   "source": [
+    "vw = vc.widget()\n",
+    "vw"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "vw.close()\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "vc.to_dict(\"\")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.9.0"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 4
+}

pyproject.toml

@@ -4,22 +4,21 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "vitessce"
-version = "3.4.0"
+version = "3.4.1"
 authors = [
   { name="Mark Keller", email="mark_keller@hms.harvard.edu" },
 ]
 description = "Jupyter widget facilitating interactive visualization of spatial single-cell data with Vitessce"
 readme = "README.md"
 license = {file = "LICENSE"}
-requires-python = ">=3.7"
+requires-python = ">=3.9"
 keywords = ["ipython", "jupyter", "widgets"]
 classifiers = [
   'Development Status :: 4 - Beta',
   'Framework :: IPython',
   'Intended Audience :: Developers',
   'Intended Audience :: Science/Research',
   'Topic :: Multimedia :: Graphics',
-  'Programming Language :: Python :: 3.8',
   'Programming Language :: Python :: 3.9',
   'Programming Language :: Python :: 3.10',
   'Programming Language :: Python :: 3.11',
@@ -34,8 +33,9 @@ dependencies = [
   'black>=21.11b1',
   'numpy>=1.21.2,<2.0',
   'anndata>=0.7.8,<0.11',
+  'spatialdata>=0.2.2',
   'scanpy>=1.9.3',
-  'ome-zarr==0.8.3',
+  'ome-zarr>=0.8.3',
   'tifffile>=2020.10.1',
   'jsonschema>=3.2',
   'tqdm>=4.1.0'
@@ -81,7 +81,7 @@ all = [
   'starlette==0.14.0',
   'generate-tiff-offsets>=0.1.7',
   'kerchunk>=0.2.6',
-  'fsspec>=2023.12.2',
+  'fsspec',
 
   # aiofiles is not explicitly referenced in our code,
   # but it is an implicit dependency of starlette==0.14.0.

setup.cfg

@@ -8,11 +8,16 @@ per-file-ignores =
   vitessce/data_utils/__init__.py: F401
   vitessce/widget_plugins/__init__.py: F401
 ignore =
-  E501, # Ignore line too long
-  W605, # Ignore invalid escape sequence '\*'
-  W503, # Ignore line break before binary operator: Skim down the left edge to understand intent.
-  E127 # Ignore continuation line over-indented for visual indent
-  E128 # Ignore continuation line under-indented for visual indent
+  # Ignore line too long
+  E501,
+  # Ignore invalid escape sequence '\*'
+  W605,
+  # Ignore line break before binary operator: Skim down the left edge to understand intent.
+  W503,
+  # Ignore continuation line over-indented for visual indent
+  E127
+  # Ignore continuation line under-indented for visual indent
+  E128
 exclude =
   ./js/node_modules/,
   ./docs/notebooks/.ipynb_checkpoints/,