0.2.0

This release marks the complete rebuild of the client with React! It has feature parity with 0.1 release, adds features like more control over image display, and improves the usability and performance of the user interface.

🚀 Features

Rework DeepCell Label with React @tddough98 (#248)

This PR provides a complete rework of user interface for DeepCell Label. We use React to build the UI. This PR also build upon previous work with XState and integrates it with React so that state is managed by actors instantiated by XState and React components use hooks to access and render that state into UI elements.

We also include some changes to the backend to better serve the React-based frontend, namely changed

React components

We bootstrapped this React app with create-react-app, so this PR contains the tooling and dependencies installed by CRA.

The UI is made up of Controls on the left, and a Canvas that fills the available space to the right.

Canvas

The Canvas folder contains everything for the interactive canvas. The top-level Canvas component manages styling and event handling, the RawCanvas folder handles adjusting and showing the raw images, the LabeledCanvas folder handles overlaying the labels on the raw image, and the ToolCanvas folder has components to show tools like brush or threshold.

canvasUtils has functions to manipulate ImageData objects.

Controls

The ImageControls folder has the controls on the far left used that control what and how images are shown. There is a SubmitButton on top, then a FrameSlider, then the LabelControls, then the RawControls.

The LabelControls include a FeatureSelect, a OutlineToggle, and a OpacitySlider.
The RawControls include a ColorModeToggle to switch between viewing one or multiple channels, then either GrayscaleControls when viewing one channel or RGBControls when viewing multiple channels.
The GrayscaleControls have a switch to invert the channel and sliders for range, brightness, and contrast, and the RGBControls have a LayerController for each channel that controls the layer's color, its range, and whether the layer is shown.

The Toolbar folder has the controls just to the left of the canvas that manage how we edit the labels (like picking a tool, selecting labels.
The UndoRedoButtons are on top, then the Toolbar, then the ActionButtons (swap/replace), and finally a palette that shows if there is a foreground/background selected.

Other components

Other simple components include the Navbar, the Footer, and the Instructions.

State management with XState

To provide the state machines to components throught the React, we define a top level ServiceContext component that wraps the whole application. In this component with define a Context that provides the top level state machine to any child of ServiceContext. These children can access the state through hooks defined in this module. At a high level, we can image the React component tree and the XState actor hierarchy as two separate trees that touch each other through these hooks.

Components subscribe to some part of these state machines and rerender when the state updates.

Actor hierarchy

The root actor is still deepCellLabelMachine, which fetches initial project data, and then spawns canvas, image, tool, undo, and api.

canvas tracks where the canvas is within the image, such as zoom, position, and mouse
position. It also manages mouse events and forwards them to tool.

image tracks the current frame, and spawns labeled and raw. feature manages the current segmentation, and spawns a feature machine to fetch the data for each segmentation. raw manages the current raw image, and spawns a channel machine to fetch the data for each channel. raw also spawns two "color modes" grayscale and color and passes them all the channel actors. The grayscale color mode tracks which single channel is displayed in grayscale mode. The color color mode spawns a layer for each displayed channel and each layer manages its own image adjustment settings for that channel.

tool tracks the current tool we're using to interact with the labels. The tool will spawns an actor for the current tool that has custom behavior for mouse events sent from canvas. The tool actor also tracks the selected labels (the foreground and the background).

api handles requests to the Label backend that edit the labels. The events that can edit labels include EDIT, BACKEND_UNDO, and BACKEND_REDO. Once api gets a response from the backend, it sends a EDITED event to the labeled machine to reload the data that changed.

undo can manage any machine that responds to SAVE and RESTORE events. When undo receives an ADD_ACTOR event, it creates a history machine for that actor that will send a SAVE event to the actor when we edit the labeling, and the actor must respond with a RESTORE event that contains the state that the actor should restore on undo/redo. Later, when undo is undoing or redoing an edit, the history machine will send that RESTORE event back to the actor and the actor is responsible for restoring their state based on this event.

Hotkeys

Hotkeys are implemented as invoked callbacks, where the hotkey listeners are setup when entering the states/machines that use the hotkeys and are torn down on exit.

Project management

We've added prettier to the project, an opinionated code formatter than enforces consistent style. Prettier formats javascript, css, html and other browser focused formats, complementing out existing linting for python. We also use husky and lint-staged to add a pre-commit hook so any code committed to the repo will be formatted with prettier.

Typescript has been added as a dependency, but we have no types defined yet. Adding typescript now improves the intellisense for the project in VSCode and will make it easier to add types later.

Changes to Flask backend

The backend changes include:

• a new URL based approach to creating projects, which includes
  • loading raw assets and labeled assets separately
  • support for loading from ZIPs, where separate files in a zip are treated as different channels or features
  • reshaping input arrays based off an 'axes' string like 'ZYXC'
• routes to serve project data separately, rather than in a bundled JSON, including

Here is a table of the new routes, what data they serve, and the data format:

Route	data served	data format
/api/project	project metadata, like image dimensions	JSON
/api/raw	raw image slices	grayscale PNG
/api/labeled	labeled image slices	color PNG
/api/array	array of label at each pixel, with label borders in negative	compressed JSON
/api/instances	label metadata at the moment, only the frames each label is present in used on frontend to know which labels in each feature	JSON

Issues

@willgraf has reviewed outstanding issues and this PR closes these issues

Fixes #11
Fixes #52
Fixes #96
Fixes #97
Fixes #99
Fixes #102
Fixes #110
Fixes #115
Fixes #131
Fixes #132
Fixes #140
Fixes #155
Fixes #158
Fixes #169
Fixes #171
Fixes #203
Fixes #234
Fixes #244
Fixes #220

Add landing page @tddough98 (#268)

Here we add a landing page to DeepCell Label with an introduction to the app, a dropzone to upload TIFFs, NPZs and PNGs, and dropdown menu with two example files. The landing page is a more user-friendly entry point to using Label, giving users another option than the Label API to create projects.

Here's an overview of the changes:

• created a base Loader class and separate URLLoader and FileLoader implements
• the app now has two pages: / for landing page and /project for interacting with a project
  • this is a breaking change for Anolytics jobs as the we'll have to add /project to our job URLs
• the project page can show a download button instead of a submit button by adding a download=true URL parameter

There are also some smaller bug fixes and improvements included:

• loading PNGs is more flexible and robust, now handling palette-based and grayscale PNGs
• the watershed state machine no longer exits its clicked state immediately due to the state selecting the foreground label
• controls for white channels are now shown in black instead so they don't blend into the background

🐛 Bug Fixes

Fix tool keybinds @tddough98 (#265)

When using tool hotkeys to switch out the brush and threshold currently causes UI crashes as the tool changes before the BrushCanvas or ThresholdCanvas is removed, causing these canvases to access a brush machine or threshold machine that no longer exists.

Previously, we spawned one tool machine at a time and deleted the previous tool machine when switch between tools. With this PR, we spawn all of the tools and keep them in the segmentMachine and switch between these pre-existing tool. Instead of a useTool hook that provides the current tool, we have specific useBrush and useThreshold hooks that provide the persistent brush or threshold actors so these components don't crash even if the component is out of sync with the current tool.

This PR also includes a couple of fixes and improvements that I noticed while making these changes.

• refactoring switching between tools in segmentMachine
• fixing the flood tool to flood immediately instead of first selecting the background and then flooding
• fixing double clicks with the select tool to deselect the foreground because is was sending the wrong event
• changing machines to start with null in their context and sending events to initialize them to

Track selected labels for undo/redo @tddough98 (#264)

Selected labels were not changing when undoing and redoing. The segmentMachine used to manage the selected labels, so they were undone and redone through this machine, so it looks like we never added the selectMachine to the undo/redo history.

Update swap and replace hotkeys @tddough98 (#262)

Swap and replace can cause drastic changes to the labeling, in particular filling all the unlabeled areas when swapping or replacing the 0 label. Changing the hotkeys from S to Shift + S and R to Shift + R makes these actions harder to accidentally do.

Upload fix @tddough98 (#245)

These are hotfixes for submitting jobs from Anolytics to call the correct route and avoid finishing projects so that project URLs remaining usable for QC.

🧰 Maintenance

Drop node v10 and add v16 and v17 support @tddough98 (#269)

Our GitHub Actions workflow have been failing as some of our node dependencies have dropped support for node v10 and now expect version >= 12 as node v10 reached end of life earlier this year. We drop node v10 from our testing workflows and add the current versions of node v16 and v17.

Add new release-drafter GitHub Workflow. @willgraf (#257)

What

• Add the release-drafter GitHub Action workflow and template

Why

• Automatically draft the next release based on the labels of the changes.
• Saves a lot of time in building releases.
• Consistent style for all the DeepCell project releases.

📚️ Documentation

Update documentation @tddough98 (#266)

This updates the README for DeepCell Label to be up to date with the current master branch. It includes updates to the README and the in-app instructions.