3D ideas

[kshannon]

Thoughts on maximizing 3d representation via interpolation w/o training a 3d model:

Slice inter-polation was introduced to mimic how radiologists integrate infor-mation from all adjacent images of a contiguous three-dimensional (3D) volume concurrently, rather than examine each single axial 2D slice in isolation. Interpolated images from adjacent slices were pro-vided to the model with a modified loss function during training to imitate the 3D integration of image interpretation by radiologists. Lee et. al. NATURE BIOMEDICAL ENGINEERING | VOL 3 | MARCH 2019 | 173–182 |

Though note this salient point from the same paper about strict 3d voxel based approaches:

Another approach to address inter-slice dependency is to build a 3D network that directly inputs the voxel data from the entire imaging volume into a 3D format rather than as pixel-data from discrete axial slices in a 2D format. To compare the 3D versus 2D approaches, we trained a 3D model using previously described methodology21 by using case-level labels aggregated from slice-level labels, as well as volume data with a standardized dimensionality (24 × 512 × 512 voxels) generated using 2D slices. The resulting 3D model, however, achieved a mAP of only 0.328 for the multi-label classification of our five ICH subtypes, which is substantially infe-rior to the mAP we obtained with our existing 2D model (mAP of 0.686). This finding is consistent with the ‘curse of dimensionality’ reported in a previous study24, which noted that the amount of data required to train a deep-learning model scales exponentially with the dimensionality of the data.