3D representation for labelmap

[floryst]

Ideally this will be a polygonal representation, as mixed volumes in vtk.js is not yet supported.

[sankhesh]

If you're going for segmentation based on the label map, polygonal representation makes sense. IMO, just for displaying the label map, it would be better to have a volumetric representation.

[PaulHax]

Polygonal Representation

Segmentations as Surfaces:
https://discourse.slicer.org/t/volume-rendering-colorized-with-segmentation/26689/14

Combining Smoothed Polygon Segmentation with opacity set to .4 and simultaneous volume rendering
https://discourse.slicer.org/t/volume-rendering-colorized-with-segmentation/26689/21

VolView implementation:

• Use a Webworker to generate surfaces from the segmentations.
• On paint tool use, would need to update surfaces
• Wire Show/hide of segmentations to 3D surfaces.
• Try to fix blending of transparent surfaces with volume rendering.

Volume Rendering of Segmentations

Weighted components

• Currently supported by vtk.js
• Limitation: If the “background” component is low opacity, the segmentation component will be low opacity too

Volume Blending

When this is merged to VTK.js main

• Add ImageResliceMapper multi component using multiple image data vtk-js#3137

https://docs.google.com/presentation/d/19M3GuPksSxZxaEKBpxPxVrm36ZpkCBCKwN3NuQtQHNI/edit#slide=id.p

Possible Future: Colorize Volume Module

generating a full-color RGBA volume, by combining a segmentation and a scalar volume using some smart filtering and masking.

https://discourse.slicer.org/t/vtk-multivolume-cinematic-volume-rendering/31981/12
https://discourse.slicer.org/t/new-colorize-volume-module/32254
https://www.kitware.com/screen-space-ambient-occlusion-for-volumes/

[sankhesh]

For volume rendering, I suggest an alternate approach that allows you to specify material properties and transfer functions per label. This method would support all the different blend modes without requiring special cases for multi-component handling as described above.

[PaulHax]

i suggest an alternate approach that allows you to specify material properties and transfer functions per label.

Sounds nice. Would you do that with this PR's "additive" preset?

• Add custom color mix functions for volume mapper fragment shader vtk-js#3029 (comment)

[sankhesh]

Nope. Ideally, there will be a separate label volume that would give each voxel a label and the mapper would then index into the label map to figure out the current sample color and alpha. This approach could be used in addition to whatever blending function is employed for multiple components.