Image Processing: segmentation of cell and nucleus in 3D

For each gene-edited intracellular structure, the data we acquire comprises high resolution 3D z-stacks containing one gene-edited protein. We then place these structures into a frame of reference based on the location of the cell boundary and nucleus by labeling cells with fluorescent membrane and DNA dyes. The goal of our image processing algorithm pipeline is thus to segment out cells and nuclei (DNA) within them with reasonable accuracy in 3 dimensions.

Cell membrane | Mitochondria | DNA (nucleus) | Transmitted
CellMask Deep Red | Tom20-GFP | Hoechst | Light image

Figure 1. The segmentation process. Top row: representative live-cell 3D fluorescent images of TOMM20 mEGFP gene-edited WTC hiPS cells stained with CellMask Deep Red (10 min) to mark the plasma membrane (far left) and NucBlue Live (30 min) to mark the DNA (center right). Bottom row: resulting segmentation of cell boundaries (plasma membrane) and nuclei (DNA) in 3D using the approach shown below. A first pass segmentation of mitochondria is shown in the center left (see structure segmentation section below). Only cells entirely contained within the image are segmented.

Rotation of segmentation in 3D

​Example of segmentation in 3D ​

Figure 2. 3D reconstructions of cell and nuclear segmentation. 3D view of the segmentation of the cells shown above. Each cell is depicted in a different color; mitochondrial segmentation is depicted in green.

Cell and nuclear segmentation workflow

Figure 3. Overview of the segmentation workflow. Nuclei are segmented and used to create seeds for the segmentation of individual cells based on the signal from the plasma membrane. The cell segmentation is then used to improve nucleus segmentation by splitting adjacent nuclei. The plasma membrane signal is boosted at the top of the cells and fluorescent endocytotic vesicles are removed, which together with the improved nucleus segmentation are used to achieve final cell segmentation.

Figure 4. Validation of cell and nuclear segmentation. Segmentation results were overlaid with the fluorescent images in XY at four different Z positions throughout the cells (indicated by orange, dashed boxes/lines; different z positions within the cell are indicated by different line thicknesses) to confirm correct segmentation in XY. Segmentation results were also overlaid with the fluorescent images in YZ at four different X positions (indicated by blue, dashed boxes/lines; different positions in x are indicated by different line thicknesses) to confirm correct segmentation in YZ. The 4th column includes 3D maximum intensity projections of the segmented cells and 3D mean projections of the binary segmentation in Z (upper left), X (upper right) and Y (lower right) within these panels. Visualizations of the 3D segmentation were generated for all segmented cells. A graphic user interface (GUI) presented them in random order for scoring. Cells were considered sufficiently accurately segmented when they were useable for further visualization and modeling purposes by two independent scorers.​

Figure 5. Additional examples of segmented hiPS cells. Top example: hiPS cells can lean over each other such that the same cell cytoplasm can look like two separate cells in the middle of the z-stack but be clearly continuous at the top of the cell colony. This emphasizes the need for accurate 3D cell segmentation to identify the cell boundary correctly such that intracellular structures are properly assigned to one cell vs. another. Bottom example: the cell and nuclear segmentation pipeline is also successfully applicable to mitotic cells. In this case the “nuclear segmentation” captures the outline of the condensed chromosomes and more than one “nucleus” is permitted per cell.

​Image Processing: First-pass segmentation of intracellular structures in 3D

The goal of our ‘first-pass’ segmentation is to achieve an initial reasonable segmentation of each of the tagged intracellular structures for feature extraction, modeling, and visualization. This segmentation may do a very good job of capturing the general location and shape of structures in the cell, while it may not capture all of the morphological details of the structures e.g. the true connectivity of the mitochondrial or microtubular networks or the intricate sheets and tubules of the ER. For other structures, such as the interphase nucleus, the first-pass segmentation is already able to capture an accurate representation of the structure that may not improve dramatically with additional segmentation. Only structures for which this first-pass segmentation leads to reasonable results are included in the 3D cell viewer. Additional rounds of segmentation for improved accuracy are in progress.

Table of intracellular structures and methods used for first-pass segmentation

Tagged protein	Primary structure labeled	Algorithm
Lamin B1	Nuclear envelope	Level-set + watershed & ​ALT for mitotic cells
Tom 20	Mitochondria	ALT + MitoGraph
α-tubulin	Microtubules	MitoGraph
Fibrillarin	Nucleolus	ALT
Sec61-ß	Endoplasmic reticulum	ALT
Desmoplakin	Desmosomes	ALT
Z01	Tight junctions	ALT

Table References:

MitoGraph: Viana et al., (2015). Quantifying mitochondrial content in living cells. Methods in Cell Biology 125:77-93

Adaptive Local Thresholding (ALT): Peng et al., (2011). Adaptive image enhancement for fluorescence microscopy. Technologies and Applications of Artificial Intelligence (TAAI). DOI: 10.1109/TAAI.2010.13

Nucleus – Lamin B1 (nuclear envelope)

Mitochondria – Tom20

Microtubules – α​-tubulin

Nucleolus – Fibrillarin

Figure 6. Example First-pass segmentations for several structures. From left to right: Nucleus via lamin B1 revealing the nuclear envelope in this case, Mitochondria via Tom20, Microtubules via α​-tubulin, and Nucleoli via fibrillarin.