Images of rabbit nuclei were denoised with a median filter and a Gaussian filter. They were subsequently segmented through two different pathways.
HP1β images (embryos), on which several nuclei are present, were analyzed with the Insight Toolkit (ITK) library. The robust automatic threshold selection method (RATS) [100] was used to compute a threshold to ‘binarize’ the HP1β images. The threshold is computed as the mean of the intensity values in the HP1β image weighted by their Gaussian gradient magnitude. To avoid the high gradient values in the nucleus caused by non-homogeneous content, the small bright and dark zones were removed with a 3D area opening and a grayscale fill hole transformation before computing the gradient. The joined masks of nuclei were separated using a watershed transform on the distance map. Truncated nuclei at the image border as well as objects smaller than 200 µm3 were removed.
A semi-automated procedure was developed to segment mammary gland nuclei from the DAPI image. DAPI signal was denoised with a median and a Gaussian filter, and manually thresholded to produce partial nuclear masks. The DAPI signal was mostly present on the border of the nuclei. As a result, thresholding this signal results in an incomplete nucleus, in which the center is not filled and the border is not continuous. The nuclear borders were thus closed with a morphological closing transform with a large round kernel, and content of the nuclei was filled with a binary hole filling transform. The masks of the different nuclei were then separated by a watershed transform on the distance map, and the nuclei from the cell types of interest were manually selected.
Confocal image stacks of A. thaliana nuclei were processed and analyzed with programs developed using the Free-D software libraries [67] (link). Each image stack contained a single nucleus. Images were automatically cropped to limit processing to a bounding box surrounding the nucleus. To separate the nucleus from the background, a preliminary intensity threshold was then computed using the isodata algorithm [101] . This algorithm is sensitive to the relative size of the nucleus within the image. As a result, the threshold was generally too high because of the larger background size. To correct for this bias, the intensity average m and standard-deviation s were computed over the nucleus region defined by the preliminary threshold and the actual threshold was set to m-2s. The resulting binary image generally contained holes, corresponding in particular to the nucleolus, and presented boundary irregularities due to noise. In addition, bumps were also observed on some nuclei at their basal and apical faces, because of blur from chromocenters [39] (link). Hole filling, opening and closing binary morphological operators [65] were therefore applied to regularize the binary image. The subsequent processing and analyses were confined to this final nucleus mask. A surface model of the nuclear envelope was generated by applying the marching cubes algorithm [102] to the binary mask.