Convolutional neural networks for image segmentation

To construct a segmentation mask for a new image, the trained conv-net is applied to the entire image to produce pixel level classification predictions. This is done by using a fully convolutional implementation of conv-nets that can be directly applied to the entire image [35 , 36 ]. We also used model parallelism (averaging the results of 5 trained networks) to improve our segmentation accuracy.
While the classification predictions produced by conv-nets were remarkably accurate, we found that they required further refinement to produce binary segmentation masks. As opposed to using the raw class scores for each pixel as the input for refinement, we instead use the softmax normalized score $\frac{e^{class score}}{{{\displaystyle \sum }}_{all classes}{e}^{class score}}$ , which we loosely interpret as the probability deduced by the conv-net that the pixel in question belongs to that particular class. For bacterial cells, we found that thresholding of the softmax normalized score for the cell interior class with a value of 0.6 was sufficient to produce accurate masks. For nuclei, we found that thresholding with a value of 0.5–0.75 in a similar fashion to bacteria worked well in most cases, with adaptive thresholding of the softmax normalized cell interior score being used for images with very dim nuclei. For mammalian cells, the conv-net prediction in combination with the nuclear marker was used to guide an active contour algorithm to further refine the segmentation[52 ]. Training error and segmentation results for E. coli, MCF10A, NIH-3T3, and HeLa-S3 cells are shown in Fig 2, S1–S8 Figs and S4 and S7 Movies.