Segmentation

Segmentation of medical images

Segmentation of medical images is a challenging task. A myriad of different methods have been proposed and implemented in recent years. In spite of the huge effort invested in this problem, there is no single approach that can generally solve the problem of segmentation for the large variety of image modalities existing today.

The most effective segmentation algorithms are obtained by carefully customizing combinations of components. The parameters of these components are tuned for the characteristics of the image modality used as input and the features of the anatomical structure to be segmented.

The ITK Software Guide

Aliza provides several region growing filters, histogram based threshold filters and paint tool. Several free programs, e.g. ITK-SNAP, provide semi-automatic segmentation. Interchange is easy with, for example, MHA format for images and STL for meshes.

File formats

Appropriate scalar image can be used as segmentation image (binary image or labels)
VTK poly-data, STL, OBJ
VTK poly-line

Examples

Create meshes from labels or binary image: Segmentation->Meshes from labels or binary image
Create contours (ROI) from binary image: Segmentation->Contours from binary image
Create binary images from meshes: Segmentation->Binary images from meshes
Create binary image from ROI (two algorithms): Segmentation->Binary image from selected ROI
Create mesh from ROI: Segmentation->Mesh from selected ROI
Create polygons from ROI: Segmentation->Polygons from selected ROI

Export contours into VTK poly-line(s) file(s): Export->Save ROIs to VTK
Export mesh as VTK, STL or OBJ files: Export->Save mesh to VTK, STL, OBJ
Export contours (ROIs) to DICOM RT Structure Set: Export->Save ROIs to RTSTRUCT

It is recommended to select an image to use as referenced image,. To write correct UIDs is important not to modify or export referenced image, e.g. to export newly created study with RT Structure Set, referenced image must be first exported and opened. For existing DICOM study select relevant image.

Export meshes to DICOM Surface Segmentation: Export->Save mesh to DICOM Segmentation

There is an option in Settings widget to use for export the retired Triangle Point Index List, VR OW (unsigned short indices) instead the default Long Triangle Point Index List (0x0066,0x0041), VR OL (unsigned int indices). Also is recommended to select a referenced image, if possible.

Fig. 1

Draw contours (ROI)

S. Draw contours (ROI)

Paint tool

Fig. 2

Fig. 3

Select brush size
Select fill value
Clear
Optionally select image to apply painting to
Apply painting

Region growing segmentation

Region growing algorithms have proven to be an effective approach for image segmentation. The basic approach of a region growing algorithm is to start from a seed region (typically one or more pixels) that are considered to be inside the object to be segmented. The pixels neighboring this region are evaluated to determine if they should also be considered part of the object. If so, they are added to the region and the process continues as long as new pixels are added to the region. Region growing algorithms vary depending on the criteria used to decide whether a pixel should be included in the region or not, the type connectivity used to determine neighbors, and the strategy used to visit neighboring pixels.

The ITK Software Guide

Fig. 4

Prepare an image

Optionally remove unneeded parts, e.g. select image region and apply Filters->Erase (inv.) selected region
Optionally, usually recommended, is to smooth, e.g. Filters->Smoothing->Curvature Anisotropic Smoothing

Connected threshold

The criterion for including pixels in a growing region is to evaluate intensity value inside a specific interval

Select lower and upper thresholds of the interval

Click Transparency button (2D view) to set mode to Discard values outside lower and upper

Fig. 5

Adjust level/window

Fig. 6

Set seed: CTRL-click with left mouse button in 2D view [1]
Press c key, having focus in the view, to clear seeds

Fig. 7

Run Segmentation->Connected threshold

Fig. 8

Neighborhood connected threshold

On one hand, the Connected Threshold Filter considers only the value of the pixel itself when determining whether it belongs to the region: if its value is within the interval it is included, otherwise it is excluded. Neighborhood Connected Filter, on the other hand, considers a user-defined neighborhood surrounding the pixel, requiring that the intensity of each neighbor be within the interval for it to be included.

The reason for considering the neighborhood intensities instead of only the current pixel intensity is that small structures are less likely to be accepted in the region.

The ITK Software Guide

Set inside seed: CTRL-click with left mouse button in 2D view
Press c key, having focus in the view, to clear seeds

Fig. 9

Neighborhood

Neighborhood radius parameter defines the neighborhood size used to determine whether a pixel lies in the region. The larger the neighborhood, the more stable this filter will be against noise in the input image, but also the longer the computing time will be.

Confidence connected

The criterion used by the Confidence Connected Filter is based on simple statistics of the current region. First, the algorithm computes the mean and standard deviation of intensity values for all the pixels currently included in the region. A user-provided factor is used to multiply the standard deviation and define a range around the mean. Neighbor pixels whose intensity values fall inside the range are accepted and included in the region. When no more neighbor pixels are found that satisfy the criterion, the algorithm is considered to have finished its first iteration. At that point, the mean and standard deviation of the intensity levels are recomputed using all the pixels currently included in the region. This mean and standard deviation defines a new intensity range that is used to visit current region neighbors and evaluate whether their intensity falls inside the range. This iterative process is repeated until no more pixels are added or the maximum number of iterations is reached.

The ITK Software Guide

The following equation illustrates the inclusion criterion used by this filter

I(X) ∈ [m − f σ, m + f σ]

where

m and σ are the mean and standard deviation of the region intensities
f is a factor defined by the user
I() is the image
X is the position of the particular neighbor pixel being considered for inclusion in the region

Set inside seeds: CTRL-click with left mouse button in 2D view, paint seed holding CTRL and left mouse button
Press c key, having focus in the view, to clear seeds

Fig. 10

Multiplier

The most important parameter. The factor f defines how large the range of intensities will be. Small values of the multiplier will restrict the inclusion of pixels to those having very similar intensities to those in the current region. Larger values of the multiplier will relax the accepting condition and will result in more generous growth of the region. Values that are too large will cause the region to grow into neighboring regions which may belong to separate anatomical structures. This is not desirable behavior.

Number of iterations

The number of iterations is specified based on the homogeneity of the intensities of the anatomical structure to be segmented. Highly homogeneous regions may only require a couple of iterations. Regions with ramp effects, like MRI images with inhomogeneous fields, may require more iterations. In practice, it seems to be more important to carefully select the multiplier factor than the number of iterations. However, keep in mind that there is no guarantee that this algorithm will converge on a stable region. It is possible that by letting the algorithm run for more iterations the region will end up engulfing the entire image.

Initial neighborhood radius

Initial neighborhood around the seed is defined with the radius. The neighborhood will be defined as an N-dimensional rectangular region with 2r + 1 pixels on the side, where r is the value passed as initial neighborhood radius.

Isolated connected

In this filter two seeds and a lower threshold are provided by the user. The filter will grow a region connected to the first seed and not connected to the second one. In order to do this, the filter finds an intensity value that could be used as upper threshold for the first seed. A binary search is used to find the value that separates both seeds.

The ITK Software Guide

Set inside seed: CTRL-click with left mouse button in 2D view
Set outside seed: SHIFT-click with left mouse button in 2D view
Press c key, having focus in the view, to clear seeds

Fig. 11

Fill holes in binary image filter

Fig. 12

Fig. 13

Run Segmentation->Fill holes in binary image

Fig. 14

Radius

With radius (r) 1 the neighborhood will have size 3 x 3 x 3

Majority threshold

With majority threshold m the required number of foreground neighbors should be at least ((r*2+1) * (r*2+1) * (r*2+1) - 1) / 2 + m

Histogram based threshold filters

Fig. 15

Otsu

Another criterion for classifying pixels is to minimize the error of misclassification. The goal is to find a threshold that classifies the image into two clusters such that we minimize the area under the histogram for one cluster that lies on the other cluster’s side of the threshold. This is equivalent to minimizing the within class variance or equivalently maximizing the between class variance.

The ITK Software Guide

Otsu multiple (labels)

Otsu multiple thresholds calculator generates thresholds for a given histogram so as to maximize the between-class variance.

Huang

Huang’s fuzzy thresholding using Shannon’s entropy function.

Li

Li’s minimum cross entropy.

Renyi

Similar to maximum entropy, but using a different entropy measure.

Iso Data

Computes average of voxels below initial threshold and above initial threshold. Threshold is set to the average of the two. Repeat until the threshold is larger than the composite average.

Kittler-Illingworth

Similar to the Otsu method. Assumes a Gaussian mixture model. Minimizes the number of misclassifications between the two normal distributions with the given means, variances and proportions.

Yen

Maximum correlation criterion.

Triangle

The triangle method constructs a line between the histogram peak and the farthest end of the histogram. The threshold is the point of maximum distance between the line and the histogram. This implementation uses robust (default is 1% and 99%) estimation of histogram ends.

Shanbhag

Extenstion of the Kapur method.

Maximum entropy

Choose threshold such that the entropies of distributions above and below threshold is maximised. One of several entropy-based approaches.

Moments

Choose threshold such that the binary image has the same first three moments as the grey level image.

Intermodes

Iteratively smooths histogram until only 2 peaks remain. Threshold is the midpoint of the two peaks. Not good for histograms with very unequal peaks.

References

R. Beare. Histogram-based thresholding - some missing methods, Insight Journal, 2011. (*)
L.K. Huang and M.J.J. Wang. Image thresholding by minimizing the measures of fuzziness. Pattern recognition, 28(1):41–51, 1995.
J.N. Kapur, P.K. Sahoo, and A.K.C Wong. A new method for gray-level picture thresholding using the entropy of the histogram. Computer vision, graphics, and image processing, 29(3):273–285, 1985.
J. Kittler and J. Illingworth. Minimum error thresholding. Pattern recognition, 19(1):41–47, 1986.
G. et al Landini. Auto threshold.
C.H. Li and C.K. Lee. Minimum cross entropy thresholding. Pattern Recognition, 26(4):617–625, 1993.
C.H. Li and P.K.S Tam. An iterative algorithm for minimum cross entropy thresholding. Pattern recognition letters, 19(8):771–776, 1998.
A. Niemist. Histthresh toolbox for matlab.
J. Prewitt and M.L. Mendelsohn. The analysis of cell images. Annals of the New York Academy of Sciences, 128(3):1035–1053, 1965.
T.W. Ridler and S. Calvard. Picture thresholding using an iterative selection method. IEEE transactions on Systems, Man and Cybernetics, 8(8):630–632, 1978.
A.G. Shanbhag. Utilization of information measure as a means of image thresholding. CVGIP: Graphical Models and Image Processing, 56(5):414–419, 1994.
W.H. Tsai. Moment-preserving thresolding: A new approach. Computer Vision, Graphics, and Image Processing, 29(3):377–393, 1985.
J.C. Yen, F.J. Chang, and S. Chang. A new criterion for automatic multilevel thresholding. Image Processing, IEEE Transactions on, 4(3):370–378, 1995.
G.W. Zack, W.E. Rogers, and S.A. Latt. Automatic measurement of sister chromatid exchange frequency. Journal of Histochemistry & Cytochemistry, 25(7):741, 1977.