Image segmentation

. It shows the outer surface (red), the surface between compact bone and spongy bone (green) and the surface of the bone marrow (blue). In digital image processing and computer vision, image segmentation is the process of partitioning a digital image into multiple image segments, also known as image regions or image objects (sets of pixels). The goal of segmentation is to simplify and/or change the representation of an image into something that is more meaningful and easier to analyze. Image segmentation is typically used to locate objects and boundaries (lines, curves, etc.) in images. More precisely, image segmentation is the process of assigning a label to every pixel in an image such that pixels with the same label share certain characteristics. The result of image segmentation is a set of segments that collectively cover the entire image, or a set of contours extracted from the image (see edge detection). Each of the pixels in a region are similar with respect to some characteristic or computed property, such as color, intensity, or texture. Adjacent regions are significantly different with respect to the same characteristic(s). When applied to a stack of images, typical in medical imaging, the resulting contours after image segmentation can be used to create 3D reconstructions with the help of geometry reconstruction algorithms like marching cubes.

. It shows the outer surface (red), the surface between compact bone and spongy bone (green) and the surface of the bone marrow (blue). In digital image processing and computer vision, image segmentation is the process of partitioning a digital image into multiple image segments, also known as image regions or image objects (sets of pixels). The goal of segmentation is to simplify and/or change the representation of an image into something that is more meaningful and easier to analyze. Image segmentation is typically used to locate objects and boundaries (lines, curves, etc.) in images. More precisely, image segmentation is the process of assigning a label to every pixel in an image such that pixels with the same label share certain characteristics. The result of image segmentation is a set of segments that collectively cover the entire image, or a set of contours extracted from the image (see edge detection). Each of the pixels in a region are similar with respect to some characteristic or computed property, such as color, intensity, or texture. Adjacent regions are significantly different with respect to the same characteristic(s). When applied to a stack of images, typical in medical imaging, the resulting contours after image segmentation can be used to create 3D reconstructions with the help of geometry reconstruction algorithms like marching cubes. of the thorax: The anterior thoracic wall, the airways and the pulmonary vessels anterior to the root of the lung have been digitally removed in order to visualize thoracic contents: – blue : pulmonary arteries – red : pulmonary veins (and also the abdominal wall) – yellow : the mediastinum – violet : the diaphragm Some of the practical applications of image segmentation are: Content-based image retrieval Machine vision Medical imaging, and imaging studies in biomedical research, including volume rendered images from computed tomography, magnetic resonance imaging, as well as volume electron microscopy techniques such as FIB-SEM. Locate tumors and other pathologies Measure tissue volumes Diagnosis, study of anatomical structure Surgery planning Virtual surgery simulation Intra-surgery navigation Radiotherapy Digital Pathology and Histopathology. Nuclei instance segmentation in variously stained whole slide images (WSIs) refers to the automatic delineation of individual cells' nuclei borders. This task is a specialized subproblem of instance segmentation, with high practical importance in biomedical research and diagnostics (e.g. cell counting, morphometric feature extraction, tumor grading, prognostic biomarker extraction). The modern approach to the problem is the use of deep learning architectures developed to encompass the main challenges of separating overlapping or fused nuclei. The availability of few humanly-annotated datasets poses another challenge to the problem. Object detection Pedestrian detection Face detection Brake light detection Locate objects in satellite images (roads, forests, crops, etc.) Recognition Tasks Face recognition Fingerprint recognition Iris recognition Prohibited Item at Airport security checkpoints Traffic control systems Video surveillance Video object co-segmentation and action localization Several general-purpose algorithms and techniques have been developed for image segmentation. To be useful, these techniques must typically be combined with a domain's specific knowledge in order to effectively solve the domain's segmentation problems.

Classes of segmentation techniques

There are two classes of segmentation techniques. Classical computer vision approaches AI based techniques \nabla^2 f(x,y) = \frac + \frac The Laplacian operator is employed such that the partial derivatives are derived from a specific equation. The second partial derivative of f(x, y) with respect to x and y are given by: \frac = f(x+1,y) + f(x-1,y) - 2f(x,y) \frac = f(x,y+1) + f(x,y-1) - 2f(x,y) These partial derivatives are then used to compute the Laplacian as: \nabla^2 f(x,y) = f(x+1,y) + f(x-1,y) + f(x,y+1) + f(x,y-1) - 4f(x,y) This mathematical expression can be implemented by convolving with an appropriate mask. If we extend this equation to three dimensions (x,y,z), the intensity at each pixel location around a central pixel at (x, y, z) is replaced by their corresponding values. This equation becomes particularly useful when we assume that all pixels have unit spacing along each axis. A sphere mask has been developed for use with three-dimensional datasets. The sphere mask is designed to use only integer arithmetic during calculations, thereby eliminating the need for floating-point hardware or software. When applying these concepts to actual images represented as arrays of numbers, we need to consider what happens when we reach an edge or border region. The function g(x, y) is defined as: g(x, y) = \begin{cases} 1 & \text{if } |R(x, y)| \geq T \\ 0 & \text{otherwise} \end{cases} This above equation is used to determine whether a point in the image is an isolated point based on the response magnitude |R(x, y)| and a threshold value T . If the response magnitude is greater than or equal to the threshold, the function returns 1, indicating the presence of an isolated point; otherwise, it returns 0. This helps in the effective detection and segmentation of isolated points in the image. This method is a combination of three characteristics of the image: partition of the image based on histogram analysis is checked by high compactness of the clusters (objects), and high gradients of their borders. For that purpose two spaces have to be introduced: one space is the one-dimensional histogram of brightness H = H(B); the second space is the dual 3-dimensional space of the original image itself B = B(x, y). The first space allows to measure how compactly the brightness of the image is distributed by calculating a minimal clustering kmin. Threshold brightness T corresponding to kmin defines the binary (black-and-white) image – bitmap b = φ(x, y), where φ(x, y) = 0, if B(x, y)  DC = G/(k × L) has to be calculated (where k is difference in brightness between the object and the background, L is length of all borders, and G is mean gradient on the borders). Maximum of MDC defines the segmentation. \Delta U = U^\text{new} - U^\text{old} \ell_i = \begin{cases} \ell^\text{new}_i, & \text{if } \Delta U \leq 0 ,\\ \ell^\text{new}_i, & \text{if } \Delta U > 0 \text{ and } \delta the algorithm selects the newly formed graph. Simulated annealing requires the input of temperature schedules which directly affects the speed of convergence of the system, as well as energy threshold for minimization to occur. P(\lambda \mid f_i) = \frac{P(f_i \mid \lambda) P(\lambda)}{\Sigma_{\lambda \in \Lambda} P(f_i \mid \lambda) P(\lambda)} Here \lambda \in \Lambda , the set of all possible labels. 3. M step: The established relevance of a given feature set to a labeling scheme is now used to compute the a priori estimate of a given label in the second part of the algorithm. Since the actual number of total labels is unknown (from a training data set), a hidden estimate of the number of labels given by the user is utilized in computations. P(\lambda) = \frac{\Sigma_{\lambda \in \Lambda} P(\lambda \mid f_i)} where \Omega is the set of all possible features.

Segmentation of related images and videos

Related images such as a photo album or a sequence of video frames often contain semantically similar objects and scenes, therefore it is often beneficial to exploit such correlations. The task of simultaneously segmenting scenes from related images or video frames is termed co-segmentation, which is typically used in human action localization. Unlike conventional bounding box-based object detection, human action localization methods provide finer-grained results, typically per-image segmentation masks delineating the human object of interest and its action category (e.g., Segment-Tube ). Techniques such as dynamic Markov Networks, CNN and LSTM are often employed to exploit the inter-frame correlations. Online demonstration of piecewise linear image segmentation by IPOL Journal

Text derived from https://en.wikipedia.org/w/index.php?oldid=1375991769 under CC BY-SA 4.0; modifications have been made.