Machine Vision System Architecture

A typical machine vision pipeline consists of illumination, optics, a camera sensor, a frame grabber or acquisision interface, an image processing unit, and a human-machine interaction layer. The hardware stack defines the quality of captured signals before any software analysis begins.

Illumination Strategies

Light sources are chosen based on surface properties and inspection goals. Common configurations include bright-field versus dark-field setups, low-angle illumination, front-directed and diffuse front lighting, backlighting, color-contrast exploitation, and polarization techniques.

Lenses and Optical Parameters

The lens performs imaging, focusing, exposure control, and zoom functions. A smaller aperture yields a longer depth of field. Shutter speed determines the exposure duration. Lens categories range from standard and wide-angle to telecentric and macro designs; mount types must match the camera interface.

Camera Technologies

Cameras convert scene radiance into digital or analog signals. Sensors are broadly classified as CCD or CMOS, with further distinctions between line-scan and area-scan architectures. Scanning can be progressive or interlaced; shutter modes include rolling and global.

Application Domains

Industrial use cases span food packaging, quality inspection, and robotic guidance. In medicine, vision assists diagnostics. Transportation applications include traffic monitoring, license plate recognition, and autonomous navigation. Agriculture employs vision for harvesting, grading, and crop analysis. Consumer-facing systems appear in smart homes and self-driving vehicles.

Digital Image Foundations

Image Categories

Images can be visible or non-visible, binary or grayscale, color or monochrome, static or dynamic, and 2D or 3D.

Digitization Pipeline

The analog-to-digital conversion proceeds through scanning, color separation, sampling, and quantization. Sampling partitions the continuous spatial domain into discrete pixels, while quantization assigns each sample a single digital code from the continuous intensity range.

Pixel Relationships

Neighborhood, adjacency, and connectivity definitions underpin region analysis. A path exists when every intermediate pixel is adjacent to its predecessor. A region is a connected subset of pixels; its boundary comprises pixels that have at least one neighbor outside the region. Distance metrics include Euclidean, Manhattan (city-block), and chessboard measures.

Intensity Histogram

The histogram records the count of pixels at each gray level. It helps assess whether quantization is adequate and guides the selection of binarization thresholds.

The Halcon Environment

Halcon provides an integrated development interface called HDevelop. The language relies on operators with names structured as Category_SpecificFunction. For example, read_image loads an image file.

Control Structures

Assignment uses :=:

a := 1

Conditional branching:

if (Condition)
    Statements
elseif (OtherCondition)
    Statements
else
    Statements
endif

switch(Expression)
case Value1:
    Statements
    break
case Value2:
    Statements
    break
default:
    Statements
endswitch

Loops:

for idx := Start to End by Step
    Statements
endfor

while (Condition)
    Statements
endwhile

repeat
    Statements
until (Condition)

Core Interface Windows

• Graphics Window – shows original and processed images aswell as regions.
• Operator Window – exposes parameters, types, and default values; partial string entry triggers autocompletion.
• Variable Window – lists image and control variables; double-clicking reveals their current values.
• Program Window – displays the script with a program counter, breakpoint capabilities, and code alongside execution indicators.

Image Acquisition Workflows

• Static images: Open the Image Acquisition assistant, select files, and insert the generated code.
• Live feed: Use the assistant to detect the interface and camera, connect, start grabbing, and then generate the corresponding code.

Data Structures

• Image – matrix of pixel intensities.
• Region – set of pixels fulfilling a condition.
• XLD – extended line descriptions, e.g., sub-pixel contours.
• Handle – reference to complex objects like models or files.
• Tuple – ordered collection:

* empty tuple
myArray := []
* assign value at index
myArray[idx] := val
* append
myArray := [myArray, val]

Image Preprocessing

Regions of Interest

ROIs can be defined programmatically or interactively (rectangles, circles, etc.). A chessboard pattern generator illustrates ROI creation:

dev_open_window(0, 0, 400, 400, 'black', WinID)
gen_image_const(ChessImg, 'byte', 400, 400)
for Row := 0 to 400 by 80
    for Col := 0 to 400 by 80
        gen_rectangle1(Rec, Row, Col, Row + 40, Col + 40)
        overpaint_region(ChessImg, Rec, 255, 'fill')
    endfor
endfor
for Row := 40 to 400 by 80
    for Col := 40 to 400 by 80
        gen_rectangle1(Rec, Row, Col, Row + 40, Col + 40)
        overpaint_region(ChessImg, Rec, 255, 'fill')
    endfor
endfor
write_image(ChessImg, 'png', 255, 'D:/chessboard.png')

Geometric Transformations

Transformations covered include Euclidean (isometric), similarity (translation, rotation, scaling), affine, and projective (homography) mappings. Applications involve perspective correction, view-point alignment, and stitching. Interpolation methods range from nearest-neighbor to bilinear and bicubic schemes.

Intensity Enhancement

Point operations modify each pixel based on its original intensity:

• Inversion: invert_image(Input : Inverted : :) computes 255 - g.
• Contrast emphasis: emphasize(Input : Enhanced : MaskW, MaskH, Factor :) applies res := round((orig - mean) * Factor) + orig.
• Linear scaling: scale_image(Input : Scaled : Mult, Add :) computes g * Mult + Add.
• Histogram equalization and specification redistribute gray levels.

Neighborhood operations process local windows:

• Mean filtering reduces Gaussian noise.
• Median filtering suppresses salt-and-pepper noise.
• Multi-image averaging combines several exposures of the same scene.

Sharpening accentuates edges:

• First-order gradient methods: simple horizontal/vertical difference and Roberts gradient.
• Sobel operator yields broad but clean edges.
• Second-order Laplacian extracts finer details by subtracting four-times the center value from the sum of its four neighbors, producing detailed but less distinct boundaries.

Frequency-domain processing isolates low-frequency (smooth regions) and high-frequency (edges, noise) components via Fourier transform, filtering with low-pass or high-pass models, and inverse transformation. Homomorphic filtering simultaneously compresses dynamic range and enhances contrast.

Image Segmentation

Thresholding

A threshold range separates pixels. Global approaches rely on a fixed interval (threshold(Image : Region : MinGray, MaxGray :)) determined manually or via histogram valley analysis. Local adaptive methods account for varying illumination.

Edge Detection

Operators include Prewitt, Krisch, Laplacian-of-Gaussian (LoG), and Canny. Sub-pixel edge extraction refines boundaries beyond integer coordinates.

Region-Based Segmentation

Region growing merges similar pixels:

read_image(Img, 'fabrik')
median_image(Img, MedImg, 'circle', 2, 'mirrored')
regiongrowing(MedImg, Regions, 1, 1, 2, 100)
shape_trans(Regions, Centers, 'inner_center')
connection(Centers, SingleCenters)
area_center(SingleCenters, Area, Row, Col)
regiongrowing_mean(MedImg, MeanRegions, Row, Col, 25, 100)

Split-and-merge partitions the image into homogeneous blocks and fuses adjacent blocks with similar statistics.

Hough Transform and Watershed

Hough-based methods (e.g., gradient-weighted Hough) detect parametric shapes. The watershed algorithm treats intensity as topography, flooding basins to separate touching objects.

Feature Extraction

Shape Descriptors

area_center computes region area and centroid. area_holes quantifies enclosed cavities. Operators select, create, or characterize regions through inscribed circles and bounding rectangles.

Intensity Features

Gray-value statistics include mean, deviation, minimum, and maximum within a region. intensity and min_max_gray provide such measurements, enabling region selection based on brightness criteria.

Texture Features

For patterns with repetitive structures, gray-level co-occurrence matrices capture spatial relationships between pixel intensities.

Morphological Processing

Set Theory Basis

Morphology relies on union, intersection, complement, difference, translation, and reflection.

Binary Operations

• Erosion: all locations where the translated structuring element fits entirely within the foreground.
• Dilation: all centers where the reflected element intersects the foreground.
• Opening (erosion followed by dilation) removes small protrusions.
• Closing (dilation followed by erosion) fills narrow gaps.
• Hit-or-miss transform (hit_or_miss) detects specific foreground-background patterns.

Grayscale Morphology

Instead of set membership, operations compute minima (erosion) or maxima (dilation) of sums and differences within the neighborhood defined by the structuring element. Opening suppresses bright peaks on a dark background; closing suppresses dark valleys on a bright background. Top-hat (original minus opening) highlights bright spots; bottom-hat (closing minus original) highlights dark regions.

Morphological Tools for Binary Images

• Boundary extraction: subtract the eroded image from the original.
• Hole filling: fill_up propagates a seed until the foreground boundary stops expansion.
• Skeletonization: iterative thinning reduces objects to their medial axes.

Template Matching

Matching locates instances of a reference pattern within a search image through a learning phase and a matching phase.

NCC-Based Matching

1. Create a normalized cross-correlation model:

create_ncc_model(Template :: NumLevels, AngleStart, AngleExtent, AngleStep, Metric : ModelID)

2. Search for the best correspondences:

find_ncc_model(SearchImg :: ModelID, AngleStart, AngleExtent, MinScore, NumMatches, MaxOverlap, SubPixel, NumLevels : Row, Column, Angle, Score)

Feature-Based Matching

Instead of raw intensities, this method exploits distinctive keypoints, offering higher precision with lower computational cost. Moment invariants supply one such feature set.