Definitions
Computer vision is the science and technology of machines that see.
Machine vision (MV) is the application of computer vision to industry and manufacturing.
Whereas computer vision is mainly focused on machine-based image processing, machine vision most often requires also digital input/output devices and computer networks to control other manufacturing equipment such as robotic arms.
Machine Vision
See also Machine Vision Glossary
Machine vision (MV) is the application of computer vision
to industry and manufacturing. Whereas computer vision is mainly
focused on machine-based image processing, machine vision most often
requires also digital input/output devices and computer networks to
control other manufacturing equipment such as robotic arms. Machine
Vision is a subfield of engineering that encompasses computer science,
optics, mechanical engineering, and industrial automation. One of the
most common applications of Machine Vision is the inspection of
manufactured goods such as semiconductor chips, automobiles, food and
pharmaceuticals. Just as human inspectors working on assembly lines visually inspect parts to judge the quality of workmanship, so machine vision systems use digital cameras, smart cameras and image processing software to perform similar inspections.
Machine vision systems are programmed to perform narrowly defined tasks such as counting objects on a conveyor, reading serial numbers, and searching for surface defects. Manufacturers favour machine vision systems for visual inspections.
that require high-speed, high-magnification, 24-hour operation, and/or
repeatability of measurements. Frequently these tasks extend roles
traditionally occupied by human beings whose degree of failure is
classically high through distraction, illness and circumstance.
However, humans may display finer perception over the short period and
greater flexibility in classification and adaptation to new defects and
quality assurance policies.
Computers do not 'see' in the same way that human beings are able
to. Cameras are not equivalent to human optics and while people can
rely on inference systems and assumptions, computing devices must 'see'
by examining individual pixels of images, processing them and
attempting to develop conclusions with the assistance of knowledge
bases and features such as Pattern recognition
engines. Although some machine vision algorithms have been developed to
mimic human visual perception, a number of unique processing methods
have been developed to process images and identify relevant image
features in an effective and consistent manner. Machine vision and
computer vision systems are capable of processing images consistently,
but computer-based image processing systems are typically designed to
perform single, repetitive tasks, and despite significant improvements
in the field, no machine vision or computer vision system can yet match
some capabilities of human vision in terms of image comprehension,
tolerance to lighting variations and image degradation, parts'
variability etc.
Components of a machine vision system
A typical machine vision system will consist of several among the following components:
- One or more digital or analog camera (black-and-white or colour) with suitable optics for acquiring images
- Camera interface for digitizing images (widely known as a "frame grabber")
- A processor (often a PC or embedded processor, such as a DSP)
- (In some cases, all of the above are combined within a single device, called a smart camera).
- Input/Output hardware (e.g. digital I/O) or communication links (e.g. network connection or RS-232) to report results
- Lenses to focus the desired field of view onto the image sensor.
- Suitable, often very specialized, light sources (LED illuminators, fluorescent or halogen lamps etc.)
- A program to process images and detect relevant features.
- A synchronizing sensor for part detection (often an optical or magnetic sensor) to trigger image acquisition and processing.
- Some form of actuators used to sort or reject defective parts.
The sync sensor determines when a part (often moving on a conveyor)
is in position to be inspected. The sensor triggers the camera to take
a picture of the part as it passes beneath the camera and often
synchronizes a lighting pulse to freeze a sharp image. The lighting
used to illuminate the part is designed to highlight features of
interest and obscure or minimize the appearance of features that are
not of interest (such as shadows or reflections). LED panels of suitable sizes and arrangement are often used to this purpose.
The camera's image is captured by the framegrabber. A framegrabber is a digitizing device (within a smart camera or as a separate computer card) that converts the output of the camera to digital
format (typically a two dimensional array of numbers, corresponding to
the luminous intensity level of the corresponding point in the field of
view, called pixel) and places the image in computer memory so that it may be processed by the machine vision software.
The software will typically take several steps to process an image.
Often the image is first manipulated to reduce noise or to convert many
shades of gray to a simple combination of black and white (binarization).
Following the initial simplification, the software will count, measure,
and/or identify objects, dimensions, defects or other features in the
image. As a final step, the software passes or fails the part according
to programmed criteria. If a part fails, the software may signal a
mechanical device to reject the part; alternately, the system may stop
the production line and warn a human worker to fix the problem that
caused the failure.
Though most machine vision systems rely on black-and-white cameras,
the use of colour cameras is becoming more common. It is also
increasingly common for Machine Vision systems to include digital camera equipment for direct connection rather than a camera and separate framegrabber, thus reducing signal degradation.
"Smart" cameras
with built-in embedded processors are capturing an increasing share of
the machine vision market. The use of an embedded (and often very
optimized) processor eliminates the need for a framegrabber card and
external computer, thus reducing cost and complexity of the system
while providing dedicated processing power to each camera. Smart cameras
are typically less expensive than systems comprising a camera and a
board and/or external computer, while the increasing power of embedded
processors and DSPs is often providing comparable or higher performance and capabilities than conventional PC-based systems.
Processing methods
Commercial and open source machine vision software packages
typically include a number of different image processing techniques
such as the following:
- Pixel counting: counts the number of light or dark pixels
- Thresholding: converts an image with gray tones to simply black and white
- Segmentation: used to locate and/or count parts
- Blob discovery & manipulation:
inspecting an image for discrete blobs of connected pixels (e.g. a
black hole in a grey object) as image landmarks. These blobs frequently
represent optical targets for machining, robotic capture, or
manufacturing failure.
- Recognition-by-components: extracting geons from visual input
- Robust pattern recognition: location of an object that may be rotated, partially hidden by another object, or varying in size
- Barcode reading: decoding of 1D and 2D codes designed to be read or scanned by machines
- Optical character recognition: automated reading of text such as serial numbers
- Gauging: measurement of object dimensions in inches or millimeters
- Edge detection: finding object edges
- Template matching: finding, matching, and/or counting specific patterns
In most cases, a machine vision system will use a sequential
combination of these processing techniques to perform a complete
inspection. E.g. A system that reads a barcode may also check a surface
for scratches or tampering and measure the length and width of a
machined component.
Applications of machine vision
The applications of Machine Vision (MV) are diverse, covering areas of endeavour including, but not limited to:
- Biometrics
- Large-scale industrial manufacture
- Short-run unique object manufacture
- Safety systems in industrial environments
- Inspection of pre-manufactured objects (e.g. quality control, failure investigation)
- Visual stock control and management systems (counting, barcode reading, store interfaces for digital systems)
- Control of Automated Guided Vehicles (AGVs)
- Automated monitoring of sites for security and safety
- Monitoring of agricultural production
- Quality control and refinement of food products
- Retail automation
- Consumer equipment control
- Medical imaging processes (e.g. Interventional Radiology)
- Medical remote examination and procedures
Machine vision systems are widely used in semiconductor fabrication;
indeed, without machine vision, yields for computer chips would be
significantly reduced. Machine vision systems inspect silicon wafers,
processor chips, and subcomponents such as resistors and capacitors.
In the automotive industry, machine vision systems are used to guide industrial robots, gauge the fit of stamped metal components, and inspect the surface of the painted vehicle for defects.
Though machine vision techniques were developed for the visible
spectrum, the same processing techniques may be applied to images
captured using imagers sensitive to other forms of spectra such as infrared light or x-ray emissions.
Related fields
Machine vision is distinct from computer vision.
Computer vision extends to topics related to autonomous robotics and
machine representation of human vision. Machine Vision refers to
automated imaging systems including a wide range of computing
disciplines aggregated to form a complete solution to visual problems
and can be considered a superset composed of Computer Vision and
elements such as equipment control, databasing, network systems,
interfacing and machine learning.
Computer Vision
Computer vision is the science and technology of machines that see.
As a scientific discipline, computer vision is concerned with the
theory and technology for building artificial systems that obtain
information from images. The image data can take many forms, such as a
video sequence, views from multiple cameras, or multi-dimensional data
from a medical scanner.
As a technological discipline, computer vision seeks to apply the theories and models of computer vision to the construction of computer vision systems . Examples of applications of computer vision systems include systems for
- Controlling processes (e.g. an industrial robot or an autonomous vehicle).
- Detecting events (e.g. for visual surveillance or people counting)
- Organizing information (e.g. for indexing databases of images and image sequences),
- Modeling objects or environments (e.g. industrial inspection, medical image analysis or topographical modeling),
- Interaction (e.g. as the input to a device for computer-human interaction).
Computer vision can also be described as a complement (but not
necessarily the opposite) of biological vision. In biological vision,
the visual perception of humans and various animals are studied,
resulting in models of how these systems operate in terms of
physiological processes. Computer vision, on the other hand, studies
and describes artificial vision system that are implemented in software
and/or hardware. Interdisciplinary exchange between biological and
computer vision has proven increasingly fruitful for both fields.
Sub-domains of computer vision include scene reconstruction, event
detection, tracking, object recognition, learning, indexing, ego-motion
and image restoration.
State of the art
The field of computer vision can be characterized as immature and
diverse. Even though earlier work exists, it was not until the late
1970s that a more focused study of the field started when computers
could manage the processing of large data sets such as images. However,
these studies usually originated from various other fields, and
consequently there is no standard formulation of "the computer vision
problem". Also, and to an even larger extent, there is no standard
formulation of how computer vision problems should be solved. Instead,
there exists an abundance of methods for solving various well-defined
computer vision tasks, where the methods often are very task specific
and seldom can be generalized over a wide range of applications. Many
of the methods and applications are still in the state of basic
research, but more and more methods have found their way into
commercial products, where they often constitute a part of a larger
system which can solve complex tasks (e.g., in the area of medical
images, or quality control and measurements in industrial processes).
In most practical computer vision applications, the computers are
pre-programmed to solve a particular task, but methods based on
learning are now becoming increasingly common.
A significant part of artificial intelligence
deals with autonomous planning or deliberation for system which can
perform mechanical actions such as moving a robot through some
environment. This type of processing typically needs input data
provided by a computer vision system, acting as a vision sensor and
providing high-level information about the environment and the robot.
Other parts which sometimes are described as belonging to artificial
intelligence and which are used in relation to computer vision is pattern recognition
and learning techniques. As a consequence, computer vision is sometimes
seen as a part of the artificial intelligence field or the computer
science field in general.
Physics
is another field that is strongly related to computer vision. A
significant part of computer vision deals with methods which require a
thorough understanding of the process in which electromagnetic
radiation, typically in the visible or the infra-red range, is
reflected by the surfaces of objects and finally is measured by the
image sensor to produce the image data. This process is based on optics
and solid state physics. More sophisticated image sensors even require
quantum mechanics to provide a complete comprehension of the image
formation process. Also, various measurement problems in physics can be
addressed using computer vision, for example related to motion in
fluids. Consequently, computer vision can also be seen as an extension
of physics.
A third field which plays an important role is neurobiology,
specifically the study of the biological vision system. Over the last
century, there has been an extensive study of eyes, neurons, and the
brain structures devoted to processing of visual stimuli in both humans
and various animals. This has led to a coarse, yet complicated,
description of how "real" vision systems operate in order to solve
certain vision related tasks. These results have led to a subfield
within computer vision where artificial systems are designed to mimic
the processing and behaviour of biological systems, at different levels
of complexity. Also, some of the learning-based methods developed
within computer vision have their background in biology.
Yet another field related to computer vision is signal processing.
Many methods for processing of one-variable signals, typically temporal
signals, can be extended in a natural way to processing of two-variable
signals or multi-variable signals in computer vision. However, because
of the specific nature of images there are many methods developed
within computer vision which have no counterpart in the processing of
one-variable signals. A distinct character of these methods is the fact
that they are non-linear which, together with the multi-dimensionality
of the signal, defines a subfield in signal processing as a part of
computer vision.
Beside the above mentioned views on computer vision, many of the
related research topics can also be studied from a purely mathematical
point of view. For example, many methods in computer vision are based
on statistics, optimization or geometry.
Finally, a significant part of the field is devoted to the
implementation aspect of computer vision; how existing methods can be
realized in various combinations of software and hardware, or how these
methods can be modified in order to gain processing speed without
losing too much performance.
Related fields
Computer vision, Image processing, Image analysis, Robot vision and Machine vision
are closely related fields. If you look inside text books which have
any of these names in the title there is a significant overlap in terms
of what techniques and applications they cover. This implies that the
basic techniques that are used and developed in these fields are more
or less identical, something which can be interpreted as there is only
one field with different names.
On the other hand, it appears to be necessary for research groups,
scientific journals, conferences and companies to present or market
themselves as belonging specifically to one of these fields and, hence,
various characterizations which distinguish each of the fields from the
others have been presented. The following characterizations appear
relevant but should not be taken as universally accepted.
Image processing and Image analysis
tend to focus on 2D images, how to transform one image to another,
e.g., by pixel-wise operations such as contrast enhancement, local
operations such as edge extraction or noise removal, or geometrical
transformations such as rotating the image. This characterization
implies that image processing/analysis neither require assumptions nor
produce interpretations about the image content.
Computer vision tends to focus on the 3D scene projected onto
one or several images, e.g., how to reconstruct structure or other
information about the 3D scene from one or several images. Computer
vision often relies on more or less complex assumptions about the scene
depicted in an image.
Machine vision
tends to focus on applications, mainly in industry, e.g., vision based
autonomous robots and systems for vision based inspection or
measurement. This implies that image sensor technologies and control
theory often are integrated with the processing of image data to
control a robot and that real-time processing is emphasized by means of
efficient implementations in hardware and software.
There is also a field called Imaging
which primarily focus on the process of producing images, but sometimes
also deals with processing and analysis of images. For example, Medical imaging contains lots of work on the analysis of image data in medical applications.
Finally, pattern recognition
is a field which uses various methods to extract information from
signals in general, mainly based on statistical approaches. A
significant part of this field is devoted to applying these methods to
image data.
A consequence of this state of affairs is that you can be working in
a lab related to one of these fields, apply methods from a second field
to solve a problem in a third field and present the result at a
conference related to a fourth field!
Examples of applications for computer vision
One of the most prominent application fields is medical computer
vision or medical image processing. This area is characterized by the
extraction of information from image data for the purpose of making a
medical diagnosis of a patient. Generally, image data is in the form of
microscopy images, X-ray images, angiography images, ultrasonic images, and tomography images. An example of information which can be extracted from such image data is detection of tumours, arteriosclerosis
or other malign changes. It can also be measurements of organ
dimensions, blood flow, etc. This application area also supports
medical research by providing new information, e.g., about the
structure of the brain, or about the quality of medical treatments.
A second application area in computer vision is in industry. Here,
information is extracted for the purpose of supporting a manufacturing
process. One example is quality control where details or final products
are being automatically inspected in order to find defects. Another
example is measurement of position and orientation of details to be
picked up by a robot arm.
Military applications are probably one of the largest areas for
computer vision. The obvious examples are detection of enemy soldiers
or vehicles and missile guidance.
More advanced systems for missile guidance send the missile to an area
rather than a specific target, and target selection is made when the
missile reaches the area based on locally acquired image data. Modern
military concepts, such as "battlefield awareness", imply that various
sensors, including image sensors, provide a rich set of information
about a combat scene which can be used to support strategic decisions.
In this case, automatic processing of the data is used to reduce
complexity and to fuse information from multiple sensors to increase
reliability.
One of the newer application areas is autonomous vehicles, which
include submersibles, land-based vehicles (small robots with wheels,
cars or trucks), aerial vehicles, and unmanned aerial vehicles (UAV).
The level of autonomy ranges from fully autonomous (unmanned) vehicles
to vehicles where computer vision based systems support a driver or a
pilot in various situations. Fully autonomous vehicles typically use
computer vision for navigation, i.e. for knowing where it is, or for
producing a map of its environment (SLAM)
and for detecting obstacles. It can also be used for detecting certain
task specific events, e. g., a UAV looking for forest fires. Examples
of supporting systems are obstacle warning systems in cars, and systems
for autonomous landing of aircraft. Several car manufacturers have
demonstrated systems for autonomous driving of cars,
but this technology has still not reached a level where it can be put
on the market. There are ample examples of military autonomous vehicles
ranging from advanced missiles, to UAVs for recon missions or missile
guidance. Space exploration is already being made with autonomous
vehicles using computer vision, e. g., NASA's Mars Exploration Rover.
Other application areas include:
- Support of visual effects creation for cinema and broadcast, e.g., camera tracking (matchmoving).
- Surveillance.
Typical tasks of computer vision
Each of the application areas described above employ a range of
computer vision tasks; more or less well-defined measurement problems
or processing problems, which can be solved using a variety of methods.
Some examples of typical computer vision tasks are presented below.
Recognition
The classical problem in computer vision, image processing and
machine vision is that of determining whether or not the image data
contains some specific object, feature, or activity. This task can
normally be solved robustly and without effort by a human, but is still
not satisfactorily solved in computer vision for the general case:
arbitrary objects in arbitrary situations. The existing methods for
dealing with this problem can at best solve it only for specific
objects, such as simple geometric objects (e.g., polyhedrons), human
faces, printed or hand-written characters, or vehicles, and in specific
situations, typically described in terms of well-defined illumination,
background, and pose of the object relative to the camera.
Different varieties of the recognition problem are described in the literature:
- Recognition: one or several pre-specified or learned objects
or object classes can be recognized, usually together with their 2D
positions in the image or 3D poses in the scene.
- Identification: An individual instance of an object is
recognized. Examples: identification of a specific person's face or
fingerprint, or identification of a specific vehicle.
- Detection: the image data is scanned for a specific
condition. Examples: detection of possible abnormal cells or tissues in
medical images or detection of a vehicle in an automatic road toll
system. Detection based on relatively simple and fast computations is
sometimes used for finding smaller regions of interesting image data
which can be further analyzed by more computationally demanding
techniques to produce a correct interpretation.
Several specialized tasks based on recognition exist, such as:
- Content-based image retrieval:
finding all images in a larger set of images which have a specific
content. The content can be specified in different ways, for example in
terms of similarity relative a target image (give me all images similar
to image X), or in terms of high-level search criteria given as text
input (give me all images which contains many houses, are taken during
winter, and have no cars in them).
- Pose estimation:
estimating the position or orientation of a specific object relative to
the camera. An example application for this technique would be
assisting a robot arm in retrieving objects from a conveyor belt in an assembly line situation.
- Optical character recognition (or OCR): identifying characters in images of printed or handwritten text, usually with a view to encoding the text in a format more amenable to editing or indexing (e.g. ASCII).
Motion
Several tasks relate to motion estimation, in which an image
sequence is processed to produce an estimate of the velocity either at
each points in the image or in the 3D scene. Examples of such tasks are:
- Egomotion: determining the 3D rigid motion of the camera.
- Tracking: following the movements of objects (e.g. vehicles or humans).
Scene reconstruction
Given one or (typically) more images of a scene, or a video, scene reconstruction aims at computing a 3D model
of the scene. In the simplest case the model can be a set of 3D points.
More sophisticated methods produce a complete 3D surface model.
Image restoration
The aim of image restoration is the removal of noise (sensor noise,
motion blur, etc.) from images. The simplest possible approach for
noise removal is various types of filters such as low-pass filters or
median filters. More sophisticated methods assume a model of how the
local image structures look like, a model which distinguishes them from
the noise. By first analysing the image data in terms of the local
image structures, such as lines or edges, and then controlling the
filtering based on local information from the analysis step, a better
level of noise removal is usually obtained compared to the simpler
approaches.
Computer vision systems
The organization of a computer vision system is highly application
dependent. Some systems are stand-alone applications which solve a
specific measurement or detection problem, while other constitute a
sub-system of a larger design which, for example, also contains
sub-systems for control of mechanical actuators, planning, information
databases, man-machine interfaces, etc. The specific implementation of
a computer vision system also depends on if its functionality is
pre-specified or if some part of it can be learned or modified during
operation. There are, however, typical functions which are found in
many computer vision systems.
- Image acquisition: A digital image is produced by one or
several image sensor which, besides various types of light-sensitive
cameras, includes range sensors, tomography devices, radar, ultra-sonic
cameras, etc. Depending on the type of sensor, the resulting image data
is an ordinary 2D image, a 3D volume, or an image sequence. The pixel
values typically correspond to light intensity in one or several
spectral bands (gray images or colour images), but can also be related
to various physical measures, such as depth, absorption or reflectance
of sonic or electromagnetic waves, or nuclear magnetic resonance.
- Pre-processing: Before a computer vision method can be
applied to image data in order to extract some specific piece of
information, it is usually necessary to process the data in order to
assure that it satisfies certain assumptions implied by the method.
Examples are
- Re-sampling in order to assure that the image coordinate system is correct.
- Noise reduction in order to assure that sensor noise does not introduce false information.
- Contrast enhancement to assure that relevant information can be detected.
- Scale-space representation to enhance image structures at locally appropriate scales.
- Feature extraction: Image features at various levels of complexity are extracted from the image data. Typical examples of such features are
- Lines, edges and ridges.
- Localized interest points such as corners, blobs or points.
- More complex features may be related to texture, shape or motion.
- Detection/Segmentation: At some point in the processing a
decision is made about which image points or regions of the image are
relevant for further processing. Examples are
- Selection of a specific set of interest points
- Segmentation of one or multiple image regions which contain a specific object of interest.
- High-level processing: At this step the input is typically a
small set of data, for example a set of points or an image region which
is assumed to contain a specific object. The remaining processing deals
with, for example:
- Verification that the data satisfy model-based and application specific assumptions.
- Estimation of application specific parameters, such as object pose or object size.
- Classifying a detected object into different categories.
Source: Wikipedia (All text is available under the terms of the GNU Free Documentation License and Creative Commons Attribution-ShareAlike License.)
|