Embed Size (px)
Citation preview
Applied Machine Vision
ME Machine Vision ClassDoug Britton – GTRI
12/1/2005
“Not everybody trusts paintings but people believe photographs”. Ansel Adams
Machine Vision ComponentsProductCamera/SensorIlluminationOpticsTriggerAcquisition card ?Processor/PCSoftwareController digital I/O
“A system capable of acquiring one or more images using an optical non-contact sensing device capable of processing, analyzing and measuring various characteristics so decisions can be made”. - Machine Vision Online
Steps Toward SuccessUnderstand the problem you are solving
Look at current processes or solutionsLearn about the environmentGather production information
Dimensions, orientation, presentation of product Types and range of expected “defects”Production rates and distribution of “defects”
Know the processing constraints
Design a MV solution that maximizes probability of success
“You don't take a photograph, you make it”. Ansel Adams
Review: EM Spectrum & Light
Is it a wave or particle? Who cares!
Characterize Product & Defects
Spectral absorption & reflectanceSurface texture – specular or diffuseFluorescence propertiesCan humans see defects?What about non-visible properties?
Spectral Characterization
0
10
20
30
40
50
60
70
80
90
100
300 500 700 900 1100 1300 1500 1700 1900
Wavelength, nanometers
Tran
smis
sion
or R
efle
ctan
ce, %
Yellow FoamReflectance
White FoamReflectance
White FoamWrapperTransmission
Cryovac BagTransmission
Clear BagTransmission
Package Inspection Example
Common Light Sources
SunlightTungstenMercury VaporHalogenFluorescentLEDLaser
Choosing A Light SourceMatch the spectral response of product & defect with light source that give good contrastAlign spectral peaks of light with spectral reflectance/absorption of product/defect
Types of Illumination
From EdmundOptics.com
Diffuse Front Pros: minimizes shadows & specular reflections Cons: surface features less distinctType: fluorescent linears& rings
Pros: strong, relatively even illuminationCons: shadows, glareType: single (shown) and dual fiber optic light guides
Directional
Illumination Cont.
From EdmundOptics.com
Glancing Pros: shows surface defects/topologyCons: hot spots, severe shadowingType: fiber optic light guides
Pros: surface feature & contour extractionCons: intense source; absorbed by some colorsType: line generating laser diodes, fiber optic line light guides
Structured Light
Illumination Cont.
From EdmundOptics.com
Pros: even illumination, removes specularitiesCons: lower intensity through polarizerType: filter attaches to many existing lenses and light sources
Pros: shadow-free, even illumination; little glareCons: lower intensity through the beamsplitterType: LED axial source, fiber optic-driven axial adapters
Polarized Light
Diffuse Axial
Illumination Cont.
From EdmundOptics.com
Brightfield/Backlight Pros: High contrast for edge detection.Cons: Eliminates surface detail.Type: Fiber optic backlights, LED backlights.
Pros: High contrast of internal & surface details.Cons: Poor edge contrast. Not useful - opaque objects.Type: Fiber optic darkfieldattachment, line light guides.
Darkfield
Illumination SummaryApplication Requirements
Type of Object Under Inspection
Illumination Type Suggested
Reduction of Specularity Shiny Object Diffuse Front, Diffuse Axial, Polarizing
Even Illumination of Object Any Type of Object Diffuse Front, Diffuse Axial, Ring Guide
Highlight Surface Defects or Topology
Nearly Flat (2-D) Object Single Directional, Structured Light
Highlight Texture of Object with Shadows
Any Type of Object Directional, Structured Light
Reduce Shadows Object with Protrusions 3-D Object Diffuse Front, Diffuse Axial, Ring Guide
Highlight Defects within Object
Transparent Object Dark Field
Silhouetting Object Any Type of Object Backlighting
3-D Shape Profiling of Object Object with Protrusions, 3-D Object
Structured Light
From EdmundOptics.com
Imaging GeometriesField of View (FOV)
Viewable area in object space
Working Distance (WD)Distance - front of lens to object
Spatial ResolutionSmallest feature size distinguished by MV system
Depth of Field (DOF)Max object depth that can be kept in focus
From EdmundOptics.com
SensorsInfra Red (IR)
VOx microbolometerNear IR
InGaAsVisible
Silicon CCD CMOS arrays
Ultra Violet CMOS arrays NMOS detectors
Filter
Photons
Detector
Pixel Data
Electronics
CCD Sensor
Pixels and CCD arraysSquare vs Rectangular pixelsLarger pixels/unit area (7.5 micron)
More photons absorbed = less noiseLower resolution
Smaller pixels/unit area (4.5 micron)Fewer photons = more noiseHigher resolution
CCD dimensions impact FOV & lens specification
Resolution & Dynamic RangeSpatial Resolution
Smallest feature size distinguishableRequire minimum 2 pixels/line pair
Dynamic rangeGoal is to maximize contrastApplication dependent
Spatial Resolution Example
Object is 4 mm Sesame SeedWant at least 2 pixels on each seed(4 mm)/(2 pixels) = 2 mm/pixel minimum
Notice nothing mentioned about: Sensor sizeWorking distanceLens focal length
Resolution requirements often dictate the choice of sensor and optics.
Review: Simple Lens Equation
fv u
Image
Optical Axis
ObjectFocal Point
Lens
Optical Center
1u
1v
1f+ =
f = lens focal lengthu = working distancev = distance to sensor
Focal Length CalculationAssume pin hole cameraGiven:
Spatial resolution Sensor size Field of view
CalculateWorking distanceFocal length
HFOV 1.75ft:=
VFOV 1.3ft:=
SENSOR_HGT 8ft:=
OFFSET 0ft:=
Calculate the required focal length for the problem!!
CCD size 1/3in ccd 4.8x3.6mm
H_PIX 1024:= V_PIX 768:=
wo 4.8 mm⋅:= wox 3.6 mm⋅:=
HFOV 1.75ft:=
VFOV 1.3ft:=
SENSOR_HGT 8ft:=
OFFSET 0ft:=
Calculate the required focal length for the problem!!
CCD size 1/3in ccd 4.8x3.6mm
H_PIX 1024:= V_PIX 768:=
wo 4.8 mm⋅:= wox 3.6 mm⋅:=
WD 8ft=
H_THETA 12.484deg=
V_THETA 9.29deg=
f 21.943mm= fx 22.154mm=
H_RES 48.7621in
=
V_RES 49.2311in
=
WD 8ft=
H_THETA 12.484deg=
V_THETA 9.29deg=
f 21.943mm= fx 22.154mm=
WD 8ft=
H_THETA 12.484deg=
V_THETA 9.29deg=
f 21.943mm= fx 22.154mm=
H_RES 48.7621in
=
V_RES 49.2311in
=
H_RES 48.7621in
=
V_RES 49.2311in
=
Lens Distortion
No object information is lost Information is only misplaced in the image.
Pincushion Barrel
CamerasArea scan
Traditional sensorInterlaced video
Legacy from TVEvery other lineIssues with freezing frames/motion
Progressive scanEach row of pixels scanned out one at a time
Line scanSingle line/array of pixelsSuccessive lines form imageTiming/trigger and lighting crucial
Color vs. Monochrome
Can you get away without color?
Frame Rate vs. Shutter Speed
Shutter speed –Exposure time
Integration time of sensor Short integration time -> freezes motion, but requires a lot of light1/60 – 1/10,000 sec
Frame rateRate that camera is produces framesTV 30 frames/sec interlaced
Color Cameras3-chip color
Uses prisms to split color to 3 CCD’sBest quality colorTrue resolution
Single chip colorBayer tile configurationFilter each pixelTwice the green pixelsMore human sensitivity in green
Color Camera Spectrum
Sony ICX-424AQ sensor 1/3” single chip color
Grayscale CamerasIntegrate across all three RGB color bandsMost CCD’s respond in NIR ~700-900nm
Filtering Application
Pharmaceutical inspectionPills are different colorsWant to make sure that each package contains the right pillsCan you use a monochrome camera?How do we make it work?Edmund optics example
Bun Imaging SystemCloudy Day Illuminator –diffuse lightSingle chip color camera with integration of 2.5 msec
100% product inspectionInspection items:
Bun color avg. & dist.Garnish coverage & dist.2D circular size/shapeGrease/contaminants
Feedback control of Oven
Bun Imaging System - Database
Bun Classifier
Local Log File
Data Aggregation
DatabaseQueuing
Database SpecificDequeuing
DatabaseServer
Remote User
Remote User
Remote User
Loca
l Use
r
Inspection Processor & System
Netw
ork
Bun Classifier
Local Log File
Data Aggregation
DatabaseQueuing
Database SpecificDequeuing
DatabaseServer
Remote User
Remote User
Remote User
Loca
l Use
r
Inspection Processor & System
Netw
ork
Provides immediate feedback to operatorsStores & catalogs quality data Available to managers over network for “real-time” analysis
Color Data Without Controller
Data collected on bun line at Bakery
50.00
55.00
60.00
65.00
70.00
11:05
11:14
11:23
11:32
11:41
11:50
11:59
12:08
12:17
12:26
12:35
12:44
12:53
13:02
13:12
13:21
Tim e
L-Va
lue
Color Data with PI ControllerTest of PI controller to regulate L-value after it was
purposely deviated from the setpoint
50.00
55.00
60.00
65.00
70.00
15:36
15:39
15:42
15:46
15:49
15:52
15:55
15:58
Time
L-Va
lue
L-valueTarget L-valueMin. RangeMax Range
