View
248
Download
0
Category
Tags:
Preview:
Citation preview
Computer vision for non-visual spectral regimes and non-
traditional applications
Rama ChellappaUMD
Opening remarks – non-visual sensors
• Sensors in visual regimes are only a small % of the possible sensors
• Infrared, hyperspectral, LADAR, SAR, FOPEN SAR, mmWave, polarimetric
• Provide thermal signatures, material properties, 3D, all weather/all day coverage, looking behind trees, walls, …
• Sensor physics is important.• Low signal to clutter ratio.• Characterization of image statistics driven by
sensor physics
Problems addressed using hyperspectral images
• Sensor design • Material classification/segmentation• Anomaly detection
– More than two decades of work• ARO MURI (2002-2007) on the science of land-based target
signatures.• DARPA DDB program, NGA• Glenn Healey: pioneer in HSI and computer vision• Larry Wolf, Equinox Corporation
• Unmixing of pixels (Remote sensing) • Compressive sensing possibilities (SPC)• Numerous books, papers, conferences…
Estimating object reflectance
4
• Radio transfer function
– irradiance to the camera
– reflectance at location (x,y)
– main light source (e.g. sun light)
– ambient light source coming from all directions, assuming constant intensity for all directions
– fraction of unblocked sky from (x,y) view
geometry factor
Nguyen and Chellappa,CVPR 2010 Workshop onBeyond visible spectrum
5
Tracking radiance
Reflectance tracking
• Robust against illumination, abrupt motion• Capable of recovering after losing track
6
Detection of land mines
• Statistical models of clutter (non-Gaussian)
• Optimal detection methods • Sup-pixel detection methods• Detection of disturbed earth
– Detection of mass graves in the Balkans
Broadwater and Chellappa, PAMI 2007, SP, 2010
Radar images
• Synthetic aperture radar (SAR), foliage-penetrating SAR (FOPEN SAR), …• Speckle noise (random fluctuations in the return signal from an object that is no
bigger than a single image-processing element)• Shape from shading – radar clinometry (USGS, Frankot, Chellappa, AI
Journal, 1990)• 3D from interferometric SAR (Zebker and Goldstein), stereo SAR• Object detection, indexing recognition (DARPA programs MSTAR, SAIP,
DDB)– MSTAR program (feature extraction, indexing, prediction, recognition) – Typical
object recognition framework– Features can correspond to scatterers (supported by physics)
• FOPEN SAR– Low signal to clutter ratio (tree trunks produce stronger returns than objects)– Symmetric alpha-stable noise
• Global hawk, TESAR sensors
R.T. Frankot and R. Chellappa, Estimation of Surface Topography from SAR Imagery Using Shape from Shading Techniques, in Physics-Based Vision: Shape Recovery, (eds.), L.B. Wolff, S.A. Shafer and G.E. Healey, Jones and Bartlett Publishers, Boston, MA, pp. 62-101, 1992.
LADAR images• Took off in the mid eighties• From machine vision to outdoor conditions• Feature (step, crease edges, surfaces,) extraction,
matching and recognition (Hypothesis prediction/verification)
– Fundamental forms I and II (Besl and Jain)
• Pulsed-Doppler LADAR (2km) for ATR• Better resolution at longer ranges• Kinect
R. Chellappa, S. Der and E.J.M. Rignot, Statistical Characterization of FLIR, LADARand SAR Imagery, in Statistics and Images, K.V Mardia, (ed.), Carfax publishers, Oxfordshire, U.K., pp. 273-312, 1994.
Opening remarks – non-traditional applications
• Road following, lane tracking and other automotive applications– Dickmanns, Pomereleu, DOT, FHWA
• Computer vision for the blind– Navigation, face/expression recognition
• Analysis of Schlieren images – Detection of oblique structures such as shock waves
and shear layers
• Understanding bee dances• Industrial inspection
Vision for Schlieren data reduction• Schlieren Imaging
– Aerodynamic visualization technique in wind tunnels -> long established
– Capture density gradients in supersonic flow
– Shock waves, shear layers and turbulent structures
– High speed imaging -> data deluge• Images complete the picture
– Offer what surface measurements may not
– Are we taking advantage of the data collected?
• Can vision extend analysis capabilities?
– Additional insight to flow unsteadiness– Desire for automation– Removal of human subjectivity
• Recast as a segmentation/feature extraction problem
Extraction of oblique structures
• Oblique flow structures ubiquitous• Physically meaningful
segmentation– Bilateral filter -> isoperimetric cut
• Shock wave and shear-layer inclination
– Canny & Hough transform• Classification
– Length & quantifiable bounds– Location enforced from labeling
• Scale implementation for robustness?
– Hough transform binning– Convergence from two Hough grids
• Success of automation 92-94%• Outer shock motion history• Recommended for Publication
in AIAA Journal• Vision: viable analysis tool
Waggle dance
Orientation of waggle axis Direction of Food source.(with respect to sun).Intensity of waggle dance Sweetness of food source.Frequency of waggle Distance of food source.Parameters of interest in the waggle dance
Waggle Axis : Average orientation of Thorax during Waggle.Duration of Waggle : Number of frames of waggle in each segment of the dance.
Anatomy/behavior modeling - prior
• Three major body parts; each body part modeled as an ellipse.
• Anatomical modeling ensures– Physical limits of body parts are
consistent; accounts for structural limitations and correlation among orientation of body parts
– Insects move in the direction of their head.
Veeraraghavan, Chellappa and Srinivasn, IEEE TPAMI, March 2008
Insects display very specific behaviors - priorsModeling behavior explicitly improve
Tracking performanceBehavioral understanding
Position tracking and behavioral analysis in a unified framework.
Result
Detect Frames of Waggle Dance by looking at
Rate of change of Abdomen OrientationAverage absolute motion of center of abdomen in the direction perpendicular to the axis of the bee.
Parameters of Interest in the Waggle Dance
Waggle Axis : Average orientation of Thorax during Waggle.Duration of Waggle : Number of frames of waggle in each segment of the dance.
Looking for a screw amid screws (MERL)
My advice to the young ones
• Look for collaborations outside EE, CS• Helps with multidisciplinary credentials• Will help with winning MURIs: best source of
support.• There are top Transactions and journals that
accept these papers!• Computer vision presents immense
opportunities outside traditional areas.
Recommended