Embed Size (px)
DESCRIPTION
Embedding computer vision SW for in-vehicle applications requires the optimization of algorithms to fit into low cost and low consumption HW. Such optimization is a task substantially centered in improving the efficiency of the implementations, typically focused on the migration of algorithms to massive parallelization HW. The development cost associated with these actions can be prohibitive depending on the nature of the algorithms, which may be highly non-linear processes, iterative and recursive, partially sequential and often not fully parallelizable. As an alternative, co-design methodologies has shown to be a successful and feasible approach, which consists on coordinated work of the SW and HW development teams, with iterative evaluations of prototype SW implementations in reconfigurable HW platforms that allow detecting memory or processing bottlenecks. The analysis then guide the next iteration which may also consists on redesigning the algorithm and not only on implementation issues. In this paper we describe this general process applied to computer vision methods optimization. As a case study we present the improvements we have achieved designing a single camera vehicle detection system using the mentioned methodology to migrate it into an embedded platform (Xilinx Zynq 7000).
Citation preview
1September 8, 2014
Optimization of computer vision algorithms in codesign methodologies
Marcos Nieto, Juan Diego Ortega, Oihana Otaegui, Andoni Cortés
Vicomtech-IK4
1- ITS World Congress 2014,
September 8 -
2September 8, 2014
Overview
• Motivation - ADAS• Computer vision for ADAS• Codesign Methodology• Test case
– Vehicle detection Prototype #1– Vehicle detection Prototype #2– Vehicle detection Prototype #3
• Results• Conclusions
1
3September 8, 2014
↑ Safety ↑ Traffic Mobility ↓ Energy Consumption
ADASAdvanced Driver Assistance Systems
Motivation - ADAS
• Advanced Driver Assistance Systems (ADAS)
2
4September 8, 2014
Computer vision for ADAS
• Camera sensors– Cheap electronics– Greater quantity and quality of information– Perception similar to human– Can be fused with other sensors/systems: GPS, RADAR, V2V, V2I
Traffic sign recognition Safety distance Driver Monitoring Lane Keeping Pedestrian detection
3
5September 8, 2014
Computer vision for ADAS
• Embedding computer vision algorithms is a challenge
– (HW) Specify (camera, illumination, algorithm...)
– (SW) Optimize algorithm
– (HW) Select target processor
– (SW/HW) Fine tune algorithm parameters
– (SW/HW) Optimize code• Fit process flow to HW capabilities• Exploit massive parallelization HW
– (SW/HW) Add installation / calibration / maintenance / data logging procedures
C++ C, Assembly (ARM)C++ VHDL, CUDA (FPGA, GPU)
4
6September 8, 2014
Codesign Methodology
• Coordinating SW and HW developments– Using Re-configurable HW platforms– Exploiting platform-independent SW libraries
User requirements
Design SW Design/Select HW
Integrate SW in HW - Prototype
Profile SW in HW
Protpotype Validation
User requirements
Design SW Design/Select HW
Integrate SW in HW - Prototype
Profile SW in HW
Protpotype Validation
Itera
te
Last iteration Itera
te
Iterate
Co-design development cycle Traditional SW/HW development cycle
5
7September 8, 2014
Codesign Methodology
• SW and HW developments can run in parallel– More efficient joint development– SW design can be revised after profiling reports in target HW– Expensive code migration/optimization is significantly minimized
“It is better to revise the algorithm rather than spend months optimizing the code”
• The development cycle– First SW prototype can be directly tested in HW– Profiling provides bottleneck information– Revision of SW design solves detected bottleneck– Iterate until SW and HW meet requirements– Optimize final SW in target HW
6
8September 8, 2014
Vehicle detection Prototype #1
Multiscale
HOG
Adaboost Clustering SVM 1.0 Tracking
Shadows, edges,
symmetry
Training Training
Images
Brute- force Mu ltiscale Sca nning
• Multiscale scanning and detection-by-classification– Traditional (and successful) approach in computer vision– A car model is trained with a large dataset of example
images– New images are scanned at different scales and each
candidate region is compared with the model
7
9September 8, 2014
Vehicle detection Prototype #2
• Bottleneck #1: – It can propose absurd hypotheses according to size
and position that are anyway compared with the model
• Solution:– Exploit perspective information to create a grid of possible locations of
vehicles
Perspective Adaboost Clustering SVM 2.0 Tracking
TrainingTraining
HOG
Shadows, edges,
symmetry
Images
8
10September 8, 2014
Vehicle detection Prototype #3
• Bottleneck #3:– Some hypotheses are really likely not cars, and the comparison with
the model is costly
• Solution:– No need to fully compare with the trained model– Run pre-analysis to discard clearly not car candidates using simpler
features: shadows, simmetry, edges, etc.
Pre-analysis / Integral
SVM 2.0 TrackingPerspective clustering
Perspective Training
HOGShadows,
edges, symmetry
Images
Pre-selected candidates9
11September 8, 2014
Results
• Profile comparison– Only CPU, without code optimization– 10x speed up from prototype #1 to #3 -> real time performance
Xilinx Zynq Dual ARM® Cortex-A9 MPCore, 866 MHz, 1 GB
10
Features (ms) HOG (ms) Adaboost Cluster SVM (ms) Av. Time (ms)
Prototype #1 135.88 252.23 18.45 0 13.67 420.23
Prototype #2 56.43 112.56 5.88 0 4.12 178.99
Prototype #3 11.23 6.19 0 10.22 4.12 31.76
25
75
125
175
225
275
325
375
425
Tim
e (m
s) p
er fr
ame
12September 8, 2014
Results
• Sample video(s)
PETS 2001 dataset
11
Frontal camera Rear camera
13September 8, 2014
Conclusions
• Co-design strategies can reduce time-to-market for computer vision systems
• It allows SW and HW teams work in parallel
• Iterative SW re-design is possible
• We have shown a vehicle detection example
• Three prototypes with x10 speed-up without code optimization nor use of special HW
12
14September 8, 2014 13
Booth 3008
Live Demo at our booth:
Thank You!
