Self-Driving Cars:Technologies & ChallengesDeep Conversations on Deep LearningA technical series hosted by IEEE Maine Section

W.D. Rawle, PhDSenior Member IEEEChair, IEEE Maine SectionOctober 21, 2020

2… Deep Conversations on Deep Learning Series

Advanced Driver Assistance SystemsLevel 2 Autonomy

ELE Times

3… Deep Conversations on Deep Learning Series

ADAS Examples

ToyotaCamry

TeslaModel 3

4… Deep Conversations on Deep Learning Series

Self Driving Cars !! Level 4 Autonomy

Waymo Aurora Voyage

AptivAurora/Peterbilt

Autonomy

2… Deep Conversations on Deep Learning Series

Level 0: full control from the driver. Vehicle has no support systems.

Level 1: involves basis assistance features. Level 1 cars are equipped with anti-

lock breaking (ABS) and cruise control. The driver is in full control

Level 2: semi-autonomous driving. Vehicle can drive straight, maintain lane,

and maintain control over the distance to vehicles in front of it..

Level 3: defines the moment when the on-board systems can take over all

driving functions, but only in certain situations. Driver must remain behind steering wheel all the time and be ready to take over

Level 4: a fully autonomous experience with driver behind the steering wheel.

Most of the time the vehicle can drive on its own and will handle even complicated situations on highway and city traffic. No need for driver to constantly observe traffic. At Level 4, vehicles will communicate and inform each other about maneuvers such as changing lanes

Level 5: truly self-driving cars. Operating autonomously in all conditions. There

is completely no need for people in the car to take any action. Such vehicles will not be equipped with a steering wheel Wikipedia

6… Deep Conversations on Deep Learning Series

Level 4 Autonomy Technology Landscape

AUTONOMY

INFRASTRUCTURE

SYSTEMS

PERCEPTION PREDICTION PLANNING CONTROL

SAFETY REDUNDANCY DIAGNOSTICS PERFORMANCE

MAPPING SIMULATION REMOTEOPERATIONS

RELIABLENETWORKS

Oliver Cameron, CEO VoyageMIT S.0694 Guest Lecture

7… Deep Conversations on Deep Learning Series

Challenges with Perception: FMVCONVOLUTIONAL

NEURAL NETWORK

CL

AS

SIF

ICA

TIO

N

IMA

GE

IN

PU

TO

NE

FR

AM

E P

ER

CO

LO

R

SIZE/STRIDE

MAX POOL

Fooled by a little distortionMIT 6.S094 Lex Friedman Deep LearningLecture 1

CNN Approximations• Size/stride- convolution approx.• Max Pool – data loss• Training – insufficient samples

8… Deep Conversations on Deep Learning Series

Perception, CNNs, and Scene Segmentation

Mapping every voxel to an identified object for prediction and path planning

SegNet: University of Cambridge

9… Deep Conversations on Deep Learning Series

Perception, CNNs, and Scene Segmentation Questions??

• Is the image sufficiently sampled to capture “high frequency” effects- Nyquist criteria

• Does the discretization of the convolution function compromise the output

• How much data is lost when using max pool compression

• Is fidelity of training data sufficient• Would alternate approaches (DCT, for

example) provide sufficient compression and maintain fidelity

• What would be the difference in compute resource requirements

10… Deep Conversations on Deep Learning Series

LIDAR & It’s Challenges

11… Deep Conversations on Deep Learning Series

LIDAR & It’s Challenges

Elastic Backscatter LIDAR Extinction Ambiguity

where ��� � is the LIDAR elastic backscattered power (W) from range (R), ��� is the

LIDAR system constant incorporating the transmitted optical pulse energy, the

effective telescope receiving area, and the optical path losses in the system. ������ and

������ comprise the two elements of the extinction coefficient: aerosol backscatter and

molecular absorption. ���� (R) represents the two-way admittance from the instrument

to the backscatter target. ��� (R) represents to so called overlap function,

incorporating the unit normalized cross over function between the laser illuminated

atmosphere at range R and the telescope’s field of view.

12… Deep Conversations on Deep Learning Series

LIDAR & It’s Challenges

A potential solution

13… Deep Conversations on Deep Learning Series

Level 4Autonomy & Safety

Twelve Principles of Automated Driving

VSFail safe (FS), Fail Degraded (FD)Capabilities Derived from Dependability Domains

Safety First for Automated Driving

Aptiv Services 2019

14… Deep Conversations on Deep Learning Series

Level 4 Autonomy and SafetyISO/PAS 21446 Safety of the Intended Function (SOTIF)

Area 1: Known safe behaviorMaximize safe function of system

Area 2: Known behavior that could bepotentially dangerous or possiblyunintended behavior in certaincircumstancesMinimize known potential unintendedscenarios

Area 3: unknown and potentially dangerousbehavior Minimize unknown unintended scenarios

15… Deep Conversations on Deep Learning Series

Level 4 Autonomy and SafetyA

RP

47

54

G

uid

elin

es f

or

the

Dev

elo

pm

ent

of

Civ

il A

ircr

aft

an

d S

yste

ms

16… Deep Conversations on Deep Learning Series

Level 4 Autonomy and Safety

ARP 4754 Guidelines for the Development of Civil Aircraft and Systems

17… Deep Conversations on Deep Learning Series

Level 4 Autonomy and SafetyARP 4761Guidelines and Methods for Conducting the Safety Assessment Process on Civil Airborne Systems and Equipment

Fault Tree Analysis Failure Modes and Effects Analysis Common Cause Analysis Time Limited Dispatch Dependence Diagrams Markov Analysis

1818… Deep Conversations on Deep Learning Series

RTCA DO254 Design Assurance Guidance for Airborne Electronic HardwareRTCA DO 178CSoftware Considerations in Airborne Systems and Equipment Certification

Level 4 Autonomy and Safety

1919… Deep Conversations on Deep Learning Series

RTCA DO 178CSoftware Considerations in Airborne Systems and Equipment Certification

Level 4 Autonomy and Safety

2020… Deep Conversations on Deep Learning Series

Level 4 Autonomy and Safety

RTCA DO 178CSoftware Considerations in Airborne Systems and Equipment Certification

2121… Deep Conversations on Deep Learning Series

Level 4 Infrastructure: 5G Networks

2222… Deep Conversations on Deep Learning Series

RAN – Radio Access Network

Level 4 Infrastructure: 5G Networks

Enabling Intelligent Transport in 5G NetworksEricsson Technology Review #9 2017

Thank you

23

W.D Rawle PhDSenior Member, IEEEChair, IEEE Maine Sectionwrawle@hotmail.com

Many thanks to Dr. Ali Abedi, NE Area Chair, IEEE Region 1 for providing Webex Resources