Towards Declarative Safety Rules forPerception Specification Architectures

Johann Thor Mogensen Ingibergsson

MMMI, University of Southern Denmarkjoint work with Ulrik Pagh Schultz and Dirk Kraft

Field Robots

• Why field robots?

– Dangerous work.

– Decreasing workforce.

– Ecological Concerns.

• SAFE Project.

2

Context: Safety Certification within Agriculture

• Why certification? Liability!– Robot causes damage due to manufacturing defects.– Robot causes damage simply by acting or reacting. 3

4

• No standard is available.

• Other Industries? Avionics?

• Interpretation for agriculture and field robots.

How to Certify Field Robots?

Issues with current standards.

• Issue: Research is solution driven.

• Issue: 20 papers in non-development-related, suggesting apporaches areinvestigated.

5[Source: Ingibergsson, Kuhrmann & Schultz, PROFES2015]

How is Certification Done within Software for Field Robots?

How to Certify Field Robots?

• Issues with current standards.– Issue: Use of standards is limited.– Issue: Loose connection between

development practices and standards.

6

Certifying Field RobotsBased on Interpretation

• ISO 13482Risk assessment

• ISO 13849Functional safety Mechanics.

• ISO/DIS 18497Performance

• ISO 25119Functional safety electronics.

• IEC 61496Electro-Sensitive ProtectiveEquipment (EPSE).

7

Implications of Standards on Development of Field Robots in Practice?

8

Functional Safety vs PerformanceFunctional Safety Performance

9

Correct Split… ?

Example: Simple Vision Pipeline

10

Rectify and

minimizedistortion

Dense Disparity

Image

Rectify and

minimizedistortion

Vision Algorithm

Segmentating

ObjectsAlgorithm

Smooth-ing /

filtering

Vision Pipeline Described with RPSL

11

Camera_Left : SensorComponent

ImageType: RawColor: Mono_Bayer

Bayer2Mono_Left: ProcessingComponent

Region: full

Rectify_Undistort_Left: ProcessingComponent

Region: full

Camera_Right : SensorComponent

ImageType: RawColor: Mono_Bayer

Bayer2Mono_Right: ProcessingComponent

Region: full

Rectify_Undistort_Right: ProcessingComponent

Region: full

DisparityMap: ProcessingComponent

Region: full

PointCloud_3D: ProcessingComponent

Region: full

Size: 480x752

Size: 480x752

SystemType: Hardware

SystemType: HardwareSystemType: Software SystemType: Software

SystemType: Software SystemType: Software

SystemType: Software SystemType: Software

Algorithm: Bayer2MonoComplexity: Low

Algorithm: Bayer2MonoComplexity: Low

Algorithm: RemapComplexity: Low

Algorithm: RemapComplexity: Low

Algorithm: DiparityMapComplexity: Medium

Algorithm: DisparityMapTo3DComplexity: Low

Rectify and

minimizedistortion

Dense Disparity

Image

Rectify and

minimizedistortion

Vision Algorithm

Segmentating

ObjectsAlgorithm

Smooth-ing /

filtering

[Source: Hochgeschwender, Schneider, Voos & Kraetzschmar, SIMPAR2014]

Camera_Left : SensorComponent

ImageType: RawColor: Mono_Bayer

Bayer2Mono_Left: ProcessingComponent

Region: full

Rectify_Undistort_Left: ProcessingComponent

Region: full

Camera_Right : SensorComponent

ImageType: RawColor: Mono_Bayer

Bayer2Mono_Right: ProcessingComponent

Region: full

Rectify_Undistort_Right: ProcessingComponent

Region: full

DisparityMap: ProcessingComponent

Region: full

PointCloud_3D: ProcessingComponent

Region: full

Size: 480x752

Size: 480x752

SystemType: Hardware

SystemType: HardwareSystemType: Software SystemType: Software

SystemType: Software SystemType: Software

SystemType: Software SystemType: Software

Algorithm: Bayer2MonoComplexity: Low

Algorithm: Bayer2MonoComplexity: Low

Algorithm: RemapComplexity: Low

Algorithm: RemapComplexity: Low

Algorithm: DiparityMapComplexity: Medium

Algorithm: DisparityMapTo3DComplexity: Low

How to Introduce Functional Safety

12

Based on Interpretation• ISO 25119 – Functional safety

electronics.– Develop software and hardware

according to the standard.– Software could be subjected to

Misra, to create a foundationacross standards.

• IEC 61496 – Electro-Sensitive Protective Equipmen (EPSE).– Fault: Shall force the system to a

safe-state, i.e. full stop.– Multiple Faults: Shall not

influence the above reaction.– Periodic tests: Ascertain

functionality.

Safety-critical

hardware

ERROR: Go to safe state.

DSL Proposal

h=Bayer2Mono_Left.output.histogram;

length(nonempty(h.bins))/length(h.bins)>0.1;

max(h)-min(h)>1000p;

length(PointCloud_3D.output.inArea

(Camera_Left_Landmark))>900 3D points;

13

DSL Test images

14

Conclusion

Contributions

• Analysis of safetystandards in the agricultural domain.

• Language concept for extending RPSL with safety annotations.

15

Future work• Code generation for

safety-criticalhardware.

• Systematic evaluationof language design for the safety domain.

• Evaluation by safetyexperts.