Robotesting,

are you ready

for that yet?7th World Congressfor Software Quality

Rik Marselis

Robotesting:

Testing of robots &

Testing with robots7th World Congressfor Software Quality

Rik Marselis, 21 March 2017, Lima - Peru.

1980

Rik Marselis@rikmarselis

2007 2012 20122008

Workgroup

-member

20142009

|

Who has a robot?

© Sogeti 2017 4

Who has a robot? In 10 years all of you will !!

|Source: octaneai.com 5

Chatbots are robots too (at least in my definition ;-)

|

Towards autonomous robots

© Sogeti 2017 6

|

Traditional robots

Low intelligence for specific tasks(“dull and/or dangerous”)

© Sogeti 2017 7

|

Robots in practice

Over 50% of the dairy-farmers in the

Netherlands uses

one or more robots.

© Sogeti 2017 8

|

Manure robot

© Sogeti 2017 9

Low intelligence for specific tasks(“dull and/or dangerous”)

|

Milking robot

© Sogeti 2017 10

|

Social impact of robots

Before the farmer had to milk the cows 6:00 – 8:30 and 17:00 – 19:30

Now the farmer and his family can have dinner together at 18:00 !!

© Sogeti 2017 11De aardappeleters. Vincent van Gogh

|

Robots in healthcare

© Sogeti 2017 12

Is this OK?

Or should we

have people

taking care?

An ethical

question…

Testing with robots

Robotesting…

7th World Congressfor Software Quality

|

Robot automatically performs regression tests

© Sogeti 2017 14

|

Robot automatically performs regression tests

© Sogeti 2017 15

|

Use robots for…

© Sogeti 2017 16

Functional testing• Generate test cases• Execute test cases• Analyse the results

Brute-force testing• Generate a huge number of test cases and

execute them

Non-functional testing• Intelligent performance testen

(not just 10.000 times the same transaction)

|

Use robots for…

© Sogeti 2017 17

Big data analytics (using artificial intelligence)• Analyse live data• Generate relevant test data

Colaboration between human and machine• Robot repetative work• Human creative work

Regression testing• Robots are very good at repetative work!!

And so much more!!!!!

Robotesting…

Testing of robots

7th World Congressfor Software Quality

|

Different angles of quality for robotics

© Sogeti 2017 19

Mechanical

Electrical

Information Processing

Machine Intelligence

Business Impact

SocialImpact

For these “traditional” tests we already

have the knowledge and experience

for decades.

|

Artificial intelligence… (from the film “Ex Machina” 2015)

© Sogeti 2017 20The rise of artificial intelligence

has long been inevitable.

The question is

when, not if…

|

Different angles of quality for robotics

© Sogeti 2017 21

Mechanical

Electrical

Information Processing

Machine Intelligence

Business Impact

SocialImpact

new

new

new

|

Testing still has the well-know foundation

© Sogeti 2017 22

Quality

Risks

ConfidenceQuality attributes

Analysis and mitigation

|

Machine learning (in the film “Short circuit” 1986)

© Sogeti 2017 23

<<< vervangen door filmclip >>>

|

Testing machine learning: it’s about the input!!

© Sogeti 2017 24

The tester as “psychologist” of the intelligent machine

|

Different angles of quality for robotics

© Sogeti 2017 25

Mechanical

Electrical

Information Processing

Machine Intelligence

Business Impact

SocialImpact

Controlling the input

|

No more programming but training …

© Sogeti 2017 26

The tester as “trainer” of the intelligent machine

|

Different angles of quality for robotics

© Sogeti 2017 27

Mechanical

Electrical

Information Processing

Machine Intelligence

Business Impact

SocialImpact

Safety

Functionality (ISO25010)

Engagement

|

Testing still has the well-know foundation

© Sogeti 2017 28

Quality

Risks

ConfidenceQuality characteristics

Analysis and mitigation

|

New risks having impact on business process

© Sogeti 2017 29

Product Risk Analysis

New risks…

Well-known approach…

with other outcomes for

• Test strategy• Test intensity

• Test coverage

• Test techniques

|

Image remains the highest impact-area…

© Sogeti 2017 30

|

When will self-driving cars be on the road?

In how many years do you expect a self-driving vehicle will beallowed on the road in the Netherlands

Think of a number…

Who thinks in … years?

© Sogeti 2017 31

105

21

|

Sogeti has already tested a self-driving vehicle

The Wepod, anautonomous busin Wageningen.

On the public road sinceOctober 2016 !!

(and tested by my

colleagues of Sogeti)

© Sogeti 2017 32

|

Non-functionals become more important

TMap NEXT has 17 quality attributes

ISO9126 has 27 quality attributes

ISO25010 has 37 quality attributes

For “the new world” we have now made a list of

over 70 possible quaility attributes.(work in progress)

© Sogeti 2017 33

ethics

embodiment

empathy

|

Augmented people

© Sogeti 2017 34

|

Will robots take over your job?

Source: www.pewinternet.org/2016/03/10/public -predictions-for-the-future-of-workforce-automation/ © Sogeti 2017 35

The tester as philosopher for applying new technology…

|

Different angles of quality for robotics

© Sogeti 2017 36

Mechanical

Electrical

Information Processing

Machine Intelligence

Business Impact

SocialImpact

Ethics, Empathy and Embodiment

|

Different angles of quality for robotics

This is R&D work in progress ...

© Sogeti 2017 37

Mechanical

Electrical

Information Processing

Machine Intelligence

Business Impact

SocialImpact

Safety

Ethics, Empathy and Embodiment

Maintainability (ISO25010)

Compatibility (ISO25010)

Portability (ISO25010)

Security (ISO25010)

Operability (ISO25010)

Performance (ISO25010)

Reliability (ISO25010)

Functionality (ISO25010)

Engagement

Personality

Human friendliness

Robots testing…Existing foundations;

New possibilities and

additions!

Make sure you’re ready for it!7th World Congressfor Software Quality