Meszaros TID 2020 Talk-Tencent templatexunitpatterns.com/~gerard/TID2020-Slides-2-up.pdf · 2020. 7. 8. · Title: Microsoft PowerPoint - Meszaros TID 2020 Talk-Tencent template.pptx

2020-07-08

1

Driving Component or Service

Development Using Business Examples

Gerard MeszarosIndependent Consultant

CTO of FeedXL.Com

[email protected]

These slides: http://tid2020.testAutomationPatterns.com

6 July 2020 Copyright Gerard Meszaros 2020 TID-1

Product &I.T. Consulting

I.T. Consulting

EmbeddedTelecom

My Background• Object Based

• Protel• Realtime Reactive OS

• Object Oriented• Smalltalk• Java• Ruby

• Agile • eXtreme Programming• Scrum• Kanban

• Unit Testing / TDD• JUnit• NUnit• TestUnit (Ruby)

• Component Spec’ing / BDD• Fit(nesse)• Robot Framework

7 July 2020 Copyright Gerard Meszaros 2020 ZIQ-2

Gerard [email protected]

•Software developer

•Development manager

•Project Manager

•Software architect

•OOA/OOD Mentor

•Requirements (Use Case)

Mentor

•XP/TDD Mentor

•Agile PM Mentor

•Test Automation Consultant

& Trainer

•Lean/Agile

Coach/Consultant

•CTO FeedXL PTY LTD

2020-07-08

2

Outline• The FeedXL Back-Story• Why Examples?• How to Drive Development using Examples?• Examples of Examples in Use• How Using Examples Affects Architecture?• Wrap-up


The FeedXL Back Story• FeedXL was built 12 years ago using Java/Struts/Velocity• As usual, timeliness was everything so test automation was an

afterthought• Original developer left 6 years ago; replacements had to learn how

the code works• Some parts are still mysterious!

• We’ve been retrofitting automated tests to improve quality• unit tests (JUnit), component tests (Fitnesse) and end-to-end workflow tests

(Robot Framework)


2020-07-08

3

Possibly Contentious Findings• While Unit Tests are useful, we aren’t finding them the most useful• While automated end-to-end tests are useful, but we mostly do this

manually• We are getting the most bang for our buck by preparing automated

component tests in the form of Executible Examples• Use of Component Examples is improving the quality of our code:

• Fitness-for-purpose (building the right system), and• Code quality (building the system right)

• simplicity, cohesiveness, reduced coupling)


Backstory Observations – Unit TestsCons:• A lot of effort to write for legacy code• Don’t provide good coverage for overall functionality• May help developers write better code but they don’t expose how

the system works to PO and business peoplePros:• Can be used with small chunks of code, possibly without refactoring

first.• Run very fast!


2020-07-08

4

Backstory Observations – End-to-End TestsPro’s:• Robot Framework allows us to build up library of domain-specific

keywords on top of Selenium so tests can be more business focussed than UI focussed

• They provide testing of overall workflows• They don’t require code to be re-organized to be tested …Cons:• They don’t improve the quality of the code!• Code needs to be deployed to server to be run; means developers

don’t run them very often• Hard to test detailed scenarios; best used for “happy path” coverage


Backstory Observations – Component Tests/SpecsPros:• Provide a good summary of the behaviour of specific parts of the

application• Force us to refactor the code to make it all accessible from the Fit

tests. • Allows the business people to see the behaviour of existing code

(once written) Cons:• Need to refactor the code to be able to use them.


2020-07-08

5

Backstory Observations - Conclusions• We are using Fitnesse Specs to describe the behaviour of existing

code to drive refactoring it into well-defined components• We can cover most of the interesting business behaviour in a way that

business people can understand it• Using Robot Framework tests sparingly to provide end-to-end

“integration” coverage• Using JUnit tests to fill in coverage for error handling, etc.

• But letting developers decide how much unit testing is required

• Get business partners to test bug fixes and new features, and to run through a few manual end-to-end scenarios before we deploy


Outline• The Back-Story• Why Examples?• How to Drive Development using Examples?• Examples of Usage• How Using Examples Affects Architecture?• Wrap-up


2020-07-08

6

5 Why’s of Test Automation Being Hard

Tests need to interact via the UI

Test Automation is

Hard!

Product isn’t designed for

testability

Why? Why?

Why?

Tests are very complex



Dev doesn’t know how (or

care)


Test Automation is

Hard!

It’s QA’s problem

Tests are automated after Dev Complete

QA isn’t involved in

product definition


testability

Why? Why?

Why?

Why?

Why?

Why?


Why?

Why?


2020-07-08

7



care)


Test Automation is

Hard!




product definition

QA is too busy testing last

release


testability

Why? Why?

Why?

Why?

Why?

Why?

Why?

Why?


Why?

Why?

QA is supposed to be “independent”

Why?


Systems Thinking - Interconnectedness


A B+

C

-

Notation:+ Increasing- Reducing

2020-07-08

8

Systems Thinking – Feedback Loops


A B+

C

-

+ +


ReinforcingCycle

BalancingCycle

Systems Thinking – Cycles: Vicious or Virtuous?


A B+

+


Vicious orVirtuous

Cycle

(Depending on whether A & B are

good or bad)

2020-07-08

9




care)


Test Automation is

Hard!




product definition


release


testability

Why? Why?

Why?

Why?

Why?

Why?

Why?

Why?


Why?

Why?


Why?

Test Automation – Systems Thinking Diagram



care)


Test Automation is

Hard!




product definition


release


testability

+ +

+

+

+

+

+

+


+

+


+

A ViciousReinforcing Cycle

2020-07-08

10

Root, Root Cause: Conway’s Law

The Architecture of System will resemble the Organization that Built it.

Development Team

TestTeam

BusinessTeam

Req’ts Code


Bus

Dev

Test

Req’ts

Resulting Architecture

Specifi-cations

Product Code

Test Code

Bus Test

Dev


2020-07-08

11

Automated Tests are Just CodeThat Tests Other Code

Who Tests This Test Code?Who Should Write This Test Code?


TDD Is One SolutionBut is hard to implement because:• Requires everyone to do it• all the time• And is a large cultural change partly because

And it doesn’t solve the whole problem:• Ensures “code quality” but not “product quality”

• Can build great software that doesn’t work as a whole

• So Business cannot directly see the value• So won’t provide the time to do it

A step in the right direction, but not far enough!


2020-07-08

12



Tests vs. Checks ; Checking vs. Testing


Learning by experimenting, including study, questioning, modeling, observation,

inference, etc.

Machine Checking

HumanChecking

Testing

https://www.satisfice.com/blog/archives/856

Human/MachineChecking

Full Automated “Testing”

Manual Script-based

“Testing”

Tools-Assisted Script-based

“Testing”

Exploratory Testing

Machine Checking

2020-07-08

13

Ways to Prepare Automated Checks• Hand-written “Test” Scripts

• Programming of unit tests by the developers (TDD)• programming of E2E tests by “testers” or “Test Engineers” (Test Automation)

• Record & Replay (or Playback) “Test” Tools (“codeless” automation)• Record while testing manually• Replay the recording (typically need to tweak it a fair bit)

• Keyword-Driven Test Tools• Essentially “programming” in a custom-designed domain-specific language

• (Potentially) Executable Specifications• Even higher level “programming” but focussing on communication and

collaboration rather than execution• A.K.A. “Executable Examples” or just “Examples”


Tests vs. Checks vs. Specifications vs. Examples?

Execution Communication

Execution

Man

ual

Auto

mat

ed

Focus

TraditionalSpecifications

Testing

Checking

2020-07-08

14

Example-Driven DevelopmentDevelopment flow

Potentially Executable

Specifications

Example Interpreter

Code ProductCode

DevTest

Dev

DevTestBus Test

Expertise or Roles , not Teams!6 July 2020 Copyright Gerard Meszaros 2020 ZIQ-27

Expertise or Roles , not Teams!

Example-Driven DevelopmentRuntime View:

ProductCode

Example Interpreter

Code

ReportCard


Specifications

DevTest

Dev

DevTestBus Test


2020-07-08

15


Example-Driven DevelopmentFeedback Loops:


Specifications ProductCode

Example Interpreter

Code

ReportCard

What they say

How they say it

Dev

DevTest

DevTestBus Test



Example-Driven DevelopmentFeedback Loops:


Specifications ProductCode

Example Interpreter

Code

ReportCard

What they say

How they say it

DevTest

DevTestBus TestDev

Isn’t This “Just” Test Automation By Another Name?


2020-07-08

16

Life Cycle of an Example

User Goal

Feature

StoryTitle

Story Narrative

Story Scenarios

Story Examples

Executable Examples

Satisfied Examples

FormalizeAutomate

DevelopProduct

Make ConcreteAdd Data

DefineAcceptance

Criteria

WorkBreakdown

Product Design & Validation

Elaborate

Each


Artifact

Activity

Next Artifact

Who Does Activities on an Example

User Goal

Feature

StoryTitle

Story Narrative

Story Scenarios

Story Examples

Executable Examples

Satisfied Examples

FormalizeAutomate

DevelopProduct

Make ConcreteAdd Data

DefineAcceptance

Criteria

WorkBreakdown

Product Design & Validation

Elaborate

DevTestBus

DevTest Bus

DevTest

Bus

Each


Artifact

Activity

Next Artifact

2020-07-08

17

Outline• The Back-Story• Why Examples?• How to Drive Development using Examples?• Examples of Usage

• Workflow Examples• Component Examples

• How Using Examples Affects Architecture?• Wrap-up


Preparing Good Examples


2020-07-08

18

Example: Notifying of Bank Transactions

Configure Notification

Rules

AccountHolder

Process Transaction

TransactionSettlement

Not

ify

base

d on

cha

rge

Type




Rules

AccountHolder

Process Transaction


Not

ify

base

d on

cha

rge

Type

Not

ify

base

d on

cha

rge

Cont

inen

tN

otif

y ba

sed

on c

harg

e Co

untr

yN

otif

y ba

sed

on c

harg

e Ci

ty &

Sta

te


2020-07-08

19



Rules

AccountHolder

Suspend Notification

Resume Notification

Process Transaction


Not

ify

base

d on

cha

rge

Type

Not

ify

base

d on

cha

rge

Cont

inen

tN

otif

y ba

sed

on c

harg

e Co

untr

yN

otif

y ba

sed

on c

harg

e Ci

ty &

Sta

te

Sus

pend

Not

ific

atio

n


Checking Notifications• Open MegaBank app• Log in as “BobMa” with password ******• Click on “Manage Notifications” tab• Click on “Add New Rule” button• Select account “10035692877”• Type “Default Rule” into field “rule name”• Type “1000” into field “threshold amount”• Click on “all transaction types” radio button• Click on “all locations” radio button• Click on “save changes” button• … • …


2020-07-08

20

Checking Notifications – 1/2


Given: User and Accounts

Another attempt:

When: Notification

Rule is Configured

Then:Notification Rule

Configured and Active

Then:Notification Rule is

Active

Checking Notifications – 2/2


Then: ExpectedNotifications

When: The Transactions to be

processed

Another attempt:

2020-07-08

21

Given: User and Accounts

When: Notification

Rule is Configured

Then:Notification Rule

Configured and Active

Then:Notification Rule is

Active

Refactoring to Workflow Example



“If it isn’t essential to conveying the essence of the behavior, it is essential to not include it.”


2020-07-08

22

Then: ExpectedNotifications

When: The Transactions to be

processed





2020-07-08

23



Refactoring to Workflow ExampleGiven: User &

Thresholds

When: Transactions

Are Processed

Then: We Expect

Notifications6 July 2020 Copyright Gerard Meszaros 2020 ZIQ-46

2020-07-08

24

9991113333

Given: User &

Thresholds

When: Transactions

Are Processed

Then: We Expect

Notifications

credit

credit

Refactoring to Workflow Example - Validating


9991113333

Refactoring to Workflow Example - Validating

Broad Scope (Multi-Actor); Minimum Detail (per Actor/Transaction);

Given: User &

Thresholds

When: Transactions

Are Processed

Then: We Expect

Notifications

credit

credit


2020-07-08

25

Outline• The Back-Story• Why Examples?• How to Drive Development using Examples?• Examples of Usage

• Workflow Examples• Component Examples

• How Using Examples Affects Architecture?• Wrap-up


Test Automation Pyramid – With Traditional Automation

Pyramid originally proposed by Mike Cohn

Exploratory Tests


ComponentTests (API)

Whole SystemEnd-to-EndTests (GUI)

UnitTests

2020-07-08

26

Test Automation Pyramid – Desired Outcome

Pyramid originally proposed by Mike Cohn

Exploratory Tests

Unit Tests


SystemTests (GUI)


Unit Tests TDD/BDD

Behavior Specification at Right Level


SystemTests (GUI)

Detail

Hig

h

Low

ATDD/EDD


2020-07-08

27



WholeSystemTestsD

etail

Hig

h

Low

Broad NarrowScope

ATDD/EDD



WholeSystemTests


Detail

Hig

h

Low

Broad NarrowScope6 July 2020 Copyright Gerard Meszaros 2020 ZIQ-54

2020-07-08

28


WholeSystemTests


Rules &Algorithms

IncompleteSpec

Toomuch detailUnmaintainable

Detail

Hig

h

Low

Broad NarrowScope

Transactions(Use Cases)

Multi-Use Case

Workflows


Filling in the Details

Rules &Algorithms

Multi-Use Case

WorkflowsTransactions

(Use Cases)

IncompleteSpec

Too much detailUnmaintainable

Broad Narrow

Detail

Hig

h

Low

Scope6 July 2020 Copyright Gerard Meszaros 2020 ZIQ-56

2020-07-08

29

Business Rule Example (Component Tests)Process Transaction

Threshold per Charge Type

Configuration


Business Rule Example (Component Tests)Process Transaction

Threshold per Charge Type

Narrow Scope (Single Rule)High Detail (Everything that matters)

Configuration Given these rules

Then: The answer should be

When we ask NotificationRequired? with

this transaction:


2020-07-08

30



Test-After Architecture

System Under Test

ProcessTransaction

ConfigureNotificationThreshold

TransactionInterface

ConfigurationUser

Interface Should weNotify

andDo

Notification?

NotificationRules

NotificationLog

Bus

Workflow Test

Test Dev

! ! ! !


2020-07-08

31


ConfigurationUser

Interface

System Under Test


Example-Driven ArchitectureWorkflow Examples Drive API

Workflow Example

Dev

Workflow Keyword

Interpreter

Bus

Test

ProcessTransaction

Process Transaction (including:

Do We Notify and How to Notify)

Dev



ConfigurationUser

Interface

System Under Test


Example-Driven Architecture... but not Component Structure

Workflow Example

Dev

Workflow Keyword

Interpreter

Bus

Test Dev

NotificationMethod Example

NotificationRule Example Process

Transaction

System Under Test


2020-07-08

32

Example-Driven ArchitectureRules Examples Drive Component Dev’t

Bus

Test


NotificationRule Example

System Under Test


System Under Test

Example-Driven ArchitectureRules Examples Drive Component Dev’t

DoNotification.

NotificationLog



Should weNotify?

ExampleInterpreter

ExampleInterpreter

DevBus

Test

Dev

NotificationRules


2020-07-08

33

NotificationRules

Example-Driven ArchitectureRules Examples Ensure Testability

Should weNotify?

NotificationRules Fake


ExampleInterpreter

DevBus

Test

Dev6 July 2020 Copyright Gerard Meszaros 2020 ZIQ-65

System Under Test

Example-Driven ArchitectureRules Examples Ensure Testability

DoNotification.

NotificationRules

NotificationLog


Bus

Test

Dev

NotificationRules


NotificationLog Fake

NM ExampleInterpreter

Dev


2020-07-08

34

System Under Test

Example-Driven ArchitectureUnit Tests Fill in Gaps in Coverage

DoNotification.

NotificationRules

NotificationLog


Bus

Test

Dev

NotificationRules


NotificationLog Fake

NM ExampleInterpreter

Dev

UnitTestsUnit

TestsUnitTestsUnit

Tests


System Under Test

Putting it All TogetherWorkflow Examples + Component Examples + Unit Checks/Tests

DoNotification.

ProcessTransaction

NotificationRules

NotificationLog



Should weNotify?

ExampleInterpreter

ExampleInterpreter

DevBus

Test Dev

UnitTestsUnit

TestsUnitTestsUnit

Tests


ConfigurationUser

Interface


Workflow Example

Workflow Keyword

Interpreter


2020-07-08

35

System Under Test

Putting it All TogetherWorkflow Examples + Component Examples + Unit Checks/Tests

DoNotification.

ProcessTransaction

NotificationRules

NotificationLog



Should weNotify?

ExampleInterpreter

ExampleInterpreter

DevBus

Test Dev

UnitTestsUnit

TestsUnitTestsUnit

Tests


ConfigurationUser

Interface


Workflow Example

Workflow Keyword

Interpreter


Unit Tests

Component

System



2020-07-08

36

tools

Tools

As adapted by Gerard Meszaros

Tools (FeedXL Preferences Only)For End-to-End

Examples

For Component

Examples

2020-07-08

37

FeedXL Results• Business people buy in because they can see the examples

• Can refer to examples when questions come up• Are asked for examples to automate when defining new functionality• Have to do a lot less manual testing for each feature

• Developers buy in because • They see value in the examples because they get the functionality right 1st time• Can point to examples when questions come up from business• It’s less work than unit tests or Robot Framework tests

• PO (i.e. Me!) likes it because:• I’m seeing better quality, sooner• I have higher confidence that we are building the right thing• I can see that the process is being followed (unlike unit tests which are invisible)


After State – Back to Our Systems Diagram

Dev knowshow (and

cares)

Examples don’tneed to interact

via the UI

Automated checking is

Easy!

Automation is Dev’s problem

Examples are automated before

Dev starts

QA is involved in product definition

QA has time for Examples and

Exploratory testing

Product isdesigned for

testability

+ +

+

+

+

+

+

+

Examples are simple

+

+

A Virtuous Cycle

2020-07-08

38

Summary of Benefits of using Executable Examples• Defining Examples first helps us understand

• What the system should do• How it should be structured to ease testability

• Using business language for examples• Reduces opportunities for “lost in translation”

• Developers responsible for automation• Ensures the design is testable• Provides immediate feedback on function & testability.• Reduces duplication with unit tests

• Can be used with new code *and* legacy code• But start with Workflow or Component tests, not unit tests

Speaker InformationGerard Meszaros

CTO FeedXL PTY LTD

Lean/Agile Coach/Consultant/Trainer

Contact me about:

• Agile development practices

• Agile product ownership

• Software craftsmanship

• Acceptance Test-Driven Development

• Executable Examples


Gerard [email protected]

Documents

Meszaros TID 2020 Talk-Tencent templatexunitpatterns.com/~gerard/TID2020-Slides-2-up.pdf · 2020. 7. 8. · Title: Microsoft PowerPoint - Meszaros TID 2020 Talk-Tencent template.pptx