Continuous Security Testing

Continuous Security TestingAcceptance Test Driven Approach

Sunday, 15 December, 13

Who am I?

• Agile, TDD Coaching, Ugly Code Cleaning Dude

• I love coding - Java, C#, Javascript, C/C++, PHP, Perl, and some weird ones

• I speak English, Cantonese, and Mandarin

2

Odd-e Pte. Ltd.Steven Mak 麥天志Agile CoachHong KongEmail: [email protected]: www.odd-e.comTwitter: stevenmak


mailto:[email protected]


http://www.odd-e.com


Do you automate your tests?

3


Is that what you feel?

4

Script Unreadable?

Keep Changing?

Time consuming to write?


Technical Activity

Workflow

Specification pyramid

5

RuleClarity

Stability

Specification

Users can understand

AutomationTechnical


Use Examples

6

With 3 judges giving scores 4, 20, and 18, the displayed score should be 42.

When the first 2 judges have given their scores, e.g. 10 and 5, the intermediate score of 15 should be displayed already.

No scores displayed as a dash (–), not zero.

Maximum score from a judge is 20 points!


Examples, Tests, and Spec

7

Examples Tests

Requirements

can become

elaborate verify


More ideas from• Threat Modelling• Session-Based Test Management / Exploratory Testing• Product Requirement• Experts

8


Avoid handoff

9


Avoid imperative• login• enter username• enter password• enter homepage• click category• choose product• put it on shopping cart• click generate order• .....

10



11

Given I selected a doll in shopping cartWhen I generate orderThen the order should contain dolland the price is 83.55



12

Given I selected a doll in shopping cartWhen I generate orderThen the order should contain dolland the price is 83.55

This “Given When Then” is a common pattern called Gherkin


Good ones• Focus on business, not software design• Not coupled with code• Not coupled with UI• Concise• Use domain languages

13

Getting us towards Living Documentation and can be executed against existing

system


Robot Frameworkwww.robotframework.org

14


http://www.robotframework.org

http://www.robotframework.org

Test Tools

Robot Architecture

15

Test Data (Tables)

Robot Framework

Test Libraries

System Under Test

Test Library API

application interfaces

Robot comes with a number of built-in test libraries and you can (should!) add your own.

Test libraries can use any test tool necessary to interact with the system under test.


It's all in the tables

16


Test Cases are composed of keyword-driven actions

17

!"#$%&'()*+%),'-./()0



17

!"#$%&'()*+%),'-./()0

this is the name of a test case



17

!"#$%&'()*+%),'-./()0

this is the name of a test casethese keywords form the test case



17

!"#$%&'()*+%),'-./()0


keywords receive arguments


2 types of keywords

18


2 types of keywords

18

We can import keyword libraries for a test case


2 types of keywords

18


...and libraries may be configured, too.


2 types of keywords

18



This keyword comes from the imported library.


2 types of keywords

18



This keyword comes from the imported library.

This is a user keyword, implemented in table format.(Think macros composed of other macros.)


19

Data-driven test cases


keywords receive arguments


20

using Template

*** Test Cases ***Email Delivered Acceptance Rule [Template] Confirm Email Delivered Workflow [email protected] [email protected] 3asyp3asy 1 [email protected] [email protected] 3asyp3asy 0

*** Keywords ***Confirm Email Delivered Workflow [Arguments] ${sender} ${recipient} ${password} ${number_of_emails_expected} Open Mail Box ${MAIL_SERVER} ${recipient} ${password} Count Mail Received ${sender} ${number_of_emails_expected}

Keyword used as template

test data feed as arguments


Given-when-then (BDD)

21

*** Test Cases ***Addition Given calculator has been cleared When user types "1 + 1" and user pushes equals Then result is "2"

*** Keywords ***Calculator has been cleared Push button C

User types "${expression}" Push buttons ${expression}

User pushes equals Push button =

Result is "${result}" Result should be ${result}

this is the name of a test case

these keywords form the test case


Variables

22

!"#$"%&'(

)#*+,-*++"./,&$.'0

!"#$%&'"(()*+,*%-."/012345167&89:&."(()*+,*%-.";400<=2>6?@89>@."A$B'.C'CD8'A-Sunday, 15 December, 13

Other choices• Cucumber• Fitnesse

23


24

An Example


25

*** Settings ***Resource resource.txt

*** Test Cases ***Checking Opened Ports [Template] Only these ports are opened 22 25 80 135 139 445

*** Keywords ***Only these ports are opened [Arguments] @{expected_ports} @{actual_ports_opened}= Scan with Fast Mode ${HOST} List Should Contain Sub List ${actual_ports_opened} ${expected_ports}

*** Settings ***Library nmapLibraryLibrary Collections

*** Variables ***${HOST} www.scrumprimer.org

import nmap

class nmapLibrary: def scan_with_fast_mode(self, host): nm = nmap.PortScanner() nm.scan(str(host), arguments="-F") return [str(port) for port in nm[str(nm.all_hosts()[0].encode())].all_tcp()]

resource.txt

port_scanning.txt

nmapLibrary.py (with python-nmap)


http://www.scrumprimer.org

http://www.scrumprimer.org

26

pybot -d output nmap.txt ==============================================================================Port Scaning ==============================================================================Checking Openned Ports | PASS |------------------------------------------------------------------------------Nmap | PASS |1 critical test, 1 passed, 0 failed1 test total, 1 passed, 0 failed==============================================================================Output: /Users/stevenmak/Work/robotframework/securityTests/2013.12.14VXCon/output/output.xmlLog: /Users/stevenmak/Work/robotframework/securityTests/2013.12.14VXCon/output/log.htmlReport: /Users/stevenmak/Work/robotframework/securityTests/2013.12.14VXCon/output/report.html

run the test:

report: (also available in xml format for Jenkins integration)


27


More to wrap & integrate• w3af• garmr• arachni• dirb• sslyze• sqlmap

28


Acceptance Test Driven Development

29

Discussin workshop

Developin concurrence

Deliverfor acceptance


30

Discussin workshop



Focus on customer collaboration and user

engagement. Try to get as many of these people attend

as you can.

Product OwnerDev Team

Users

IT operations

Help DeskTech Writers

?


31

Discussin workshop








31

Discussin workshop







Robot tests are written in tables so that computers can read them


32


Discussin workshop



Collaboration is key

33

team gets feedback earlier

scope of work is clear and

understood by all

team understands what they're implementing

shared language and vocabulary is

built

team collaborates closely with

product owner


CITCON Hong Kong

34

• When: Apr 11 & 12, 2014• Cost: Free• Registration: contact me• Sponsorship Welcome!

http://citconf.com/hongkong2014/




Thank you for spending time with me this evening.More feedback can be sent to:

35

Odd-e Hong Kong Ltd.Steven Mak 麥天志Agile CoachHong KongEmail: [email protected]: www.odd-e.comTwitter: stevenmak






Software

Continuous Security Testing