Smoke Tests @ DevOps-Hamburg 06.02.2017

SmokeTestsWhy you should try to burn down

your production environment

Comments & Feedback

Sebastian ThoßChapter lead Backend

Disclaimer

Agenda• Types Of Tests

• SmokeTests

• Server Architecture

• Questions & Answers

Types Of Tests

Test Pyramid

UNIT TESTS

Class 2

Class 3

Mock

Mock

Unit Tests

Class 1

Class 4

Test Pyramid

INTEGRATION TESTS

UNIT TESTS

Class 3 Class 4Mock Mock

Integration Tests

Class 1 Class 2

Test Pyramid

INTEGRATION TESTS

UNIT TESTS

100% CODE COVER

AGE

Test Pyramid

ACCEPTANCE TESTS

INTEGRATION TESTS

UNIT TESTS

Acceptance Tests

Class 1 Class 2

Class 3 Class 4

Test Pyramid

ACCEPTANCE TESTS

INTEGRATION TESTS

UNIT TESTS

?

$$$

¢

SmokeTests

What are SmokeTests?

In computer programming and software testing, smoke testing is preliminary testing to reveal simple failures severe enough to reject a prospective software release. Source: https://en.wikipedia.org/wiki/Smoke_testing_(software)

SmokeTests should…• … be simple

• … be fast

• … test pages with optional parameters too

• … cover at least all URLs in google index

• … use a manual maintained list of URLs

How do SmokeTests work?

https://www.my-application.com/foo

<html><body>…</body></html>

TTFB: 65ms

HTTP 1.1/200 OK

SmokeTest Client

CI Server

Application Under Test

Production Server

How do SmokeTests work?


<html><body>…</body></html>

TTFB: 320ms

HTTP 1.1/200 OK

SmokeTest Client

CI Server


Production Server

What should SmokeTests validate?• Correct Server

• Status Code

• Time To First Byte

• If Body Is Provided

SmokeTests are NOT Acceptance Tests

SmokeTest Client

HTTP 1.1/200 OK

<html> <head> <title>Foo</title> <body> <div id="bar"><span>foobar</span></div> </body> </html>

namespace Kartenmacherei\Testing;

use PHPUnit_Framework_TestCase;

class SmokeTest extends PHPUnit_Framework_TestCase { /** * @dataProvider furyUrlProvider * * @param Url $url */ public function testFuryUrl(Url $url) { $result = $this->sendGetRequest($url);

$this->assertSame(200, $result->getStatusCode()); $this->assertNotEmpty($result->getBody()); $this->assertLessThanOrEqual(100, $result->getTimeToFirstByteInMilliseconds()); }

public function furyUrlProvider() { $urls = ['http://www.kartenmacherei.de', …]; $urlCollection = UrlCollection::fromStrings($urls);

return $urlCollection->asDataProviderArray($urlCollection); }

How to speed up SmokeTests?

Concurrent SmokeTests

SmokeTest Client

CI Server


Production Server

https://www.my-application.com/baz

https://www.my-application.com/bar



https://www.my-application.com/bazSmokeTest

Client

CI Server


Production Server



TTFB: 34ms


https://www.my-application.com/foobazSmokeTest

Client

CI Server


Production Server




SmokeTest Client

CI Server


Production Server

TTFB: 65ms

https://www.my-application.com/foobaz




SmokeTest Client

CI Server


Production Serverhttps://www.my-application.com/123




SmokeTest Client

CI Server


Production Server

TTFB: 620ms

https://www.my-application.com/123



There is a Lib for SmokeTests

DjThossi/Smoke-Testing-PHP

DataProvider

Test

Application

HTTP Requests

HTTP Responses

PHPUnit

Call

Result[]

Result

class SmokeTest extends PHPUnit_Framework_TestCase { use SmokeTestTrait;

/** * @dataProvider myDataProvider */ public function testExample(Result $result) { $this->assertSuccess($result); $this->assertTimeToFirstByte(new TimeToFirstByte(100), $result); $this->assertBodyNotEmpty($result); $this->assertHeaderExists(Header::fromPrimitives(‘App-Server', ‘Fury’), $result); }

public function myDataProvider() { $urls = ['http://www.kartenmacherei.de', …]; $options = new SmokeTestOptions( UrlCollection::fromStrings($urls), new RequestTimeout(2), new FollowRedirects(true), new Concurrency(3), new BodyLength(500) );

return $this->runSmokeTests($options); }

Server Architecture

Webserver (Router)

Webserver

FURY Frontend

Server A

K/V StoreSearch

FURY Backend

Webserver

FURY Frontend

Server B

K/V StoreSearch

FURY Backend

Webserver (Router)

Webserver

FURY Frontend

Server A

K/V StoreSearch

FURY Backend

Webserver

FURY Frontend

Server B

K/V StoreSearch

FURY Backend

active = A

Webserver (Router)

Webserver

FURY Frontend

Server A

K/V StoreSearch

FURY Backend

Webserver

FURY Frontend

Server B

K/V StoreSearch

FURY Backend

active = B

Webserver (Router)

Webserver

FURY Frontend

Server A

K/V StoreSearch

FURY Backend

active = B

RDBMS Read Slave

Webserver

FURY Frontend

Server A

K/V StoreSearch

FURY Backend

Webserver (Router)

Build Server

Deploy Code

active = B

RDBMS Read Slave

Webserver

FURY Frontend

Server A

K/V StoreSearch

FURY Backend

Webserver (Router)

Build Server

Deploy Code

Collect & Export

active = B

RDBMS Read Slave

Webserver

FURY Frontend

Server A

K/V StoreSearch

FURY Backend

Webserver (Router)

Build Server

Deploy Code

Collect & Export

Smoke Tests

active = B

RDBMS Read Slave

Webserver

FURY Frontend

Server A

K/V StoreSearch

FURY Backend

Webserver (Router)

Build Server

Deploy Code

Collect & Export

Smoke Tests

active = B

RDBMS Read Slave

Webserver

FURY Frontend

Server A

K/V StoreSearch

FURY Backend

Webserver (Router)

Build Server

Deploy Code

Collect & Export

Smoke Tests

active = B

RDBMS Read Slave

Webserver

FURY Frontend

Server A

K/V StoreSearch

FURY Backend

Webserver (Router)

Build Server

Deploy Code

Collect & Export

Smoke Tests

Switch to A

active = Bactive = A

Conclusion• Write tests

• Get 100% coverage

• SmokeTest your Website

• Only activate server if it didn't start smoking

Q&A

https://www.facebook.com/kartenmacherei/

[email protected]

http://inside.kartenmacherei.de/job.html

https://tech.kartenmacherei.de/

@techdotkam

Comments & Feedback

Technology

Smoke Tests @ DevOps-Hamburg 06.02.2017