ThoughtWorks Technology Radar Roadshow - Sydney

TECHNOLOGYRADARMay 2015 — Our thoughts on the technology and trends that are shaping the future

1

2

3

TECHNOLOGY ADVISORY BOARD

4

5

6

THEMES FOR THIS ISSUE

7

TECHNIQUES8

TECHNIQUES8

9

ADOPT 1. Consumer-driven contract testing NEW

2. Focus on mean time to recovery 3. Generated infrastructure diagrams NEW 4. Structured logging

TRIAL 5. Canary builds 6. Datensparsamkeit 7. Local storage sync 8. NoPSD 9. Offline-first web applications NEW 10. Products over projects NEW 11. Threat Modelling NEW

ASSESS 12. Append-only data store 13. Blockchain beyond Bitcoin 14. Enterprise Data Lake 15. Flux NEW 16. “git-based CMS” NEW 17. Phoenix environments NEW 18. Reactive architectures NEW

HOLD 19. Long lived branches with Gitflow 20. Microservice envy 21. Programming in your CI/CD tool 22. SAFe™ 23. Security sandwich 24. Separate DevOps team

TECHNIQUES

10

TECHNIQUESArchitectures for the digital world (theme: innovations in architecture)

10

TECHNIQUES18

15

14

12

REACTIVE ARCHITECTURES

FLUX

ENTERPRISE DATA LAKE

APPEND-ONLY DATA STORE

Architectures for the digital world (theme: innovations in architecture)

Attrition

Acquisition

Retention

Activation

Referral

Ads

ARCHITECTURES FOR THE DIGITAL JOURNEY

11

Emails

Google

Legacy Systems

RIGHT DATA, RIGHT PLACE, RIGHT TIME

12

Legacy SystemsLegacy Systems Web Analytics Operational Metrics Insights

13

Business Logic

(functions)

Legacy Systems

Archivers

File Store (S3)

Insights Analytics

Reports, Model

Parameters

Spark, Hadoop

Microservices

Append-only

Database

Flux-based Web Application

User Actions

View Rendering

(react.js)

Transactions, Web Analytics,

Operational Logs

Subscribed Events Event Queue

(Time Series Database, Apache Kafka, AWS Kinesis,

Eventstore, …)

13

Business Logic

(functions)

Legacy Systems

Archivers

File Store (S3)

Insights Analytics

Reports, Model

Parameters

Spark, Hadoop

Microservices

Append-only

Database


User Actions

View Rendering

(react.js)


Operational Logs

Subscribed Events

Event Queue (Time Series Database,

Apache Kafka, AWS Kinesis,

Eventstore, …)

14

Insights Analytics

Reports, Model

Parameters

Spark, Hadoop



Eventstore, …)


Operational Logs

Subscribed Events

Business Logic

(functions)

Legacy Systems

Archivers

File Store (S3)Microservices

Append-only

Database


User Actions

View Rendering

(react.js)

14

Insights Analytics

Reports, Model

Parameters

Spark, Hadoop


Operational Logs

Subscribed Events

Business Logic

(functions)

Legacy Systems

Archivers


Append-only

Database


User Actions

View Rendering

(react.js)

15



Eventstore, …)

Insights Analytics

Reports, Model

Parameters

Spark, Hadoop

Append-only

Database


User Actions

View Rendering

(react.js)


Operational Logs

Subscribed Events

Business Logic

(functions)

Legacy Systems

Archivers


15


Operational Logs

Subscribed Events

Business Logic

(functions)

Legacy Systems

Archivers


16

Insights Analytics

Reports, Model

Parameters

Spark, Hadoop

File Store (S3)



Eventstore, …)

Microservices

Append-only

Database


User Actions

View Rendering

(react.js)


Operational Logs

Subscribed Events

Business Logic

(functions)

Legacy Systems

Archivers

16

Insights Analytics

Reports, Model

Parameters

Spark, Hadoop

File Store (S3)



Eventstore, …)

16

Insights Analytics

Reports, Model

Parameters

Spark, Hadoop

File Store (S3)



Eventstore, …)

ALL DATA IN MOTION IS IMMUTABLE

16

Insights Analytics

Reports, Model

Parameters

Spark, Hadoop

File Store (S3)



Eventstore, …)


FIT-FOR-PURPOSE “STATE” IS COMPUTED BY THE CONSUMER

16

Insights Analytics

Reports, Model

Parameters

Spark, Hadoop

File Store (S3)



Eventstore, …)



MANAGING AND PUBLISHING EVENTS BRINGS COMPLEXITY

16

Insights Analytics

Reports, Model

Parameters

Spark, Hadoop

File Store (S3)



Eventstore, …)PUSHES RESPONSIBILITY FOR DATA QUALITY BACK ON THE SOURCE SYSTEMS



MANAGING AND PUBLISHING EVENTS BRINGS COMPLEXITY

17

ADOPT 1. Consumer-driven contract testing NEW

2. Focus on mean time to recovery 3. Generated infrastructure diagrams NEW 4. Structured logging

TRIAL 5. Canary builds 6. Datensparsamkeit 7. Local storage sync 8. NoPSD 9. Offline-first web applications NEW 10. Products over projects NEW 11. Threat Modelling NEW

ASSESS 12. Append-only data store 13. Blockchain beyond Bitcoin 14. Enterprise Data Lake 15. Flux NEW 16. “git-based CMS” NEW 17. Phoenix environments NEW 18. Reactive architectures NEW

HOLD 19. Long lived branches with Gitflow 20. Microservice envy 21. Programming in your CI/CD tool 22. SAFe™ 23. Security sandwich 24. Separate DevOps team

TECHNIQUES

TOOLS18

TOOLS18

19

TOOLSADOPT 48. Composer 49. Go CD 50. Mountebank 51. Postman

TRIAL 52. Boot2docker 53. Brighter NEW 54. Consul

55. Cursive 56. Gitlab 57. Hamms NEW

58. IndexedDB 59. POLLY NEW 60. Rest-assured NEW 61. Swagger 62. Xamarin 63. ZAP NEW

ASSESS 64. Apache Kafka NEW 65. Blackbox 66. Bokeh/Vega NEW 67. Gor NEW 68. NaCL NEW 69. Origami NEW 70. Packet beat 71. pdfmake NEW 72. PlantUML NEW 73. Prometheus NEW 74. Quick NEW 75. Security Monkey NEW

HOLD 76. Citrix for development

20

TOOLS

20

TOOLS

7563

65

68

BLACKBOX

ZED ATTACK PROXYSECURITY MONKEY

NACL

SECURITY AWARENESS AMONG SENIOR DEVELOPERS*

21*Source: http://jemurai.com/developer-survey-1-results-part-2.html

37% think security isa small concern

8% think it is a top concern

67%

haver never heard of OWASP, OWASP top 10, or

CWE top 25

25%

of projects reported had security training, pen test or security embedded in

development

Overwhelmingly, the only security practices in place are manual code and design reviews.

http://jemurai.com/developer-survey-1-results-part-2.html

OWASP ZED ATTACK PROXY

22

The Main Features

All the essentials for web application testing

■ Intercepting Proxy

■ Active and Passive Scanners

■ Traditional and Ajax Spiders

■ WebSockets support

■ Forced Browsing (using OWASP DirBuster code)

■ Fuzzing (using fuzzdb & OWASP JBroFuzz)

■ Online Add-ons Marketplace

Browser configured to use proxy

Browser

Primary OS

Web Proxy

Your Computer

VM

Web Server

Browser Web Proxy

Web Server

http://www.slideshare.net/dgsweigert/using-the http://www.slideshare.net/tabaradetestare/owasp-2013-zapquickintro

ARE YOUR REPOS AND BUILD SERVERS SECURE?

23

http://www.wired.com/2012/09/adobe-digital-cert-hacked/

ARE YOUR REPOS AND BUILD SERVERS SECURE?

23

http://www.wired.com/2012/09/adobe-digital-cert-hacked/

PROTECTING DEV SECRETS WITH BLACKBOX

Git Repo

Keys

Shhhh

secret

ShhhhBlackbox

Repo seen by all

Secrets readable by few

25

TOOLSADOPT 48. Composer 49. Go CD 50. Mountebank 51. Postman

TRIAL 52. Boot2docker 53. Brighter NEW 54. Consul

55. Cursive 56. Gitlab 57. HAMMS NEW

58. IndexedDB 59. POLLY NEW 60. Rest-assured NEW 61. Swagger 62. Xamarin 63. ZAP NEW

ASSESS 64. Apache Kafka NEW 65. Blackbox 66. Bokeh/Vega NEW 67. Gor NEW 68. NaCL NEW 69. Origami NEW 70. Packet beat 71. pdfmake NEW 72. PlantUML NEW 73. Prometheus NEW 74. Quick NEW 75. Security Monkey NEW

HOLD 76. Citrix for development

LANGUAGES & FRAMEWORKS

26


26

27


ADOPT 77. Nancy

TRIAL 78. Dashing 79. Django Rest 80. Ionic Framework 81. Nashorn 82. Om 83. React.js 84. Retrofit 85. Spring Boot

ASSESS 86. Ember.js NEW 87. Flight.js 88. Haskell Hadoop library 89. Lotus 90. Reagent 91. Swift

HOLD 92. JSF

28


28


85 SPRING BOOT

NANCY77

A TALE OF TWO WEB FRAMEWORKS

29

Java/Spring C#/.NET

Lightweight ✓ ✓

Low-ceremony ✓ ✓

Self-hosted ✓ ✓

Opinionated ✓ ✓

boot

ON THE SURFACE, VERY SIMILAR

30

ON THE SURFACE, VERY SIMILAR

30

BUT WHAT’S UNDER THE COVERS?

31

Spring Boot’s pom.xml

1847 lines in total!


31

Nancy’s Nuget page


31

Nancy’s Nuget page

FRAMEWORKS VS. COMPOSITION

32

Spring Framework

Your Spring Boot App

JettyYour App

Code

Owin

Nancy.Owin

Nancy

Composes

Calls higher-order functions

33


ADOPT 77. Nancy

TRIAL 78. Dashing 79. Django Rest 80. Ionic Framework 81. Nashorn 82. Om 83. React.js 84. Retrofit 85. Spring Boot

ASSESS 86. Ember.js NEW 87. Flight.js 88. Haskell Hadoop library 89. Lotus 90. Reagent 91. Swift

HOLD 92. JSF

PLATFORMS34

PLATFORMS34

35

PLATFORMSADOPT TRIAL 25. Apache Spark NEW 26. Cloudera Impala NEW 27. DigitalOcean 28. TOTP Two-Factor Authentication

HOLD 45. Application Servers NEW 46. OSGi 47. SPDY NEW

ASSESS 29. Apache Kylin NEW 30. Apache Mesos 31. CoreCLR and CoreFX NEW 32. CoreOS 33. Deis NEW 34. H2O NEW 35. Jackrabbit Oak 36. Linux security modules 37. MariaDB 38. Netflix OSS Full stack 39. OpenAM 40. SDN 41. Spark.io 42. Text it as a service / Rapidpro.io 43. Time-series Databases NEW 44. U2F

36

PLATFORMSDeployment architectures keep evolving.

36

PLATFORMS

33 DEIS

30 APACHE MESOS

32 COREOS45APPLICATION SERVERS

Deployment architectures keep evolving.

THE RISE OF DOCKER

37

http://blog.docker.com/2014/11/docker-governance-advisory-board-output-of-first-meeting/

GitHub Starts by Date and Project Config Management GitHub Totals

EXPLOSION OF TOOLS AND PLATFORMS

38

CoreOS Fleet

Docker Swarm

DEIS: DOCKER-BASED PAAS — ANYWHERE

39http://docs.deis.io/en/v0.9.0/gettingstarted/architecture/

Developer Application Consumers

Load Balancer

Controller Load Balancer

Cluster (Test)

ContainersScheduler Router

Cluster (Dev)


Cluster (Prod)


Monitoring Logging Backing Services

Containers

Containers

Containers

Containers

Containers

Containers

Router

Router

Router

APACHE MESOS

40http://abhishek-tiwari.com/post/building-distributed-systems-with-mesos

batch services Workloads

Apps

Frameworks

Kernel

DFS

Cluster

C++ BASH Python

Scalding Impala Shark MySQL Kafka JBoss Django Rails

MPI Hadoop Spark Storm

Marathon

Chronos

RubyPythonJVMC++

distributed file system

distributed resources: CPU, RAM, I/O, FS, rack locality, etc.

WHERE DOES THIS LEAVE APPLICATION SERVERS?

41

42

PLATFORMSADOPT TRIAL 25. Apache Spark NEW 26. Cloudera Impala NEW 27. DigitalOcean 28. TOTP Two-Factor Authentication

HOLD 45. Application Servers NEW 46. OSGi 47. SPDY NEW

ASSESS 29. Apache Kylin NEW 30. Apache Mesos 31. CoreCLR and CoreFX NEW 32. CoreOS 33. Deis NEW 34. H2O NEW 35. Jackrabbit Oak 36. Linux security modules 37. MariaDB 38. Netflix OSS Full stack 39. OpenAM 40. SDN 41. Spark.io 42. Text it as a service / Rapidpro.io 43. Time-series Databases NEW 44. U2F

43

Scott Shaw

@scottwshaw

Evan Bottcher

@evanbottcher

thoughtworks.com/radar

http://thoughtworks.com/radar