Upload
eduard-lazar
View
55
Download
0
Embed Size (px)
Citation preview
Stockholm
Get Social#devsum16
Who am I ?
Sr. Solution Consultant with LastMileLink Technologies, a CitySprint UK Innovation Lab, the leading Same Day UK network, often cited as amongst top 5 in Europe.
: #Lazaredd
:https://uk.linkedin.com/in/lazared
Process Flow.
Topics
• Software Defined “Everything”
• Geo-Spatial Fleet Analysis using Big-Data
• Truly Reusable Code – FlowBaseProgramming
Architect
TADev
CommunityNetwork
Marketplace
TechnologyInfrastructure
Data
Source: Sangeet Paul Chaudry , http://platformed.info/platform-stack/
Community, Network, Marketplace
TechnologyInfrastructure
Data
CommunityNetwork
Marketplace
TechnologyInfrastructure
Data
Defunct or Trailing Hanging On Or Declining Thriving & Growing
Leveraging Data produced by Community, Network & Marketplaceis Key to
Platform Success.
From Pipes to Platforms & API Economy
Platform Security – Emphasis on Data
Platform Security – Data Component
Software Defined “Everything”
Process Flow.
“Software is Eating the World”
Image : Pivotal
Marc Andreessen , WSJ 2011
Cloud Natives
$6B $50B $41B
$25B $33.5B
11
SPEED
UBIQUITY*SCALE
SAFETY
(MOBILE and APIs
Four key patterns
12
Day One Day Two and BeyondDeliver Continuously
DevSecOps and Rugged
How we do it
13
Build Reliable Systemsfrom
Unreliable Components
Invite Chaos
14
What a Cloud Native is NOT:
•Just Configuring Infrastructure
•Just Orchestrating Containers
•Composing Distributed Systems
•Supporting Ad-Hoc General Purpose Automation
Conway’s Law.
Any organization that designs a system (defined broadly) will produce a design whose structure is a copy of the
organization's communication structure.
Melvyn Conway, 1967
Invoke the Inverse Conway Maneuver
Breaking the Monolith & Scale
How deep is deep enough?
An example - Mantl.io
Image Source: Mantle.io
Infrastructure Metrics, Monitoring and Microservices Simulation
Image Source: Instana, Image: Spigo and SIMIANVIZ
https://github.com/adrianco/spigo
Command and Control
Omakase
omakaseəʊməˈkaseɪ,əʊˈmakəseɪ/noun1.(in a Japanese restaurant) a type of meal consisting of dishes selected by the chef.2."we had the five-course omakase"
Source:Wikipedia
“ Rails is omakase (DHH) - David Heinemeier Hansson “
SPRING CLOUD
23
http://cloud.spring.iohttps://network.pivotal.io/products/p-spring-cloud-services
Omakase Distibuted Systems
Zeta Architecture - Technologies That Work
• Dynamic compute resources• Common storage platform• Real-time application support• Flexible programming
models• Deployment management• Solution based approach• Applications to operate a
business
* This is a pluggable architectureImage Source: courtesy of Jim Scott @ MapR
PolyglotProgramming?
Fleet Geospatial Analysis
Process Flow.
Blue signals a pick-upRed signals a drop-off
Sample of how one driver’s journey looks like
Used for:• Viewing the base unit of analysis
Process Flow.Demand heat map
Demand heat mapHeat map of pickup locations density
Used for:• Optimising resource allocation• Identifying areas for potential expansion
Process Flow.K-means clustering analysis – 40 centres
K-means clustering analysis – 40 centresEmployed the K-means algorithm to identify clusters of pickup points
Used for:• Validating against current service centres map• Identifying areas for potential expansion
Process Flow.K-means 100 centres
Can we be better placed ? Where can we plan the next M&A?
K-means 100 centresHigher granularity clustering
Used for:• Assessing the frequency of pickups for
micro-clusters (e.g. villages, neighbourhoods)
• Directing drivers to hotter waiting areas
Process Flow.Geographical supply & demand
Geographical supply & demandPickup locations shown vs to routes
Used for:• Improving likelihood of parcel pickup while on-route
Demand variation across time
0 3 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 230.0
4.5
9.0
13.5
18.0
Expected parcels allocated to cluster 41 (Stevenage)
Time of day
Exp
ecte
d pa
rcel
sDemand variation across time
Used for:• Positioning couriers in the right place at the right
time
For each demand cluster we calculated the frequency of pickups per hour
Solution Summary
The solution outline
• Data science capabilities of Spark, easy to use with SQL knowledge• Map plotting on Esri– heat mapping, zoom in/out capabilities, real-time• High-performance due to in-memory processing capabilities of Spark• Can work with large data sets due to high performance disk-based
data access in Hadoop File System (HDFS)• Can import data from EDW• All delivered with Bigstep on their high performance Full Metal Cloud
IaaS-Paas
Shout out to Bigstep
Process Flow.Heavy Lifting done by:
Shout out to Bigstep
Process Flow.
Shout out to Bigstep
84 Lines of Scala code in Spark
Process Flow.
FlowBasedProgramming(FBP)
Switch and …
DevSum 2016
Stockholm
Please vote for this session on DevSum App
: #Lazaredd
:https://uk.linkedin.com/in/lazared