ザニケエフ マラット [email protected] maratishe.github.io
2016/01/14 @PRO研@博多
3-Way
Scripts
PDF: bit.do/160114
as a Practical Platform for
Secure Distributed Code
in Clouds
.
Things You Might've Noticed
• most coding today is done in scripting languages -- php, python, ruby, ....
• DiY deploy is becoming more common than platforms -- Docker deploy via filesystem diffs
• smartphone-centric coding is already a reality and reaches further intoIoT, IoV, etc.
• .... a demo?
M.Zhanikeev -- [email protected] -- 3-Way Scripts as a Practical Platform for Secure Distributed Code in Clouds -- bit.do/160114 2/24...
2/24
.
3-Way Scripts
function callme (…) …
Class NAME
Other code, Manager CLI mode
$a = new NAME () $a ->callme (); HTTP mode $a = new NAME () echo toJSON ( $a ->callme ());
Operator Remote code
Use in Object mode
01 M.Zhanikeev "Github Public Repository for the 3-Way Scripting Project" https://github.com/maratishe/3wayscripting (current)
M.Zhanikeev -- [email protected] -- 3-Way Scripts as a Practical Platform for Secure Distributed Code in Clouds -- bit.do/160114 3/24...
3/24
.
The Need (also for Speed)• 2 ways wanted 06 .. 詳細 : high-thru local vs low-thru remote calls to A
A
B
C Many Many
HTTP REQ HTTP REP (small data)
new, call
CLI
High volume
Big Data
Manager
06 M.Zhanikeev "Streaming Algorithms for Big Data Processing on Multicore" CRC (2015)
M.Zhanikeev -- [email protected] -- 3-Way Scripts as a Practical Platform for Secure Distributed Code in Clouds -- bit.do/160114 4/24...
4/24
.
Objectives1. heterogeneity
◦ in clouds known as federated and fog clouds 03◦ in coding, ambient 18 and p2p networking between instances/agents 15◦ ultimately VNE .. 詳細 for distributed software agents
2. volatility◦ again, fog clouds 03 ambient programming 18◦ hierarchical structures -- like Fractal 14
3. autonomy◦ not much in programming but more actively discussed in clouds, networkmanagement, etc. 09
◦ distant goal is Truly Autonomous .. 詳細
03 M.Zhanikeev "A Cloud Visitation Platform to Facilitate Cloud Federation and Fog Computing" IEEE Computer (2015)
18 J.Dedecker+4 "Ambient-Oriented Programming in AmbientTalk" 20th ECOOP (2006)
15 M.Albano+2 "Hierarchical P2P Overlays for DVE: An Additively Weighted Voronoi Based Approach" ICUMT (2009)
14 F.Baude+2 "From Distributed Objects to Hierarchical Grid Components" CoopIS, DOA, and ODBASE, Springer LNCS (2003)
09 M.Zhanikeev "Population Management in Clouds is a Do-It-Yourself Technology" IETF94/NMRG (2015)
M.Zhanikeev -- [email protected] -- 3-Way Scripts as a Practical Platform for Secure Distributed Code in Clouds -- bit.do/160114 5/24...
5/24
.
Theory of Distributed Computing
A
Platform Platform
B c C
Run
A. CORBA model
A B
B. Current (traditional) Model
Platform
C. Distributed Code (proposed)
A
Platform Platform
B
C Run
3-way code Distribution borders (network, etc.)
Code (objects, scripts) Data, messages Runtime Invocation routes
XYZ
• CORBA used tostringify andsend objects
• modern platformsmostly send data
• 3-way code isunique◦ platformless◦ autonomous◦ volatility OK =adaptible
◦ ...
M.Zhanikeev -- [email protected] -- 3-Way Scripts as a Practical Platform for Secure Distributed Code in Clouds -- bit.do/160114 6/24...
6/24
.
Class/Function Call-by-Variable
• most scripting languages (PHP) allow for some nifty referencing techniques
• key to using the same code in 3+ ways
$a = new NAME();$b = ‘callme’;$a->$b( $param, $defaultparam = 10);
M.Zhanikeev -- [email protected] -- 3-Way Scripts as a Practical Platform for Secure Distributed Code in Clouds -- bit.do/160114 7/24...
7/24
.
Security Features
M.Zhanikeev -- [email protected] -- 3-Way Scripts as a Practical Platform for Secure Distributed Code in Clouds -- bit.do/160114 8/24...
8/24
.
Basic Security for Web API
A B
Many Many
REQ( md5p , …)
Make secure
Operator, Manager
mp5p
mp5 (file) Attacker
Manager
M.Zhanikeev -- [email protected] -- 3-Way Scripts as a Practical Platform for Secure Distributed Code in Clouds -- bit.do/160114 9/24...
9/24
.
More Specific on MD5 Keys
B A
Each Each
REQ( key , …)
keyfile Operator/ automation (once) c
web.php make()
verify
same dir.
Attacker
Manager (key exchange)
M.Zhanikeev -- [email protected] -- 3-Way Scripts as a Practical Platform for Secure Distributed Code in Clouds -- bit.do/160114 10/24...
10/24
.
web.php key management
• the ultimate objective is : it has to be humanly simple• see source code at 01
• key made by make() is mapped to tag, then to place• load() is internal, used for de-mapping from place/tag
make( $libdir,$stuff,$length=10){ // returns $keytag( $key,$tag,$cldir=‘.’){ // outgoingplace( $name,$iport,$tag,$cldir=‘.’){ // outgoingprivate load( $place,$tag,$cldir=‘.’){ // outgoing
01 M.Zhanikeev "Github Public Repository for the 3-Way Scripting Project" https://github.com/maratishe/3wayscripting (current)
M.Zhanikeev -- [email protected] -- 3-Way Scripts as a Practical Platform for Secure Distributed Code in Clouds -- bit.do/160114 11/24...
11/24
.
Cloud Distribution/Deploy
M.Zhanikeev -- [email protected] -- 3-Way Scripts as a Practical Platform for Secure Distributed Code in Clouds -- bit.do/160114 12/24...
12/24
.
(1) 3-Way Scripts on Docker
• Docker is mostly best for autonomy andisolation of execution environment
• requireme.php has all the dependencies that I usefor my PHP scripting -- no need for any other files
• ... but need at least PHP 5.6 -- it has built-inweb server
• the magic: pack example.php,my3wayscript.php, web.php,requireme.php into allinone.tbz for deploy
M.Zhanikeev -- [email protected] -- 3-Way Scripts as a Practical Platform for Secure Distributed Code in Clouds -- bit.do/160114 13/24...
13/24
.
(1) Deploy via Dockerfile• based on official PHP 5.6• adds some utilities
• downloads and unpacksallinone.tbz
• prepare: download github.com/maratishe/3wayscripting,unzip, cd
• build: docker build -tmy3way .
• run: docker run -ti -p8001:8001 --rm my3wayphp example.php test
M.Zhanikeev -- [email protected] -- 3-Way Scripts as a Practical Platform for Secure Distributed Code in Clouds -- bit.do/160114 14/24...
14/24
.
(2) Application: SimpleCV (vision lib) API• example of applying 3-way scripts to vision and therelated AI
• SimpleCV is better/simpler than the traditionalOpenCV or other tools
• we can pack it together with a 3-way API• the guys from sightmachine already createdDocker container, but we can make it even better
M.Zhanikeev -- [email protected] -- 3-Way Scripts as a Practical Platform for Secure Distributed Code in Clouds -- bit.do/160114 15/24...
15/24
.
(2) Deploy via Dockerfile
• has no PHP, so need to add it
• prepare: download github.com/maratishe/simplecvapi,unzip, cd
• build: docker build -t cvapi .• run: docker run -ti -p8001:8001--rm cvapipython test.py
M.Zhanikeev -- [email protected] -- 3-Way Scripts as a Practical Platform for Secure Distributed Code in Clouds -- bit.do/160114 16/24...
16/24
.
Wrapup : Feature Comparison
• note the learning curve -- very important for modern programming and edgeboxes .. 詳細
• destined for wireless fog boxes which need platformless agents .. 詳細
• SimpleCV API is a natural application of the 3-way scripting method -- but notlimited by this particular usecase
(Web)Sockets
RESTfulcalls
Hierarchicalstructure
Heterogeneous environments
Learningcurve
(simplicity)Needs a platform?
Traffic encoding optional?
Traditional(Corba, ESB, MPI,…) YES NO NO NO NO YES NO
Advanced/modern(SOC, Ibis, …) YES YES YES YES NO YES NO
Distributed code (proposed) YES YES YES YES YES NO YES
M.Zhanikeev -- [email protected] -- 3-Way Scripts as a Practical Platform for Secure Distributed Code in Clouds -- bit.do/160114 17/24...
17/24
.
That’s all, thank you ...
M.Zhanikeev -- [email protected] -- 3-Way Scripts as a Practical Platform for Secure Distributed Code in Clouds -- bit.do/160114 18/24...
18/24
.
Application: Hadoop Interface.Hadoop Agent..
.
... is software that runs locally as an agent of theremote client
Storage Node (shard)
Time-Aware Sub-Store(s)
Manager
Client Machine
Client
Your Sketcher
You
Start Use
Schedule
Multicore Replay
Replay Node
many
• Hadoop interface is veryintensive, cannot be doneremotely
• there are also alternativesto Hadoop 06
• one of applications for 3-wayscripts .. 戻る
06 M.Zhanikeev "Streaming Algorithms for Big Data Processing on Multicore" CRC (2015)
M.Zhanikeev -- [email protected] -- 3-Way Scripts as a Practical Platform for Secure Distributed Code in Clouds -- bit.do/160114 19/24...
19/24
.
VNE: Virtual Network Embedding.VNE..
.
... is the best way to formulate optimization ofmultiple overlayed virtual (service) networks
Physical layer
Virtual Layer 1
VN Request
2 Make VN
3 VN Reply
4 Virtual
Virtual
Physical
• still active area of research,good heuristics wanted 07
• best for service networks inclouds .. 戻る
07 M.Zhanikeev "A New VNE Method for More Responsive Networking in Many-to-Many Groups" 7th ICUFN (2015)
M.Zhanikeev -- [email protected] -- 3-Way Scripts as a Practical Platform for Secure Distributed Code in Clouds -- bit.do/160114 20/24...
20/24
.
Truly Autonomous Agents.True Autonomy..
.
... is alternative to VNE in that optimization isdone autonomously at service side
Network Provider’s (NOC)
Agent
Service Traditional agents True Autonomy
• part of current standardsdiscussion on futurenetwork management 09
• traditional: NOC + agentsbut NOC is interface =bottleneck
• true autonomy: agents areopen for directcoordination with removeservices .. 戻る
09
09 M.Zhanikeev "Population Management in Clouds is a Do-It-Yourself Technology" IETF94/NMRG (2015)
M.Zhanikeev -- [email protected] -- 3-Way Scripts as a Practical Platform for Secure Distributed Code in Clouds -- bit.do/160114 21/24...
21/24
.
Wireless Network Edge.Reachable Devices..
.
... are boxes that talk to your smartphone usingP2P WiFi (WiFi Direct is best today)
AP
User
Internet
User
Internet WiFiCongestion
User
Internet 3G
Smartphone
Device
Device
Device
AP Design
WLAN Design
Reachable Devices Design
AP
Device WiFi User Internet AP Design (2)
Congestion
WiFiP2P WiFi
Cannot use at the same time
Reachable Platform
• modern boxes are mostly based onthe AP Design -- connectivity problem
• even if conenctivity problem isresolved, you have congestionproblem
• with Reachable Technology,smartphone is the realtime mid-wayinterface between clouds anddevices
• link to code: software agent isrunning at (1) smartphone and (2)the box .. 戻る
M.Zhanikeev -- [email protected] -- 3-Way Scripts as a Practical Platform for Secure Distributed Code in Clouds -- bit.do/160114 22/24...
22/24
.
Example Edge Box for 3-Way Scripts• currently trying to build this box based on conventional Xen• LHAP is Local Hardware Awareness Platform 03
• software agents are running at the box, those are Beacons, Wireless DataHubs, etc. .. 戻る
WiFi
Wireless users
Wireless AP
Physical Device
LHAP
VM VM Con. Con. Con. Storage
Sensors
…
Beacon
WiFi AP WiFi Client
P2P WiFi
03 M.Zhanikeev "A Cloud Visitation Platform to Facilitate Cloud Federation and Fog Computing" IEEE Computer (2015)
M.Zhanikeev -- [email protected] -- 3-Way Scripts as a Practical Platform for Secure Distributed Code in Clouds -- bit.do/160114 23/24...
23/24
.
Binary Diffs and DiffHub Idea.Binary Diffs..
.
... are the best way to achieve trafficefficiency in a datatype-indiferent way
Cloud Drive
User A
User B
New
Old
Old
Diff Diff
Old version New version Binary diff
DiffHub
• exampe: Docker uses filesystemdiffs
• recent study: binary diffs are betterthan filesystem diffs
• connection: efficient deploy/maintain of scale-out code .. 戻る
M.Zhanikeev -- [email protected] -- 3-Way Scripts as a Practical Platform for Secure Distributed Code in Clouds -- bit.do/160114 24/24...
24/24