Upload
pivorak-meetup
View
433
Download
1
Embed Size (px)
Citation preview
Building Distributed Systems
decision guide
Oleksiy Kurnenkov
Core InfrastructureDomain leadOnApp
Building Distributed Systems
decision guide
Introduction
Part I
Distributed system
is a set of interconnected functional units with separate runtime context
MONOLITH : DISTRIBUTED
1 Runtime context
Coroutines
Threads
Process
N Runtime contexts
Nodes
Components
Clusters
Grids
SCALE
Small ->
Middle ->
Global ->
OS, LAN
LAN
WAN
TYPES
CDN
Grids
Clouds
Clusters
Enterprise
CDN
Grid
Globus Toolkit 6
BOINC
x-Torrent
Cloud
Cluster
Enterprise
TASKS
MMOG
Analysis
Sensors
Real Time Enterprise
Media
Storage
Computation
Transactions
Challenges
AvailabilityScalabilitySecurityOpennessHeterogeneityConcurrencyTransparencyFeasibility
It IS there
99.9999999%
Availability
It grows and shrinks
Scalability
U ain’ gonna crack it >:|
Security
It is open for development and integration
Opennes
Everything changes
Heterogeneity
Parallel workloads
Concurrency
Less know - less depend
Transparency
Simplify as possible
Feasibility
Fallacies
The network is reliableLatency is zeroBandwidth is infiniteThe network is secureTopology doesn't changeThere is one administratorTransport cost is zeroThe network is homogeneous
Mechanics
Part II
MechanicsI
MembershipConcurrency / ParallelismSynchronizationCoordinationConsensusClustering
Who’s there?
Membership
Membership
Heartbeats
Up-2-date list of nodes
We are Legion
Concurrency / Parallelism
Concurrency / Parallelism
Actors
Process Calculi
Shared memory
MY precious!
Synchronization
Synchronisation
Mutex | Semaphore
Distributed Lock
Roger that!
Coordination
Coordination
Leader election
Orchestration
the ‘Two Generals Problem’
Deal
Consensus
Consensus
Quorum
Consistency Availability Partition tolerance
Altogether
Clustering
Clustering
Load Balancing
Redundancy
Availability
Replication
Partition tolerance
MechanicsII
ScalingFault DetectionFailoverRecoverFail back
Scaling
Vertical
Horisontal
Automatic
Fault Detection
Monitoring
Supervision
Activity Analysis
Failover
Data | State Accessibility
Resource allocation
Spare Capacity
Business Continuity
Recovery
State Reconstruction
Data backup
State dump
Failback
Data consistency
Conflict resolution
Apache YARN
Apache Zookeeper
Apache Mesos
Apache Drill
Google Chubby
CoreOS
Google Kubernetes
Google Dremel
Architecture
Part III
Client - Server
N-tier
Service oriented Architecture
Event Driven Architecture
P2P
Context
System Allocation
Architecturalentitiesdecomposition
SubsystemServiceComponentProcessObjectMiddleware
Fractal
Subsystem -> Component -> Process -> Object
Subsystem -> Component -> Process -> Object
Subsystem -> Component -> Process -> Object
CommunicationInfrastructure
aka Middleware
IPC-----------------------------Remote InvocationGroup CommunicationMessaging
IPC
Share data
Send data
Remote Invocation
Request - Reply
RPC
RMI
Group Communication
{ Uni | Multi | Broad } cast
Publish | Subscribe
Messaging
Message Queue
Message Broker
Pipe | Filter | Translator
Implementation
Part IV
Microcosm -> Macrocosm
Middleware
[ * ] Process
Host
Network
Within a process
Threads
Shared memory
Mutex | Monitor
require 'thread'
class RaceCondition def initialize(resource, concurrency_level) @resource = resource @concurrency_level = concurrency_level end
def perform @concurrency_level.times do threads << Thread.new { action } end
threads.each(&:join) end
private
def threads @threads ||= [] end
def action @resource += 1 puts @resource endend
RaceCondition.new(0, 10).perform
# 12# 3# 5# 6# 7# 8# 910# 4
require 'thread'
class Sycnhronizer def initialize(resource, concurrency_level) @resource = resource @concurrency_level = concurrency_level end
def perform @concurrency_level.times do threads << Thread.new { action } end
threads.each(&:join) end
private
def threads @threads ||= [] end
def action lock.synchronize do @resource += 1 puts @resource end end
def lock @lock ||= Mutex.new endend
Sycnhronizer.new(0, 10).perform
# 1# 2# 3# 4# 5# 6# 7# 8# 9# 10
Microcosm -> Macrocosm
Process
[ * ] Host
NetworkMiddleware
Within a host
Processes
MQ | Shared memory | Pipes | UNIX Socket
Semaphore
# http://bogomips.org/ruby_posix_mq/
require 'posix_mq'
class Producer attr_reader :mq
def initialize(mq_name) @mq = POSIX_MQ.new("/foo", :rw) end
def send(message, prio = 0) puts "Send: #{message}. Priority #{prio}" mq.send("#{message} #{prio}", prio) endend
p = Producer.new("/test_ipc")
p.send("Hello from #{Process.pid}", 10)p.send("Hello from #{Process.pid}", 2)p.send("Hello from #{Process.pid}", 0)p.send("Hello from #{Process.pid}", 1)p.send("Hello from #{Process.pid}", 20)
# ruby posix_mq/producer.rb## Send: Hello from 12635. Priority 10# Send: Hello from 12635. Priority 2# Send: Hello from 12635. Priority 0# Send: Hello from 12635. Priority 1# Send: Hello from 12635. Priority 20
# http://bogomips.org/ruby_posix_mq/
require 'posix_mq'
class Consumer attr_reader :mq
def initialize(mq_name) @mq = POSIX_MQ.new("/foo", :rw) end
def receive mq.receive.first end
def receive_non_block mq.nonblock = true
begin receive rescue Errno::EAGAIN mq.nonblock = false puts "Nothing" end end
def shift mq.tryshift endend
c = Consumer.new("/test_ipc")
while m = c.shift puts "got: #{m}"end
# ruby posix_mq/consumer.rb
# got: Hello from 12635 10# got: Hello from 12635 20# got: Hello from 12635 2# got: Hello from 12635 1# got: Hello from 12635 0
# https://github.com/pmahoney/process_shared
require 'process_shared'
mutex = ProcessShared::Mutex.newmem = ProcessShared::SharedMemory.new(:int)mem.put_int(0, 0)
pid1 = fork do puts "in process 1 (#{Process.pid})" 10.times do sleep 0.01 mutex.synchronize do value = mem.get_int(0) sleep 0.01 puts "process 1 (#{Process.pid}) incrementing" mem.put_int(0, value + 1) end endend
pid2 = fork do puts "in process 2 (#{Process.pid})" 10.times do sleep 0.01 mutex.synchronize do value = mem.get_int(0) sleep 0.01 puts "process 2 (#{Process.pid}) decrementing" mem.put_int(0, value - 1) end endend
Process.wait(pid1)Process.wait(pid2)
# ruby shm.rb## in process 1 (8038)# in process 2 (8041)# process 1 (8038) incrementing# process 2 (8041) decrementing# process 1 (8038) incrementing# process 2 (8041) decrementing# process 1 (8038) incrementing# process 2 (8041) decrementing# process 1 (8038) incrementing# process 2 (8041) decrementing# process 1 (8038) incrementing# process 2 (8041) decrementing# process 1 (8038) incrementing# process 2 (8041) decrementing# process 1 (8038) incrementing# process 2 (8041) decrementing# process 1 (8038) incrementing# process 2 (8041) decrementing# process 1 (8038) incrementing# process 2 (8041) decrementing# process 1 (8038) incrementing# process 2 (8041) decrementing# value should be zero: 0
rd_child, wr_parent = IO.piperd_parent, wr_child = IO.pipe
pid = fork do rd_parent.close wr_parent.close
wr_child.puts "sent from child process" puts rd_child.getsend
rd_child.closewr_child.close
wr_parent.write "sent from parent process"puts rd_parent.gets
# ruby pipes/pipes.rb## sent from child process# sent from parent process
require 'eventmachine'
module UnixServer def receive_data(data) puts data
EM.stop if data.chomp == "exit"
send_data("Server #{Process.pid}: Got #{data.chomp} from you") close_connection_after_writing endend
EM.run do puts "Started UNIX socket server on /tmp/sock"
EM::start_unix_domain_server("/tmp/sock", UnixServer)end
# ruby server.rb## Started UNIX socket server on /tmp/sock## HELLO! My pid is 13847
require 'socket'
UNIXSocket.open("/tmp/sock") do |c| c.write("HELLO! My pid is #{Process.pid}")
puts c.readend
# ruby client.rb## Server 13843: Got HELLO! My pid is 13847 from you
Microcosm -> Macrocosm
Process
Host
[ * ] NetworkMiddleware
Over the Network
{ TCP | UDP } Socket
HTTP Client - Server
Async messaging
require "bunny"
conn = Bunny.newconn.start
ch = conn.create_channelq = ch.queue("hello")
puts " [*] Waiting for messages in #{q.name}. To exit press CTRL+C"q.subscribe(:block => true) do |delivery_info, properties, body| puts " [x] Received #{body}"
# cancel the consumer to exit delivery_info.consumer.cancelend
require "bunny"
conn = Bunny.new(:hostname => "rabbit.local")conn.start
ch = conn.create_channel
q = ch.queue("hello")ch.default_exchange.publish("Hello World!", :routing_key => q.name)puts " [x] Sent 'Hello World!'"
conn.close
require "bunny"
conn = Bunny.newconn.start
ch = conn.create_channelx = ch.fanout("logs")
msg = ARGV.empty? ? "Hello World!" : ARGV.join(" ")
x.publish(msg)puts " [x] Sent #{msg}"
conn.close
require "bunny"
conn = Bunny.newconn.start
ch = conn.create_channelx = ch.fanout("logs")q = ch.queue("", :exclusive => true)
q.bind(x)
puts " [*] Waiting for logs. To exit press CTRL+C"
begin q.subscribe(:block => true) do |delivery_info, properties, body| puts " [x] #{body}" endrescue Interrupt => _ ch.close conn.closeend
Apache Kafka
RabbitMQ
Next time
To be continued:
Algorithms, patterns,
code!
Thank you!
Q&A