Communication in Python and the C10k problem

Jose Ignacio Galarza @igalarzab

Communication in Python and the C10k problem

MANOLOOOOO!!!!

select the best method to

communicate

select the best method to scale

Index✤ From polling… to pushing ✤ Concurrency ✤ C10k ✤ Asynchronous I/O

PYTHON 3

PYTHON 3 EVERYWHERE

From polling to pushing

polling

Client Server

polling➡ Nothing new to implement ➡ Good support in all platforms ➡ Proxy friendly ➡ Uni-directional ➡ Very low efficiency ➡ No cross-domain

while True: r = requests.get('http://api.yo.com/messages') messages.extend(r.json()) time.sleep(5)

polling@app.route("/messages") def messages(): message = check_new_message() return message

long polling

ServerClient

long polling➡ Good support in all platforms ➡ “Proxy friendly” ➡ Uni-directional ➡ Low efficiency (better than polling) ➡ No cross-domain

long polling

while True: try: r = requests.get('http://yo.com/messages', timeout=60) except TimeOut: continue

messages.extend(r.json())

@app.route("/messages") def messages(): message = wait_until_new_message() return message

HTTP Streaming

ServerClient

HTTP Streaming➡ Better efficiency than (long) polling ➡ “Proxy friendly” ➡ Uni-directional ➡ You need to parse the data manually

HTTP Streaming➡ Type 1: Connection Close

HTTP/1.1 200 OK Content-Type: text/plain Connection: close

Hello world This a connection close response

HTTP Streaming➡ Type 2: Chunked response

HTTP/1.1 200 OK Content-Type: text/plain Transfer-Encoding: chunked

E Hello World!

19 I am a chunked response

0

HTTP Streaming

r = requests.get('http://yo.com/messages', stream=True) for message in r.iter_lines(): messages.extend(message.decode(‘utf8’))

@app.route("/messages") def messages(): def content(): for sentence in wait_until_next_sentence(): yield sentence

return Response(content())

SSE

ServerClient

SSE➡ Well known protocol (HTTP) ➡ Good efficiency ➡ JS API ➡ Few client implementations ➡ Uni-directional

WebSockets

ServerClient

WebSockets➡ Bidirectional ➡ Great efficiency ➡ A lot of implementations (JS API) ➡ Cross-domain ➡ Handshake to update from HTTP ➡ Complete different protocol

GET /chat HTTP/1.1 Host: server.example.com Upgrade: websocket Connection: Upgrade Sec-WebSocket-Key: x3JJHMbDL1EzLkh9GBhXDw== Sec-WebSocket-Protocol: chat, superchat Sec-WebSocket-Version: 13 Origin: http://example.com

WS Handshake

HTTP/1.1 101 Switching Protocols Upgrade: websocket Connection: Upgrade Sec-WebSocket-Accept: HSmrc0sMlYUkAGmm5OPpG2HaGWk= Sec-WebSocket-Protocol: chat

Wikipedia

WebSockets

def msg(ws, msg): messages.extend(msg)

ws = websocket.WebSocketApp(‘ws://api.yo.com/', on_message=msg) ws.on_open = on_open ws.run_forever()

@sockets.route('/messages') def messages(ws): while True: message = wait_until_new_message() ws.send(message)

Bytes IN Bytes Out Total Bytes Time (seconds)

Polling 14640 13564 28204 159

Polling 2 7503 7636 15139 200

Long Polling 7503 7636 15139 120

Streaming 183 1549 1732 120

WebSockets 209 1605 1814 120

0

3000

6000

9000

12000

15000

18000

21000

24000

27000

30000

Polling

Polling 2

Long Polling

Streaming

WebSocketsTotal Bytes Bytes Out Bytes IN

Concurrency and the

C10k

C1how to handle more than

10k connections simultaneously?

0k?

concurrency is…

the ability of running in overlapping time periods,

not necessarily at the same time

concurrency !=

parallelism

processes➡ Preemptive scheduling by the OS ➡ Separate memory space ➡ No GIL related issues ➡ How to communicate them? ➡ They are really heavy

threads➡ Preemptive scheduling by the OS ➡ Same memory space ➡ Faster and lighter than processes ➡ You may suffer the GIL ➡ What about synchronisation? ➡ Race conditions

user threads➡ Same space address ➡ Lightest alternative ➡ No race-conditions ➡ You may suffer the GIL ➡ Very bad with CPU bound tasks

GIL

I/O methods

again…how to handle more than

10k connections simultaneously?

option 1

use 10k machines, one per client, VIP service

option 2use blocking and

synchronous calls, one per client

option 3non-blocking calls to start

I/O and then readiness notifications to know

when the socket is ready

option 4asynchronous calls to

start I/O and completion notifications to know when they’ve finished

but… in Python?

twistedtornadogevent

asyncio (aka tulip)

asyncio➡ Single-threaded ➡ Async I/O ➡ Coroutines ➡ Multiplexes I/O events ➡ …

COROUTINES

websockets

asyncio

demo

XGame Of Life

10K connections

questions?@igalarzab

thanks!