Twisted (Python)
/ /
""
CPU time/wall time
Twisted Framework
Python
: async IO (select/poll)
Deferred
:DNS, HTTP, IMAP, memcached, AMQP,
Twisted
,
- (memcached, nginx, haproxy, )
/ 1
Python
Python -
10x C
90% 10%
Cython, Nukitka, Unladen Swallow,
kcachegrind
Python
gc, heapy
: __slots__, Cython, ...
/
, NUMA,
NUMA
CPU
Interconnection
CPUMemory
CPU
CPUMemoryCPU binding
Node binding
Local alloc
+10-15%
DB1DB2App1App2App3Proxy
DB1DB2App1App2App3Proxy
CacheCacheCache
App1CacheApp2CacheApp3Cache
Message bus
AMQP:
:fanout ( )
origin ( )
RabbitMQ
txAMQP
ZeroMQ
App2App3Proxy
App1
Message bus
IPIPIP
Key-value
Memcached memcached, MemcacheDB,
MemcacheQ, membase, ...
txRedis
txMongo
...
pickling
Object(id=199)key1=v1key2=v2...key-valuestorageObject_199
pickledObject_378 pickledObject _988 pickled
user_199_posts
(id, date)
(197, 11:03)
(180, 10:45)
(111, 9:50)
(120, 9:01)
(98, 8:15)
(60, 7:05)
(42, 7:04)
(37, 7:01)
text="Nice
shoes..."author="john"tags=['dev','schedule']post_197
text="Nice
shoes..."author="john"tags=['dev','schedule']post_180
...post_111
...post_120
originMemcached
origins[]"" increment/add
append/add
originsAggApp1App2stats_1stats_2Memcached
Twisted: manhole + telnet/ssh
$ telnet localhost 2000
C100k+
Firewall stateless mode
twistd: reactor=epoll (kqueue)
?
"" ( .. )?
""
memcached, MongoDB, RabbitMQ, cPickle, ...
JSON-RPC, "hello world"890 req/s @ 0.022 sec
530 req/s @ 0.006 sec
()JSON-RPC: 710 req/sec
memcachedX: 11833 req/sec
AMQP: 800 req/sec
connections: 600K+ (670 reconnects/sec)
web- Qik
[email protected], [email protected]
http://www.smira.ru/, @smira
24.10.10
24.10.10
