Embed Size (px)
Citation preview
DPDK Summit China 2017
OVS-DPDK Practices in Meituan cloud
2
Huai, Huang
Network Platforms Group
3
MOS:experiences large-scale business practices
A public cloud provider based on the world's largest O2O platform
云计算平台组件
Basic monitoring
Maintenance data falcon
IDC
Network Physical Server
…DockerData Offline
Data Pipeling
ElastiCache
Message Queue
RDS NoSQL
S3
CDN
Image video
OVS EBS
ECSLBSecu
rityDNSMirrorEIP
WebConsole EC2 APIConsole
Logi
n &
Serv
ice
Operation
Billing
Alert
Cloud Platform Architecture
Usa
ge
BDS-
Mon
itor
Network Platforms Group
6
Introducing OVS-DPDK PerformanceOur works Further works
ONTETSC
Network Platforms Group
7
OVS & OVS-DPDK
DPDK
vhost-user
Network Platforms Group
8
OVS-DPDK PerformanceOur works Further works
ONTETSC
Network Platforms Group
9
NIC-BR-NIC E5-2650 v2 @ 2.6G
1 core
128GB mem
10-Gigabit X540-AT2
kernel 3.10.0
Open vSwitch 2.4.90
qemu 2.6.0
64bytes0
4,000,000
8,000,000
12,000,000
16,000,000
NIC-BR-NIC
Pack
ets/
seco
nds
OVS-kernel OVS-DPDK
4.2x
in_port=2,action=output:1
Network Platforms Group
10
NIC-BR-VM
ip,in_port=2,nw_src=192.168.102.0/27 actions=output:3ip,in_port=2,nw_src=192.168.102.32/27 actions=output:4ip,in_port=2,nw_src=192.168.102.64/27 actions=output:5ip,in_port=2,nw_src=192.168.102.96/27 actions=output:6…
0
1,500,000
3,000,000
4,500,000
6,000,000
1 vm 2 vm 4 vm 8 vm
PPS
NIC-BR-VM
ovs ovs vxlan ovs-dpdk ovs-dpdk vxlan
Network Platforms Group
11
OVS-DPDK PerformanceOur works Further
ONTETSC
Network Platforms Group
12
Question 1: stability
stability problem
OVS-DPDK, DPDP, qemu
fix ovs, dpdk, qemu 16 bugs
debug status
rte resource: rte_heap
vhost: virtqueue, fd_set, reconnect_list, virtio_dev…
NIC: link status, bond status…
Network Platforms Group
13
Question 2: Function Defect
raise
vhost recovery
live migration
windows vm run
finished by Intel
Network Platforms Group
14
Problem 3: Compatibility
DPDK-BOND port dump QoS
Network Platforms Group
15
Problem 3: Compatibility dump ovs-filter framework
filter hook load/unload module
kernel module transplant L4 session
Network Platforms Group
16
Problem 4: LOCAL capability
DPDK current method TAP
KNI
2-3Gbps big packets
Network Platforms Group
17
Problem 4: LOCAL capability Local port
forward bottlenecks
Case. Distributed file system（EBS）
I/O -> network
Offline calculation，Data collection
OVS-DPDKVM1 VM 2
VIF VIFVPort
VPort
PF Port
linux kernel
NIantic/82599
FDBVLAN300
KNI Device
DPDK Interface
I/O
EBS
Pool
Network Platforms Group
18
SR-IOV Approach
SRIOV PF：DPDK NIC
VF ：VM
eswitch
Switch controll
user isolation
live migration
Network Platforms Group
19
LOCAL capability
SR-IOV method
linux PF + DPDK VF
DPDK PF + linux VF
CHANGES
DPDK VF code
ixgbevf code
PF、VF queue set
bond4
OVS-DPDKVM1 VM 2
Pool
Pool
Pool
VIF VIFVPort
VPort
PF Port
linux
NIantic/82599
FDBVLAN300
VM 0
VIF
VPort
PFPF: dpdk0 PF: dpdk1 VF: eth0 VF:
ethic
dpdk bond
VM0port
VM1port
linux bond
brApplication
mode=4
mode=2
Network Platforms Group
20
OVS-DPDK PerformanceOur works Further
ONTETSC
Network Platforms Group
21
Problem 5: OVS -> OVS-DPDK
old OVS update OVS-DPDK Method 1：native update
Method 2: offline migration
question：VM need shutdown
live migration vhost-kernel vm -> vhost-user vm
Network Platforms Group
22
Problem 6: Hot Update
Question
ovs-vswtichd restart 2-3min
last time section
hugepage init
nic port restart
link auto negotiation
bond4 handshake
vhost-user port reinit
step 1:
2+min -> 10- s
step 2:
10- s -> 2- s
Primary
Secondary
rte_res
ovs-vswitchd
Solution.
Network Platforms Group
DPDK China Summit 2017 Shanghai，
25
Thanks!!
欢迎关注DPDK开源社区
