RT-WiFi: Real-Time High-Speed Communication Protocol for
Wireless Cyber-Physical Control Applications
1
Yi-Hung Wei†, Quan Leng†, Song Han‡, Aloysius K. Mok†,
Wenlong Zhang§, Masayoshi Tomizuka§
†Department of Computer Science, The University of Texas at Austin
‡Department of Computer Science and Engineering, University of Connecticut
§Department of Mechanical Engineering, University of California, Berkeley
Wireless Control Systems
• Wireless Control System
– A control system where the control loops are through wireless communication network
• Advantages:
– Enhanced mobility
– Reduced deployment
and maintenance cost
• Applications:
2
Industrial Process Control and Automation Healthcare Robotic Control
Outline
• Introduction
• RT-WiFi Design
• Performance Evaluation
• Case Study
• Conclusion and Future Work
4
1. Real-time data delivery and high sampling rate
– Aim to provide at least 1 kHz sampling rate • minimum requirement for many mechanical control systems
2. Flexible Configuration
– Configurable parameters: sampling rate, predictability of real-time data delivery, reliability, co-existence with regular WiFi networks
3. Transparent System Design
– Use commercial-off-the-shelf network card
– Transparent to upper layer protocols
5
RT-WiFi Design Goals
• Channel access method in regular WiFi: CSMA/CA
– Carrier Sense
– Random Backoff
7
Review of Regular WiFi
data
datawaitwait
waitwait
STA2
STA1 busy
busy
Data ready
Data ready
Random Backoff
Random Backoff
Carrier Sense
Carrier Sense
Main Design of RT-WiFi: TDMA on WiFi
• TDMA: Time Division Multiple Access
• Divide the channel into time slots and assign time slots to each station.
• With TDMA, we do not need carrier sense and random backoff. Each station knows when it can send its packets.
8
STA 1 Send
STA 2 Send
STA 1 Send
STA 2 Send
STA 1 Send
STA 2 Send
An example of TDMA schedule
Architecture of RT-WiFi Protocol
10
•Based on Timing Synchronization function (TSF) in IEEE802.11 •Synchronize all the stations in the RT-WiFi network. •Generate timer interrupt
System Architecture of RT-WiFi Protocol
11
• Coordinate channel access among the stations •Link:
•Broadcast link, transmit link, receive link, shared link •Superframe:
Slot 0
AP Broadcast
Shared
Slot 1 Slot 2 Slot 3 Slot 4 Slot 5 Slot 6 Slot 7
STA1
AP
STA2
AP
STA3
AP
AP
STA1
AP
STA2
AP
STA3
System Architecture of RT-WiFi Protocol
12
Configuration parameters: Sampling rate Co-existence with regular WiFi network Reliability (Retransmission)
Hard Real-Time Traffic How to Enforce TDMA in the Presence of CSMA/CA?
13
For RT-WiFi to grab the channel, we use shorter interframe space (IFS)
After RT-WiFi grabs the channel, RT-WiFi traffic will have precedence over regular WiFi traffic for as long as needed. When bounded latency communication is not needed, regular WiFi can use the channel.
RT-IFSBusy Channel RT-WiFi Data
DIFSRegular WiFi TrafficRegular WiFi Traffic
Deterred
Bounded Length
RT-WiFi
Regular WiFi
Soft Real-Time Traffic (Enabling Co-existence with Regular WiFi Network)
14
1. Assume bounded length for regular WiFi data frames to ensure bounded latency
• Limit maximum transmission unit to at most some upper bound
• Limit lowest data rate to at least some lower bound
2. For RT-WiFi:
– Enable carrier sense
– Use shorter interframe space (IFS)
Guard Interval
RT-IFS
Time Slot
Busy Channel RT-WiFi Data ACK
DIFSRegular WiFi Traffic Regular WiFi Traffic
Bounded Length
RT-WiFi
Regular WiFi
How to Increase Reliability in TDMA?
• Retransmission – In-slot retransmission
– Out-of-slot retransmission
15
(a) Regular time slot
(b) Time slot with in-slot retransmission
Guard Interval
Time Slot
Data Ack
Ack Wait
Guard Interval
Time Slot
Data ACK
ACK Wait
Data Retransmission
ACK Wait
Outline
• Introduction
• RT-WiFi Design
• Performance Evaluation
• Case Study
• Conclusion and Future Work
16
Testbed setup
17
Setup: RT-WiFi baseline vs. regular WiFi UDP socket program to emulate control applications Metrics: MAC layer to MAC layer latency Packet loss ratio Environment: Interference free environment vs. office environment
Timeslot size: 500 μs
Results of RT-WiFi baseline (TDMA only)
in Interference Free Environment
Link Max Latency(μs) Mean Latency(μs) Latency stdev. (μs) Loss Ratio
RT-WiFi Wi-Fi RT-WiFi Wi-Fi RT-WiFi Wi-Fi RT-WiFi Wi-Fi
STA1->AP 535 16448 173 191 4.88 231.60 0.19% 0%
STA2->AP 529 13387 172 181 2.52 214.15 0.16% 0%
STA3->AP 525 13589 174 202 3.01 236.75 0.21% 0%
AP->STA1 827 16472 184 250 5.29 342.24 0.06% 0%
AP->STA2 544 17465 187 298 3.98 360.66 0.04% 0%
AP->STA3 1055 17049 188 248 4.87 325.50 0.01% 0%
18
16-32x 47-90x
The delay in RT-WiFi is caused by the delay of the physical layer in initiating transmission due to factors such as listening to other beacons that we do not fully understand at this point.
Results of RT-WiFi baseline (TDMA only)
in Office Environment
19
Link Max Latency(μs) Mean Latency(μs) Latency stdev. (μs) Loss Ratio
RT-WiFi Wi-Fi RT-WiFi Wi-Fi RT-WiFi Wi-Fi RT-WiFi Wi-Fi
STA1->AP 3865 100078 176 401 25.86 1491.69 8.64% 0%
STA2->AP 4193 81499 171 348 27.62 1000.60 9.97% 0%
STA3->AP 3861 75298 174 429 25.16 1221.72 7.55% 0%
AP->STA1 1197 78089 184 788 16.86 2861.42 9.52% 0%
AP->STA2 1342 78923 189 790 15.19 2806.56 8.09% 0%
AP->STA3 2186 77860 189 799 19.03 2855.89 9.31% 0%
36-184x 2.0-4.3x
The delay in RT-WiFi is caused by the delay of the physical layer in initiating transmission due to factors such as listening to other beacons that we do not fully understand at this point.
• Network A: – Regular WiFi network – 10 Mbps UDP traffic generated by iperf
• Network B: – UDP program to emulate control
applications – Compare:
• Regular WiFi • RT-WiFi baseline • RT-WiFi with co-existence enabled • RT-WiFi with co-existence enabled and
one in-slot retransmission enabled
• Metrics: MAC to MAC layer latency Packet loss ratio
20
Flexible Channel Access Controller Experiment Setup
Flexible Channel Access Controller Experiment Results
Max Latency
(μs)
Mean Latency
(μs)
Latency Stdev. (μs)
Loss Ratio
Regular WiFi 62629 580 1679.04 0%
RT-WiFi baseline 953 183 27.75 50.21%
RT-WiFi co-ex 2507 220 149.27 10.92%
RT-WiFi co-ex + retry 2831 254 166.37 4.96%
21
Outline
• Introduction
• RT-WiFi Design
• Performance Evaluation
• Case Study
• Conclusion and Future Work
22
Case Study: A Mobile Gait Rehabilitation System
23
Wireless Motion Sensors
Mobile Robotic Device
Wireless Smart Shoes
A Host Computer Running
Control Applications
Remote Doctors and
Physical Therapist
Sensing Signals for
Health Monitoring (100Hz)
Sensing S
ignals for
Health M
onitorin
g (100Hz)
Sensing Signals for
Controlling Assistive Robot (1KHz)
Control
Signals
RT-WiFi Station
RT-WiFi Station
RT-WiFi Station
RT-WiFi AP
Control Signals for
Mobile Assistive Robot (1KHz)
Sensing
Signals
Outline
• Introduction
• RT-WiFi Design
• Performance Evaluation
• Case Study
• Conclusion and Future Work
26
Conclusion and Future Work
• Designed and implemented RT-WiFi protocol
– Up to 6 kHz sampling rate
– Predictable real-time data delivery
– Flexible wireless communication platform
• Future work
– Dynamic network management
– Power management
– Extension to mesh topology
– Fault tolerance
27
References
• [1] Y.-H. Wei, Q. Leng, S. Han, A. K. Mok, W. Zhang, and M. Tomizuka, “RT-WiFi: Real-Time High-Speed Communication Protocol for Wireless Cyber-Physical Control Applications”, Technical Report http://www.cs.utexas.edu/users/yhwei/pub/rt-wifi-TR.pdf.
• [2] J. Song, S. Han, A. K. Mok, D. Chen, M. Lucas, M. Nixon, and W. Pratt, “WirelessHART: Applying wireless technology in real-time industrial process control”, in IEEE Real-Time and Embedded Technology and Applications Symposium (RTAS), 2008, pp. 377-386.
• [3] W. Zhang, X, Zhu, S. Han, N. Byl, A. K. Mok, and M. Tomizuka, “Design of a network-based gait rehabilitation system,” in IEEE International Conference on Robotics and Biomimetics (ROBIO), 2012, pp. 1773-1778.
• [4] S. Han, A. K. Mok, J. Meng, Y.-H. Wei, P.-C. Huang, Q. Leng, X. Zhu, L. Sentis, K. S. Kim, and R. Miikkulainen, “Architecture of a cyberphysical avatar”, in International Conference on Cyber-Physical Systems (ICCPS), 2013.
29
References
• [5] Y.-H. Wei, Q. Leng, S. Han, A. K. Mok, W. Zhang, M. Tomizuka, T. Li, D. Leith, and D. Malone, “RT-WiFi: Real-time high speed communication protocol for wireless control systems”, in IEEE Real-Time Systems Symposium Work-in-Progress Session (RTSS WiP), 2012.
• [6] “ISA 100”, http://www.isa.org/isa100.
• [7] X. Zhu, S. Han, P.-C. Huang, A. K. Mok, and D. Chen, “MBstar: A real-time communication protocol for wireless body area networks”, in IEEE Euromicro Conference on Real-Time Systems (ECRTS), 2011, pp. 57-66.
• [8] “IEEE 802.11 working group”, http://www.ieee802.org/11/.
• [9] J. C. Eidson, Measurement, control and communication using IEEE1588, Springer Publishing Company, Incorporated, 2010.
30
Communication Requirements for Future Wireless Control Systems
• Real-time data delivery
• High sampling rate
32
Tracking performance of PD control systems with different sampling rates (partial amplification)
RT-WiFi Time slot
34
ACKSIFSDatavalGuardInterTimeslot TTTTT Time slot size:
• Example: – UDP socket with 200 bytes application layer payload
– IEEE 802.11g with 54Mbps data rate
– 802.11 ERP-OFDM coding scheme
PLCP Header APP UDP IP MAC
Sampling Rate
• Higher sampling rate is better for control applications, but requires smaller time slot size.
• Sampling rate depends on:
– Packet size and data rate
– Synchronization accuracy
– Computational resource
35
ACKSIFSDatavalGuardInterTimeslot TTTTT Time slot size:
Reliability
• In-slot retransmission
• Out-of-slot retransmission
– Retransmit in the next time slot of this link
36
Co-existence with Regular WiFi Network
• Assumption on WiFi:
– Conforms CSMA/CA
– Limit maximum transmit unit (MTU) size
– Limit lowest available data rate
• For RT-WiFi:
– Perform carrier sense
– Use shorter inter-frame spacing
37
System Implementation
• Hardware and software platform
– Atheros AR9285 Wi-Fi card
– Ubuntu 12.04 with Linux kernel 3.2.0
– Open source Linux compat-wireless driver
• Two software modules from compat-wireless driver, mac80211 and ath9k, are incorporated with the TDMA design to build the MAC layer of RT-WiFi.
• Highest sampling rate supported in experiments: 6 kHz
38
39
Results in Interference Free Environment
Latency Empirical CDF for uplink (STAs->AP)
Latency Empirical CDF for downlinks(AP->STAs)
40
Results in Office Environment
Latency Empirical CDF for uplink (STAs->AP)
Latency Empirical CDF for downlinks(AP->STAs)
Simulation of a Wireless Control System
• After integrate RT-WiFi with smart shoes, we collect traces of each sensor data packets to get the network dynamics.
• Then, we run simulation of the controller based on the traces to evaluate the performance of the wireless control system.
• A proportional-derivative (PD) controller was implemented in the simulation to track a sinusoidal signal with unit magnitude.
• Simulation were conducted with different controller gains and reference frequencies.
41
42
Simulation of a Wireless Control System
ECDF of tracking errors with Kp=0.5, Kd = 0.005
ECDF of tracking errors with Kp=1, Kd = 0.01
Transmission Time vs. Payload Size
43
0
50
100
150
200
250
300
350
0 500 1000 1500 2000
Tran
smis
sio
n T
ime
(μ
s)
Payload Size (bytes)
Transmission Time vs Payload Size