1

Learning State Selection for ReconfigurableAntennas: A Multi-Armed Bandit Approach

Nikhil Gulati, Member, IEEE, Kapil R. Dandekar, Senior Member, IEEE,

Abstract—Reconfigurable antennas are capable of dynamicallyre-shaping their radiation patterns in response to the needs ofa wireless link or a network. In order to utilize the benefits ofreconfigurable antennas, selecting an optimal antenna state forcommunication is essential and depends on the availability of fullchannel state information for all the available antenna states. Weconsider the problem of reconfigurable antenna state selection ina single user MIMO system. We first formulate the state selectionas a multi-armed bandit problem that aims to optimize arbitrarylink quality metrics. We then show that by using online learningunder a multi-armed bandit framework, a sequential decisionpolicy can be employed to learn optimal antenna states withoutinstantaneous full CSI and without apriori knowledge of wirelesschannel statistics. Our objective is to devise an adaptive stateselection technique when the channels corresponding to all thestates are not directly observable and compare our results againstthe case of a known model or genie with full information. Weevaluate the performance of the proposed antenna state selectiontechnique by identifying key link quality metrics and usingmeasured channels in a 2×2 MIMO OFDM system. We show thatthe proposed technique maximizes long term link performancewith reduced channel training frequency.

Index Terms—Multi-Armed Bandit, Online Learning, Recon-figurable Antennas, Beamsteering, MIMO, OFDM, CognitiveRadio.

I. INTRODUCTION

RECONFIGURABLE antenna technology has gained a lotof attention in recent years for applications in wireless

communications. Both theoretical and experimental studieshave shown that reconfigurable antennas can offer additionalperformance gains in Multiple Input Multiple Output (MIMO)systems by increasing channel capacity [2], [3]. They havealso been shown to perform well in low SNR regimes [4].Gradually making their way into commercial wireless sys-tems [5], [6], these antennas bring two major benefits totraditional multi-element wireless systems. First, the additionaldegree of freedom to dynamically alter radiation patterns en-able MIMO systems to adapt to physical link conditions. Thisadaptation leads to enhanced performance and can provide ro-bustness to varying channel conditions. Second, these antennasprovide space and cost benefits either by incorporating mul-tiple elements in a single physical device [7] or by reducingthe number of RF chains [8], [9]. Beyond single user MIMOsystems, reconfigurable antennas have more recently been used

Manuscript received October 15, 2012; revised February 06, 2013; acceptedApril 23, 2013. Date of publication November, 2013. This work was supportedby NSF Grant # 0916480. The authors are with the Department of Electricaland Computer Engineering, Drexel University, Philadelphia, PA, 19104 USAe-mail:(nikhil.gulati@drexel.edu, dandekar@coe.drexel.edu). Preliminary ver-sion of this work appeared in IEEE RWS, 2012 [1].

for improving advanced interference management techniquessuch as Interference Alignment (IA) [10], [11], [12]. Withthe continued interest in developing practical cognitive radioshaving greater autonomy, learning, and inference capabilities,the integration of reconfigurable antennas in cognitive radiosalong with intelligent algorithms to control them, is going toplay a significant role.

One of the key challenge to effectively use the recon-figurability offered by these antennas and integrate them inpractical wireless systems, is to select an optimal radiationstate1(in terms of capacity, SNR, diversity etc.) among allthe available states for a wireless transceiver in a givenwireless environment. There are two fundamental challengesto achieve this goal: (i) the requirement of additional channelstate information (CSI) corresponding to each state for eachtransmitter-receiver pair; (ii) The amount and the frequencyof the channel training required and the associated overhead.These challenges become even more difficult to overcomewhen reconfigurable antennas are employed at both the endsof the RF link thus creating a large search space in order tofind an optimal radiation state for communication. Moreover,the effect of node mobility, changes in physical antennaorientation, and the dynamic nature of the wireless channelcan render previously found “optimal” states suboptimal overtime.

Translating the benefits of reconfigurable antennas into apractical realizable MIMO system is thus a highly challengingtask. Existing antenna state selection techniques (see Sec II-B)have primarily relied on the availability of perfect instanta-neous CSI coupled with modified periodic pilot based trainingto perform optimal state selection. Besides the performanceloss caused by imperfect or incomplete CSI, the additionalproblem of changing the data frame to enable additionalchannel training render these approaches impractical for usein systems such as IEEE 802.11x and IEEE 802.16 as thenumber of available states increase.

In this work, instead of relying on the availability of full,instantaneous CSI and periodic channel training, we ask thefollowing questions:

• Can a reconfigurable antenna system learn to select anoptimal radiation state without the availability of instan-taneous full CSI and periodic training while optimizingan arbitrary link quality metric?

• Can a reconfigurable antenna system also adapt to vary-

1State refers to either the radiation pattern selected at the receiver or thetransmitter or a combination of radiation patterns selected at the receiver andtransmitter respectively.

2

ing channel conditions without extensive channel trainingand parameter optimization?

Towards addressing these challenges, we present an onlinelearning framework for reconfigurable antenna state selection,based on the theory of multi-armed bandit. We model a systemwhere each transmitter-receiver pair (i.e., a link) develops asequential decision policy to maximize the link throughput.For each decision, the system receives some reward from theenvironment which is assumed to be an i.i.d random processwith an arbitrary distribution. It is also assumed that meanreward for each link is unknown and is to be determined onlineby the learning process. Specifically, we provide the followingcontributions in this work:

1) We first present a learning framework to learn theoptimal state selection based on the theory of multi-armed bandit, in order to maximize system performance.

2) We identify key link quality metrics which can be usedwith the learning framework to assess the long-termperformance of the system.

3) We implement and evaluate the learning algorithm in apractical IEEE 802.11x based single user MIMO-OFDMsystem to evaluate the performance over measured wire-less channels.

The rest of our paper is organized as follows: In section II,we provide a background on reconfigurable antennas andmulti-armed bandit theory along with related work on boththe topics. Section III describes the MIMO system modeland explains the multi-armed bandit formulation for recon-figurable antenna state selection. In Section IV, we describethe selection policies and the reward metrics used to evaluatethe performance of the proposed schemes. In Section V,we provide a description of experimental setup, hardwarecomponents, and the implementation of the learning policies.Section VI provides detailed performance analysis and empir-ical evaluation followed by the conclusion in Section VII.

II. BACKGROUND AND RELATED WORK

A. Background

Pattern Reconfigurable Antennas: With the introduction ofreconfigurable antennas, there was a departure from the notionthat antennas can only have fixed radiation characteristics.Reconfigurable antennas are capable of changing the oper-ating characteristics of the radiating elements through eitherelectrical, mechanical or other means. Over the last ten years,research has primarily been focused on designing reconfig-urable antennas with the ability to dynamically change eitherfrequency [13], radiation pattern [3] and polarization [14] orthe combination of one of these properties. Reconfigurability isbased on the fact that the change in the current distribution inthe antenna structure effects the spatial distribution of radiationfrom the antenna element [15]. Current distribution in the re-configurable antenna can be modified by mechanical/structuralchanges, material variations or using electronic componentslike PIN diodes [16] and MEMS switches [17]. In this paper,we focus only on the pattern reconfigurable antennas and usingthe pattern diversity offered by them.

Multi-Armed Bandit Theory: The Multi-armed banditproblem (MAB) is a fundamental mathematical frameworkfor learning unknown variables. It embodies the well knownexploitation vs. exploration trade off seen in reinforcementlearning and limited feedback learning. In the classic formu-lation [18], [19], [20], there are N independent arms witha single player, playing arm i (i = 1, . . . N). The trade-offinvolves choosing an arm with the highest expected payoffusing current knowledge and exploring the other arms toacquire more knowledge about the expected pay-offs of therest of the arms. On each play of a single arm, the playerreceives a random reward. The goal is to design a policyto play one arm at each time sequentially to maximize thetotal expected reward in the long run. In the seminal work ofGittins [21], it was shown that the optimal policy of MAB withBayesian formulation where reward distributions are known,has a simple index structure i.e., Gittins index. Within thenon-Bayesian framework, Lai and Robbins [18] provided aperformance measure of an arm selection policy referred to asregret or cost of learning. Regret is defined as the differencein the expected reward gained by always selecting the optimalchoice and the reward obtained by a given policy. Since thebest arm cannot always be identified in most cases using afinite number of prior observations, the player will always haveto keep learning. Due to the continuous learning process, theplayer will make mistakes which will grow the regret overtime. The goal of the learning policies under multi-armedbandit framework is to keep the regret as low as possible andbound the growth of regret over time.

B. Related Work

1) Pattern Reconfigurable Antennas and Beam Selection:Though the architecture of pattern reconfigurable antennas andtheir performance in MIMO systems have received significantattention, there are only a handful of studies which focuson practical control algorithms and optimal state selectiontechniques. In [22], the authors estimate the channel responsefor each antenna state at the transmitter and receiver using pilotbased training and select an optimal state combination. Theytheoretically show the effect of channel estimation error onlink capacity and power consumption. Since, their techniquerelies on the availability of full CSI at every time slot, a changein the data frame is required to enable additional channeltraining which can lead to a loss of capacity as the number ofantenna states increase. The authors in [23], provide a selectiontechnique based on second order channel statistics and averageSNR information for optimal state selection without changingthe OFDM data frame. This technique maximizes the averagesystem performance over long run. Their selection techniquerelies on building offline lookup tables for a given SNR andpower angular spread for a given environment and is notadaptive in nature. Periodic exhaustive training techniqueswith reduced overhead is presented in [24] where the authorshighlight the effect of channel training frequency on thecapacity and the bit-error rate (BER) of a MIMO system.In order to reduce the overhead of exhaustive training forall the beam combinations, the authors make assumptions

3

------

Preamble OFDM Data 1 OFDM Data 2 OFDM Data N

T1

T2

T3

T4

4 Training

SymbolsN Data

Symbols

Pilots for Phase Correction

Fig. 1. OFDM frame structure. Preamble is loaded with 4 training symbols for channel training of a single selected antenna state

on the prior availability of statistics of the channel in orderto eliminate sub-optimal beams. But in their work it is notclear how to re-introduce the excluded states which maybecome optimal over time due to channel variations. Thoughsome of these techniques were successful in showing thebenefits of multi-beam selection and motivated the need fora selection algorithm, none solved the challenges mentionedabove. Moreover, all the schemes mentioned above providesimulated results only without any practical implementationand assume the availability of full CSI from the periodicchannel training. In our work, we neither assume full CSIfor all states at every time slot nor do we perform periodicexhaustive training. Our approach does not require a changein the frame structure and can still adapt on a per packet basis.

2) Multi-Armed Bandit (MAB) for Cognitive Radio:Stochastic online learning via multi-armed bandit has gainedsignificant attention for applications in opportunistic anddynamic spectrum access. The authors in [25] applied themulti-armed bandit formulation to the secondary user channelaccess problem in cognitive radio network. Later, a restlessMAB formulation for opportunistic multi-channel sensing forsecondary user network was presented and evaluated in [26].Further in [27], a combinatorial version of MAB was proposedfor a network with multiple primary and secondary userstaking into account the collisions among the secondary users.Distributed channel allocation among multiple secondary userswas further studied and proposed in [28] and [29]. Anotherapplication of multi-armed bandit for cognitive radio wasproposed in [30] for adaptive modulation and coding. Theapplication of online learning for cognitive radio for dynamicspectrum access can be enhanced by either using patternreconfigurable antennas to avoid interference using spatialmeans or can be combined with spectrum sensing where asecondary user can opportunistically select a radiation stateonce an unoccupied channel is found using channel sensing.

Notation: We use capital bold letters to denote matrices andsmall bold letters for vectors. H−1, H† and HT denote thematrix inverse, Hermitian and transpose operation respectively.‖H‖F represents the Frobenius norm of H respectively. Thed × d identity matrix is represented by Id.

III. SYSTEM MODEL AND BANDIT FORMULATION

A. MIMO System Model with Reconfigurable Antennas

Consider a point to point MIMO link with M patternreconfigurable antennas at the transmitter (Tx) and N pattern

reconfigurable antennas at the receiver (Rx). We assume thatreconfigurable antennas at the transmitter have J such uniquestates and the antennas at the receiver are capable of switchingbetween K such states. The degree of correlation betweenall the resulting J × K combination of channel realizationsis governed by the physical structure of the reconfigurableantenna. We employ the V-BLAST [31] architecture for trans-mission of the spatial multiplexed input symbol x (∈ CM×1)(i.e., each antenna element carries an independent stream ofinformation). Further, in order to approximate the channelas having flat fading, we employ OFDM as the multicarriermodulation scheme. Under such a setting, the received signalis given as:

y(f) = Hk,j(f)x(f) + n(f), (1)

where f denotes the OFDM subcarrier index, k, j representthe antenna state of the reconfigurable antenna selected at thereceiver and transmitter respectively, y is the N × 1 receivedsignal vector, Hk,j is the N × M MIMO channel betweenthe transmitter and the receiver, x is M × 1 and n representsthe N × 1 vector of complex zero mean Gaussian noise withcovariance matrix E

[nn†]

= σ2IN . For brevity, we will dropthe symbols k, j and f . The input vector x is subject to anaverage power constraint, E

[Tr(xx†)

]= P with total power

equally distributed across the input streams, i.e. the inputcovariance matrix is given by Q = P

M IM . The set containingall the combinations of states of the reconfigurable antenna atthe transmitter and receiver will be represented by the vectorΩ =j× k: j ∈ J , k ∈ K.

Frame Structure and Channel Estimation: To enable fre-quency domain estimation of MIMO channel coefficients at thereceiver, every OFDM frame carries four training symbols inthe preamble, two for each antenna element as shown in Fig. 1.Each transmit antenna is trained orthogonally in time i.e.while the training symbols are transmitted from one transmitantenna, the other transmit antenna stays silent. The receivedtraining symbols are then used to estimate the MIMO channelusing a Least Squares Estimator. Let t1 and t2 represent thevector of training symbols and T represent the matrix oftraining symbols, i.e. T = [t1 t2]. Then, the least squareestimate of a single tap channel is given as:

HLS = YT†(TT†

)−1

(2)

where Y represents the matrix of received signal vectorscorresponding to both the training symbols. It should be noted

4

VBLAST

Encoder

Modulation

Modulation

S/P

1:F

S/P

1:F

IFFT

Add CP

IFFT

Add CP

Extract Antenna

Source

Bits

Demod

Demod

Remove CP

FFT

Remove CP

FFT

VBLAST

DetectionP/S

Received

BitsExtract

Reward

Metrics

Learning

Module

Antenna

Control Unit

(ACU)

Bits

Set State

Fig. 2. System diagram of 2x2 MIMO OFDM System with Reconfigurable Antennas and the receiver employing Learning Module for Antenna StateSelection

that (2) is evaluated for every OFDM subcarrier f. In additionto the channel estimates obtained via (2), each OFDM symbolcarries 4 evenly placed pilot tones, which are used for phasecorrection at the receiver.

Hphase-corrected = HLSe−jφi (3)

where φi represents the average phase of the 4 pilot tonesfor the ith OFDM symbol in a frame. These frequency domainchannel estimates are then used to carry out channel equaliza-tion using Zero Forcing-Successive Interference Cancelation(ZF-SIC) equalizer. To further enhance the performance ofZF-SIC, symbols are decoded based on optimal SNR orderingand then finally combined using Maximal Ratio Combining(MRC). We estimate the channel for a single antenna stateusing the OFDM frame structure described above. Therefore,this frame structure does not change as the number of antennastates are increased. Once the channel estimates are availablefor the selected antenna state at each time slot, the rewardmetrics are extracted and used as input to the learning modelto select the antenna state for next time slot as shown in Fig. 2

B. Bandit Formulation for Antenna State Selection

Our work is influenced by the formulation in [20] wherearms have non-negative rewards that are i.i.d over time withan arbitrary un-parametrized distribution. We consider theset up where there is a single transmitter and L wirelessreceivers and both the transmitter and the receiver employpattern reconfigurable antennas. The receivers can select fromK available antenna states and the state at the transmitter isfixed which reduces the problem to selecting an antenna stateonly at the receiver where each receiver can select state iindependently. It can be shown that this framework can beeasily extended to the case where the antenna state at thetransmitter is not fixed and state selection is also performedfor the transmitter. In that case an antenna state will refer toa combination of the radiation state j at the transmitter andradiation state i at the receiver. In the context of multi-armedbandit a radiation state i is interchangeably referred to as anarm i. The decision is made at every time slot (packet) n toselect the antenna state to be used for the next reception. Ifa receiver selects a state i and assuming the transmission issuccessful, an instantaneous random reward is achieved whichwe denote as Ri (n). This reward is assumed to evolve as an

i.i.d random process and the mean of this random processis unknown to the receiver. Without loss of generality, wenormalize Ri (n) ∈ [0, 1] . When a receiver selects an antennastate i, the reward Ri (n) is only observed by that receiverand the decision is made only based on the locally observedhistory. Also, the reward is only observed for the selected statei and not for the other states. In other words, a receiver receiveschannel state information for only the selected radiation stateat a given time slot and acquires no new information aboutthe other available radiation states. In this way our proposedtechnique differs from the other techniques in the literature asit does not rely on the availability of instantaneous CSI for allthe radiation states at each time slot.

We represent the unknown mean for a given state i as xi =E [Ri]. The collection of these means for all states is thenrepresented as X = E xi, 1 ≤ i ≤ K. We further define thedeterministic policy π (n) at each time serving as a mappingbetween the reward history Rkn−1

k=1 and the vector of antennastates r (n) to be selected at time slot n where receiver l selectsantenna state rl (n). The goal is to design policies for thismulti-armed bandit problem that perform well with respect toregret. Intuitively, the regret should be low for a policy andthere should be some guarantees on the growth of regret overtime. Formally, the regret of a policy after n plays is givenby (4).

µ∗n−K∑i=1

µiE [Ti (n)] (4)

where,µ∗ = max1≤i≤Kµi (5)

µ∗ is the average reward of the optimal antenna arm, µi isthe average reward for arm i, n is the number of total trials.E [·] is the expectation operator and Ti (n) is the number oftimes arm i has been sampled upto time slot n. It has beenshown in [18] that the minimum rate at which regret growsis of logarithmic order under certain regularity conditions.The authors established that for some families of rewarddistributions there are policies that can satisfy

E [Ti (n)] ≤(

1

D (µi||µ∗)+ o(1)

)ln(n) (6)

where o (1)→ 0 as n→∞ and

5

D (µ||µ∗) ≡∫µiln

µiµ∗

(7)

is the Kullback-Leibler divergence between the rewarddensity µi of a suboptimal arm i and the reward density ofthe optimal machine µ∗. Therefore, over infinite horizon, theoptimal arm is played exponentially more often than any otherarm.

IV. POLICIES AND REWARD

A. Selection Policies

A learning policy for antenna state selection (also referred tointerchangeably as selection technique) must overcome certainchallenges. We identify such challenges below:

1) Optimal antenna state for each wireless link (betweena single transmitter and a receiver location) is unknownapriori. Moreover, each wireless link may have a differ-ent optimal state. A selection technique should be ableto learn and find the optimal state for a given link.

2) For a given wireless link, there might be several stateswhich are near optimal over time based on channel con-ditions and multipath propagation. A selection techniqueshould provide a policy to balance between exploitinga known successful state and exploring other availablestates without excessive retraining to account for thedynamic behavior of the channel.

3) For the purpose of real-time implementation in a prac-tical wireless system, a selection technique must becomputationally efficient and employ simple metricswhich can be extracted from the channel without largeoverhead or extensive feedback data.

Most of the learning policies for the non-Bayesian multi-armed bandit problem in the literature, works by associatingan index called upper confidence index to each arm. Thecalculation of such an index relies on the entire sequence ofrewards obtained up to a point from selecting a given arm.The computed index for each arm is then used as an estimatefor the corresponding reward expectations and is used to selectthe arm with highest index.

We base the antenna state selection technique on the deter-ministic policy UCB1 and its variants as given in [20].

1) UCB1 - Selection Policy: To implement the UCB1policy, each receiver stores and updates two variables; theaverage of all the instantaneous reward values observed forstate i up to the current packet n denoted as Ri (n) (samplemean) and the number of times antenna state i has beenselected up to the current packet n, denoted as ni (n). The twoquantities Ri (n) and ni (n) are updated using the followingupdate rule:

Ri (n) =

Ri(n−1)ni(n−1)+Ri(n)

ni(n−1)+1 if state i is selected

Ri (n− 1) else

(8)

ni (n) =

ni (n− 1) + 1 if state i is selectedni (n− 1) else (9)

The UCB1 policy as shown in Algorithm 1, first beginsby selecting each antenna state at least once and Ri (n) andni (n) are then updated using 8 and 9. Once the initializationis completed, the policy selects the state that maximizes thecriteria on line 6. From line 6, it can be seen that the index ofthe policy is the sum of two terms. The first term is simply thecurrent estimated average reward. The second term is the sizeof the one-sided confidence interval of the estimated averagereward within which the true expected value of the meanfalls with a very high probability. The estimate of the averagereward improves and the confidence interval size reduces asthe number of times an antenna state is selected increases.Eventually, the estimated average reward reaches as close aspossible to the true mean. The size of the confidence intervalalso governs the index of the arm for future exploration.

−

1R

−

2R

1

)ln(2

n

n

2

)ln(2

n

n 12 where nn <

Select arm 2Arm 2

Arm 1

Fig. 3. Illustration of UCB1 Policy for a 2-armed bandit problem

For an instance, consider a two-armed bandit problemshown in Fig. 3. In order to select between the two arms, ifonly the average reward R1 and R2 are considered, clearly arm1 will be selected. However, this will ignore the confidence inthe other arm in order to explore higher pay-offs. By addingthe size of the confidence interval to the index term, even thethough the R2 < R1, arm 2 is selected for further explorationinstead of arm 1. Since the confidence interval size dependson the number of times an arm has been played as well as thetotal number of trials, an arm which has been played only afew times and has estimated average reward near to an optimal,will be chosen to be played.

Algorithm 1 UCB1 Policy, Auer et al. [20]1: // Initialization2: ni, Ri ← 03: Select each antenna state at least once and update ni, Ri

accordingly.4: // Main Loop5: while 1 do6: Select antenna state i that maximizes Ri +

√2ln(n)ni

7: Update ni, Ri for antenna state i8: end while

The UCB1 policy has an expected regret of at most [20]:

6

8∑

i:µi<µ∗

lnn∆i

+

(1 +

π2

3

) ∑i:µi<µ∗

∆i

(10)

where ∆i = µ∗ − µi2) UCB1-Tuned Policy: For practical implementations, it

has been shown that by replacing the upper confidence boundof UCB1 with a different bound to account for the variancein the reward yields better results [20]. In UCB1-Tuned, theexploration term is given as:

√ln(n)

nimin

1

4, Vi (ni)

(11)

where Vi is defined as:

Vi (s) ≡(

1

s

∑R2i,s

)− R2

i,s +

√2ln(t)

s(12)

when state i has been selected s times during the first ttime slots. Therefore, the UCB1-Tuned policy can now bewritten as Algorithm II. Even though UCB1-Tuned policy hasbeen shown to work experimentally well, there are no proofsavailable for the regret bounds in the literature. We expect thispolicy to work best for the scenarios where the link qualitymetrics have large variance due to dynamic channel variations.

Algorithm 2 UCB1-Tuned Policy, Auer et al. [20]1: // Initialization2: ni, Ri ← 03: Select each antenna state at least once and update ni, Ri

accordingly.4: // Main Loop5: while 1 do6: Select antenna state i that maximizes Ri +√

ln(n)ni

min

14 , Vi (ni)

7: Update ni, Ri for antenna state i8: end while

3) UCB1-Normal Policy: The two UCB1 polices describedabove made no assumptions on the reward distributions. In theUCB1-Normal policy, it is assumed that the reward follows anormal distribution with unknown mean and variance. Thisis a special case and it is shown in [20] that the policy inAlgorithm 3 achieves logarithmic regret uniformly over n timeslots. The index associated with each arm is still calculatedbased on the one sided confidence interval of the averagereward, but since the reward distribution is known, insteadof using the Chernoff-Hoeffding bound, to compute the index,the sample variance is used as an estimate of the unknownvariance. Thus, the UCB1 policy can be modified as shown inAlgorithm 3 and the highest index is calculated using line 8.

Algorithm 3 UCB1-Normal Policy, Auer et al. [20]1: // Initialization2: ni, Ri ← 03: // Main Loop4: while 1 do5: Select the antenna state which has not been selected

at least 8logn times6: Otherwise select the antenna state i that maximizes7:

8: Ri +√

16 qi−niRi2

ni−1ln(n−1)ni

9:10: Update ni, Ri for antenna state i11: end while

where qi is the sum of squared rewards for state i.UCB1-Normal policy has an expected regret of at most:

256 (logn)

∑i:µi<µ∗

σ2i

∆i

+

(1 +

π2

2+ 8logn

)( K∑i=1

∆i

)(13)

B. Reward Metrics

In this section, we discuss the link quality metrics that weuse as instantaneous reward for the selection policies describedabove. The selection of reward metrics is dependent on thespecific system implementation and based on the desiredobjective, the system designer can identify a relevant rewardmetric. In this paper, we evaluate two commonly used linkquality metrics for MIMO systems.

1) Post-Processing SNR (PPSNR): We first use Post-Processing SNR as the reward metric to perform the antennastate selection. PPSNR can be defined as the inverse of theError Vector Magnitude (EVM) [32], [33]. As true SNR isnot easily available for MIMO systems on a per-packet basis,PPSNR can be used to approximate the received SNR andcan be used as the link quality metric. EVM for a MIMOspatial multiplexed system is defined as the squared symbolestimation error calculated after the MIMO decoding of thespatial streams. At every OFDM packet reception, we calculatethe PPSNR for all the subcarriers and separately for all thespatial streams. Then, the instantaneous reward Ri (n) for timeslot (packet) n can be calculated as:

Ri (n) =1

S

S∑s=1

1

F

F∑f=1

PPSNR[s]f (14)

where F is the number of subcarriers and S is the number ofspatial streams.

2) Demmel Condition Number (DCN): For MIMO systemsemploying spatial multiplexing (SM) technique, the separateantenna streams on which data is modulated are required to beas uncorrelated as possible for achieving maximum capacity.The correlation among the spatial streams is influenced bythe propagation effects such as multipath propagation andthe amount of scattering. In MIMO systems equipped withreconfigurable antennas, the additional degree of freedom to

7

select the radiation states can potentially reduce the correlationbetween the spatial streams. Previously, regular conditionnumber or its reciprocal have been used to evaluate the qualityof MIMO channel matrix. However, motivated by resultsin [34], we use the Demmel condition number as it can berelated to a sufficient condition for multiplexing to be betterthan diversity. If the Demmel condition number is high, itrepresents high correlation between the streams. We calculatethe Demmel condition number per subcarrier as:

κf =||Hf ||2Fλk

(15)

where || · ||F is the Frobenius norm of the channel and λk isthe smallest eigenvalue of the MIMO channel matrix Hf . Wethen calculate instantaneous reward Ri (n) as

Ri (n) =1

κ(16)

where,

κ =1

S

S∑s=1

1

F

F∑f=1

κ[s]f (17)

V. EXPERIMENTAL SETUP

A. Pattern Reconfigurable Leaky Wave Antennas

For our experiments, we use the Reconfigurable LeakyWave Antenna (RLWA) which is a two port antenna designedto electronically steer two highly directional independentbeams over a wide angular range. Initially proposed by theauthors in [7], the prototype shown in Fig. 4 is a com-posite right/left-handed leaky wave antenna composed of 25cascaded metamaterial unit cells [35] loaded with varactordiodes. In order to achieve the CRLH behavior, a unit cellis implemented by inserting an artificial series capacitanceand a shunt inductance into a conventional microstrip lineusing an interdigital capacitor and a shorted stub. Further, thereare two varactor diodes (DS) in parallel with the microstripseries interdigital capacitor and one varactor diode (DSH)is placed in series with the shunt inductor. The applicationof various combinations of bias voltages “S” and “SH” tothe independent bias networks, controls the beam directionallowing for symmetrical steering of the two radiation beamsat the two ports over a 140 range. As the two ports are locatedon the same antenna structure, it is used as a two-element arrayin a MIMO setup. The radiation states were selected so that allthe states have approximately similar measured gain with aslow pattern correlation as possible. All the radiation states atthe two ports are matched for a target return loss of 10dB withthe isolation between the two ports being higher than 10dB.

Fig. 4. Two port reconfigurable leaky wave antenna [7]

Though the antenna in [35] is ideally capable of switchingbetween an infinite number of radiation states, in order tocharacterize the effect of beam direction on the efficacy ofa wireless system with RLWAs deployed at both ends of alink, a subset of 5 states was selected to allow the beam tosteer over a range of 140 in the azimuthal plane. Fig. 5 showsthe measured radiation patterns for the selected states and theircorresponding bias voltages are shown in Table. I.

State

State

State

State

State

1

2

3

4

5

Fig. 5. Measured radiation patterns for port 1 & 2(Gain is shown in dB andis ≈ −3dB)

TABLE IMAIN RADIATION CHARACTERISTICS OF FIVE ANTENNA STATES

Index Bias Voltage(V) Gain(dB) Direction(expected)1 S = 38 SH = 5 -3 0

2 S = 5 SH = 30 -3 18

3 S = 10 SH = 5 -3.2 36

4 S = 2 SH = 10 -3.4 60

5 S = 2 SH = 2 -3 72

B. Measurement Setup

In our experiments we make use of the Wireless OpenAccess Research Platform (WARP), an FPGA-based softwaredefined radio testbed and WARPLab, the software develop-ment environment used to control WARP nodes from MAT-LAB [36]. Four WARP nodes were distributed throughoutthe fifth floor of the Drexel University Bossone ResearchCenter as shown in Fig. 7. This setup allowed us to captureperformance for both Line-of-Sight (LOS) and Non-Line-of-Sight (NLOS) links and also cover a wide SNR regime. Asshown in Fig 6, the PPSNR varies significantly from locationto location and for each antenna state. When node 1 is active, itmeans that node 1 is transmitting and the other three nodes arereceiving. Once all the channels are collected for node 1, themeasurement controller makes node 2 as the transmitting nodeand other remaining nodes receive. This process is repeateduntil all four nodes have finished transmitting and the datais collected. Since we have 4 nodes and at a given time 1node transmits and 3 nodes receive, we have a set of total of12 links each corresponding to a unique transmitter-receivercombination. For certain links, there are more than one antennastates which are near optimal thus making the state selectionmore challenging. By using WARPLab, each of the nodes werecentrally controlled for the synchronization of the transmission

8

1 2 3 4 5 6 7 8 9 10 11 12

−5

0

5

10

15

20

Link Number

Aver

age P

ost P

roce

ssing

SNR

(dB)

State 1State 2State 3State 4State 5

Fig. 6. Average received Post Processing SNR for 5 antenna states at the receiver measured for 12 links. Link 1 - 3 is when node 1 is active, Link 4 - 6is when node 2 is active, Link 7 - 9 is when node 3 is active and Link 10 - 12 is when node 4 is active. Standard deviation is also shown for all links andantenna states.

and reception process and to provide control over the antennastates selected at each of the nodes. Although, the nodes werecontrolled centrally for data collection purposes, the learningalgorithm was decentralized. Specifically, no information dur-ing the learning process was shared with the transmitter.

Open Air18°

18°

Rx: Most selected optimal stateTx: Fixed antenna state

Node 4

Node 1

36° Node 2

Node 318°

Fig. 7. Node positions on the 5th floor of the Drexel University BossoneResearch Center.

The performance of the RLWA was evaluated in a 2 × 2MIMO system with spatial multiplexing as the transmissiontechnique [31]. The implemented communication system fol-lowed an OFDM PHY as described in the IEEE 802.11nstandard with total 64 subcarriers. 48 subscribers were usedfor loading data symbols, 4 for Carrier Frequency Offsetcorrection and 12 left blank (i.e., F=48, S=2). For collect-ing channel realizations, we use a broadcast scheme whereeach designated WARP node transmitter broadcasts packetsmodulated using BPSK. For each packet transmission, thereceiver nodes stored channel estimates and extracted thereward metrics as described above. Furthermore, the antennastates for each receiver node were switched after each packetuntil all 5 possible antenna states between the transmitter andreceivers were tested. This process was repeated until 200channel realizations were stored for all the state combinationsand for each node acting as a transmitter. The beam directionsin Fig. 7 corresponds to the optimal state selected most often

at each of the receivers when node 4 was transmitting. Theselection policies described in Section III-B are online learningpolicies but we note that we collected the channel realizationscorresponding to each state and evaluated the algorithm inpost-processing. This is essential in order to benchmark theperformance of different selection policies under the samechannel conditions and to make sure that channel conditionsdo not bias performance results.

VI. PERFORMANCE ANALYSIS AND RESULTS

We evaluate the performance of the proposed online selec-tion policies using the measurement setup described above.We compare these polices with three policies 1) Genie Policy:In this policy it is assumed that the true mean rewards forall the antenna states are known apriori and a genie alwaysselects the optimal antenna state. This closely represents theideal case where instantaneous full CSI corresponding to allthe states are available to select the optimal antenna state.2) Exhaustive Search with Periodic Training (ESPT): In thisselection scheme, all the antenna states are selected period-ically in sequence and the corresponding channel estimatesare then used to select the optimal state. The frequency andthe amount of the training is fixed and are given by τ andφ. Since, we do not change the frame structure to enableperiodic training for all antenna states, this represents analternative where all antenna states are periodically selectedboth for channel training and sending data. This can be viewedas the process of consecutive exploration and exploitation,except that the duration of the exploration and exploitationis fixed and exploration occurs uniformly across all the states3) Random Selection: In this selection scheme, at each timeslot, one antenna state is randomly selected with uniformprobability from the available K states.

A. Regret Analysis

The goal of the learning policies is to maximize the long-term average reward, but in order to analyze the cost oflearning, the regret of a system is an essential metric. Regret is

9

0 20 40 60 80 100 120 140 160 180 2000

200

400

600

800

1000

1200

Packet Number

Re

gre

t(n

)

Regret

UCB1UCB1−TunedUCB1−NormalESPT: τ=5,φ=1ESPT: τ=10,φ=1ESPT: τ=10,φ=2Random

(a) Regret vs time slot (n), K=5, Reward: PPSNR

0 20 40 60 80 100 120 140 160 180 2000

20

40

60

80

100

120

140

160

180

200

Packet Number

Re

gre

t(n

)

Regret

UCB1UCB1−TunedUCB1−NormalESPT: τ=5,φ=1ESPT: τ=10,φ=1ESPT: τ=10,φ=2Random

(b) Regret vs time slot (n), K=5, Reward: DCN

Fig. 8. Regret R (n)

0 50 100 150 2000

10

20

30

40

50

60

70

80

90

100

Time Slot (n)

Op

tim

al S

tate

Se

lecte

d (

%)

UCB1UCB1−TunedUCB1−NormalESPT: τ=5,φ=1

ESPT: τ=10,φ=1

ESPT: τ=10,φ=2Random

(a) K=5, Reward: PPSNR

0 50 100 150 2000

10

20

30

40

50

60

70

80

Time Slot (n)

Op

tim

al S

tate

Se

lecte

d (

%)

UCB1UCB1−TunedUCB1−NormalESPT: τ=5,φ=1

ESPT: τ=10,φ=1

ESPT: τ=10,φ=2Random

(b) K=5, Reward: DCN

Fig. 9. Percentage of the optimal state is selected

a finer performance criteria as compared to long-term averagereward and it indicates the growth of the loss incurred dueto learning without complete state information. In Fig. 8(a)and 8(b), we show the regret with respect to the number ofpackets for all the selection policies corresponding to twodifferent reward metrics respectively. The regret is averagedacross all the transmitter and receiver locations shown inFig. 7. It can be seen that the UCB1 policy and its variantshave a sublinear regret as compared to the ESPT and Randomselection. Further, it can be seen that varying the parametersof ESPT has direct impact on the regret of the system. ESPTwith τ = 10 and φ = 1 shows minimum regret amongother ESPT policies, where a single round of periodic trainingwas performed every 10 OFDM symbols. If there are manysuboptimal states and exhaustive training is performed withhigh frequency (τ = 5,φ = 1), suboptimal states will beselected much more often thus incurring higher regret. Also,if the frequency is kept constant and the amount of trainingis increased in order to get a better estimate of the reward(τ = 10,φ = 2), it will reduce the rate of regret but it will stillbe linear. This indicates the trade-off between the amount andthe frequency of the channel training which makes tuning theparameters τ and φ very challenging for a given environment.

On the other hand, UCB1 polices do not require parametertuning and inherently adjusts the exploration rate based onthe acquired estimate of the average reward and the associatedconfidence in the estimate.

Further in the Fig. 9(a) and 9(b), we show the percentageof time optimal state is selected by a policy up to time slot n.For reward metric instantaneous PPSNR, UCB1-Tuned policyoutperforms other policies, selecting the optimal state upto90% of the time.

While UCB1 policy performs closer to UCB1-Tuned, theUCB1-Normal performs suboptimally due to the assumptionthat the reward is normally distributed. As shown in [32],PPSNR is chi-squared distributed, therefore the UCB1-Normalpolicy is not expected to perform as well as the rest ofthe UCB1 policies. Further, as shown in the case of regret,performance of the ESPT policy varies as τ and φ are varied,where in this case, best ESPT policy selected the optimal stateonly upto 50%.

In Fig. 10, we show the impact of increasing the numberof antenna states on regret. As the number of antenna statesincrease, the regret for ESPT polices increase with higherleading constant while the regret for UCB1 policies increaseonly slightly. More interesting is the fact that the regret for

10

1 2 3 4 5 6 7 8 9 10 11 12−5

0

5

10

15

20

Link Number

Aver

age P

ost P

roce

ssing

SNR

(dB)

UCB1−Tuned (Best UCB1)UCB1−Normal (Worst UCB1)ESPT: τ=10,φ=1 (Best ESPT)

ESPT: τ=5,φ=1 (Worst ESPT)

Fig. 11. Average received Post Processing SNR for best and worst UCB and ESPT policies for 12 links. Link 1 - 3 is when node 1 is active, Link 4 - 6is when node 2 is active, Link 7 - 9 is when node 3 is active and Link 10 - 12 is when node 4 is active. Standard deviation is also shown for all links andantenna states.

K=2 K=3 K=4 K=50

100

200

300

400

500

600

700

800

900

1000

Number of Available Antenna States

Ave

rag

e R

eg

ret

UCB1UCB1−TunedUCB1−NormalESPT: τ=5,φ=1ESPT: τ=10,φ=1ESPT: τ=10,φ=2Random

Fig. 10. Regret R (n) vs Number of Available Antenna States available ateach receiver

random policy decreases as the number of available statesto select, increases. This decrease is due to the fact that therandom policy explores uniformly and if there are more thanone near optimal states, the regret will decrease. The regretfor the random policy will eventually grow in a situationwhere there is only one constantly optimal state and the restof the states are sub optimal as the probability of samplingsub optimal states will increase.

B. Average PPSNR

We first study the impact of increased pattern diversity onthe average PPSNR. In Fig 12, it can be seen that as the num-ber of antenna states at the receiver is increased, a higher gainin average PPSNR is achieved. There is a 25% improvementachieved by learning policies in average PPSNR across all thelinks when the number of antenna states is increased from 2

to 5. On the other hand, the gain in average PPSNR achievedby the best ESPT policy is only 10% while the worst caseESPT polocy shows negative improvement. This is due to thefact that, as the number of antenna states are increased, theexhaustive search with periodic training negates the benefit ofincreased pattern diversity due to increased training overhead.Interesingly, the random policy shows higher gain since therandom policy explores more frequently to find the optimalantenna state.

K=2 K=3 K=4 K=50

1

2

3

4

5

6

7

8

9

10

11

Number of Available Antenna States

Ave

rag

e P

PS

NR

(d

B)

UCB1

UCB1−Tuned

UCB1−Normal

ESPT: τ=5,φ=1

ESPT: τ=10,φ=1

ESPT: τ=10,φ=2

Random

Fig. 12. Gain in Average PPSNR (dB) vs Number of Available AntennaStates available at each receiver

We further study the improvement in the average PPSNRby evaluating the gain in the average PPSNR for both thereward metrics; instantaneous PPSNR and Demmel ConditionNumber (DCN). In Fig. 11, we compare the performanceof best case and worst UCB1 policies with respective ESPTpolicies. It can be seen that the percentage improvement inaverage PPSNR is significant between the worst case UCB1and ESPT policies. Also, the two UCB1 polices have only

11

marginial performance difference. In Fig. 13 and 14, we showthe empirical CDF of average PPSNR for all the selectionpolicies averaged across all the links in the network for thetwo reward metrics respectively. Additionally, we also showthe average PPSNR achieved by the genie policy as a referenceideal case which defines the upper bound. It can be seen fromFig. 13 that all UCB1 policies have better performance thanthe rest of the policies.

4 5 6 7 8 9 10 110

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Average Post−Processing SNR (dB)

F(x

)

UCB1UCB1−TunedUCB1−NormalESPT: τ=5,φ=1ESPT: τ=10,φ=1ESPT: τ=10,φ=2RandomUpper Bound

Fig. 13. Empirical CDF of Post-Processing SNR (PPSNR) (db) averagedacross all links. Reward: Instantaneous PPSNR

The average PPSNR for all UCB1 policies lie within1dB of the best case scenario which is shown as upperbound. The relative performance of the ESPT policies followa similar trend as seen above. The best performing UCB1policy achieves 2.3dB higher average PPSNR providing 31%improvement over the best ESPT (τ = 10,φ = 1) policywhile the worst UCB1 policy acheives 4.4dB higher averagePPSNR providing 82% improvement over the worst case ESPT(τ = 5,φ = 1).

3.5 4 4.5 5 5.5 6 6.5 7 7.5 8 8.50

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Average Post−Processing SNR (dB)

F(x

)

UCB1UCB1−TunedUCB1−NormalESPT: τ=5,φ=1ESPT: τ=10,φ=1ESPT: τ=10,φ=2RandomUpper Bound

Fig. 14. Empirical CDF of Post-Processing SNR (PPSNR) (db) averagedacross all links. Reward: Instantaneous DCN

Further, in Fig. 14, when instantaneous DCN is selected asthe reward metric, the overall gain in average PPSNR is less

than the case when instantaneous PPSNR is selected as thereward metric. This difference in average PPSNR indicates thatthe optimal state which provides the most correlation betweenthe streams may not always provide the best PPSNR, sincereceived PPSNR also depends on the power in the channeland is influenced by propagation effect such as fading. Inthis case, the best performing UCB1 policy achieves 1.4dBhigher average PPSNR providing 22% improvement over thebest ESPT (τ = 10,φ = 1) policy while the worst UCB1policy acheives 2.4dB higher average PPSNR providing 46%improvement over worst case ESPT (τ = 5,φ = 1).

C. Sum Throughput

We further assess the impact of improved PPSNR on thethroughput of the network. To analyze throughput improve-ment, we perform a simple discrete rate look up table basedadaptive modulation and coding (AMC) technique. The fixedlook up table is defined by SNR ranges obtained by usingthe upper bound expression for symbol error probability inAWGN channels [37].

P√M ≈ 2

(1− 1√

M

)Q

(√3m

(M − 1)

EbN0

1

r

)≤ E (18)

where,

E = 2 exp

(3m

(M − 1)

EbN0

1

r

)(19)

where M is the constellation order, r is the coding rate,Eb

N0is the SNR per bit, and m = log2 (M). The selected

AMC scheme is used for all the subcarriers. Based on themeasured received PPSNR of each antenna state, we calculatethe throughput each link would achieve when using the AMCschemes shown in Table. II.

TABLE IIAMC SCHEMES WITH DATA RATES (BPS/HZ)

Index Modulation(M) Coding Rate (r) Data rate (bps/Hz)1 BPSK 1/2 0.52 4-QAM 1/2 13 4-QAM 3/4 1.54 16-QAM 1/2 25 16-QAM 3/4 36 64-QAM 2/3 47 64-QAM 3/4 4.5

In Fig. 15 and 16, we show the empirical CDF of calculatedsum throughput using the AMC scheme described above fortwo reward metrics respectively. The PPSNR improvementrealized by using the learning policies allow each receiver toselect an appropriate AMC scheme to maximize link through-put, thereby improving the sum throughput of the network. Asshown in Fig. 15, the best performing UCB1 policies achieve17% improvment over best case ESPT policy, while the worstcase UCB1 policy achieves 38% improvement over worst caseESPT policy.

For the scenario where DCN is used as reward metric UCB1policy achieves 8% and 13% improvement over best case andworst case respectively.

12

13 14 15 16 17 18 19 20 21 220

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Sum Throughput (bits/sec/Hz)

F(x

)

UCB1UCB1−TunedUCB1−NormalESPT: τ=5,φ=1ESPT: τ=10,φ=1ESPT: τ=10,φ=2Random

Fig. 15. Empirical CDF of Sum throughput (bits/sec/Hz) across all links.Reward: Instantaneous PPSNR

13 14 15 16 17 180

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Sum Throughput (bits/sec/Hz)

F(x

)

UCB1UCB1−TunedUCB1−NormalESPT: τ=5,φ=1ESPT: τ=10,φ=1ESPT: τ=10,φ=2Random

Fig. 16. Empirical CDF of Sum throughput (bits/sec/Hz) across all links.Reward: Instantaneous DCN

VII. CONCLUSION

We have proposed a novel online learning based antennastate selection technique and have shown that wireless sys-tems employing reconfigurable antennas can benefit fromsuch technique. The proposed selection technique allows theoptimal state selection without requiring instantaneous CSIfor all the antenna states and does not require modificationto OFDM frame for periodic training. This leads to reducedoverhead of channel training. The performance of the proposedselection technique is empirically evaluated in a practicalwireless system covering wide SNR range and both LOSand NLOS links. We show the impact of available antennastates on system regret and long time average reward. Arelative average PPSNR gain and corresponding throughputgain is achieved and the performance is compared to the idealselection technique utilizing instantaneous full CSI. Futurework will involve devising new learning policies which utl-ize multiple reward metrics at the same time for sequentialdecision making. In addition, investigating the application ofantenna state selection in conjunction with optimal channel

selection in a cognitive radio network or multi-user MIMOnetwork are possible future directions.

ACKNOWLEDGMENT

The authors wish to acknowledge Kevin Wanuga, DavidGonzalez, and Rohit Bahl for their valuable suggestions andfeedback. This material is based upon work supported by theNational Science Foundation under Grant No. 0916480.

REFERENCES

[1] N. Gulati, D. Gonzalez, and K. Dandekar, “Learning algorithm for re-configurable antenna state selection,” in Radio and Wireless Symposium(RWS), 2012 IEEE. IEEE, 2012, pp. 31–34.

[2] J. Boerman and J. Bernhard, “Performance study of pattern recon-figurable antennas in MIMO communication systems,” Antennas andPropagation, IEEE Transactions on, vol. 56, no. 1, pp. 231–236, 2008.

[3] D. Piazza, N. Kirsch, A. Forenza, R. Heath, and K. Dandekar, “Designand evaluation of a reconfigurable antenna array for MIMO systems,”Antennas and Propagation, IEEE Transactions on, vol. 56, no. 3, pp.869–881, 2008.

[4] A. Sayeed and V. Raghavan, “Maximizing MIMO capacity in sparsemultipath with reconfigurable antenna arrays,” Selected Topics in SignalProcessing, IEEE Journal of, vol. 1, no. 1, pp. 156–166, 2007.

[5] Y. Jung, “Dual-band reconfigurable antenna for base-station applica-tions,” Electronics Letters, vol. 46, no. 3, pp. 195 –196, 4 2010.

[6] Adant. [Online]. Available: http://www.adant.com/[7] D. Piazza, M. D’Amico, and K. Dandekar, “Two port reconfigurable

CRLH leaky wave antenna with improved impedance matching andbeam tuning,” in Antennas and Propagation, 2009. EuCAP 2009. 3rdEuropean Conference on. IEEE, 2009, pp. 2046–2049.

[8] C. Sukumar, H. Eslami, A. Eltawil, and B. Cetiner, “Link performanceimprovement using reconfigurable multiantenna systems,” Antennas andWireless Propagation Letters, IEEE, vol. 8, pp. 873–876, 2009.

[9] J. Kountouriotis, D. Piazza, K. Dandekar, M. D’Amico, andC. Guardiani, “Performance analysis of a reconfigurable antenna systemfor MIMO communications,” in Antennas and Propagation (EUCAP),Proceedings of the 5th European Conference on. IEEE, 2011, pp.543–547.

[10] R. Bahl, N. Gulati, K. Dandekar, and D. Jaggard, “Impact of reconfig-urable antennas on interference alignment over measured channels,” inGLOBECOM Workshops (GC Wkshps), IEEE. IEEE, 2012.

[11] T. Gou, C. Wang, and S. Jafar, “Aiming perfectly in the dark-blind in-terference alignment through staggered antenna switching,” in GLOBE-COM 2010, 2010 IEEE Global Telecommunications Conference. IEEE,2010, pp. 1–5.

[12] L. Ke and Z. Wang, “Degrees of freedom regions of Two-User MIMO Zand full interference channels: The benefit of reconfigurable antennas,”Information Theory, IEEE Transactions on, vol. 58, no. 6, pp. 3766–3779, june 2012.

[13] G. Huff, J. Feng, S. Zhang, and J. Bernhard, “A novel radiationpattern and frequency reconfigurable single turn square spiral microstripantenna,” Microwave and Wireless Components Letters, IEEE, vol. 13,no. 2, pp. 57–59, 2003.

[14] M. Fries, M. Grani, and R. Vahldieck, “A reconfigurable slot antennawith switchable polarization,” Microwave and Wireless ComponentsLetters, IEEE, vol. 13, no. 11, pp. 490–492, 2003.

[15] J. Bernhard, “Reconfigurable antennas,” Synthesis Lectures on Antennas,vol. 2, no. 1, pp. 1–66, 2007.

[16] D. Piazza, P. Mookiah, M. D’Amico, and K. Dandekar, “Two portreconfigurable circular patch antenna for MIMO systems,” in Antennasand Propagation, 2007. EuCAP 2007. The Second European Conferenceon. IET, 2007, pp. 1–7.

[17] D. Anagnostou, G. Zheng, M. Chryssomallis, J. Lyke, G. Ponchak,J. Papapolymerou, and C. Christodoulou, “Design, fabrication, and mea-surements of an RF-MEMS-based self-similar reconfigurable antenna,”Antennas and Propagation, IEEE Transactions on, vol. 54, no. 2, pp.422–432, 2006.

[18] T. L. Lai and H. Robbins, “Asymptotically efficient adaptive allocationrules,” Advances in Applied Mathematics, vol. 6, no. 1, pp. 4 – 22, 1985.

[19] V. Anantharam, P. Varaiya, and J. Walrand, “Asymptotically efficientallocation rules for the multiarmed bandit problem with multiple plays-part I: I.I.D rewards,” Automatic Control, IEEE Transactions on, vol. 32,no. 11, pp. 968–976, 1987.

13

[20] P. Auer, N. Cesa-Bianchi, and P. Fischer, “Finite-time analysis of themultiarmed bandit problem,” Machine learning, vol. 47, no. 2, pp. 235–256, 2002.

[21] J. Gittins, “Bandit processes and dynamic allocation indices,” Journalof the Royal Statistical Society. Series B (Methodological), pp. 148–177,1979.

[22] A. Grau, H. Jafarkhani, and F. De Flaviis, “A reconfigurable multiple-input multiple-output communication system,” Wireless Communica-tions, IEEE Transactions on, vol. 7, no. 5, pp. 1719–1733, 2008.

[23] D. Piazza, J. Kountouriotis, M. D’Amico, and K. Dandekar, “A techniquefor antenna configuration selection for reconfigurable circular patcharrays,” Wireless Communications, IEEE Transactions on, vol. 8, no. 3,pp. 1456–1467, 2009.

[24] H. Eslami, C. Sukumar, D. Rodrigo, S. Mopidevi, A. Eltawil, L. Jofre,and B. Cetiner, “Reduced overhead training for multi reconfigurableantennas with beam-tilting capability,” Wireless Communications, IEEETransactions on, vol. 9, no. 12, pp. 3810–3821, 2010.

[25] L. Lai, H. El Gamal, H. Jiang, and H. Poor, “Cognitive medium access:Exploration, exploitation, and competition,” Mobile Computing, IEEETransactions on, vol. 10, no. 2, pp. 239–253, 2011.

[26] Q. Zhao, B. Krishnamachari, and K. Liu, “On myopic sensing for multi-channel opportunistic access: Structure, optimality, and performance,”Wireless Communications, IEEE Transactions on, vol. 7, no. 12, pp.5431–5440, 2008.

[27] Y. Gai, B. Krishnamachari, and R. Jain, “Learning multiuser channelallocations in cognitive radio networks: a combinatorial multi-armedbandit formulation,” in New Frontiers in Dynamic Spectrum, 2010 IEEESymposium on. IEEE, 2010, pp. 1–9.

[28] Y. Gai and B. Krishnamachari, “Decentralized online learning algorithmsfor opportunistic spectrum access,” in Global Telecommunications Con-ference (GLOBECOM 2011), 2011 IEEE. IEEE, 2011, pp. 1–6.

[29] K. Liu and Q. Zhao, “Distributed learning in multi-armed bandit withmultiple players,” Signal Processing, IEEE Transactions on, vol. 58,no. 11, pp. 5667–5681, 2010.

[30] H. Volos and R. Buehrer, “Cognitive engine design for link adaptation:an application to multi-antenna systems,” Wireless Communications,IEEE Transactions on, vol. 9, no. 9, pp. 2902–2913, 2010.

[31] P. Wolniansky, G. Foschini, G. Golden, and R. Valenzuela, “V-blast: Anarchitecture for realizing very high data rates over the rich-scatteringwireless channel,” in Signals, Systems, and Electronics, 1998. ISSSE98. 1998 URSI International Symposium on. IEEE, pp. 295–300.

[32] R. Shafik, S. Rahman, R. Islam, and N. Ashraf, “On the error vectormagnitude as a performance metric and comparative analysis,” inEmerging Technologies, 2006. ICET’06. International Conference on.IEEE, 2006, pp. 27–31.

[33] H. Arslan and H. Mahmoud, “Error vector magnitude to SNR conversionfor nondata-aided receivers,” Wireless Communications, IEEE Transac-tions on, vol. 8, no. 5, pp. 2694–2704, 2009.

[34] R. Heath and A. Paulraj, “Switching between diversity and multiplexingin MIMO systems,” Communications, IEEE Transactions on, vol. 53,no. 6, pp. 962–968, 2005.

[35] D. Piazza, M. D’Amico, and K. Dandekar, “Performance improvementof a wideband MIMO system by using two-port RLWA,” Antennas andWireless Propagation Letters, IEEE, vol. 8, pp. 830–834, 2009.

[36] Rice University WARP project. [Online]. Available: http://warp.rice.edu[37] A. Goldsmith, Wireless communications. Cambridge university press,

2005.

Nikhil Gulati received the B.Eng. degree in Elec-tronics Instrumentation and Control from Universityof Rajasthan, India, in 2005. In 2007, he receivedthe M.S degree specializing in Control Theory andRobotics from Drexel University in Philadelphiawhere he conducted research on sensor networks, au-tonomous systems and real-time control. He workedas a Software Engineer from 2007 - 2010.

He is currently with Drexel University, where he ispursuing his PhD. degree in electrical and computerengineering. His research is focused on developing

adaptive algorithms for cognitive radios employing electrical reconfigurableantennas. He also works on applying benefits of electrically reconfigurableantennas to Interference Alignment and Physical Layer Security.

Kapil R. Dandekar received the B.S. degree inelectrical engineering from the University of Vir-ginia in 1997 with specializations in communica-tions and signal processing, applied electrophysics,and computer engineering. He received the M.S. andPh.D. degrees in Electrical and Computer Engineer-ing from the University of Texas at Austin in 1998and 2001, respectively. In 1992, he worked at theU.S. Naval Observatory and from 1993-1997, heworked at the U.S. Naval Research Laboratory. In2001, Dandekar joined the Electrical and Computer

Engineering Department at Drexel University in Philadelphia, Pennsylvania.He is currently an Associate Professor and the Director of the Drexel WirelessSystems Laboratory (DWSL). DWSL has been supported by the U.S. NationalScience Foundation, Army CERDEC, National Security Agency, Office ofNaval Research, and private industry. Dandekars current research interestsinvolve MIMO ad hoc networks, reconfigurable antennas, free space opticalcommunications, ultrasonic communications, and sensor networks. He haspublished articles in several journals including IEEE TRANSACTIONS ONANTENNAS AND PROPAGATION, IEEE TRANSACTIONS ON WIRE-LESS COMMUNICATIONS, IEEE TRANSACTIONS ON COMMUNICA-TIONS, IEEE TRANSACTIONS ON VEHICULAR TECHNOLOGY, andIEE ELECTRONICS LETTERS.