arX

iv:1

509.

0188

6v2

[cs.

IT]

3 O

ct 2

015

1

A Lyapunov Optimization Approach for GreenCellular Networks with Hybrid Energy SuppliesYuyi Mao, Student Member, IEEE, Jun Zhang,Senior Member, IEEE, and Khaled B. Letaief,Fellow, IEEE

Abstract—Powering cellular networks with renewable energysources via energy harvesting (EH) has recently been proposedas a promising solution for green networking. However, withintermittent and random energy arrivals, it is challenging toprovide satisfactory quality of service (QoS) in EH networks.To enjoy the greenness brought by EH while overcoming theinstability of the renewable energy sources, hybrid energysupply(HES) networks that are powered by both EH and the electricgrid have emerged as a new paradigm for green communications.In this paper, we will propose new design methodologies for HESgreen cellular networks with the help of Lyapunov optimizationtechniques. The network service cost, which addresses boththegrid energy consumption and achievable QoS, is adopted asthe performance metric, and it is optimized via base stationassignment and power control (BAPC). Our main contributionis a low-complexity online algorithm to minimize the long-termaverage network service cost, namely, theLyapunov optimization-based BAPC(LBAPC) algorithm. One main advantage of thisalgorithm is that the decisions depend only on the instantaneousside information without requiring distribution informat ion ofchannels and EH processes. To determine the network operation,we only need to solve a deterministic per-time slot problem,forwhich an efficient inner-outer optimization algorithm is pr oposed.Moreover, the proposed algorithm is shown to be asymptoticallyoptimal via rigorous analysis. Finally, sample simulationresultsare presented to verify the theoretical analysis as well as validatethe effectiveness of the proposed algorithm.

Index Terms—Green communications, energy harvesting, hy-brid energy supply, base station assignment, power control, QoS,Lyapunov optimization.

I. I NTRODUCTION

T HE continuous growth of wireless applications combinedwith the proliferation of smart mobile devices has re-

sulted in an unprecedented growth of wireless data traffic,which has contributed to a dramatic increase in the energyconsumption and carbon emission of the Information andCommunication Technology (ICT) sector. It is estimated thatthe annual carbon emissions and electric power consumptionof the ICT industry will reach up to 235 Mto [1] and 414TWh [2], respectively, in 2020. Heterogeneous and small cellnetworks (HetSNets) provide an energy-efficient paradigm toimprove the network capacity, and thus have been regardedas one of the most promising solutions for realizing greenradio [3], [4]. However, with the dense deployment of basestations (BSs) in HetSNets, the overall energy consumptionand carbon footprint will still be high [5]. Consequently, it isurgent to seek alternative green energy sources for wirelessnetworks.

The authors are with the Department of Electronic and Computer Engineer-ing, the Hong Kong University of Science and Technology, Clear Water Bay,Kowloon, Hong Kong. E-mail:ymaoac, eejzhang, eekhaled@ust.hk.

The recent advances of energy harvesting (EH) technologiesenable the BSs with EH components to capture ambientrecyclable energy, e.g., solar radiation and wind energy, whichis promising to achieve green networking [6], [7]. By introduc-ing EH capabilities to the next-generation cellular networks,potentially 20% of theirCO2 emission can be reduced [8].Nevertheless, since the surrounding harvestable energy de-pends highly on environmental factors such as location andweather, the harvested energy is unstable by nature. As aresult, it is challenging to maintain satisfactory qualityofservice (QoS) if communication nodes are solely powered bythe harvested renewable energy. To enjoy the environmentalfriendliness of EH, and also to overcome the unreliabilityof the renewable energy sources, wireless networks with ahybrid energy supply (HES), where EH and the electricgrid coexist, will be an ideal solution. While HES networkshave attracted recent attention, they also bring new designchallenges. In particular, communication protocols developedeither for conventional grid-powered cellular networks orEHsystems cannot take the full benefits of the heterogeneousenergy sources in HES networks. In this paper, we shallpropose new design methodologies for HES wireless networks,which will provide valuable guidelines for developing greencellular networks supported by renewable energy sources inthe near future.

A. Related Works and Motivations

EH communications have attracted significant attentionfrom academia in recent years. It was revealed that, with eitherthe save-then-transmit protocol or the best-effort protocol,the capacity of the point-to-pointadditive white Gaussiannoise (AWGN) channel can be achieved if the transmitter ispowered by EH [9]. This result indicates the benefits of EHcommunications from the information theoretical perspective.However, as the harvested energy is intermittent and sporadic,on one hand, energy management in EH systems should bebased on thechannel side information(CSI) as in conventionalsystems, but on the other hand, it should be adaptive to theen-ergy side information(ESI). With non-causal side information(SI)1, including the CSI and ESI, the maximum throughputof point-to-point EH fading channels can be achieved by thedirectional water-filling(DWF) algorithm [10]. The study waslater extended to broadcast channels [11], multiple accesschannels [12], and cooperative communications systems [13],

1‘Causal SI’ refers to the case that, at any time instant, onlythe past andcurrent SI is known, while ‘non-causal SI’ means that the future SI is alsoavailable.

2

[14]. Besides these, scenarios with more practical assumptionson SI have been investigated in [15]–[17].

Transmission protocols for HES systems have also beenrecently studied, where the main focus is on point-to-pointsystems. Given the grid energy budget, atwo-stageDWFalgorithm was proposed in [18] to achieve the optimal through-put with non-causal SI at the transmitter. A similar problemwas investigated in [19], where a low-complexityrecursivegeometric water-fillingalgorithm was derived. In [20], powerallocation strategies for weighted energy cost minimization ina point-to-point HES link were proposed. For wireless linkswith hybrid energy sources, optimal power allocation policiesto minimize the non-harvested energy consumption with delay-constrained data traffic requirement were proposed in [21].And resource allocation policies to maximize the energyefficiency in HES OFDMA systems were developed in [22].To realize green networking, more recently, powering cellularnetworks with hybrid energy supplies has been proposed [7],[23]–[26]. The design in the network setting becomes morechallenging since more decisions should be made, and moreSI will be needed. To save the grid energy consumption, thegreen energy utilization optimization problem was solved in[23] assuming full ESI was available, and a green energyand latency-aware user association scheme was proposed in[24]. In [25], a sleep control scheme for HES networks wasdeveloped, while joint energy cooperation and communicationcooperation for HEScoordinated multi-point(CoMP) systemswas proposed in [26].

To simplify the design, previous studies on HES networkseither ignore the accumulation of harvested energy at BSs[24], [26], or assume non-causal ESI is available [23], [25],which cannot fully capture the intermittency and randomnessof EH. In general, for more practical online scenarios withcausal SI, the optimal transmission policies remain unknown.For many online cases, the design problem can be formulatedas aMarkov Decision Process(MDP) problem, and thus canbe solved in principle. However, due to the huge dimensionof system states in HES networks, the complexity of theMDP solutions is unacceptable. Although heuristic policiescan be developed, they generally do not have any performanceguarantees. Motivated by these limitations in existing works,in this paper, we will investigate how to design practicalonline transmission protocols for HES cellular networks withdesirable properties such as low complexity and theoreticalperformance guarantees. Specifically, Lyapunov optimizationwill be used as the main tool. Such techniques have a longhistory in the field of discrete stochastic processes and Markovchains [27]. Moreover, it has been one of the most importantmethods for delay-aware resource control problems in wirelesssystems [28], while application in EH networks was firstproposed by Huanget al. [15]. The algorithms developed fromthe Lyapunov optimization techniques enjoy various attractiveproperties, e.g., little requirement of prior knowledge, lowcomputational complexity, and quantifiable worst-case perfor-mance, which make them a good fit for HES networks.

B. Contributions

In this paper, we will develop effective online algorithms tooptimize green cellular networks powered by hybrid energysources based on Lyapunov optimization techniques. Ourmajor contributions are summarized as follows:

• We consider a multi-user HES cellular network, anda network service cost that incorporates both the gridenergy consumption and achievable QoS is adopted asthe performance metric. Thenetwork service cost mini-mization(NSCM) problem, which is an intractable high-dimension Markov decision problem, is first formulatedassuming causal SI. A modified NSCM problem withtightened battery output power constraints is then pro-posed, which will assist the algorithm design based onLyapunov optimization techniques.

• A low-complexity onlineLyapunov optimization-basedbase stationassignment andpower control (LBAPC)algorithm is proposed for the modified NSCM problem,which also provides a feasible solution to the originalproblem. In each time slot, the network operation onlydepends on the optimal solution of a deterministic opti-mization problem, which can be solved efficiently by aproposed inner-outer optimization algorithm.

• Performance analysis for the LBAPC algorithm is con-ducted. It is shown that the proposed algorithm canachieve asymptotically optimal performance of the origi-nal NSCM problem by tuning a set of control parameters.Moreover, it does not require statistical information of theinvolved stochastic processes including both the channeland EH processes, which makes it also applicable inunpredictable environments.

• Simulation results are provided to verify the theoreti-cal analysis, especially the asymptotic optimality of theLBAPC algorithm. Moreover, the effectiveness of theproposed policy is demonstrated by comparison with agreedy transmission scheme. It will be shown that theLBAPC algorithm not only achieves significant perfor-mance improvement in terms of the network service cost,but also greatly reduces both the network grid energyconsumption as well as the packet drop ratio. Moreover,it can more efficiently utilize the available spectrum.

The organization of this paper is as follows. In Section II, weintroduce the system model. The NSCM problem is formulatedin Section III. The LBAPC algorithm for the NSCM problemis proposed in Section IV and its performance analysis isconducted in Section V. We present the simulation results inSection VI and conclude this paper in Section VII.

II. SYSTEM MODEL

We consider a multi-user HES wireless network with anEH-BS (B1), an HES-BS (B2), andK mobile users (MUs),as shown in Fig. 1. The index set of the MUs is denoted asK = 1, · · · ,K. Two BSs coordinate to serve the MUs. TheEH-BS, which can be a small cell to increase system capacity[7], is equipped with an EH component and powered purely bythe harvested renewable energy, with the maximum transmitpower given bypmax

B1. The HES-BS is not only mounted with

3

Fig. 1. A multi-user HES wireless network with one EH-BS, oneHES-BSand four mobile users.

an EH component, but also connected to the electric grid, i.e.,it can utilize both the harvested energy and the grid energy,with the maximum transmit power denoted aspmax

B2. This HES-

BS may serve as a macro-BS to guarantee coverage. BothpmaxB1

andpmaxB2

are assumed to be bounded. In this paper, the HES-BS acts as the decision center and collects all the SI neededfor decision making. For ease of reference, we list the keynotations of our system model in TABLE I.

Time is slotted, and denoteτ as the time slot length andT = 0, 1, · · · as the set of time slot indices. Downlinktransmission is considered. Particularly, we assume that at thebeginning of each time slot, a packet withR bits arrives fromupper layers at the BS side for eachMU. These packets maybe transmitted via either the EH-BS or the HES-BS in thefollowing time slot. It may also happen that neither of the BSsis able to deliver the packet. BS assignment and power controlwill be adopted to optimize the network. DenoteItj,k ∈ 0, 1,with j ∈ B1,B2, D, as the BS assignment indicator forMUk

in the tth time slot, whereItB1,k= 1 (ItB2,k

= 1) indicatesthat the EH-BS (HES-BS) is assigned to serveMUk, andItD,k = 1 means neither of the BSs will transmit the packet,i.e., the packet is dropped. These indicators are subjectedtothe following operation constraint:

∑

j∈B1,B2,D

Itj,k = 1, ∀k ∈ K, t ∈ T . (1)

The transmit powers of the EH-BS and the HES-BS forMUk in time slot t are denoted asptB1,k

and ptB2,k, re-

spectively. As ptB1,koriginates from the harvested energy

at the EH-BS, for convenience it is also denoted asptH1,k,

while ptB2,kconsists of both the harvested energy, denoted

as ptH2,k, and the grid energy, denoted asptG,k, i.e., ptB2,k

=ptH2,k

+ptG,k. In this work, the energy consumed for purposesother than transmission, e.g., cooling and baseband signalprocessing, is neglected.

The EH processes are modeled as successive energy packetarrivals, i.e., at the beginning of each time slot, energypackets withEt

1 and Et2 arrive at the EH-BS and the HES-

BS, respectively. We assumeEt1’s (Et

2’s) are independent andidentically distributed (i.i.d.) among different time slots withthe maximum valueEmax

H1(Emax

H2). Although the i.i.d. EH

model is idealized, it captures the intermittent nature of theEH processes, and thus it has been widely adopted in the

TABLE ISUMMARY OF KEY NOTATIONS

Notation DescriptionB1 (B2) The EH-BS (HES-BS)K Index set of the MUsT Index set of the time slotsR Packet sizeIt

j,k BS assignment indicator forMUk in time slot t

pmax

B1(pmax

B2) Maximum transmit power at the EH-BS (HES-BS)

ptB1,k

(ptB2,k

) Transmit power of the EH-BS (HES-BS) forMUk in time slot t

ptH1,k

(ptH2,k

) Power consumption of harvested energy at theEH-BS (HES-BS) forMUk in time slot t

ptG,k

Power consumption of grid energy at theHES-BS forMUk in time slot t

Bt1

(Bt2)

Battery energy level at the EH-BS (HES-BS)in time slot t

Et1

(Et2)

Harvestable energy at the EH-BS (HES-BS)in time slot t

Emax

H1(Emax

H2) Maximum value ofEt

1(Et

2)

et1

(et2)

Harvested energy at the EH-BS (HES-BS)in time slot t

htB1,k

(htB2,k

) Channel gain from the EH-BS (HES-BS)to MUk in time slot t

NB1(NB2

) Number of available orthogonal channelsat the EH-BS (HES-BS)

ϕG (ϕD)Cost incurred by per Joule of grid energyconsumption (per packet drop)

wG (wD) The weight of the grid energy cost (packet drop cost)

literature, e.g., [9], [15], [20]. In each time slot, part ofthearrived energy, denoted asetj, satisfying

0 ≤ etj ≤ Etj , j ∈ 1, 2, t ∈ T , (2)

will be harvested and stored in a battery, and it will beavailable for transmission from the next time slot. We startbyassuming that the battery capacity is sufficiently large. Laterwe will show that by picking the values ofetj ’s, the batteryenergy levels are deterministically upper-bounded under theproposed algorithm, thus we only need finite-capacity batteriesin the actual implementation. More importantly, includingetj ’sas variables in the optimization facilitates the derivation andperformance analysis of the proposed Lyapunov optimization-based algorithm, which will be elaborated in the followingsections. Similar techniques were adopted in previous studies,such as [15] and [29]. Denote the battery energy levels ofthe EH-BS and the HES-BS at the beginning of time slottas Bt

1 and Bt2, respectively. Without loss of generality, we

assumeB0j = 0 andBt

j < +∞, j = 1, 2. Since the renewableenergy that has not yet been harvested can not be utilized, thefollowing energy causality constraint should be satisfied:

∑

k∈K

ptHj ,kτ ≤ Bt

j < +∞, j ∈ 1, 2, ∀t ∈ T . (3)

Thus,Btj evolves according to

Bt+1j = Bt

j −∑

k∈K

ptHj ,kτ + etj, j ∈ 1, 2, t ∈ T . (4)

The BSs will serve the MUs with multiple orthogonalchannels, e.g., by adopting OFDMA as in the LTE standard[30]. Static and orthogonal spectrum allocation is adoptedforthe two BSs, which is a popular scheme for tiered cellularnetworks [31], [32]. Specifically, we assumeNB1

+ NB2

orthogonal channels with equal bandwidthw Hz are available,

4

whereNB1≥ 1 of them are allocated for the EH-BS while the

remainingNB2≥ 1 channels are reserved for the HES-BS. In

this paper,NB1andNB2

are fixed and pre-determined. It isworthwhile to note that,NB1

andNB2can be further optimized

with the assistance oforthogonal access spectrum allocationstrategies based on various system parameters, such as theEH conditions at both BSs and the network traffic demand[31], [32]. This is beyond the scope of this paper and will behandled in our future works. As a result, at each time slot, theEH-BS and HES-BS can serve at mostNB1

andNB2MUs,

respectively. So the following channel assignment constraintshould be met:

∑

k∈K

Itj,k ≤ Nj , j ∈ B1,B2, t ∈ T . (5)

The channels are assumed to be flat fading within theconsidered bandwidth and the channel gains are i.i.d. amongdifferent time slots. We denote the channel gains from theEH-BS (HES-BS) toMUk as ht

B1,k(ht

B2,k). For simplic-

ity, we further assume the channel gains from the BSs tothe MUs are statistically independent and identical. Thus,htBj ,k

∼ FBj(x) , j = 1, 2, ∀k ∈ K, whereFBj

(x) denotes thecumulative distribution functions ofht

Bj ,k. As is the case in

most of the existing works on EH communications, error-freechannel estimation and feedback are assumed. Thus, perfectCSI is available at the transmitters. Therefore, the results inthis paper can serve as a design guideline and a performanceupper bound for cases where only imperfect CSI is available.Consequently, if the EH-BS (or HES-BS) servesMUk intime slot t, the throughput is given byr

(

htB1,k

, ptB1,k

)

(or

r(

htB2,k

, ptB2,k

)

), wherer (h, p) = wτ log2

(

1 + hpσ

)

is theShannon-Hartley formula andσ is the noise power at thereceiver.

III. PROBLEM FORMULATION

In this section, we will first introduce the performance met-ric, namely, the network service cost. Then the network servicecost minimization (NSCM) problem will be formulated, andits unique technical challenges will be identified.

A. The Network Service Cost Minimization Problem

As mentioned in Section II, it may happen that some datapackets can not be successfully delivered to the correspondingMUs. For instance, the EH-BS does not have enough harvestedenergy while the channel from the HES-BS is in deep fading,or all the available channels are occupied. In such circum-stances, neither of the BSs is capable of transmitting the datapacket, which induces apacket drop costϕD. In some real-time applications, this packet may indeed be dropped, whilefor other applications, this will increase the delay. On theotherhand, due to the intermittent and sporadic nature of energyharvesting, the HES-BS will need to use the grid energy fortransmission from time to time, which incurs agrid energycostϕG per Joule. Minimizing the grid energy consumptionand maximizing the provided QoS are two important designobjectives for HES networks. Thus, it is desirable to minimize

both types of costs in order to optimize the system. In thispaper, we will adopt theservice costas the performancemetric, which is the weighted sum of the grid energy cost andpacket drop cost. This metric was introduced in [33], whereit was shown to be capable of adjusting the tradeoff betweenthe grid energy consumption and the achievable QoS in HESnetworks. We will use thenetwork service cost(NSC) as theperformance metric for the studied HES network, which is thetotal service cost for all MUs and defined for thetth time slotas

NSCt,∑

k∈K

(

wGϕGptG,kτ + wDϕDItD,k

)

, (6)

where the first term presents the grid energy cost, the secondterm stands for the packet drop cost, andwG and wD arethe weights of the grid energy cost and packet drop cost,respectively. WhenwG ≫ wD, the network is grid energysensitive. On the other hand, whenwD ≫ wG, the networkplaces more emphasis on the successful packet delivery, i.e.,addresses on QoS more. Without loss of generality, we assumeϕG, ϕD, wG andwD are positive and bounded.

Remark1: Since the channel gains from the EH-BS (HES-BS) to different MUs are assumed to be i.i.d. in this paper, inorder to guarantee fairness, the weights of the packet dropcost and grid energy cost are set to be the same for allMUs in (6), i.e.,wG and wD. For cases where the channelgains from the BSs to the MUs are not statistically identical,the weights can be adjusted accordingly, and the proposedLyapunov optimization approach still applies.

Our design objective will be to minimize the long-termaverage network service cost, which addresses both the net-work grid energy consumption and the achievable QoS, i.e.,the percentage of successfully transmitted data packets oftheMUs. For each MU, the following QoS constraint should besatisfied:

∑

j∈B1,B2

Itj,kr(

htj,k, p

tj,k

)

≥(

1− ItD,k

)

R, k ∈ K, t ∈ T ,

(7)i.e., if it is decided to transmit the packet, the throughputshould be greater than the packet size. Consequently, theNSCM problem can be formulated as

P1 : minIt,pt,et

limT→+∞

1

T

T−1∑

t=0

E

[

K∑

k=1

(

ϕGptG,kτ + ϕDItD,k

)

]

s.t. (1)− (3), (5), (7)

0 ≤ ptj,k ≤ Itj,kpmaxj , ∀j ∈ B1,B2, t ∈ T , k ∈ K

(8)∑

k∈K

ptj,k ≤ pmaxj , ∀j ∈ B1,B2, t ∈ T (9)

Itj,k ∈ 0, 1, ∀j ∈ B1,B2, D, t ∈ T , k ∈ K(10)

ptH1,k, ptH2,k

, ptG,k ≥ 0, ∀t ∈ T , k ∈ K, (11)

where ϕG , wGϕG, ϕD , wDϕD, It =[

ItB1,k, ItB2,k

, ItD,k]

, pt =[

ptH1,k, ptH2,k

, ptG,k]

and et = [et1, et2]. Thus we need to determine the BS

assignment indicatorsIt, the power allocationpt, and the

5

harvested energy at both BSs, i.e.,et1 and et2. In P1, (8)indicates that there will be no power allocated if an MU isnot being served. The peak transmit power constraint and thepower non-negativity constraint are imposed in (9) and (11),respectively. Moreover, the zero-one indicator constraint forthe BS assignment indicators is represented by (10).

B. Problem Analysis

In the considered HES network, the system state includesthe battery energy levels, harvestable energy at each BS andthe channel states, and the action is the energy harvestingas well as the BS assignment and power control. It can bechecked easily that the allowable action set in each time slotonly depends on the current system state. Also, the statetransition is Markovian, which depends on the current systemstate and action, and is irrelevant with the state and actionhistory. Besides, the objective function inP1 is the long-termaverage network service cost. Therefore,P1 is an infinite-horizon Markov decision process (MDP) problem. In principle,the optimal solution ofP1 can be obtained by standardMDP algorithms, e.g., thevalue iteration algorithmandlinearprogramming reformulationapproach [34]. Nevertheless, inboth algorithms, we need to use finite states to characterizethe system for practical implementation. For example, whenK = 4, if we useM = 10 states to quantize the energy level ateach BS,E = 5 states to describe the harvestable energy, andL = 10 states to represent the channel gain, overall, there willbe M2E2L2K = 2.5 × 1011 possible system states. For thevalue iteration algorithm, it will take unacceptably long timeto converge to the optimal value function, while for thelinearprogramming reformulation, a large-scale linear programming(LP) problem with more than2.5 × 1011 variables has tobe solved, which is practically infeasible. Apart from thecomputational complexity, the memory requirement for storingthe optimal policies is also a big challenge. Thus, it is criticalto develop alternative approaches to handleP1.

In the next section, we will propose aLyapunovoptimization-based BS assignment and power control(LBAPC) algorithm to solveP1, which enjoys the followingfavorable properties:

• The decision of the LBAPC algorithm within each timeslot is of low complexity, and there is no memoryrequirement for storing the optimal policy.

• The LBAPC algorithm has no prior information require-ment on the channel statistics or the distribution of therenewable energy processes.

• The performance of the LBAPC algorithm is controlledby a triplet of control parameters. Theoretically, byadjusting these parameters, the proposed algorithm canbehave arbitrarily close to the optimal performance ofP1.

• An upper bound of the required battery capacity isobtained, which shall provide guidelines for practicalinstallation of the EH devices and storage units.

IV. ONLINE BS ASSIGNMENT AND POWER CONTROL: THE

LBAPC ALGORITHM

In this section, we will develop the LBAPC algorithm tosolveP1. In order to take the advantage of Lyapunov opti-mization, we will first introduce a modified NSCM problem toassist the algorithm design. Then the LBAPC algorithm willbe proposed for the modified problem, which also provides afeasible solution toP1. In Section V, we will show that thissolution is asymptotically optimal forP1.

A. The Modified NSCM Problem

Due to the energy causality constraint (3), the system’s de-cisions are not independent among different time slots, whichmakes the design challenging. This is a common difficultyfor the design of EH and HES communication systems. Wefind that by introducing a non-zero lower bound,ǫHj

, on thebattery output power, such coupling effects can be eliminatedand the network operations can be optimized by ignoring (3)at each time slot. Thus, we first introduce a modified versionof P1 as follows:

P2 : minIt,pt,et

limT→+∞

1

T

T−1∑

t=0

E

[

K∑

k=1

(

ϕGptG,kτ + ϕDItD,k

)

]

s.t. (1)− (3), (5), (7)− (11)∑

k∈K

ptHj ,k∈ Ωj , j = 1, 2, t ∈ T , (12)

whereΩj , 0⋃

[

ǫHj, pmax

Bj

]

and 0 < ǫHj≤ pmax

Bj, j =

1, 2. Compared toP1, an additional constraint on the batteryoutput power is imposed in (12), i.e., it is not allowed to bewithin

(

0, ǫHj

)

, j = 1, 2. Hence,P2 is a tightened versionof P1, and thus any feasible solution forP2 is also feasiblefor P1. Denote the optimal values ofP1 andP2 asNSC∗

P1

and NSC∗P2, respectively. The following proposition reveals

the relationship betweenNSC∗P1 and NSC∗

P2, which willlater help show the asymptotic optimality of the proposedalgorithm.

Proposition1: The optimal value ofP2 is greater than thatof P1, but no worse than the optimal value ofP1 plus a posi-tive constantν (ǫH1

, ǫH2), i.e.,NSC∗

P1 ≤ NSC∗P2 ≤ NSC∗

P1+ν (ǫH1

, ǫH2), where ν (ǫH1

, ǫH2) ,

(

1− FKB1

(η))

KϕD +

ǫH2τ · ϕG andη =

(

2Rwτ − 1

)

σǫ−1H1

.Proof: See Appendix A.

In general, the upper bound ofNSC∗P2 in Proposition 1 is

not tight. However, asǫHj, j = 1, 2 goes to zero,ν (ǫH1

, ǫH2)

diminishes, as shown in the following corollary.Corollary 1: By letting ǫH1

and ǫH2approach zero,

NSC∗P2 can be made arbitrarily close toNSC∗

P1, i.e.,lim

ǫH1,ǫH2

→0ν (ǫH1

, ǫH2) = 0.

Proof: First, limǫH2

→0ǫH2

τ · ϕG = 0. Meanwhile, since

limǫH1

→0η = +∞ and lim

x→+∞FB1

(x) = 1, by combin-

ing the two equations, we havelimǫH1

→0FKB1

(η) = 1, i.e.,

limǫH1

→0

(

1− FKB1

(η))

= 0. Thus, the desired result is obtained.

6

Proposition 1 bounds the performance ofP2 by that ofP1, while Corollary 1 shows that the performance of bothproblems can be made arbitrarily close. Actually, Corollary 1fits our intuition, since whenǫHj

→ 0, P2 reduces toP1.Moreover, we see from Proposition 1 that the upper boundof NSC∗

P2 equalsNSC∗P1 + ν (ǫH1

, ǫH2), which is a linear

function of ǫH2. As a result, it converges toNSC∗

P1 linearlywith respect toǫH2

. Nevertheless, the convergence speed withrespect toǫH1

depends on the channel statistics.

B. The LBAPC Algorithm

In this subsection, we will propose the LBAPC algorithmfor P2 based on Lyapunov optimization techniques. It is worthmentioning that the conventional Lyapunov optimization tech-niques, where the decisions are i.i.d., can not be applied toP2directly. As mentioned in the last subsection, this is becauseof the temporal correlation of the battery energy levels whichmakes the system’s decisions time dependent. Fortunately,for the modified NSCM problem, the weighted perturbationmethod provides an effective solution to circumvent this issue[35].

To present the algorithm, we will first define the perturbationparameters and the virtual energy queues, which are twocritical elements.

Definition 1: The perturbation parametersθ1 and θ2 forthe EH-BS and HES-BS are bounded constants satisfying

θ1 ≥ pmaxB1

τ +(

V KϕD + 1K 6= 1EmaxH2

pmaxB2

τ)

(ǫH1τ)−1 ,

(13)

θ2 ≥ pmaxB2

τ +(

V KϕD + 1K 6= 1EmaxH1

pmaxB1

τ)

(ǫH2τ)

−1,

(14)respectively. In (13) and (14),1· denotes the indicatorfunction and0 < V < +∞ is a control parameter in theLBAPC algorithm with unitJ2 · cost−1. Here, “cost” denotesthe unit of the network service cost.

Definition 2: The virtual energy queuesBt1 and Bt

2 aredefined as

Btj = Bt

j − θj , j ∈ 1, 2, (15)

which are shifted versions of the actual battery energy levels.We denoteBt , 〈Bt

1, Bt2〉 for convenience.

As will be elaborated later, the proposed algorithm mini-mizes the weighted sum of the virtual queue lengths and thenetwork service cost in each time slot, which shall stabilizeBt

j

around the perturbed energy levelθj and meanwhile minimizethe network service cost. The LBAPC algorithm is summa-rized in Algorithm 1. In each time slot, the network operationsare determined by solving the per-time slot problem, which isparameterized by the current system state, including the CSIandBt, and with all constraints inP2 except (3). (A Lyapunovdrift-plus-penalty function will be defined in Section V, andone of its upper bounds will be derived, which happens to bethe objective function of the per-time slot problem. Also, wewill show that ignoring (3) in the per-time slot problem willnot affect the feasibility of the LBAPC algorithm forP2.)This confirms that the LBAPC algorithm does not requireprior knowledge of the harvesting processes and the channel

statistics, which makes it also applicable for unpredictableenvironments. We will discuss the solution of the per-time slotproblem in the next subsection, and analyze the feasibilityaswell as the performance of the proposed algorithm in SectionV.

Algorithm 1 The LBAPC Algorithm1: At the beginning of each time slot, obtain the virtual

energy queue stateBtj , harvestable energyEt

j and channelgainsht

Bj,k, ∀k ∈ K, j = 1, 2.

2: Decideet, It andpt by solving the per-time slot problem,i.e.,

minIt,pt,et

2∑

j=1

Btj

(

etj −∑

k∈K

ptHj ,kτ

)

+ V

K∑

k=1

(

ϕGptG,kτ + ϕDItD,k

)

s.t. (1), (2), (5), (7)− (12).

3: Update the virtual energy queues according to (4) and(15).

4: Set t = t+ 1.

C. Solving the Per-Time Slot Problem

In this subsection, we will develop the optimal solution tothe per-time slot problem, which consists of two components:the optimal energy harvesting, i.e., to determineet, as wellas the optimal BS assignment and power control, i.e., todetermineIt andpt. We find the closed-form solution of theoptimal energy harvesting decision, and propose an efficientinner-outer optimization (IOO) algorithm to obtain the optimalBS assignment and power control decision. The results ob-tained in this subsection are essential for feasibility verificationand for performance analysis of the LBAPC algorithm inSection V.

Optimal Energy Harvesting: It is straightforward to showthat the optimal amount of harvested energyet∗ is obtainedby solving the following LP problem:

min0≤et

j≤Et

j

2∑

j=1

Btje

tj (16)

and its optimal solution is given by

et∗j = Etj · 1B

tj ≤ 0, j = 1, 2. (17)

Optimal BS Assignment and Power Control:Once decou-pling et from the objective function, we can then simplify theper-time slot problem into the following optimization problemPBPAC:

PBAPC : minIt,pt

−2∑

j=1

Btj

∑

k∈K

ptHj ,kτ

+ VK∑

k=1

(

ϕGptG,kτ + ϕDItD,k

)

s.t. (1), (5), (7)− (12),

7

which is a difficult mixed integer nonlinear programming(MINLP) problem. For each MU, there are 3 choices of BSassignment, so there are in total3K possible combinations.Moreover, for a fixedIt, a non-convex power control problemshould be solved, which further complicates the solution. Thus,the complexity of solvingPBAPC with exhaustive search isprohibitively high.

We will develop an IOO algorithm to solvePBAPC, whichreduces the search space and avoids solving the non-convexpower control problem. In the IOO algorithm, the inneroptimization finds the optimalItB2,k

, ItD,k, ptH2,k and

ptG,k for given ItB1,k and ptH1,k

, while the outer opti-mization determines the global optimalIt∗B1,k

andpt∗H1,k.

DenoteH , k ∈ K|ItB1,k= 1 and thus,ItB2,k

= 0,ptH2,k

= ptG,k = 0, ∀k ∈ H. (Hc , K \ H.) 2

1) The Inner Problem:Given ItB1,k and ptH1,k

, theinner problem can be written as

Pin : minX t

k

∑

k∈Hc

(

−Bt2p

tH2,k

τ + V ϕGptG,kτ − V ϕDItB2,k

)

s.t.∑

k∈Hc

ItB2,k≤ NB2

, ItB2,k∈ 0, 1, k ∈ Hc (18)

0 ≤ ptB2,k≤ ItB2,k

pmaxB2

,

ItB2,kptB2,k

≥ ItB2,kρtB2,k

, k ∈ Hc (19)∑

k∈Hc

ptB2,k≤ pmax

B2,∑

k∈Hc

ptH2,k∈ Ω2 (20)

ptH2,k, ptG,k ≥ 0, k ∈ Hc, (21)

whereX tk ,

[

ItB2,k, ptH2,k

, ptG,k

]

andρtB2,k,

(

2Rwτ − 1

)

σ/

htB2,k

is called the channel inversion power of the HES-BS toMUk channel.Pin is obtained by plugging the givenItB1,k

and ptH1,k

into the objective function ofPBAPC, utilizingthe fact that the transmit power forMUk should be greater thanthe channel inversion power, and eliminatingItD,k with (1).We denote the optimal value ofPin asJin (H). However,Pin

is still a combinatorial optimization problem and not easy tosolve. Fortunately, we find that, without loss of optimality, theMUs with lower channel inversion powers should be assignedto the HES-BS with higher priority, as shown in the followingproposition.

Proposition2: There exists an optimal solution forPin

satisfying

ρtB2,k1≤ ρtB2,k0

, ∀k1 ∈ S1, k0 ∈ S0, (22)

whereS1 , k ∈ Hc|ItB2,k= 1 andS0 , Hc \ S1 = k ∈

Hc|ItB2,k= 0. In other words, the MUs served by the HES-

BS have better channel conditions than the remaining MUs.

Proof: Suppose for an optimal solution ofPin, ∃k1 ∈S1, k0 ∈ S0 such thatρtB2,k1

> ρtB2,k0, we can always serve

MUk0instead ofMUk1

with the channel and transmit powerthat are originally allocated forMUk1

, which will not increasethe value of the objective function. Hence, there is also anoptimal solution forPin satisfying (22), which ends the proof.

2We focus on the non-trivial cases withHc 6= ∅, where∅ is the empty set.

Therefore, we may concentrate on the solutions that satisfytheproperty in Proposition 2, which means that only the optimalnumber of MUs that are assigned to the HES-BS, denoted asm∗, need to be identified. We denote[i] as the index of theMU in Hc with the ith smallestρtB2,[i]

, and Hcl (0 ≤ l ≤

|Hc|,Hc0 , ∅) as the set of MUs inHc with the l smallest

ρtB2,[l]. Besides,N , minNB2

, |Hc|.

SincePin is parameterized byBt2, we will investigate the

solution ofPin in the following three disjoint cases: 1)−Bt2 >

V ϕG, 2) Bt2 ≥ 0 and 3) 0 < −Bt

2 ≤ V ϕG. The optimalsolution to the inner problem for these three cases will beshown later in Corollary 2, 3 and 4, respectively.

The following lemma reveals a useful property for thesolution in case 1).

Lemma1: When−Bt2 > V ϕG, for an optimal solution to

Pin, we haveptH2,k= 0, ∀k ∈ Hc, i.e., no harvested energy

in the HES-BS will be consumed.Proof: Suppose for an optimal solution,∃k ∈ Hc such

that ptH2,k> 0, it is feasible to construct a new solution with

pt′

H2,k= 0 and pt

′

G,k = ptG,k + ptH2,k, where the value of the

objective function will decrease by(

−Bt2 − V ϕG

)

ptH2,kτ >

0. By contradiction, the result is obtained.According to Lemma 1, the transmit power of the HES-BS

comes from the electric grid, with weightV ϕG > 0. Thus, itis optimal to serve the MUs with the channel inversion power.Hence, the optimal solution toPin is given by Corollary 2.

Corollary 2: For case 1), i.e.,−Bt2 > V ϕG, the opti-

mal number of MUs assigned to the HES-BS ism∗ =max

i∈Acase1

i · 1Acase1 6= ∅, where Acase1 = i ∈

1, · · · , N∣

∣

∑i

k=1 ρtB2,[k]

≤ pmaxB2

, ϕGρtB2,[i]

τ ≤ ϕD. Andthe optimal solution toPin is given by

ItB2,k= 1k ∈ Hc

m∗

ptH2,k= 0

ptG,k = ItB2,kρtB2,k

, ∀k ∈ K. (23)

For case 2), as the weight of the harvested energy consump-tion is non-positive, i.e.,Bt

2 ≥ 0, we can use∑

k∈Hc ptH2,k=

pmaxB2

to decrease the length of the virtual energy queue byserving as many MUs as possible. Thus, the optimal solutionto Pin can be summarized in Corollary 3.

Corollary 3: For case 2), i.e.,Bt2 ≥ 0, the opti-

mal number of MUs assigned to the HES-BS ism∗ =max

i∈Acase2

i · 1Acase2 6= ∅, where Acase2 = i ∈

1, · · · , N∣

∣

∑i

k=1 ρtB2,[k]

≤ pmaxB2

. If m∗ = 0, which meansthat all MUs inHc are experiencing deep fading, the HES-BSis not able to provide service. Otherwise, the optimal solutionto Pin is given by

ItB2,k= 1k ∈ Hc

m∗

ptH2,k=

ρtB2,kItB2,k

∑

k∈Hcm∗

ρtB2,k

pmaxB2

ptG,k = 0

, ∀k ∈ K. (24)

Denote the total channel inversion powers of the MUsassigned to the HES-BS asρΣ, i.e., ρΣ ,

∑

k∈Hc ρtB2,kItB2,k

.

8

For case 3), the following result reveals the relationship amongρΣ, ptH2,k

andptG,k.Lemma2: When 0 < −Bt

2 ≤ V ϕG, given ItB2,k

(∑

k∈Hc ItB2,k> 0)3, ∀k ∈ K

ptH2,k=

0, ρΣ ∈[

0,−Bt

2ǫH2

V ϕG

]

ρtB2,kI

tB2,k

ρΣǫH2

, ρΣ ∈(

−Bt2ǫH2

V ϕG, ǫH2

]

ρtB2,kItB2,k

, ρΣ ∈(

ǫH2, pmax

B2

]

,

ptG,k =

ρtB2,kItB2,k

, ρΣ ∈[

0,−Bt

2ǫH2

V ϕG

]

0, ρΣ ∈(

−Bt2ǫH2

V ϕG, pmax

B2

]

. (25)

Proof: See Appendix B.Lemma 2 shows that the transmit power comes either from

harvested energy or grid energy, i.e., it can not be a mixtureof them. Inspired by this property, given the number of MUsserved by the HES-BSi, we haveItB2,k

= 1k ∈ Hci, ∀k ∈

K, and the optimal power allocation is determined by (25).Thus, we provide the optimal solution toPin in Corollary 4.

Corollary 4: For case 3), i.e.,0 < −Bt2 ≤ V ϕG, the

optimal number of MUs assigned to the HES-BS ism∗ =

arg mini∈Acase3

∑

k∈Hc

(

−Bt2p

tH2,k

+ V ϕGptG,k

)

τ − V ϕDi,

where Acase3 = 0⋃

i ∈ 1, · · · , N∣

∣

∑i

k=1 ρtB2,[k]

≤

pmaxB2

. The optimal ItB2,k to Pin is given by

ItB2,k= 1k ∈ Hc

m∗, ∀k ∈ K, and the optimalptH2,k and

ptG,k can be obtained by using (25) accordingly.Thus, the inner problem is solved by obtainingm∗ and

the associated power allocation policy by Corollary 2, 3 or4 depending on the value ofBt

2.2) The Outer Problem:For the EH-BS, givenH, the

optimal transmit power is given by

ptH1,k=

0, H = ∅ρtB1,kI

tB1,k∑

k∈HρtB1,k

pmaxB1

, H 6= ∅, Bt1 ≥ 0

ρtB1,kI

tB1,k

min∑

k∈HρtB1,k

,ǫH1ǫH1

, H 6= ∅, Bt1 < 0

, k ∈ H,

(26)whereρtB1,k

,

(

2Rwτ − 1

)

σ/htB1,k

. Thus, the outer problemcan be formulated as

Pout : minH∈FH

Φ (H) + Jin (H) (27)

where

Φ (H) =

0, H = ∅

−Bt1p

maxB1

τ − V ϕD|H|, H 6= ∅, Bt1 ≥ 0

−Bt1 max

∑

k∈H

ρtB1,k, ǫH1

τ − V ϕD|H|, H 6= ∅, Bt1 < 0

(28)

andFH = Ks ⊆ K∣

∣|Ks| ≤ NB1,∑

k∈KsρtB1,k

≤ pmaxB1

.The global optimalH∗, i.e., It∗B1,k

, can be obtained viasearching all subsets ofFH, and the associatedpt∗H1,k

canbe determined by (26). Basically, the IOO algorithm performs

3If∑

k∈Hc ItB2,k

= 0, ptH2,k

= ptG,k

= 0, ∀k ∈ K.

a reduced-size search, which eliminates part of the possiblecombinations ofIt’s that are not optimal. More importantly,for fixed It, closed-form expressions for power allocationare derived to avoid solving the non-convex power controlproblem. In the worst case, there are2K subsets ofFH, andfor the inner problem, we have to searchm∗ from 1 toK, i.e.,the complexity isO

(

2K ·K)

. Such exponential complexityis inevitable in BS assignment problems which are typicallyNP-hard. However, with a reasonable number of MUs, suchcomplexity is acceptable given the increasing computationpower at BSs. Overall, the IOO algorithm brings benefits ofaccelerating the searching processes compared to exhaustivesearch, while maintaining the optimality. In practice, at eachtime slot, the decision center, i.e., the HES-BS, collects the SI,runs the IOO algorithm and notifies the EH-BS of its decision.Details of the IOO algorithm are summarized in Algorithm 2.

Algorithm 2 The IOO Algorithm

1: Compute the channel inversion powerρtBj ,kbased on

htBj ,k

, j = 1, 2, k ∈ K.2: SetF = FH, J ∗ = 0, It∗D,k = 1, It∗B1,k

= 0, It∗B2,k= 0,

pt∗H1,k= pt∗H2,k

= pt∗G,k = 0, ∀k ∈ K.3: While F 6= ∅ do4: Arbitrarily pick H ∈ F , setItB1,k

= 1k ∈ H andobtainptH1,k

by (26).5: Based on the value ofBt

2, obtainItB2,k, ItD,k,

ptH2,k, ptG,k and the associatedJin (H) with either

Corollary 2, 3 or 4.6: If Φ (H) + Jin (H) < J ∗ do7: J ∗ = Φ(H) + Jin (H).8: UpdateIt∗, pt∗ with It, pt.9: Endif

10: F = F \ H.11: Endwhile

V. PERFORMANCEANALYSIS

One unique advantage of the proposed algorithm is that wecan provide theoretical performance analysis and characterizeits asymptotic optimality, which will be pursued in this section.We will first prove the feasibility of the LBAPC algorithmfor P2, which will be followed by the optimality charac-terization. Our analysis is based on Lyapunov optimizationtheory, where the Lyapunov drift function is the key element.During the analysis, an auxiliary optimization problemP3will be introduced, which bridges the optimal performance ofP2 and the performance achieved by the proposed algorithm.Together with Proposition 1, this will establish the asymptoticoptimality of the LBAPC algorithm forP1.

We verify the feasibility of the LBAPC algorithm by show-ing that the battery energy level is confined within a giveninterval, as demonstrated in the following proposition.

Proposition3: Under the LBAPC algorithm, the batteryenergy levelBt

j is confined within[

0, θj + EmaxHj

]

, j = 1, 2.Proof: See Appendix C.

9

Proposition 3 shows that the energy causality constraint (3)will not be violated, which indicates that the LBAPC algorithmis feasible forP2. It also implies that the required batterycapacity in the proposed algorithm isθj + Emax

Hj, j = 1, 2.

In other words, given the size of the available energy storageat the BSs, i.e.,CBj

, j = 1, 2, we can determine the controlparameterV = minV1, V24 as

Vj = (KϕD)−1

[

(

CBj− Emax

Hj− pmax

Bjτ)

ǫHjτ

− 1K 6= 1EmaxH3−j

pmaxB3−j

τ

]

, j = 1, 2.

(29)

Furthermore, the bounds of the battery energy levels areuseful for deriving the main result on the performance of theproposed algorithm.

Next we will proceed to show the asymptotic optimality ofthe LBAPC algorithm, for which we first define the Lyapunovfunction as

L(

Bt)

=1

2

2∑

j=1

(

Btj − θj

)2=

1

2

[

(

Bt1

)2

+(

Bt2

)2]

. (30)

It is a sum of squares of the virtual queue lengths, i.e., thedistance between the battery energy levels and the perturbedenergy levels. Accordingly, the Lyapunov drift function canbe written as

∆(

Bt)

= E

[

L(

Bt+1)

− L(

Bt)

|Bt]

. (31)

Moreover, the Lyapunov drift-plus-penalty function can beexpressed as

∆V

(

Bt)

= ∆(

Bt)

+V E

[

∑

k∈K

(

ϕGptG,kτ + ϕDItD,k

)

|Bt

]

.

(32)In the following lemma, we derive an upper bound for∆V

(

Bt)

, which will play a critical role throughout theanalysis of the LBAPC algorithm.

Lemma3: For arbitrary feasible decision variableset, It,pt for P2, ∆V

(

Bt)

is upper bounded by

∆V

(

Bt)

≤ E

[

2∑

j=1

Btj

(

etj −∑

k∈K

ptHj ,kτ

)

+ V∑

k∈K

(

ϕGptG,kτ + ϕDItD,k

) ∣

∣Bt

]

+ C,

(33)

whereC = 12

∑2j=1

(

EmaxHj

)2

+ 12

∑2j=1

(

pmaxBj

τ)2

.

Proof: By subtractingθj on both sides of (4),

Bt+1j = Bt

j + etj −∑

k∈K

ptHj ,kτ, j = 1, 2. (34)

4Note that to guaranteeVj > 0, j = 1, 2, the values ofCBj, j = 1, 2 can

not be arbitrarily small.

Squaring both sides of (34) and adding up the equalities withj = 1, 2, we have

2∑

j=1

(

Bt+1j

)2

=

2∑

j=1

(

Btj + etj −

∑

k∈K

ptHj ,kτ

)2

=2∑

j=1

[

(

Btj

)2

+

(

etj −∑

k∈K

ptHj ,kτ

)2

+ 2Btj

(

etj −∑

k∈K

ptHj ,kτ

)]

≤2∑

j=1

[

(

Btj

)2

+(

EmaxHj

)2

+(

pmaxBj

τ)2

+ 2Btj

(

etj −∑

k∈K

ptHj ,kτ

)]

.

(35)

Dividing both sides of (35) by 2, addingV∑

k∈K

(

ϕGptG,kτ + ϕDItD,k

)

, as well as taking the

expectation conditioned onBt, we can obtain the desiredresult.

Notice that the upper bound of∆V

(

Bt)

derived in Lemma3 coincides with the objective function of the per-time slotproblem5 in the LBAPC algorithm. To facilitate the perfor-mance analysis, we define the following auxiliary problemP3:

P3 : minIt,pt,et

limT→+∞

1

T

T−1∑

t=0

E

[

K∑

k=1

ϕGptG,kτ + ϕDItD,k

]

s.t. (1), (2), (5), (7)− (12),

limT→+∞

1

T

T−1∑

t=0

E

[

∑

k∈K

ptHj ,kτ − etj

]

= 0, j = 1, 2.

(36)

In P3, the energy causality constraint (3) inP2 is replacedby (36), i.e., the average harvested energy consumption equalsthe average harvested energy. In the following lemma, we willshow thatP3 is a relaxation ofP2.

Lemma4: P3 is a relaxation ofP2, i.e., NSC∗P3 ≤

NSC∗P2, whereNSC∗

P3 is the optimal value ofP3.Proof: For any feasible solution ofP2, based on the

battery dynamics, we have

Bt+1j = Bt

j −∑

k∈K

ptHj ,kτ + etj, j = 1, 2, t = 0, 1, · · · , T − 1.

(37)Summing up both sides of the aboveT equalities, taking theexpectation, dividing both sides byT and lettingT go toinfinity, we have

limT→+∞

1

TE[

BTj

]

=

limT→+∞

1

TE[

B0j

]

− limT→+∞

1

T

T−1∑

t=0

E

[

∑

k∈K

ptHj ,kτ − etj

]

.

(38)

5We drop the constantC in the objective function of the per-time slotproblem.

10

SinceBtj < +∞, we have lim

T→+∞

E[BTj ]

T= 0, i.e., (36) is

satisfied. Hence, any feasible solution ofP2 is also feasibleto P3, which ends the proof.

Besides, we find that there exists a stationary and ran-domized policy [34], where the decisions are i.i.d. amongdifferent time slots, that behaves arbitrarily close to theoptimalsolution of P3. Meanwhile, the difference betweenE

[

etj]

and E

[

∑

k∈K ptHj ,kτ]

is arbitrarily small. It can be statedmathematically in the following lemma, which will help showthe asymptotic optimality of the LBAPC algorithm.

Lemma5: For an arbitraryδ > 0, there exists a stationaryand randomized policyΠ for P3, which decidesetΠ, ItΠ andptΠ, such that (1), (2), (5), (7)-(12) are met, and the followinginequalities are satisfied:

E

[

∑

k∈K

(

ϕGptΠG,kτ + ϕDItΠD,k

)

]

≤ NSC∗P3 + δ, t ∈ T , (39)

∣

∣

∣

∣

∣

E

[

∑

k∈K

ptΠHj ,kτ − etΠj

]∣

∣

∣

∣

∣

≤ δ, j = 1, 2, t ∈ T , (40)

where is a scaling constant.Proof: The proof can be obtained by Theorem 4.5 in [36],

which is omitted for brevity.In Section IV, we bounded the optimal performance of

the modified NSCM problemP2 with that of the originalNSCM problemP1, while in Lemma 4, we showed theauxiliary problemP3 is a relaxation ofP2. With the assistanceof these results, next, we will provide the main result inthe section, which shows the worst-case performance of theLBAPC algorithm.

Theorem1: The network service cost achieved by theLBAPC algorithm, denoted asNSCLBAPC, is upper boundedby

NSCLBAPC ≤ NSC∗P1 + ν (ǫH1

, ǫH2) +

C

V. (41)

Proof: See Appendix D.Remark2: Theorem 1 implies that the performance of the

LBAPC algorithm is controlled by the triplet〈ǫH1, ǫH2

, V 〉.By letting V → +∞, ǫHj

→ 0, j = 1, 2, the cost upperbound can be made arbitrarily tight, that is, the proposedalgorithm asymptotically achieves the optimal performanceof the original design problemP1. Note that in each timeslot, the computational complexity of the LBAPC algorithmcomes from the IOO algorithm, which isO

(

K · 2K)

in theworst case regardless of the choice of〈ǫH1

, ǫH2, V 〉. However,

approaching the optimal performance ofP1 comes at theexpense of a higher battery capacity requirement and longerconvergence time to the optimal performance. The reason isthat, under the LBAPC algorithm, the battery energy levelswill be stabilized atθj . As ǫHj

decreases orV increases,θj increases accordingly, i.e., the perturbed energy levels arehigher, and it will take longer time to accumulate the harvestedenergy, which postpones the arrival of the network stabilityand therefore delays the convergence. Thus, by tuning thecontrol parameters, we can achieve different tradeoffs between

the system performance and the battery capacity/convergencetime.

In this paper, the studied HES network consists of two typ-ical types of renewable energy-powered BSs, and the LBAPCalgorithm provides an effective methodology for designingsuch a network. It is worthwhile noting that the proposedLyapunov optimization approach can be generalized to HESnetworks with multiple BSs. However, as the network size in-creases, the computational complexity of obtaining the optimalsolution of the per-time slot problem increases accordingly.Hence, low-complexity algorithms with performance guaran-tees for the per-time slot problem deserve further investigation.

VI. SIMULATION RESULTS

In this section, we will verify the theoretical results derivedin Section V and evaluate the performance of the proposedLBAPC algorithm through simulations. We consider an HESwireless network with 4 MUs unless otherwise specified. Insimulations,Et

j is uniformly distributed between0 andEmaxHj

with the average EH power given byPHj= Emax

Hj(2τ)−1, the

channel gains are exponentially distributed with meang0d−4,

whereg0 = −40 dB is the path-loss constant andd = 50 mis the distance from the MUs to the BSs. In addition,τ = 1ms, wG = 5, R = 2 Kbits6, w = 1 MHz, σ = 10−13 W,pmaxB1

= pmaxB2

= 1 W andNB2= K = 4. The costs of the

grid energy and packet drop are normalized, i.e.,ϕG = 1per Joule andϕD = 1 per packet drop. For comparison, weintroduce the Cost-aware Greedy algorithm as a performancebenchmark, which gives higher priority to using the harvestedenergy and optimizes the system cost at the current time slot.It works as follows:

• First, MUs will be assigned to the EH-BS one by one untilthe harvested energy is used up. The MUs with highervalues ofht

B1,kwill have a higher priority to be served.

• Second, the remaining MUs will be assigned to the HES-BS until the harvested energy is used up. The MUs withhigher values ofht

B2,kwill have a higher priority to be

served.• The remaining MUs will be assigned to the HES-BS one

by one if the increment of grid energy cost is less thanthe decrement of the packet drop cost. Similarly, the MUswith higherht

B2,kwill be assigned first.

A. Theoretical Results Verification

In this subsection, we will verify the feasibility and asymp-totic optimality of the LBAPC algorithm developed in Propo-sition 3 and Theorem 1, respectively. The values ofθ1 andθ2 are chosen as the values of the right hand side of (13)and (14), respectively. To verify the feasibility, we show thebattery energy levels in Fig. 2. First, we observe that theharvested energy keeps accumulating at the beginning, andfinally stabilizes at the perturbed energy levels. The reasonis that in the proposed algorithm the Lyapunov drift-plus-penalty function is minimized at each time slot. From the

6If a packet is transmitted, the data rate of the link between its destineduser and the assigned BS isR/τ = 2 Mbps in the time slot right after thepacket’s arrival.

11

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

20

40

60

80

Time (minutes)

Batt

ery

Ener

gy

at

the

EH

-BS

(mJ) (a)

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

20

40

60

80

Time (minutes)Batt

ery

Ener

gy

at

the

HE

S-B

S(m

J) (b)

ǫH2= 40 mW

V = 6 × 10−6 J2· cost−1

V = 1 × 10−4 J2· cost−1

ǫH1= 80 mWV = 6 × 10−6 J2

· cost−1

V = 1 × 10−4 J2· cost−1

ǫH2= 80 mW

ǫH1= 40 mW

Fig. 2. Battery energy levels,PH1= PH2

= 30 mW andwD = 0.01.

curves, with a larger value ofV or a smaller value ofǫHj,

the stabilized energy levels become higher, which coincideswith the definition of the perturbation parameters in (13) and(14). Also, we see that the energy levels are confined within[

0, θj + EmaxHj

]

, which verifies Proposition 3 and confirmsthat the energy causality constraint is not violated under theproposed algorithm. The evolution of network service costwith respect to time is shown in Fig. 3. We see that, alarger value ofV or a smaller value ofǫHj

leads to betterlong-term average performance. Nevertheless, the algorithmconverges more slowly to the stable performance. Besides, if〈ǫH1

, ǫH2, V 〉 are tuned properly, the proposed algorithm will

greatly outperform the Cost-aware Greedy policy.

0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 54

5

6

7

8

9

10

11

12

13

14x 10

−4

Time (minutes)

Net

wor

kSer

vic

eC

ost

Cost-aware Greedy

NSC-based BAPC

V = 6 × 10−6 J2· cost−1, ǫH1

= ǫH2= 80 mW

V = 6 × 10−6 J2· cost−1, ǫH1

= ǫH2= 40 mW

V = 1 × 10−4 J2· cost−1, ǫH1

= ǫH2= 80 mW

V = 1 × 10−4 J2· cost−1, ǫH1

= ǫH2= 40 mW

Fig. 3. Network service cost vs. time,PH1= PH2

= 30 mW andwD =

0.01.

The relationship between the network service cost/requiredbattery capacity andV is shown in Fig. 4. From Fig. 4(a), we see that the network service cost achieved by theproposed algorithm decreases inversely proportional toV , andeventually it converges to the optimal value ofP2, whichverifies Theorem 1, i.e., the asymptotic optimality. However,as shown in Fig. 4 (b), the required battery capacity grows

0 0.2 0.4 0.6 0.8 1

x 10−4

5

6

7

8

9

10

11x 10

−4

V (J2· cost−1)

Net

wor

kSer

vic

eC

ost

(a)

0 0.2 0.4 0.6 0.8 1

x 10−4

0

20

40

60

80

100

120

V (J2· cost−1)

Req

uired

Bat

tery

Siz

e(m

J)

(b)

Cost-aware Greedy

LBAPC

EH-BS

HES-BS

Fig. 4. Network service cost and required battery capacity vs. V , PH1=

PH2= 30 mW andwD = 0.01.

linearly with V , which is because the value ofθj is linearlyincreasing withV . Thus, V should be chosen to balancethe achievable performance, convergence time and requiredbattery capacity. For instance, if batteries with 100 mJ capacityare available, we can chooseV = 1.0×10−4 J2 ·cost−1 for theLBAPC algorithm, and then47% performance gain comparedto the benchmark will be obtained.

B. Performance Evaluation

We will show the effectiveness of the proposed algorithmand demonstrate the impacts of various system parametersin this subsection. The relationship between the networkservice cost and the harvesting power at the EH-BS, i.e.,PH1

, is shown in Fig. 5. We see that the network servicecost achieved by either policy is non-increasing withPH1

,which is in accordance with our intuition since consuming theharvested energy incurs no cost. Also, the LBAPC algorithmsignificantly reduces the network service cost compared tothe greedy algorithm. Besides, the influence of the numberof channels at the EH-BS is also revealed. WithNB1

= 1,i.e., in the spectrum limited scenario, increasingPH1

bringsnegligible benefit to the performance in the benchmark policy,since the MU with the best channel condition to the EH-BSis served and this consumes little harvested energy, i.e.,< 15mW. On the other hand, in the proposed algorithm, the networkservice cost keeps decreasing withPH1

because it minimizesthe Lyapunov drift-plus-penalty function at each time slotandthe MU being served by the EH-BS is not necessarily theone with the highest channel gain. By increasingNB1

from 1to 2, the network service cost for both algorithms is greatlyreduced, but the Cost-aware Greedy algorithm experiencesperformance saturation whenPH1

≥ 40 mW. This showsthat the proposed algorithm can not only reduce the systemcost, but also better utilize the spectrum resource. Meanwhile,increasingNB1

from 2 to 4 has noticeable impact on thegreedy algorithm while the benefit to the LBAPC algorithm isminor, which is because the EH-BS becomes energy limitedin this region.

12

15 20 25 30 35 40 45 50 550

0.25

0.5

0.75

1

1.25

1.5x 10

−3

PH1(mW)

Net

work

Ser

vic

eC

ost

Cost-aware Greedy (NB1= 1)

Cost-aware Greedy (NB1= 2)

Cost-aware Greedy (NB1= 4)

LBAPC (NB1= 1)

LBAPC (NB1= 2)

LBAPC (NB1= 4)

Fig. 5. Network service cost vs.PH1, PH2

= 30 mW, ǫH1= ǫH2

= 40

mW, CB1= CB2

= 150 mJ.

The network service cost versus the harvesting power atthe HES-BS, i.e.,PH2

, is shown in Fig. 6. Similarly, thecost performance decreases asPH2

increases. Nevertheless,there is no performance saturation at highPH2

since withNB2

= K, the MUs can always be served by the HES-BS. From Fig. 6, we also see the linear relationship betweenthe network service cost andPH2

, which is because theharvesting energy and grid energy are co-located at the HES-BS. Additionally, by comparing Fig. 5 and Fig. 6, we observethat the performance improvement by increasingPH1

is moreobvious than increasingPH2

, which is due to the diversitygain obtained from independent channels from the two BSs.

15 20 25 30 35 40 45 50 552

4

6

8

10

12

14

16x 10

−4

PH2(mW)

Net

work

Ser

vic

eC

ost

Cost-aware Greedy (NB1= 1)

Cost-aware Greedy (NB1= 2)

Cost-aware Greedy (NB1= 4)

LBAPC (NB1= 1)

LBAPC (NB1= 2)

LBAPC (NB1= 4)

Fig. 6. Network service cost vs.PH2, PH1

= 30 mW, ǫH1= ǫH2

= 40

mW, CB1= CB2

= 150 mJ.

The grid energy consumption and packet drop ratio achievedby different algorithms are shown in Fig. 7 (a) and Fig. 7 (b),respectively. AswD increases, the grid energy consumptionincreases, meanwhile, the packet drop ratio decreases. Thus,by adjusting the weights of the grid energy cost and packetdrop cost, different tradeoffs can be achieved. WhenwD

is sufficiently large, the packet drop ratio achieved by theproposed scheme approaches zero while that achieved by the

benchmark remains upon 1.0%, i.e., the LBAPC algorithm hasthe potential to meet a higher QoS requirement. From Fig. 7,the proposed algorithm not only outperforms the Cost-awareGreedy algorithm in terms of the network service cost, butit is also more competent to suppress both the grid energyconsumption and the packet drops. This indicates that, in HESwireless networks, the optimal energy management shouldbalance the current and future performance, as well as fullyutilize the available SI, and the LBAPC algorithm is such apromising solution.

0 0.02 0.04 0.06 0.0840

60

80

100

120

140

160

wD

Ave

rage

Grid

Pow

erC

onsu

mption

(mW

)

(a)

0 0.02 0.04 0.06 0.080

0.5

1.0

1.5

2.0

2.5

wD

Pac

ket

Dro

pR

atio

(%)

(b)

Cost-aware Greedy

LBAPC

Cost-aware Greedy

LBAPC

Fig. 7. Grid power consumption and packet drop ratio vs.wD, PHj= 30

mW, ǫHj= 40 mW, CBj

= 150 mJ, j = 1, 2. The curves with no markersare with NB1

= 1, while the curves marked with “+” and “” are withNB1

= 2 and4, respectively.

Finally, we show the relationship between the networkservice cost and the number of MUs in Fig. 8. We see that thenetwork service cost achieved by either policy is increasingwith K. This is because the network traffic demand growsasK increases. Due to the insufficient amount of harvestedenergy, the networks with more MUs are prone to consumemore grid energy and drop more packets, both of whichcontribute to the increase of the network service cost. Besides,the effectiveness of the proposed policy is again validated,and the performance gain compared to the benchmark policyexpands asK increases. This highlights the importance ofoptimal energy management and the benefits of the proposedalgorithm for HES cellular networks, especially when therenewable energy resource is scarce, or the traffic load isheavy.

VII. C ONCLUSIONS

In this paper, we proposed new design methodologies forHES green cellular networks based on Lyapunov optimizationtechniques. The network service cost, which is comprised ofthe grid energy cost and the packet drop cost, was adoptedas the performance metric, and a BS assignment and powercontrol (BAPC) policy was then developed to optimize the net-work. A practical online Lyapunov optimization-based BAPC(LBAPC) algorithm was proposed, which requires little priorknowledge and enjoys low computational complexity. Perfor-mance analysis was conducted which revealed the asymp-

13

1 2 3 4 50

0.5

1

1.5x 10

−3

K

Net

work

Ser

vic

eC

ost

Cost-aware Greedy (NB1= 2)

Cost-aware Greedy (NB1= 4)

LBAPC (NB1= 2)

LBAPC (NB1= 4)

Fig. 8. Network service cost vs.K, PH1= PH2

= 30 mW, NB2= K,

ǫH1= ǫH2

= 40 mW, CB1= CB2

= 150 mJ.

totic optimality of the proposed algorithm. Simulation resultsshowed that the proposed LBAPC algorithm significantlyoutperformed the greedy transmission scheme in terms of thenetwork service cost, grid energy consumption, as well as theachievable QoS. Our results demonstrated the effectivenessof Lyapunov optimization techniques to overcome the curseof dimensionality in MDP solutions, which was the primarybarrier in online transmission protocols design for HES greencellular networks. It will be interesting to extend the proposedalgorithm to more general HES networks as well as incorpo-rate more realistic EH models and multi-antenna techniquesinto consideration, and investigate other design problemssuchas interference management and user scheduling.

APPENDIX

A. Proof of Proposition 1

SinceP2 is a tightened version ofP1, we haveNSC∗P1 ≤

NSC∗P2. The proof for the other side of the inequality can be

obtained by constructing a feasible solution forP2 based onthe optimal solution ofP1:

• If∑

k∈K

ptH1,k∈ (0, ǫH1

), then the harvested energy of the

EH-BS will not be used in the constructed solution, andthe MUs assigned to the EH-BS will experience packetdrop.

• If∑

k∈K

ptH2,k∈ (0, ǫH2

), in the constructed solution, the

harvested energy from the HES-BS will be replaced bythe grid energy.

It is straightforward to verify that the constructed solution isfeasible toP2. Thus,

NSC∗P2 ≤ NSC∗

P1 + ǫH2τ · ϕG +

K∑

k=1

pk · kϕD

≤ NSC∗P1 + ǫH2

τ · ϕG + (1− p0)KϕD,

(42)

wherepk is the probability thatk MUs can be served by theEH-BS with total transmit powerǫH1

. Because of the couplingamong the transmit power and the available channels, an exact

expression ofp0 is difficult to obtain. However, we can obtaina lower bound onp0 by ignoring the coupling effects, i.e.,p0 ≥

∏

k∈K

P

[

r(

htB1,k

, ǫH1

)

≤ R]

= FKB1

(η). By substituting

the lower bound ofp0 into (42), the result is obtained.

B. Proof of Lemma 2

WhenρΣ ∈[

0,−Bt

2ǫH2

V ϕG

]

, suppose there is a solution with

k ∈ k ∈ Hc|ItB2,k= 1, ptH2,k

> 0. Due to (12), with thesolution in (25) instead, the value of the objective function willdecrease by−Bt

2max∑

k∈Hc

ptH2,kItB2,k

, ǫH2

τ−V ϕGρΣτ ≥

0, i.e.,ptH2,k= 0, ∀k ∈ K is optimal. Also, asV ϕG > 0, it is

optimal to transmit with the channel inversion power.

When ρΣ ∈(

−Bt2ǫH2

V ϕG, ǫH2

]

, since −Bt2 > 0, either

∑

k∈Hc

ptH2,k= 0 or ǫH2

is optimal. When∑

k∈Hc

ptH2,k= 0,

we haveptG,k = ρtB2,kItB2,k

. With the solution in (25) in-stead, the value of the objective function will decrease byV ϕGρΣτ −

(

−Bt2

)

ǫH2τ > 0, i.e.,

∑

k∈Hc

ptH2,k= ǫH2

is

optimal.When ρΣ ∈

(

ǫH2, pmax

B2

]

, by contradiction, the optimalsolution should satisfy

∑

k∈Hc ptG,kItB2,k

= λρΣ∑

k∈Hc ptH2,kItB2,k

= max(1− λ) ρΣ, ǫH2, (43)

whereλ ∈ [0, 1). Suppose there is a solution withk ∈ k ∈Hc|ItB2,k

= 1, ptG,k > 0, i.e., λ > 0. By using (25) instead,i.e., λ = 0, the value of the objective function will decreaseby

V ϕGλρΣτ − Bt2 max(1− λ) ρΣ, ǫH2

τ −(

−Bt2

)

ρΣτ

≥ V ϕGλρΣτ − Bt2 (1− λ) ρΣτ −

(

−Bt2

)

ρΣτ

=(

V ϕG −(

−Bt2

))

ρΣλτ ≥ 0,

i.e., λ = 0 is optimal. Similarly, as−Bt2 > 0, it is optimal

to transmit with the channel inversion power. To summarize,(25) is optimal and thus Lemma 2 is obtained.

C. Proof of Proposition 3

We shall first prove thatBtj is upper bounded byθj +

EmaxHj

, j = 1, 2, based on the optimal harvested energy in (17).Supposeθj ≤ Bt

j ≤ θj + EmaxHj

, and sinceet∗j = 0, we haveBt+1

j ≤ Btj ≤ θj + Emax

Hj. Otherwise, if0 ≤ Bt

j < θj , sinceet∗j = Et

j , we haveBt+1j ≤ Bt

j + et∗j < θj + EmaxHj

.Next, we reveal an important property of the optimal

solution of the per-time slot problem to assist the proof forthe lower bound. It indicates that if the battery energy level ata BS is below a threshold, no harvested energy in this BS willparticipate in transmission, as shown in the following lemma.

Lemma6: If Btj < pmax

Bjτ , then pt∗Hj ,k

= 0, j = 1, 2, k ∈K, wherept∗Hj ,k

is the optimal battery output power obtainedin the per-time slot problem.

14

Proof: We start with the single user case and omit theuser index. Suppose whenBt

j < pmaxBj

τ , there is an optimalsolution whereptHj

> 0 (ItBj= 1, ItB3−j

= ItD = 0). SinceptHj

∈ Ωj , thusptHj≥ ǫHj

. With this solution, the value of theobjective function in the per-time slot problem is no less than−Bt

jǫHjτ > V ϕD, which is greater than what is achieved by

the solution withItD = 1, ptHj= ptH3−j

= ptG = 0, i.e.,pt∗Hj=

0. For the multi-user scenario, again, suppose whenBtj <

pmaxBj

τ , there is an optimal solution where∑

k∈K ptHj ,k> 0,

i.e.,∑

k∈K ptHj ,k≥ ǫHj

. With this solution, the minimum

value of the objective function is−BtjǫHj

τ − EmaxH3−j

pmaxB3−j

τ ,which is achieved whenBt

3−j = θ3−j + EmaxH3−j

, meanwhileall the MUs are served with the harvested energy, and at leastone of them is served by BSB3−j. With the definition ofθj ,we are able to show−Bt

jǫHjτ − Emax

H3−jpmaxB3−j

τ > KV ϕD,which is achieved by the solution withItD,k = 1, ∀k ∈ K, and∑

k∈K ptHj ,k= 0, i.e., pt∗Hj ,k

= 0, ∀k ∈ K.Based on the property ofpt∗Hj ,k

derived in Lemma 6, nowwe proceed to showBt

j is lower bounded by zero. Suppose0 ≤ Bt

j < pmaxBj

τ , according to Lemma 6,pt∗Hj ,k= 0, ∀k ∈ K,

thusBt+1j ≥ Bt

j ≥ 0. Otherwise, ifBtj > pmax

Bjτ , with (9),

∑

k∈K

pt∗Hj ,k≤ pmax

Bj, thusBt+1

j ≥ Btj −

∑

k∈K

pt∗Hj ,kτ ≥ 0. As a

result,Btj ∈

[

0, θj + EmaxHj

]

.

D. Proof of Theorem 1

Since the LBAPC algorithm obtains the optimal solution ofthe per-time slot problem,

∆V

(

Bt)

≤ E

[

2∑

j=1

Btj

(

et∗j −∑

k∈K

pt∗Hj ,kτ

)

+ V∑

k∈K

(

ϕGpt∗G,kτ + ϕDIt∗D,k

) ∣

∣Bt

]

+ C

≤ E

[

2∑

j=1

Btj

(

etΠj −∑

k∈K

ptΠHj ,kτ

)

+ V∑

k∈K

(

ϕGptΠG,kτ + ϕDItΠD,k

)∣

∣Bt

]

+ C

= E

2∑

j=1

Btj

(

etΠj −∑

k∈K

ptΠHj ,kτ

)

∣

∣Bt

+ V E

[

∑

k∈K

(

ϕGptΠG,kτ + ϕDItΠD,k

)

]

+ C

(†)

≤ δ

2∑

j=1

maxθj , EmaxHj

+ V (NSC∗P3 + δ) + C,

(44)

where (†) is due to Lemma 5. By lettingδ go to zero, weobtain

∆V

(

Bt)

≤ VNSC∗P3 + C. (45)

Taking the expectation on both sides of (45), summing upthe equations fort = 0, · · ·T − 1, dividing by T and letting

T → +∞, we haveNSCLBAPC ≤ NSC∗P3 +

CV

. By furtherutilizing Proposition 1 and Lemma 4, the theorem is proved.

REFERENCES

[1] A. Fehske, G. Fettweis, J. Malmodin, and G. Biczok, “The globalfootprint of mobile communications: The ecological and economic per-spective,”IEEE Commun. Mag., vol. 49, no. 8, pp. 55-62, Aug. 2011.

[2] S. Lambertet al., “World-wide electricity consumption of communicationnetworks,”Optical Express, vol. 20, no. 26, pp. B513-524, Mar. 2012.

[3] I. Hwang, B. Song, and S. Soliman, “A historic view on hyper-denseheterogeneous and small cell networks,”IEEE Commun. Mag., vol. 51,no. 6, pp. 20-27, Jun. 2013.

[4] C. Li, J. Zhang, and K. B. Letaief, “Throughput and energyefficiencyanalysis of small cell networks with multi-antenna base stations,” IEEETrans. Wireless Commun., vol. 13, no. 5, pp. 2502-2517, May 2014.

[5] C. Li, J. Zhang, and K. B. Letaief, “Energy efficiency analysis of smallcell networks,” in Proc. IEEE Int. Conf. Commun. (ICC), Budapest,Hungary, Jun. 2013, pp. 4404-4408.

[6] S. Ulukus, A. Yener, E. Erkip, O. Simeone, M. Zorzi, and K.Huang, “En-ergy harvesting wireless communications: A review of recent advances,”IEEE J. Sel. Areas Commun., vol. 33, no. 3, pp. 360-381, Mar. 2015.

[7] Y. Mao, Y. Luo, J. Zhang, and K. B. Letaief, “Energy harvesting smallcell networks: Feasibility, deployment and operation,”IEEE Commun.Mag., vol. 53, no. 6, pp. 94-101, Jun. 2015.

[8] G. Piro et al., “Hetnets powered by renewable energy sources: Substain-able next generation cellular networks,”IEEE Internet Comput., vol. 17,no. 1, pp. 32-39, Jan. 2013.

[9] O. Ozel and S. Ulukus, “Achieving AWGN capactity under stochasticenergy harvesting,”IEEE Trans. Inf. Theory, vol. 58, no. 10, pp. 6471-6483, Oct. 2012.

[10] O. Ozel, K. Tutuncuoglu, J. Yang, S. Ulukus and A. Yener,“Transmis-sion with energy harvesting nodes in fading wireless channels: Optimalpolicies,” IEEE J. Sel. Areas Commun., vol. 29, no. 8, pp. 1732-1743,Sep. 2011.

[11] J. Yang, O. Ozel and S. Ulukus, “Broadcasting with an energy harvestingrechargeable transmitter,”IEEE Trans. Wireless Commun., vol. 11, no. 2,pp. 571-583, Feb. 2012.

[12] J. Yang and S. Ulukus, “Optimal packet scheduling in a multiple accesswith energy harvesting transmitter,”J. Commun. & Networks, vol. 14, no.2, pp. 140-150, Apr. 2012.

[13] C. Huang, R. Zhang, and S. Cui, “Throughput maximization for theGaussian relay channel with energy harvesting constraints,” IEEE J. Sel.Areas Commun., vol. 31, no. 8, pp. 1469-1479, Aug. 2013.

[14] Y. Luo, J. Zhang, and K. B. Letaief, “Optimal schedulingand powerallocation for two-hop energy harvesting communication systems,”IEEETrans. Wireless Commun., vol. 11, no. 9, pp. 4729-4741, Sep. 2013.

[15] L. Huang and M. J. Neely, “Utility optimal scheduling inenergy-harvesting networks,”IEEE/ACM Trans. Netw., vol. 21, no. 4, pp. 1117-1130, Aug. 2013.

[16] P. Blasco, D. Gunduz, and M. Dohler, “A learning theoretic approachto energy harvesting communication system optimization,”IEEE Trans.Wireless Commun., vol. 12, no. 4, pp. 1872-1882, Apr. 2013.

[17] Z. Wang, V. Aggarwal, and X. Wang, “Power allocation forenergyharvesting transmitter with causal information,”IEEE Trans. Commun.,vol. 62, no. 11, pp. 4080-4093, Nov. 2014.

[18] J. Gong, S. Zhou, and Z. Niu, “Optimal power allocation for energyharvesting and power grid coexisting wireless communication systems,”IEEE Trans. Commun., vol. 61, no. 7, pp. 3040-3049, Jul. 2013.

[19] P. He, L. Zhao, S. Zhou, and Z. Niu, “Recursive geometricwater-fillingfor wireless links with hybrid energy systems,”Int. Conf. on WirelessCommun. and Signal Process. (WCSP), Hangzhou, China, Oct. 2013, pp.1-6.

[20] X. Kang, Y. -K. Chia, C. -K. Ho, and S. Sun, “Cost minimization forfading channels with energy harvesting and conventional energy,” IEEETrans. Wireless Commun., vol. 13, no. 8, pp. 4586-4598, Aug. 2014.

[21] I. Ahmed, A. Ikhlef, D. W. K. Ng, and R. Schober, “Power allocationfor an energy harvesting transmitter with hybrid energy sources,” IEEETrans. Wireless Commun., vol.12, no.12, pp. 6255-6267, Dec. 2013.

[22] D. W. K. Ng, E. S. Lo, and R. Schober, “Energy-efficient resource allo-cation in OFDMA systems with hybrid energy harvesting base station,”IEEE Trans. Wireless Commun., vol. 12, no. 7, pp. 3412-3427, Jul. 2013.

[23] T. Han and N. Ansari, “On optimizing green energy utilization forcellular networks with hybrid energy supplies,”IEEE Trans. WirelessCommun., vol. 12, no. 8, pp. 3872-3882, Aug. 2013.

15

[24] T. Han and N. Ansari, “Green-energy aware and latency aware userassociation in heterogeneous cellular networks,” inProc. IEEE Globecom,Atlanta, GA, USA, Dec. 2013, pp. 4946-4951.

[25] J. Gong, J. S. Thompson, S. Zhou, and Z. Niu, “Base station sleepingand resource allocation in renewable energy powered cellular networks,”IEEE Trans. Commun., vol. 62, no. 11, pp. 3801-3813, Nov. 2014.

[26] J. Xu and R. Zhang, “CoMP meets smart grid: a new communicationand energy cooperation paradigm,”IEEE Trans. on Veh. Technol., vol.64, no. 6, pp. 2476-2488, Jun. 2015.

[27] S. Meyn and R. Tweedie,Markov Chains and Stochastic Stability, 2nded. Cambridge University Press, 2009.

[28] Y. Cui, V. K. N. Lau, R. Wang, H. Huang and S. Zhang, “A surveyon delay-aware resource control for wireless systems–Large derivationtheory, stochastic Lyapunov drift and distributed stochastic learning,”IEEE Trans. Info. Theory, vol. 58, no. 3, pp. 1677-1701, Mar. 2012.

[29] S. Lakshminarayana, T. Q. S. Quek and H. V. Poor, “Cooperation andstorage tradeoffs in power grids with renewable energy resources,” IEEEJ. Sel. Areas Commun., vol. 32, no. 7, pp. 1386-1397, Jul. 2014.

[30] A. Ghosh, J. Zhang, J. G. Andrews, and R. Muhamed,Fundamentals ofLTE. Prentice-Hall, 2010.

[31] V. Chandrasekhar and J. G. Andrews, “Spectrum allocation in tieredcellular networks,”IEEE Trans. Commun., vol. 57, no. 10, pp. 3059-3068, Oct. 2009.

[32] W. Cheung, T. Q. S. Quek, and M. Kountouris, “Throughputopti-mization, spectrum allocation, and access control in two-tier femtocellnetworks,” IEEE J. Select. Areas Commun., vol. 30, no. 3, pp. 561-574,Apr. 2012.

[33] Y. Mao, J. Zhang, and K. B. Letaief, “Joint base station assignmentand power control for hybrid energy supply wireless networks,” in Proc.IEEE Wireless Commun. and Networking Conf. (WCNC), New Orleans,LA, Mar. 2015, pp. 1177-1182.

[34] D. P. Bertsekas,Dynamic Programming and Optimal Control, 3rd ed.Belmonth, MA, USA: Athens Scientific, 2005.

[35] M. J. Neely and L. Huang, “Dynamic product assembly and inventorycontrol for maximum profit,”IEEE Conference on Decision and Control(CDC), Atlanta, GA, Dec. 2010, pp. 2805-2812.

[36] M. J. Neely, Stochastic Network Optimization with Application toCommunication and Queueing Systems. San Rafael, CA, USA: Morgan& Calypool., 2010.