1

A Novel Data-Driven Situation AwarenessApproach for Future Grids—Using Large Random

Matrices for Big Data ModelingXing He, Lei Chu, Robert C. Qiu, Fellow, IEEE, Qian Ai, Senior Member, IEEE, Zenan Ling

Abstract—Data-driven approaches, when tasked with situationawareness, are suitable for complex grids with massive datasets. Itis a challenge, however, to efficiently turn these massive datasetsinto useful big data analytics. To address such a challenge, thispaper, based on random matrix theory (RMT), proposes a data-driven approach. The approach models massive datasets as largerandom matrices; it is model-free and requiring no knowledgeabout physical model parameters. In particular, the large datadimension N and the large time span T , from the spatial aspectand the temporal aspect respectively, lead to favorable results.The beautiful thing lies in that these linear eigenvalue statistics(LESs) built from data matrices follow Gaussian distributionsfor very general conditions, due to the latest breakthroughsin probability on the central limit theorems of those LESs.Numerous case studies, with both simulated data and field data,are given to validate the proposed new algorithms.

Index Terms—Big data analytics, situation awareness, randommatrix theory, linear eigenvalue statistics, statistical indicator

I. INTRODUCTION

S ITUATION awareness (SA) is of great significance forpower system operation, and a reconsideration of SA is

essential for future grids [1]. These future grids are alwayshuge in size and complex in topology. Operating under a novelregulation, their management mode is much different [2].Data are more and more easily accessible, on the other hand,and data-driven approaches become natural for future grids.Towards this vision, we are facing the following challenges:• There are massive data in power grids. The so-called

curse of dimensionality [3] occurs inevitably.• The resource cost (time, hardware, human, etc.) for

extracting big data analytics should be tolerable.• For a massive data source, there often exist realistic

“bad” data, e.g. the incomplete, the inaccurate, the asyn-chronous, and the unavailable. For system operations,decisions such as relay actions, should be highly reliable.

This paper is built upon our previous work in the lastseveral years. See Section I-B for details. Motivated for datamining, our line of research is based on the high-dimensionalstatistics. By high-dimensionality, we mean that the datasetsare represented in terms of large random matrices. These data

This work was partly supported by National Natural Science Foundationof China (No. 61571296). c© 20xx IEEE. Personal use of this material ispermitted. Permission from IEEE must be obtained for all other uses, in anycurrent or future media, including reprinting/republishing this material foradvertising or promotional purposes, creating new collective works, for resaleor redistribution to servers or lists, or reuse of any copyrighted component ofthis work in other works.

matrices can be viewed as data points in high-dimensionalvector space—each vector is very long.

Data-driven approaches and data utilization for smart gridsare current stressing topics, as evidenced in the special issueof “Big Data Analytics for Grid Modernization” [1]. Thisspecial issue is most relevant to our paper in spirit. SeveralSA topics are discussed. We highlight the anomaly detectionand classification [4, 5], the estimation of active ingredientssuch as PV installations [6, 7], and finally the online transientstability evaluation using real-time data [8].

In addition, we point out research about the improvementin wide-area monitoring, protection and control (WAMPAC)and the utilization of PMU data [9–11], together with the faultdetection and location [12, 13]. Xie et al. based on principalcomponent analysis (PCA), propose an online application forearly event detection by introducing a reduced dimensionality[14]. The work has a special connection to our paper. Lim etal. study the quasi-steady-state operational problem relevant tothe voltage instability phenomenon based on SVD using PMUdata [15].

A. Contributions of Our Paper

Randomness is critical to future grids since rapid fluctu-ations in voltages and currents are ubiquitous. Often, thesefluctuations exhibit Gaussian statistical properties [15]. Ourcentral interest in this paper is to model these rapid fluctuationsusing the framework of random matrix theory (RMT). Our newalgorithms are made possible due to the latest breakthroughsin probability on the central limit theorems of the lineareigenvalue statistics (LESs) [16, Chapter 7]. See [17] for arecent review.

1) Starting from fundamental formulas of power systems,a theoretical justification is given for the validity ofmodeling complex grids as large random matrices. Thisdata modeling framework ties together the RMT and thepower system analysis. This part is basic in nature.

2) We study numerous basic problems including the tech-nical route and applied framework, data-processing andrelevant procedures, evaluation system and indicator sets,and the advantages over classical methodologies.

3) We make a comparison between RMT-based approachand PCA-based one.

4) On the basis of big data analytics, we study some powersystem applications: anomaly detection and location, em-pirical spectral density test, sensitivity analysis, statistical

arX

iv:1

610.

0507

6v2

[st

at.M

E]

16

Jan

2018

2

indicator system and its visualization, and, finally, robust-ness against asynchronous data.

B. Relationship to Our Previous Work

Our work [2] is the first attempt to introduce the math-ematical tool of RMT into power systems. Later, numerouspapers demonstrate the power of this tool. Ring Law andMarchenko-Pastur (M-P) Law are regarded as the statisticalfoundation, and Mean Spectral Radius (MSR) is proposedas the high-dimensional indicator. Then we move forward tothe second stage—paper [18] studies the correlation analysisunder the above framework. The concatenated matrix Ai isthe object of interest. It consists of the basic matrix B anda factor matrix Ci, i.e., Ai = [B; Ci]. In order to seek thesensitive factors, we compute the advanced indicators thatare based on the LESs of these concatenated matrices Ai.This study contributes to fault detection and location, line-loss reduction, and power-stealing prevention. We also conductanalysis for power transmission equipment based on the sametheoretical foundation [19]. Paper [20] is the third step inwhich the LES set is studied. Based on the LES set, a statisticaland data-driven indicator system, rather than its deterministicand model-based counterpart, is built to describe the systemfrom a high-dimensional perspective. The robustness againstspatial data error, precisely, data losses in the core area, isemphasized.

C. Advantages of RMT-based Approach

The data-driven approach conducts analysis requiringno prior knowledge of the system topology, the unitoperation/control mechanism, the causal relationship, etc.Comparing with classical data-driven methodologies such asPCA-based method, the RMT-based counterpart has someunique advantages:1) The massive dataset of power systems are in a high-dimensional vector space; the temporal variations (Tsampling instants) are simultaneously observed togetherwith spatial variations (N grid nodes). The extraction ofinformation from the above temporal-spatial variations isa challenge that does not meet the prerequisites of mostclassical statistical algorithms. Unifying time and spacethrough their ratio c = T/N , RMT deal with such kind ofdata mathematically rigorously.2) The statistical indicator is generated from all the datain the form of matrix entries. This is not true to principalcomponents—we really do not know the rank of thecovariance matrix. The large size of the data enhancesthe robustness of the final decision against the bad data(inaccuracy, losses, or asynchronization), as well as thosechallenges in classical data-driven methods, such as erroraccumulations and spurious correlations [18].3) For the statistical indicator, a theoretical or empirical valuecan be obtained in advance. The statistical indicator suchas LES follows a Gaussian distribution, and its variance isbounded [21] and decays very fast in the order of O(N−2)for a given data dimension N, say N = 118.4) We can flexibly handle heterogenous data to realize data

fusion via matrix operations, such as the blocking [2], thesum [22], the product [22], and the concatenation [18] of thematrices. Data fusion is guided by the latest mathematicalresearch [16, Chapter 7].5) Only eigenvalues are used for further analyses, while theeigenvectors are omitted. This leads to a much faster data-processing speed and less required memory space. Althoughsome information is lost, there is still rich informationcontained in the eigenvalues [23], especially those outliers[24, 25].6) Particularly, for a certain RMM, various forms of LES,in the form of τF =

∑Ni=1 ϕF (λM,i), can be constructed

by designing test functions ϕF (·) without introducing anysystem error. Each LES, similar to a filter, provides a uniqueview-angle. As a result, the system is understood piece bypiece. With a proper LES, we can trace some specific signal.

Section II gives the mathematical background and theoret-ical foundation. Spectrum test is introduced as a novel tool.Section III studies the details about the RMT-based method.Section IV and Section V, using the simulated data and fielddata respectively, study the function designing based on theproposed method. Section VI concludes this paper.

II. MATHEMATICAL BACKGROUND AND THEORETICALFOUNDATION

A. Random Matrix Modeling

Operating in a balance situation, power grids obey∆Pi = Pis − Pi (V,θ)

∆Qi = Qis −Qi (V,θ), (1)

where Pis and Qis are the power injections of node i,and Pi (V,θ) and Qi (V,θ) are the power injections of thenetwork, satisfying

Pi = Vi

n∑j=1

Vj (Gij cos θij +Bij sin θij)

Qi = Vi

n∑j=1

Vj (Gij sin θij −Bij cos θij)

. (2)

Combining (1) and (2), we obtain

w0 = f (x0,y0) , (3)

where w0 is the vector of nodes’ power injections dependingon Pis, Qis, x0 is the system status variables depending onVi, θi, and y0 is the network topology parameters dependingon Bij , Gij .

Then, the system fluctuations, thus randomness in datasets,are formulated as

w0+∆w=f (x0+∆x,y0+∆y) . (4)

With a Taylor expansion, (4) is rewitten as

w0+∆w=f(x0,y0)+f ′x(x0,y0) ∆x+f ′y(x0,y0) ∆y

+1

2f ′′xx(x0,y0)(∆x)

2+

1

2f ′′yy(x0,y0)(∆y)

2

+ f ′′xy(x0,y0) ∆x∆y+· · · .

(5)

3

The value of system status variables x are relatively stable,which means we can ignore the second-order term (∆x)

2 andhigher-order terms. Besides, (2) shows that f ′′yy(x,y)=0. Asa result, (5) is turned into

∆w=f ′x(x0,y0) ∆x+f ′y(x0,y0) ∆y

+f ′′xy(x0,y0) ∆x∆y.(6)

Suppose the network topology is unchanged, i.e., ∆y = 0.From (6), we deduce that

∆x=(f ′x(x0,y0))−1

(∆w)=S0∆w. (7)

On the other hand, suppose the power demands is un-changed, i.e., ∆w=0. From (6), we deduce that

∆x=S0∆wy, (8)

where wy=[I+f ′′xy(x0,y0)∆ys0]−1[f ′y(x0,y0)].

Note that S0 = (f ′x(x0,y0))−1, i.e., the inversion of the

Jacobian matrix J0.Thus, we describe the power system operation using a

random vector. If there exists an unexpected active powerchange or short circuit, the corresponding change of systemstatus variables x0, i.e. Vi, θi, will obey (7) or (8) respectively.

For a practical system without dramatic changes, rich sta-tistical empirical evidence indicates that the Jacobian matrixJ keeps nearly constant, so does s0. Considering T randomvectors observed at time instants i = 1, · · · , T, we build arelationship in the form of ∆Xs = S0∆W with a similarprocedure as (3) to (8), where ∆Xs denotes the variationof state [∆x1, · · · ,∆xT ] , and ∆W denotes the variation ofpower injections or topology parameters accordingly.

Taking the case in [20] as an example, for an equilibrium op-eration system (the topology is unchanged, the reactive poweris almost constant or changes much more slowly than theactive one), the relationship model between voltage magnitudeand active power is just like the Multiple Input Multiple Output(MIMO) model in wireless communication [16, 22]. We writeV = ΞP. Note that most variables of vector V are random dueto the ubiquitous noises, e.g., small random fluctuations in P.Furthermore, with the normalization, we can build the standardrandom matrix model (RMM) in the form of V = ΞR, whereR is a standard Gaussian random matrix.

B. Anomaly Detection Based on Asymptotic Empirical Spec-tral Distribution

Often, these rapid fluctuations exhibit Gaussian statisticalproperties [15], as pointed out above. In practice, Gaus-sian unitary ensemble (GUE) and Laguerre unitary ensemble(LUE) are used in our models:

A =

1

2

(X + XH

),X ∈ XN×N ,GUE;

1

NXXH ,X ∈ XN×T ,LUE.

, (9)

where X is the standard Gaussian random matrix whoseentries are independent identically distributed (i.i.d.) complexGaussian random variables.

Let fA (x) be the empirical density of A, and define itsempirical spectral distribution (ESD) FA (x):

FA (x) =1

N

N∑i=1

Iλi≤x, (10)

where A is GUE or LUE matrix, and I (·) represents the eventindicator function. We investigate the rate of convergence ofthe expected ESD E FA (x) to the Wigner’s Semicircle Lawor Wishart’s M-P Law.

Let gA (x) and GA (x) denote the true eigenvalue densityand the true spectral distribution of A, and the Wigner’sSemicircle Law and Wishart’s M-P Law say:

gA (x) =

1

2π

√4− x2 , x ∈ [−2, 2] ,GUE;

1

2πcx

√(x− a) (b− x) , x ∈ [a, b] ,LUE;

,

(11)where a = (1−

√c)

2, b = (1 +

√c)

2.

GA (x) =

∫ x

−∞gA (u) du. (12)

Then, we denote the Kolmogorov distance betweenE FA (x) and GA (x) as ∆:

∆ = supx|E FA (x) −GA (x)| . (13)

Gotze and Tikhomirov, in their work [26], prove an optimalbound for ∆ of order O

(N−1

).

Lemma II.1. There exists a positive constant C such that, forany N ≥ 1,

∆ ≤ CN−1. (14)

They also prove that the convergence of the density ofstandard Semicircle Law and M-P Law to the expected spectraldensity fA(x) satisfies following lemmas.

Lemma II.2. For GUE matrix, there exists a positive constantε and C such that, for any x ∈

[−2 +N−

13 ε, 2−N− 1

3 ε],

|fA (x)− g (x)| ≤ C

N (4− x2). (15)

Lemma II.3. For LUE matrix, let β = N/T , there existssome positive constant β1 and β2 such that 0 < β1 ≤ β ≤β2 < 1, for all N ≥ 1. Then there exists a positive constantC and ε depending on β1 and β2 and for any N ≥ 1 andx ∈

[a+N−

23 ε, b−N− 2

3 ε],

|fA (x)− h (x)| ≤ C

N (x− a) (b− x). (16)

Lemma II.2 and II.3 also describe how fast the populationdistribution functions converge to the asymptotic empiricalspectral distribution limit. This ESD-based test is interestingfor anomaly detection about a complex grid; the effectivenessis validated in Section IV. We exploit the mathematicalknowledge that the ESD converges to its limit with a optimalconvergence rate of N−1.

4

III. THE METHOD OF SITUATION AWARENESS

A. Technical Route and Practical Procedures

The proposed RMT-based method consists of three proce-dures as illustrated in Fig. 1: 1) big data model—to modelthe system using experimental data for the RMM; 2) big dataanalysis—to conduct big data anlytics for the indicator system;3) engineering interpretation—to visualize and interpret thestatistical results to operators for the decision-making.

ΔX=S0ΔW Other Models: X,Y,…

Voltage Stability (PQVθ) Other SAs

Actual State+ST&ICT Present State+SA Perceived State

Physical Power System

Situation Awareness

Mathematical Random Matrix Models

Analytics Based on Random Matrix Theory

Ring Law LES Visualization

Engi

neer

ing

Inte

rpre

tatio

n fo

r Dec

ision

-mak

ing

ms Level

years

Transient Stability

Grid Planning

Static Stability

Dynamic Stability

...…

…

Fig. 1: RMT-based Method for SA

This method is universal. We have already made numeroussuccessful attempts in the field of anomaly detection and di-agnosis for both the grid network [2, 18] and the transmissionequipment [19]. In addition, Zhang et al., based on RMT,study the steady stability and transient stability in [27] and[28] respectively.

B. Paradigms and Method

We would like to refer to Fig. 2 in book [29] as a clue.We are now entering the age of 4th-paradigm—data-intensivescientific discovery. Besides, the summaries for the classicaldecision-making approaches and for our proposed ones, ob-tained initially in [2], are improved as Fig. 3.

xviii

escience: WhaT is iT?

eScience is where “IT meets scientists.” Researchers are using many different meth-ods to collect or generate data—from sensors and CCDs to supercomputers and particle colliders. When the data finally shows up in your computer, what do you do with all this information that is now in your digital shoebox? People are continually seeking me out and saying, “Help! I’ve got all this data. What am I supposed to do with it? My Excel spreadsheets are getting out of hand!” So what comes next? What happens when you have 10,000 Excel spreadsheets, each with 50 workbooks in them? Okay, so I have been systematically naming them, but now what do I do?

science Paradigms

I show this slide [Figure 1] every time I talk. I think it is fair to say that this insight dawned on me in a CSTB study of computing futures. We said, “Look, computa-tional science is a third leg.” Originally, there was just experimental science, and then there was theoretical science, with Kepler’s Laws, Newton’s Laws of Motion, Maxwell’s equations, and so on. Then, for many problems, the theoretical mod-els grew too complicated to solve analytically, and people had to start simulating. These simulations have carried us through much of the last half of the last millen-nium. At this point, these simulations are generating a whole lot of data, along with

FIGURE 1

4πGp3 K c2

a2=aa

2.

• Thousand years ago:science was empirical

describing natural phenomena

• Last few hundred years:theoretical branch

using models, generalizations

• Last few decades:a computational branch

simulating complex phenomena

• Today: data exploration (eScience) unify theory, experiment, and simulation

– Data captured by instrumentsor generated by simulator

– Processed by software– Information/knowledge stored in computer– Scientist analyzes database / files

using data management and statistics

Science Paradigms

Jim graY oN eSCiENCE

Fig. 2: Science paradigms [30]

The second and third paradigms are typically model-based—they use equations, formulas, and simulations to de-scribe the system. The blue line in Fig. 3 depicts the generalprocedure and corresponding tools. These tools cannot deal

with massive data due to the essence of mechanism models—the models are in low dimensions, leading to deterministicresults which are fully dependent upon only a few parame-ters1. It will raise some problems, causing inefficient or evenincorrect big data analytics. For instance, only under idealconditions, is the wind power proportional to the cube of windspeed. Moreover, some physical parameters, e.g., admittancematrixes, will introduce system error due to the ubiquitousrandomness and uncertainty.

Under classical statistical framework, only two typical datamatrices in the form of X∈RN×T are at our disposal: 1) N,Tare small, and 2) N is small, T is very large (compare with N ).This prerequisite greatly restricts the utilization of the massivedata; we should enable more data to speak for themselves[31]. In other words, model-based framework is not able toturn massive data into useful big data analytics. Althoughthese massive data can contribute to model improvement andparameters correction, we can hardly conduct analysis moreprecisely with extremely large data volumes. Even worse,more data mean more errors; if we take those bad data into thefixed model, poor results are obtained almost surely. Besides,the bias, caused by challenges such as error accumulationsand spurious correlations, will not be alleviated via a low-dimensional procedure [18]—the dimensions of the procedureare limited by the dimensions of the model. The belief thatdata-driven mode is adapted to the future grid’s analysis agreeswith the core viewpoint of the 4th-paradigm. The classical datautilization methodology needs be revisited.

C. Classical Dimensionality Reduction Algorithm—PCA

Data-driven methodology is an alternative; it is model-free and able to process massive data in a holistic way.Principal component analysis (PCA) is one of the classical dataprocessing algorithms which are sensitive to relative scalingoriginal variables. It uses an orthogonal transformation toconvert a set of possibly correlated raw variables into a setof linearly uncorrelated variables called principal components.The number of principal components is often much less thanthe number of original variables. In [14], PCA is used fordimensionality reduction from 14 PMU datasets to extractthe event indicators. For PCA, the procedure consists ofthree parts: 1) Singular Value Decomposition (SVD) [15], 2)Projection, and 3) Indicators.

This procedure is applied to conduct early event detection;details can be found in [14]. We will make a comparison be-tween the PCA-based approach and the RMT-based approach,and the advantages of the later is summarized in I-C.

D. Data-Driven Approach Based on Random Matrix Theory

The procedure based on RMT is outlined below.1) Ring Law and MSR: Ring Law Analysis conducts SA

as follows:

1E.g., y=ax2+bx+c is a 3-dimensional model—the relationship betweenx and y fully depends on a, b, and c.

5

Data processing

in high-dimsion

Pre-Processing

Data Source

Platform

(e.g. WAMS)

Data Integration

& Storage

Unified Modeling

Data Gathering

Cloud Storage

Data Calculation

& Analysis

Mechanism

Model

Parallel Computing

Cloud Computing

Functions and

Applications

Modelling Building

Parameter Identification

Modelling Modification

Decoupling

Build

RMM

Indicators

System and

Visualization

Conduct

Engineering

Interpretation

Relay protection

Prediction

Fault

diagnosis

State estimation

Sample Data A Analysis 1Acquisition 1 Application 1

Sample Data B Analysis 2Acquisition 2 Application 2

Sample Data N Analysis nAcquisition n Application n

Sample Data A

Sample Data B

Sample Data N

G1: Small-scale Isolated Grids

Decentralized Control System

Individual-work Mode

Model-based

G2: Large-scale Connected Grids

(Half-)Centralized Control System

Team-work Mode

Model-based

G3: Smart Grids

Distributed Control System

Group-work Mode

Data-driven

Condition

monitoring

Pure mathematic procedure based on probability

In high dimensions

With tolerable time and predictable errors

1. Law of Large

Number

2. Central Limit

Theorems

1. Data Modeling

2. Data fusion

(blocking, augmented,

& polynomial matrix) View perspective

No bad data

No heterogenous

Sample Data A, B, , N

4Vs data

Only a few data

are available

Decoupling

Low dimensional

Applying Fileds

Concrete Services

Practical Functions

Certain Physical Model

with Assumptions &

Simplifications

Model-Based

Da

ta-d

riv

en

Modeling Analysis Visualization Interpretation

Fig. 3: Data utilization method for power systems. The above, middle, and below parts indicate the data processing proceduresand the work modes for G1, G2, and G3, respectively.

Steps of Ring Law Analysis1) Select arbitrary raw data (or all available data) as data source Ω.2) At a certain time ti, form X as random matrix.3) Obtain Z by matrix transformations (X → X → Xu → Z → Z [2]).4) Calculate eigenvalues λZ and plot the Ring on the complex plane.5) Conduct high-dimensional analysis.

5a) Observe the experimental ring and compare it with the reference.5b) Calculate τMSR =

∑Ni=1

∣∣λZ,i

∣∣/N as the statistical indicators.5c) Compare τMSR with the theoretical value E(τMSR ).

6) Repeat 2)-5) at the next time point (ti= ti + 1).7) Visualize τ on the time series, i.e. draw τ–t curve.8) Make engineering explanations.

In Steps 2–7, with a high-dimensional procedure, oneconducts SA without any prior knowledge, assumption, orsimplification. In step 2, arbitrary raw data, even those fromdistributed nodes or intermittent time periods, are at ourdisposal. The size of X is controllable, and as a result thedimensionality curse is relieved in some ways.

2) M-P Law and LES: For the M-P Law Analysis, the stepsare very similar, except for the following differences:

Partial Steps of M-P Law Analysis

3] Obtain M by matrix transformations (M = 1N

XXH

).4] Calculate eigenvalues λM.

5b] Calculate τ=∑N

i=1 ϕ(λM,i

)as the statistical indicators.

5c] Compare τ with the theoretical value E(τ ).

Notice that Ring Law maps the information from datasetsto the complex plane (CN×T 7→ C), while M-P law does thisto the right half real-axis (CN×T 7→ R+). This fundamentaldifference plays a critical role in data visualization.

IV. CASE STUDIES USING SIMULATED DATA

A. Background and Assumption of the Case

We adopt a standard IEEE 118-node system as Fig. 16 andassume the events as Table II. Thus, the power demand oneach node is obtained as the system injections (Fig. 4a); thevoltage is also obtained (Fig. 4b). Suppose that the powerdemand data is unknown or unqualified for SA due to the lowsampling frequency or the bad quality. For further analysis, wejust start with data source ΩV : vi,j ∈ R118×2500 and assignthe analysis matrix as X ∈ R118×240 (4 minutes’ time span).Firstly, we conduct category for the system operation status;the results are given in Fig. 4c. In general, according to the

6

data feature (events on time-series) and the matrix length (timespan, i.e., T ), we divide the operation satus into 8 stages. Notethat S4,S5, and S6 are transition stages, and their time spanis right equal to the analysis matrix length minus ones, i.e,T−1=239. These stages are described as follows:• For S0,S1,S2, white noises play a dominant part.PNode-52 is rising in turn.

• For S3, PNode-52 keeps a sustained and stable growth.• S4, transition stage. Ramping signal exists.• S5,S6, transition stages. Step signal exists.• For S7, voltage collapse.Two typical data sections, at stage S0 and S6 respectively,

are selected: X0 ∈ R118×240, covering period t=[61:300] andat sampling time tend = 300, and 2) X6 ∈ R118×240, coveringperiod t=[662:901] and at sampling time tend =901.

(a) Assumed event, unavailable.

0 500 1000 1500 2000 2500

Sample Time(s)0.5

0.6

0.7

0.8

0.9

1

1.1

1.2

1.3

1.4

Vol

tage

Mag

nitu

de(p

.u.) Voltage ||Ver=002

X: 2253Y: 0.6021 X: 2254

Y: 0.5396

VBus52

(b) Raw voltage, ΩV for analysis.

0 500 1000 1500 2000 25000.5

0.6

0.7

0.8

0.9

1

1.1

1.2

1.3

1.4Voltage ||Ver=002

S1

S4

S7

!

S3

X6

t=900

S5

S0

X0

VBus52

[740,900] [1140,1300] [1540,2253][240,500]

[2254, --][1301,1539]

S6 S

2 [901,1139][501,739]

X: 2254Y: 0.5396

X: 740Y: 1.05

t=901t=300

Vol

tage

Mag

nitu

de(p

.u.)

Sample Time(s)

(c) Category for operation status.

Fig. 4: Background of Case 1.

B. Anomaly Detection

1) Based on Ring Law and M-P Law:According to our previous work [2], RMM V is build from

the raw voltage data. Then, τMSR is employed as a statisticalindicator to conduct anomaly detection. For the selected datasection X0 and X6, their M-P Law and Ring Law Analysisare shown as Fig 5a, 5b, 5c and 5d. With sliding-window, theτMSR-t curve is obtained as Fig. 5e.

Fig. 5 shows that when there is no signal in the system,the experimental RMM well matches Ring Law and M-PLaw, and the experimental value of LES is approximatelyequal to the theoretical value. This validates the theoreticaljustification for modeling rapid fluctuation of each node usingwhite Gaussian noises, as shown in Section II-A. On theother hand, Ring Law and M-P Law are violated at the very

-1.5 -1 -0.5 0 0.5 1 1.5

Real(Z)-1.5

-1

-0.5

0

0.5

1

1.5

Imag

(Z)

St=300||Ver=5#N=118, T=240 ||L=1

eigenvaluerinner

=0.71297

5MSR

=0.86087

(a) Ring Law for X0

0 1 2 3 4 5 6 70

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

0.45

PD

F St=300, ||Ver=5#T=240, N=118 a=0.18085, b=5.8617

HistogramMarcenko-Pastur Law: fcx

(b) M-P Law for X0

-1.5 -1 -0.5 0 0.5 1 1.5

Real(Z)-1.5

-1

-0.5

0

0.5

1

1.5

Imag

(Z)

St=901||Ver=5#N=118, T=240 ||L=1

eigenvaluerinner

=0.71297

5MSR

=0.81796

(c) Ring Law for X6

-2 0 2 4 6 8 10 12 140

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

0.45

PD

F St=901, ||Ver=5#T=240, N=118 a=0.18085, b=5.8617

HistogramMarcenko-Pastur Law: fcx

Outlier

(d) M-P Law for X6

0 500 1000 1500 2000 2500

Sample Time(s)0.2

0.4

0.6

0.8

1

= MS

R

-200

0

200

400

600

=MSR

PLoad52

X: 500Y: 0.8629

X: 740Y: 0.8586

X: 901Y: 0.818

X: 900Y: 0.8569

X: 2269Y: 0.7372

X: 2270Y: 0.2885

X: 500Y: -0.02856

X: 501Y: 29.9

X: 900Y: 30.12

X: 901Y: 119.9

X: 1300Y: 119.7

X: 2269Y: 361.9

X: 1240Y: 0.854

Pow

er D

eman

d(M

W)

(e) τMSR-t curve using MSW method on time series.

Fig. 5: Anomaly detection results.

0 500 1000 1500 2000 2500

Sample Time(s)-1

0

1

2

4

8

16

=R==

X/7

T

=T2

=T3

=T4

=DET

=LRF

=MSR

=MSR

Fig. 6: Illustration of various LES indicators.

beginning (tend =901) of the step signal. Besides, the proposedhigh-dimensional indicator τMSR, is extremely sensitive to theanomaly—at tend = 901, the τMSR starts the dramatic change(Fig. 5e, τMSR-t curve), while the raw voltage magnitudesare still in the normal range (Fig. 4c). Moreover, followingthe previous work [20], we design numerous kinds of LESτ and define µ0 = τ/E(τ). The results are shown in Fig. 6,proving that different indicators have different effectiveness;this suggests another topic to explore in the future.

2) Based on Spectrum Test:

7

We still set the sampling time at tend =300 and tend =901.Following Lemma II.2 and Lemma II.3, Y0,Y6 ∈ R118×240

(span t=[61:300] and t=[662:901]), and Z0,Z6 ∈ R118×118

(span t = [183 : 300] and t = [784 : 901]) are selected. Theresults are shown in Fig. 7 and Fig. 8. These results validatethat empirical spectral density test is competent to conductanomaly detection—when the power grid is under a normalcondition, the empirical spectral density fA (x) and the ESDfunction FA (x) are almost strictly bounded between the upperbound and the lower bound of their asymptotic limits. On theother hand, these results also validate that GUE and LUE areproper mathematical tools to model the power grid operation.

x

0 0.5 1 1.5 2 2.5 3 3.5

F(x

)

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

ESDDistribution Function Upper Bound

Lower Bound

(a) ESD of Y0 (Normal)

x

0 0.5 1 1.5 2 2.5 3 3.5

F(x

)

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

ESDDistribution Function

Upper Bound

Lower Bound

(b) ESD of Y6 (Abnormal)

Fig. 7: Anomaly Detection Using LUE matrices

x

-2.5 -2 -1.5 -1 -0.5 0 0.5 1 1.5 2 2.5

p(x

)

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

0.45

0.5

0.55Empirical Eigenvalue Density

Semi-Circle Law

Lower Bound

Upper Bound

(a) Density of Z0 (Normal)

x

-2.5 -2 -1.5 -1 -0.5 0 0.5 1 1.5 2 2.5

p(x

)

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

1.8

Empirical Eigenvalue Density

Semi-Circle Law

Lower Bound

Upper Bound

(b) Density of Z6 (Abnormal)

x-2 -1.5 -1 -0.5 0 0.5 1 1.5 2

F(x

)

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

ESDDistribution Function Lower BoundUp Bound

(c) ESD of Z0 (Normal)

x

-2 -1.5 -1 -0.5 0 0.5 1 1.5 2

F(x

)

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

ESDDistribution Function Lower BoundUp Bound

(d) ESD of Z6 (Abnormal)

Fig. 8: Anomaly detection using GUE matrices

C. Steady Stability Evaluation

The V − P curve (also called nose curve) and the smallesteigenvalue of the Jacobian Matrix [15] are two clues for steadystability evaluation. In this case, we focus on E4 stage duringwhich PNode-52 keeps increasing until the system exceeds itssteady stability limit. The V − P curve and λ − P curve,respectively, are given in Fig. 9a and Fig. 9b. Furthermore,we choose some data section, T1 : [1601:1840]; T2 : [1901:

2140]; T3 : [2101 : 2340], shown as Fig. 9a. The RMT-based results are shown as Fig. 10. The outliers become moreevident as the stability degree decreases. The statistics of theoutliers, in some sense, are similar to the smallest eigenvalueof Jacobian Matrix, Lyapunov Exponent or the entropy.

0 50 100 150 200 250 300 350 400 450 500

Load(MW)0.2

0.4

0.6

0.8

1

1.2

1.4

Vol

tage

Mag

nitu

de(p

.u.) Nose Curve ||Ver=002

X: 119.7Y: 0.9148

T2

X: 358.1Y: 0.6021

T3

T1

(a) V − P Curve

100 150 200 250 300 350 400 450

Load(MW)0

0.2

J Jaccobian Matrx ||Ver=002

0

5

10

15

20

%

min 6J

max 6%

X: 357.8Y: 0.0959

X: 350.5Y: 0.6769

X: 357.8Y: 3.682

X: 350.5Y: 0.1772

(b) λ− P Curve

Fig. 9: The V − P curve and λ− P curve.

-1.5 -1 -0.5 0 0.5 1 1.5

Real(Z)-1.5

-1

-0.5

0

0.5

1

1.5

Imag

(Z)

St=1840||Ver=48#N=118, T=240 ||L=1

eigenvaluerinner

=0.71297

5MSR

=0.77584

(a) Ring Law for T1

-5 0 5 10 15 20 250

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

0.45

PD

F St=1840, ||Ver=48#T=240, N=118 a=0.18085, b=5.8617

HistogramMarcenko-Pastur Law: fcx

(b) M-P Law for T1

-1.5 -1 -0.5 0 0.5 1 1.5

Real(Z)-1.5

-1

-0.5

0

0.5

1

1.5

Imag

(Z)

St=2140||Ver=48#N=118, T=240 ||L=1

eigenvaluerinner

=0.71297

5MSR

=0.75984

(c) Ring Law for T2

-5 0 5 10 15 20 25 300

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

0.45

PD

F St=2140, ||Ver=48#T=240, N=118 a=0.18085, b=5.8617

HistogramMarcenko-Pastur Law: fcx

(d) M-P Law for T2

-1.5 -1 -0.5 0 0.5 1 1.5

Real(Z)-1.5

-1

-0.5

0

0.5

1

1.5

Imag

(Z)

St=2340||Ver=48#N=118, T=240 ||L=1

eigenvaluerinner

=0.71297

5MSR

=0.32165

(e) Ring Law for T3

-10 0 10 20 30 40 50 60 700

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

0.45

PD

F St=2340, ||Ver=48#T=240, N=118 a=0.18085, b=5.8617

HistogramMarcenko-Pastur Law: fcx

(f) M-P Law for T3

Fig. 10: RMT-based results for voltage stability evaluation.

For further analysis, we take the signal and stage divisioninto account. In general, sorted by the stability degree, thestages are ordered as S0 > S1 > S2 max(S3,S4,S5) >min(S3,S4,S5) S6 S7. According to Fig. 6, we makethe Table I. The high-dimensional indicators τXR has the sametrend as the stability degree order. These statistics have thepotential for data-driven stability evaluation. Moreover, basedon the Gaussian property of LES indicators, hypothesis testsare designed for the anomaly detection; see [32] for details.

8

TABLE I: Indicator of Various LESs at Each Stage.

MSR T2 T3 T4 DET LRFE0: Theoretical ValueE(τ) 0.8645 1338.3 10069 8.35E4 48.322 73.678

S0 [0240:0500, 261]: Small fluctuations around 0 MW 1©τXR 0.995 1.010 1.040 1.080 0.959 1.014

S5 [0501:0739, 239]: A step signal (0 MW ↑ 30 MW) is included 4©τXR 0.9331 1.280 2.565 7.661 0.5453 1.284

S1 [0740:0900, 161]: Small fluctuations around 30 MW 2©τXR 0.9943 1.010 1.039 1.084 0.9568 1.015

S6 [0901:1139, 239]: A step signal (30 MW ↑ 120 MW) is included 7©τXR 0.8742 2.054 1.06E1 7.22E1 7E−2 1.597

S2 [1140:1300, 161]: Small fluctuations around 120 MW 3©τXR 0.9930 1.019 1.067 1.135 0.9488 1.021

S4 [1301:1539, 239]: A ramp signal (119.7 MW ) is included 4©τXR 0.9337 1.295 2.787 9.615 0.5316 1.294

S3 [1540:2253, 714]: Steady increase ( 358.1 MW) 4©τXR 0.8906 1.717 6.530 3.48E1 0.1483 1.545

S7 [2254:2500, 247]: Static voltage collapse (361.9 MW ) 8©τXR 0.4259 1.02E1 2.11E2 4.65E3 −1.4E1 1.08E1

*τXR = τX/E(τ).

D. Correlation Analysis

The key for correlation analysis is the concatenated matrixAi, which consist of two part—the basic matrix B and acertain factor matrix Ci, i.e., Ai = [B; Ci]. For details, seeour previous work [18]. The LES of each Ai is computed inparallel, and Fig. 11 shows the results.

0 500 1000 1500 2000 2500

Sampling Time(s)

1

2

=T

2(A

j)

LES = of Concatenated Matrix Aj |Ver: 71-87

NoneRandom115245464748495051535455565758

C52 C51

C53C58

None

Random

Fig. 11: Sensitivity Analysis based on Concatenated Matrix.

In Fig. 11, the blue dot line (marked with None) showsthe LES of basic matrix B, and the orange line (markedwith Random) shows the LES of the concatenated matrix[B; R] (R is the standard Gaussian Random Matrix). Fig.11 demonstrates that: 1) node 52 is the causing factor of theanomaly; 2) sensitive nodes are 51, 53, and 58; and 3) nodes11, 45, 46, etc, are not affected by the anomaly. Based on thisalgorithm, we can conduct behavior analysis, e.g., detectionand estimation of residential PV installations [6]. It is anotherhot topic, and we expand it in [32].

E. SA with Asynchronous Data

The proposed data-driven method is robust against bad databoth in space and in time. In our previous work [20], wehave successfully conducted SA with data loss in the corearea. This part we talk about SA with asynchronous data. It iscommon that asynchronous data exists in the data platformssuch as SCADA or WAMS. The problem is mainly causedby erroneous time-tags or communication delays. Sometimes,for a certain signal, the proper delay protection or inter-action/response mechanism will also lead to asynchronousdata. It is hard to measure or even detect the time delay viatraditional methods. Our approach has a special meaning here.

Using the simulated data, we make an artificial delay of 25sampling points for 7 nodes—11, 14, 50, 52, 53, 77, and 81.With the concatenation operation introduced above, similarly,we obtain the results shown as Fig. 12. It is an interestingdiscovery that the approach is robust against asynchronousdata: 1) the anomalies are detected at t = 501 and t = 901;2) node 52 is the most sensitive node; 3) with more detailedobservation, we can even quantitatively draw the conclusionthat there exists a 25 sampling points delay (925 − 900) fornode 52. It is surprising that the exact delay value can berecovered for the particular node! The power of our proposedapproach is vividly exhibited here.

0 500 1000 1500 2000 2500

Sampling Time(20ms)0.5

1

2

=T

2(A

j)

LES = of Concatenated Matrix with Asynchronous Data NoneRandom115245464748495051535455565758

900 950 10000.5

1

1.5

2

2.5

3

3.5

4

C52 C51C53

C58

X: 925Y: 1.142

Fig. 12: Situation awareness with asynchronous data.

V. CASE STUDIES USING FIELD DATA

Some power grid of China are selected, with 34 PMUscollecting power flow data. The raw data are shown as Fig.13; it is quite obvious that the fault begins at samplingtime ts = 3271. The ring distribution and M-P law pre-fault(3101 − 3100), during fault (3173 − 3272), and post-fault(7201 − 7300) are given as Fig 14. This implies that thereal-world data do follow the Ring Law and M-P Law undernormal condition, and they violate these laws when the faultis occurring. Moreover, the LES t− τ curves of basic matrixB and concatenated matrix Ci are obtain as Fig. 15. It showsthat Node 8, 9, 26, 27, 28, 10, 11, 12 are most relevant to thisfault; while Node 1− 7 are not so sensitive.

9

2000 4000 6000 8000 10000 12000 14000

Sample Time (0.02s)-2000

-1500

-1000

-500

0

500

1000

1500

2000

2500

3000

Pow

er F

low

(M

W) Power Flow of 34 PMU of Some Grid

3000 3200 3400 3600 3800 4000 4200

-500

0

500

data1

data2

data3

data4

data5

data6

data7

data8

data9

data10

data11

data12

data13

data14

data15

data16

data17

data18

data19

data20

data21

data22

data23

data24

data25

data26

data27

data28

data29

data30

data31

data32

data33

data34

Fig. 13: Raw power flow data of 34 PMUs.

-1.5 -1 -0.5 0 0.5 1 1.5

Real(Z)-1.5

-1

-0.5

0

0.5

1

1.5

Imag

(Z)

St=3100||Ver=2

N=34, T=100 ||L=1

eigenvaluerinner

=0.8124

MSR=0.90797

(a) Pre-fault: Ring Law

St=3100, ||Ver=2T=100, N=34 a=0.50609, b=7.2974

0 2 4 60

0.05

0.1

0.15

0.2

0.25

0.3

PD

F

HistogramMarcenko-Pastur Law: fcx

(b) Pre-fault: M-P Law

-1.5 -1 -0.5 0 0.5 1 1.5

Real(Z)-1.5

-1

-0.5

0

0.5

1

1.5

Imag

(Z)

St=3272||Ver=6 N=34, T=100 ||L=1

eigenvaluerinner

=0.8124

MSR=0.79623

(c) During fault: Ring Law

St=3272, ||Ver=6T=100, N=34 a=0.50609, b=7.2974

0 5 10 15 20 250

0.05

0.1

0.15

0.2

0.25

0.3

PD

F HistogramMarcenko-Pastur Law: fcxdata1

(d) During fault: M-P Law

-1.5 -1 -0.5 0 0.5 1 1.5

Real(Z)-1.5

-1

-0.5

0

0.5

1

1.5

Imag

(Z)

St=7300||Ver=4 N=34, T=100 ||L=1

eigenvaluerinner

=0.8124

MSR=0.8788

(e) Post-fault: Ring Law

St=7300, ||Ver=4T=100, N=34 a=0.50609, b=7.2974

0 2 4 6 80

0.05

0.1

0.15

0.2

0.25

0.3

0.35

PD

F

HistogramMarcenko-Pastur Law: fcx

(f) Post-fault: M-P Law

Fig. 14: Ring Law and M-P Law for the fault.

2000 4000 6000 8000 10000 12000 14000

Sampling Frequncy (50Hz)

-0.5

0

0.5

1

1.5

2

2.5

3

3.5

=k*

log

k

LES =k*logk

for Filed Datadata1data2data3data4data5data6data7data8data9data10data11data12data13data14data15data16data17data18data19data20data21data22data23data24data25data26data27data28data29data30data31data32data33data34data35

3250 3255 3260 3265 3270 3275 3280 3285 3290 3295 3300

-0.2

0

0.2

0.4

0.6

Fig. 15: LES t− τ curves.

VI. CONCLUSION

This paper has made significant progress on the basis of ourprevious work in the context of big data analytics for futuregrids. Randomness and uncertainty are at the heart of this

data modeling and analysis. Our approach exploits the massivespatial-temporal datasets of power systems. Random matrixtheory (RMT) appears very natural for the problem at hands;in a random matrix of CN×T , we use N nodes to representthe spatial nodes and T data samples to represent the temporalsamples. When the number of nodes N is large, very uniquemathematical phenomenon occurs, such as concentration ofmeasure [16]. Phase transition as a function of data size Nis a result of this deep mathematical phenomenon. This is thevery reason why the proposed algorithms are so powerful inpractice.

Explicitly expressed in forms of linear eigenvalue statistics(LESs) [17], the proposed RMT-based algorithms have nu-merous unique advantages. They are especially suitable forcomplex systems. In the form of a large random matrix, theyhandle massive data that are in high dimensions and withina wide time span all at once. The trick is to treat thesedata as a whole at the disposal of RMT. In this way, highlyreliable decisions are still attainable with some imperfect data,e.g., the asynchronous data. Moreover, with the statisticalprocessing such as test function setting, the proposed data-driven approach has the potential to balance the perspectivesof the speed, the sensitivity, and the reliability in practice.

The stability evaluation and behavior analysis are two bigtopics along this direction. Besides, the statistical indicatorsare good starting points for artificial intelligence and machinelearning. For example, we can extract the linear eigenvaluestatistics as features; those extracted features are used forfurther data processing in the pipeline using algorithms suchas random forest, decision trees, and support vector machine.Our whole framework starts with the use of sample covariancematrix to replace the true covariance matrix. It is well knownthat this replacement is far from optimal. The almost optimalestimation of large covariance matrices using tools fromRMT [33] can be used, instead.

APPENDIX A

11 2

3

4 11

12 117

14

1513

16

6 7

5

G

G

G

G

G

G

G

G

G G G

G

G

GG

G

G

G

G

G

G

G

G

G

G

G

G

G

G

G G

GGGG

G

G G

G

G

G

GG

G

G

G

G

G

G

G

G

G G

G

1917

18

30113

32

3129

20

21

35

33

34

36

24

2227

8

9

10

28

114 115

26

25

23

39

40 41 42 53 54 5556 59

58

51

57

50

48

49

45

4647

69

38

43

52

68

67

66

65

62

61

64

60

63

73

71

7072

74

116

79

8178

82

75

118 76

77 97

96

88 8985

8483

86

87

90 91

92

93

9495

102101 103

111

110112

109

108

107

105104

100

98

80 99

106

443752

Fig. 16: Partitioning network for the IEEE 118-node system.

10

APPENDIX B

TABLE II: Series of Events

Stage E1 E2 E3 E4Time (s) 1–500 501–900 901–1300 1301–2500PNode-52 (MW) 0 30 120 t/4− 205

P52 is the power demand of node 52.

The power demand of other nodes are assigned as

yload nt=yload nt × (1 + γMul×r1) + γAcc×r2, (17)

where r1 and r2 are the element of standard Gaussian randommatrix; γAcc=0.1, γMul=0.001.

REFERENCES

[1] T. Hong, C. Chen, J. Huang et al., “Guest editorial big dataanalytics for grid modernization,” IEEE Transactions on SmartGrid, vol. 7, no. 5, pp. 2395–2396, Sept 2016.

[2] X. He, Q. Ai, R. C. Qiu et al., “A big data architecture design forsmart grids based on random matrix theory,” Jan. 2015, acceptedby IEEE Trans on Smart Grid.

[3] L. Moulin, A. Alves da Silva, M. El-Sharkawi et al., “Supportvector machines for transient stability analysis of large-scalepower systems,” Power Systems, IEEE Transactions on, vol. 19,no. 2, pp. 818–825, 2004.

[4] M. Rafferty, X. Liu, D. M. Laverty, and S. McLoone, “Real-time multiple event detection and classification using movingwindow pca,” IEEE Transactions on Smart Grid, vol. 7, no. 5,pp. 2537–2548, Sept 2016.

[5] H. Jiang, X. Dai, D. W. Gao et al., “Spatial-temporal syn-chrophasor data characterization and analytics in smart gridfault detection, identification, and impact causal analysis,” IEEETransactions on Smart Grid, vol. 7, no. 5, pp. 2525–2536, Sept2016.

[6] X. Zhang and S. Grijalva, “A data-driven approach for detectionand estimation of residential pv installations,” IEEE Transac-tions on Smart Grid, vol. 7, no. 5, pp. 2477–2485, Sept 2016.

[7] H. Shaker, H. Zareipour, and D. Wood, “A data-driven approachfor estimating the power generation of invisible solar sites,”IEEE Transactions on Smart Grid, vol. 7, no. 5, pp. 2466–2476,Sept 2016.

[8] B. Wang, B. Fang, Y. Wang, H. Liu, and Y. Liu, “Power systemtransient stability assessment based on big data and the corevector machine,” IEEE Transactions on Smart Grid, vol. 7,no. 5, pp. 2561–2570, Sept 2016.

[9] A. Phadke and R. M. de Moraes, “The wide world of wide-area measurement,” Power and Energy Magazine, IEEE, vol. 6,no. 5, pp. 52–65, 2008.

[10] V. Terzija, G. Valverde, D. Cai, P. Regulski, V. Madani, J. Fitch,S. Skok, M. M. Begovic, and A. Phadke, “Wide-area monitor-ing, protection, and control of future electric power networks,”Proceedings of the IEEE, vol. 99, no. 1, pp. 80–93, 2011.

[11] L. Xie, Y. Chen, and H. Liao, “Distributed online monitoringof quasi-static voltage collapse in multi-area power systems,”Power Systems, IEEE Transactions on, vol. 27, no. 4, pp. 2271–2279, 2012.

[12] Q. Jiang, X. Li, B. Wang, and H. Wang, “Pmu-based faultlocation using voltage measurements in large transmission net-works,” Power Delivery, IEEE Transactions on, vol. 27, no. 3,pp. 1644–1652, 2012.

[13] A. H. Al-Mohammed and M. Abido, “A fully adaptive pmu-based fault location algorithm for series-compensated lines,”Power Systems, IEEE Transactions on, vol. 29, no. 5, pp. 2129–2137, 2014.

[14] L. Xie, Y. Chen, and P. Kumar, “Dimensionality reductionof synchrophasor data for early event detection: Linearized

analysis,” Power Systems, IEEE Transactions on, vol. 29, no. 6,pp. 2784–2794, 2014.

[15] J. M. Lim and C. L. DeMarco, “Svd-based voltage stabilityassessment from phasor measurement unit data,” IEEE Trans-actions on Power Systems, vol. PP, no. 99, pp. 1–9, 2015.

[16] R. Qiu and P. Antonik, Smart Grid and Big Data. John Wileyand Sons, 2015.

[17] R. C. Qiu, Big Data for Complex Network. CRC, 2016, ch.Large Random Matrices and Big Data Analytics.

[18] X. Xu, X. He, Q. Ai, and C. Qiu, “A correlation analysis methodfor power systems based on random matrix theory,” Jun. 2015,accepted by IEEE Trans on Smart Grid.

[19] Y. Yan, G. Sheng, H. Wang, Y. Liu, Y. Chen, X. Jiang, andZ. Guo, “The key state assessment method of power transmis-sion equipment using big data analyzing model based on largedimensional random matrix,” Proceedings of the CSEE, vol. 36,no. 2, pp. 435–445, Jan. 2016.

[20] X. He, R. C. Qiu, Q. Ai, L. Chu, and X. Xu, “Linear eigenvaluestatistics: An indicator ensemble design for situation awarenessof power systems,” Dec. 2015.

[21] M. Shcherbina, “Central limit theorem for linear eigenvaluestatistics of the wigner and sample covariance randommatrices,” ArXiv e-prints, Jan. 2011. [Online]. Available:http://arxiv.org/pdf/1101.3249.pdf

[22] C. Zhang and R. C. Qiu, “Massive mimo as a big data system:Random matrix models and testbed,” IEEE Access, vol. 3, pp.837–851, 2015.

[23] J. R. Ipsen and M. Kieburg, “Weak commutation relations andeigenvalue statistics for products of rectangular random matri-ces,” Physical Review E, vol. 89, no. 3, 2014, Art. ID 032106.

[24] F. Benaych-Georges and J. Rochet, “Outliers in the singlering theorem,” Probability Theory and Related Fields, pp.1–51, May 2015. [Online]. Available: http://dx.doi.org/10.1007/s00440-015-0632-x

[25] T. Tao, “Outliers in the spectrum of iid matrices with boundedrank perturbations,” Probability Theory and Related Fields, vol.155, no. 1-2, pp. 231–263, 2013.

[26] F. Gotze and A. Tikhomirov, “The rate of convergence forspectra of gue and lue matrix ensembles,” Open Mathematics,vol. 3, no. 4, pp. 666–704, 2005.

[27] Q. Wu, D. Zhang, D. Liu, W. Liu, and C. Deng, “A methodfor power system steady stability situation assessment based onrandom matrix theory,” 2016.

[28] W. Liu, D. Zhang, X. Wang, D. Liu, and Q. Wu, “Power systemtransient stability analysis based on random matrix theory,”2016.

[29] A. J. Hey, S. Tansley, K. M. Tolle et al., The fourth paradigm:data-intensive scientific discovery. Microsoft Research Red-mond, WA, 2009, vol. 1.

[30] J. Gray, “Jim gray on escience: A transformed scientificmethod,” The fourth paradigm: Data-intensive scientific discov-ery, pp. xvii–xxxi, 2009.

[31] R. Kitchin, “Big data and human geography opportunities,challenges and risks,” Dialogues in human geography, vol. 3,no. 3, pp. 262–267, 2013.

[32] X. He, R. Qiu, L. Chu, Q. Ai, Z. Ling, and J. Zhan, “Detectionand estimation of the invisible units using utility data basedon random matrix theory,” ArXiv e-prints, 2017. [Online].Available: http://arxiv.org/pdf/1710.10745.pdf

[33] J. Bun, J.-P. Bouchaud, and M. Potters, “Cleaning large correla-tion matrices: tools from random matrix theory,” arXiv preprintarXiv:1610.08104, 2016.