Slycat Ensemble AnalysisPatricia J. Crossno, Timothy M. Shead, Milosz A. Sielicki,

Warren L. Hunt, Shawn Martin, and Ming-Yu Hsieh Sandia National Laboratories

Sandia National Laboratories is a multi-program laboratory managed and operated by Sandia Corporation, a wholly owned subsidiary of Lockheed Martin Corporation, for the U.S. Department of Energy’s National Nuclear Security Administration under contract DE-AC04-94AL85000.SAND 2014-1399P

Patricia J. Crossno: pjcross@sandia.gov Timothy M. Shead: tshead@sandia.gov Milosz A. Sielicki: masieli@sandia.gov Warren L. Hunt: wlhunt@sandia.gov Shawn Martin: smartin@sandia.govMing-Yu Hsieh: myhsieh@sandia.gov

Analysis Tasks:• Find strongest input/output correlations• Find inputs with least impact on outputs

• Find anomalous simulation runs

CCA Visual Representations

Scatterplot: Each Simulation Relative to Ensemble

Distance off diagonal shows difference from

ensemble as a whole, plus potential anomalies.

Purple = Outputs

Bar chart: Ensemble-wide Relationships

Viewing 1st CCA component in

both views

Input x1 has the least impact on

outputs y1 and y2

250 simulations, each color-coded by its y1 output value

Selected simulation

Positive many-to-

many correlation (bar color the same)

between X25 & X14 and

Y2 & Y1

Green = Inputs

Inputs x25 & x14 have the most impact on both

outputs y1 and y2

Viewing 2nd CCA component in

both bar chart & scatterplot

250 simulations, each color-coded by its x23 input value

Inputs and outputs sorted by

correlation strength within

CCA2 component

X23 selected for scatterplot color-

coding (dark green row highlight)

Three distinct groups of

input values

Inverse correlation (red vs. blue)

between x23 & y4; CCA3 captures

relationship between x8 & y3

Scatterplot color-coding changed by clicking on y4

row (darker purple highlight)

Three output value groupsmap to the 3 input groups

250 simulations, each color-coded by its y4 output value

Click CCA column header to select CCA component in

views

Viewing 3rd CCA component in

both bar chart & scatterplot

Inverse correlation between x8 & y3; CCA2 captures

relationship between x23 & y4

250 simulations, each color-coded by

its x8 input value

X8 inputs range from low (blue)

to high (red)

X8 selected for scatterplot color-

coding (dark green row highlight)

Click header triangle to sort

variables (toggles fromdecreasing to

increasing)

250 simulations, each color-coded by its y3 output value

Corresponding y3 outputs

inversely range from high (red)

to low (blue)

Scatterplot color-coding changed by clicking on y3

row (darker purple highlight)

Approach: Canonical Correlation Analysis (CCA)

feat

ure

s

simulations

ou

tpu

tsin

pu

ts

s1 s2 sn

o2

i1

om

…s3 s4

ik

o1

.

.

.

.

.

.

CCA

feat

ure

s

inp

uts

i1

ik

.

.

.

ou

tpu

ts

o2

o1

.

.

.om

CCA components

c1 ck…

CCA1

input meta-features

ou

tpu

t m

eta

-fe

atu

res

s1

s2

sns4

s3

Structure Correlations

Slycat Sensitivity Analysis

Input parameters

SimulationEnsemble

Simple Regression(1-to-1)

Multiple Regression(Many-to-1)

Model Confidence

How About Many-to-Many Correlations?

Problem: Electrical Circuit Simulation Sensitivity Analysis

Rerun CCA analysis between all inputs and y4 to find strongest correlations (all-to-1)

All to y4 analysis4 anomalous runs

share common x248 values

2641 simulations, each color-coded by its x248 input value

(strongest)

All to y4 analysis4 anomalous runs

share common x255 values

2641 simulations, each color-coded by its x255

input value (2nd strongest)

2641 simulations, each color-coded by its y4 output value

4 anomalous runs in y4 values

All to all analysis

Finding Anomalous SimulationsFinding Most Significant Inputs

Objectives:• Map Output Variability Back to Inputs• Reduce Number of Input Parameters

• Reduce Number of Simulations to Run• Identify Anomalous Runs

• Increase Model Confidence

266 scrollable

Inputs

Note R2 is increasing & P is decreasing with

each CCA component

Available Open Sourcehttps://github.com/sandialabs/slycat

Reduce Inputs & Simulations

In the 2641 run ensemble above, analysis allowed input parameters to be reduced from 266 to 21,

decreasing simulation time ten-fold.