Exploratory Factor Analysis. Principal components analysis seeks linear combinations that best capture the variation in the original variables. Factor

Exploratory Factor Analysis

• Principal components analysis seeks linear combinations that best capture the variation in the original variables.

• Factor analysis seeks linear combinations that best capture the correlations among the original variables.

Review: FA vs. PCA

Factor Analysis: Seeks to identify common factors that influence correlations among items, it is not a correlation matrix that is analyzed thus, the main diagonal is replaced by the communalities - the variances in X that are due only to the common factors. But, communalities are not known in advance of the factor analysis, giving rise to the communality problem and the need to solve for the common factors iteratively.

Review: FA vs. PCAPrincipal Components Analysis: The correlation matrix contains ones on the main diagonal, accounting for all of the variance in X—the goal of principal components analysis.

In the principal axes approach to factor analysis, the only difference compared to principal components is that the matrix being analyzed is a correlation matrix (which is also a variance-covariance matrix for standardized variables) in which the main diagonal contains the communalities rather than the variances.

FA: Principal Axes Factoring

• Most of the loadings on any given factor are small and a few loadings are large in absolute value

• Most of the loadings for any given variable are small, with ideally only one loading being large in absolute value.

• Any pair of factors have dissimilar patterns of loadings.

Simple Structure

The ideal loadings are the simple structure, which might look like this for the factor loadings:

To get closer to the ideal, we rotate the factors

F1 F2 F3

X1 1 0 0

X2 1 0 0

X3 1 0 0

X4 0 1 0

X5 0 1 0

X6 0 1 0

X7 0 0 1

X8 0 0 1

X9 0 0 1

Varimax One way to approach this ideal pattern is to find the rotation that maximizes the variance of the loadings in the columns of the factor structure matrix. This approach was suggested by Kaiser and is called varimax rotation.

F1 F2 F3

X1 1 0 0

X2 1 0 0

X3 1 0 0

X4 0 1 0

X5 0 1 0

X6 0 1 0

X7 0 0 1

X8 0 0 1

X9 0 0 1

Quartimax A second way to approach this ideal pattern is to find the rotation that maximizes the variance of the loadings in the rows of the factor structure matrix. This approach is called quartimax rotation.

F1 F2 F3

X1 1 0 0

X2 1 0 0

X3 1 0 0

X4 0 1 0

X5 0 1 0

X6 0 1 0

X7 0 0 1

X8 0 0 1

X9 0 0 1

Equimax For those who want the best of both worlds, equimax rotation attempts to satisfy both goals.

Varimax is the most commonly used and the three rarely produce results that are very discrepant.

F1 F2 F3

X1 1 0 0

X2 1 0 0

X3 1 0 0

X4 0 1 0

X5 0 1 0

X6 0 1 0

X7 0 0 1

X8 0 0 1

X9 0 0 1

Correlation Matrixa

1.000 .447 .422 .435 .114 .203 .091 .082 .513 .304 .245 .101 .245

.447 1.000 .619 .604 .068 .146 .103 .063 .445 .318 .203 .088 .199

.422 .619 1.000 .583 .053 .139 .110 .066 .365 .240 .183 .074 .184

.435 .604 .583 1.000 .115 .258 .122 .097 .482 .368 .255 .139 .293

.114 .068 .053 .115 1.000 .349 .209 .321 .186 .303 .272 .279 .278

.203 .146 .139 .258 .349 1.000 .221 .355 .315 .377 .323 .367 .545

.091 .103 .110 .122 .209 .221 1.000 .201 .150 .163 .310 .232 .232

.082 .063 .066 .097 .321 .355 .201 1.000 .154 .219 .288 .320 .314

.513 .445 .365 .482 .186 .315 .150 .154 1.000 .534 .301 .204 .394

.304 .318 .240 .368 .303 .377 .163 .219 .534 1.000 .302 .368 .467

.245 .203 .183 .255 .272 .323 .310 .288 .301 .302 1.000 .340 .392

.101 .088 .074 .139 .279 .367 .232 .320 .204 .368 .340 1.000 .511

.245 .199 .184 .293 .278 .545 .232 .314 .394 .467 .392 .511 1.000

.000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000

.000 .000 .000 .003 .000 .000 .005 .000 .000 .000 .000 .000

.000 .000 .000 .016 .000 .000 .004 .000 .000 .000 .001 .000

.000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000

.000 .003 .016 .000 .000 .000 .000 .000 .000 .000 .000 .000

.000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000

.000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000

.000 .005 .004 .000 .000 .000 .000 .000 .000 .000 .000 .000

.000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000

.000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000

.000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000

.000 .000 .001 .000 .000 .000 .000 .000 .000 .000 .000 .000

.000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000

CIGS

BEER

WINE

LIQUOR

COCAINE

TRANQS

DRUGSTR

HEROIN

MARIJUAN

HASHISH

INHALE

HALLUCIN

AMPHETS

CIGS

BEER

WINE

LIQUOR

COCAINE

TRANQS

DRUGSTR

HEROIN

MARIJUAN

HASHISH

INHALE

HALLUCIN

AMPHETS

Correlation

Sig. (1-tailed)

CIGS BEER WINE LIQUOR COCAINE TRANQS DRUGSTR HEROIN MARIJUAN HASHISH INHALE HALLUCIN AMPHETS

Determinant = 1.742E-02a.

Data on drug use reported by 1634 students in Los Angeles. Participants rated their use on a 5-point scale: 1 = never tried, 2 = only once, 3 = a few times, 4 = many times, 5 = regularly.

Example: Principal Components Analysis

FACTOR /VARIABLES cigs beer wine liquor cocaine tranqs drugstr heroin marijuan hashish inhale hallucin amphets /PRINT INITIAL KMO EXTRACTION ROTATION /PLOT EIGEN ROTATION /CRITERIA MINEIGEN(1) ITERATE(25) /EXTRACTION PC /CRITERIA ITERATE(25) /ROTATION VARIMAX /METHOD=CORRELATION .

Syntax for Factor Analysis

PC = Principal ComponentsPAF = Principal Axis Factoring

KMO asks for a test of multicollinearity – this will tell whether there is enough shared variance to warrant a factor analysis

KMO and Bartlett's Test

.868

6592.802

78

.000

Kaiser-Meyer-Olkin Measure of SamplingAdequacy.

Approx. Chi-Square

df

Sig.

Bartlett's Test ofSphericity

Should the matrix be analyzed?

This can vary between 0 and 1, indicating whether there is sufficient multicollinearity to warrant an analysis. Higher values indicate the desirability of a principal components analysis.

Bartlett's test indicates the correlation matrix is clearly not an identity matrix

Total Variance Explained

4.382 33.705 33.705 4.382 33.705 33.705

2.046 15.736 49.440 2.046 15.736 49.440

.953 7.328 56.769 .953 7.328 56.769

.817 6.288 63.057 .817 6.288 63.057

.766 5.889 68.946 .766 5.889 68.946

.687 5.286 74.231 .687 5.286 74.231

.644 4.950 79.182 .644 4.950 79.182

.615 4.729 83.911 .615 4.729 83.911

.561 4.314 88.225 .561 4.314 88.225

.399 3.067 91.293 .399 3.067 91.293

.395 3.037 94.330 .395 3.037 94.330

.374 2.879 97.209 .374 2.879 97.209

.363 2.791 100.000 .363 2.791 100.000

Component1

2

3

4

5

6

7

8

9

10

11

12

13

Total % of Variance Cumulative % Total % of Variance Cumulative %

Initial Eigenvalues Extraction Sums of Squared Loadings

Extraction Method: Principal Component Analysis.

Two principal components account for nearly half of the information in the original variables

Communalities

1.000 1.000

1.000 1.000

1.000 1.000

1.000 1.000

1.000 1.000

1.000 1.000

1.000 1.000

1.000 1.000

1.000 1.000

1.000 1.000

1.000 1.000

1.000 1.000

1.000 1.000

CIGS

BEER

WINE

LIQUOR

COCAINE

TRANQS

DRUGSTR

HEROIN

MARIJUAN

HASHISH

INHALE

HALLUCIN

AMPHETS

Initial Extraction


Initial Extraction All principal components are extracted so all of the variance in the variables is accounted for.

Component Matrix

.582 -.400 -5.81E-02 1.457E-02

.598 -.567 .127 9.098E-02

.555 -.561 .215 .128

.665 -.465 5.158E-02 5.725E-02

.436 .412 5.078E-02 .534

.613 .370 -.167 7.766E-02

.369 .271 .710 -.299

.422 .451 .143 .481

.710 -.234 -.231 -9.89E-02

.688 7.239E-02 -.346 -.107

.578 .242 .306 -.176

.520 .471 -.107 -.260

.688 .332 -.227 -.239

CIGS

BEER

WINE

LIQUOR

COCAINE

TRANQS

DRUGSTR

HEROIN

MARIJUAN

HASHISH

INHALE

HALLUCIN

AMPHETS

1 2 3 4

Component


13 components extracted.a.

Part of the loading matrix. How should the first two principal components be interpreted?

Scree Plot

Component Number

13121110987654321

Eig

enva

lue

5

4

3

2

1

0

Only two principal components are indicated by the scree test.


4.382 33.705 33.705 4.382 33.705 33.705

2.046 15.736 49.440 2.046 15.736 49.440

.953 7.328 56.769

.817 6.288 63.057

.766 5.889 68.946

.687 5.286 74.231

.644 4.950 79.182

.615 4.729 83.911

.561 4.314 88.225

.399 3.067 91.293

.395 3.037 94.330

.374 2.879 97.209

.363 2.791 100.000

Component1

2

3

4

5

6

7

8

9

10

11

12

13

Total % of Variance Cumulative % Total % of Variance Cumulative %

Initial Eigenvalues Extraction Sums of Squared Loadings


Only two principal components are indicated by the Kaiser rule as well.

Communalities

1.000 .499

1.000 .680

1.000 .622

1.000 .658

1.000 .360

1.000 .513

1.000 .209

1.000 .381

1.000 .559

1.000 .479

1.000 .393

1.000 .492

1.000 .583

CIGS

BEER

WINE

LIQUOR

COCAINE

TRANQS

DRUGSTR

HEROIN

MARIJUAN

HASHISH

INHALE

HALLUCIN

AMPHETS

Initial Extraction


With only two principal components, much less of the variance in each variable is accounted for.

Component Matrixa

.582 -.400

.598 -.567

.555 -.561

.665 -.465

.436 .412

.613 .370

.369 .271

.422 .451

.710 -.234

.688 7.239E-02

.578 .242

.520 .471

.688 .332

CIGS

BEER

WINE

LIQUOR

COCAINE

TRANQS

DRUGSTR

HEROIN

MARIJUAN

HASHISH

INHALE

HALLUCIN

AMPHETS

1 2

Component



The loadings for the first two components do not change when they are the only ones extracted.

Correlation Matrixa

1.000 .447 .422 .435 .114 .203 .091 .082 .513 .304 .245 .101 .245

.447 1.000 .619 .604 .068 .146 .103 .063 .445 .318 .203 .088 .199

.422 .619 1.000 .583 .053 .139 .110 .066 .365 .240 .183 .074 .184

.435 .604 .583 1.000 .115 .258 .122 .097 .482 .368 .255 .139 .293

.114 .068 .053 .115 1.000 .349 .209 .321 .186 .303 .272 .279 .278

.203 .146 .139 .258 .349 1.000 .221 .355 .315 .377 .323 .367 .545

.091 .103 .110 .122 .209 .221 1.000 .201 .150 .163 .310 .232 .232

.082 .063 .066 .097 .321 .355 .201 1.000 .154 .219 .288 .320 .314

.513 .445 .365 .482 .186 .315 .150 .154 1.000 .534 .301 .204 .394

.304 .318 .240 .368 .303 .377 .163 .219 .534 1.000 .302 .368 .467

.245 .203 .183 .255 .272 .323 .310 .288 .301 .302 1.000 .340 .392

.101 .088 .074 .139 .279 .367 .232 .320 .204 .368 .340 1.000 .511

.245 .199 .184 .293 .278 .545 .232 .314 .394 .467 .392 .511 1.000

.000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000

.000 .000 .000 .003 .000 .000 .005 .000 .000 .000 .000 .000

.000 .000 .000 .016 .000 .000 .004 .000 .000 .000 .001 .000

.000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000

.000 .003 .016 .000 .000 .000 .000 .000 .000 .000 .000 .000

.000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000

.000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000

.000 .005 .004 .000 .000 .000 .000 .000 .000 .000 .000 .000

.000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000

.000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000

.000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000

.000 .000 .001 .000 .000 .000 .000 .000 .000 .000 .000 .000

.000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000

Cigarettes

Beer

Wine

Liquor

Cocaine

Tranquilizers

Drug Store Medication

Heroin

Marijuana

Hashish

Inhalants

Hallucinogenics

Amphetamines

Cigarettes

Beer

Wine

Liquor

Cocaine

Tranquilizers


Heroin

Marijuana

Hashish

Inhalants

Hallucinogenics

Amphetamines

Correlation

Sig. (1-tailed)

Cigarettes Beer Wine Liquor Cocaine TranquilizersDrug StoreMedication Heroin Marijuana Hashish Inhalants

Hallucinogenics

Amphetamines

Determinant = 1.742E-02a.

Drug use data by 1634 students. Participants rated their use on a 5-point scale: 1 = never tried, 2 = only once, 3 = a few times, 4 = many times, 5 = regularly.

Common Factor Analysis


.868

6592.802

78

.000


Approx. Chi-Square

df

Sig.


The analysis begins in the same way as principal components analysis. It would make little sense to search for common factors in an identity matrix:

Communalities

.354 .380

.504 .617

.465 .524

.498 .589

.215 .246

.382 .440

.136 .135

.217 .258

.476 .478

.415 .416

.278 .310

.337 .393

.478 .545

Cigarettes

Beer

Wine

Liquor

Cocaine

Tranquilizers


Heroin

Marijuana

Hashish

Inhalants

Hallucinogenics

Amphetamines

Initial Extraction

Extraction Method: Principal Axis Factoring.

Unlike principal components analysis, factor analysis will not attempt to explain all of the variance in each variable. Only common factor variance is of interest. This creates the need for some initial estimates of communalities.


4.382 33.705 33.705 3.832 29.477 29.477 2.687 20.670 20.670

2.046 15.736 49.440 1.498 11.526 41.003 2.643 20.333 41.003

.953 7.328 56.769

.817 6.288 63.057

.766 5.889 68.946

.687 5.286 74.231

.644 4.950 79.182

.615 4.729 83.911

.561 4.314 88.225

.399 3.067 91.293

.395 3.037 94.330

.374 2.879 97.209

.363 2.791 100.000

Factor1

2

3

4

5

6

7

8

9

10

11

12

13

Total % of Variance Cumulative % Total % of Variance Cumulative % Total % of Variance Cumulative %

Initial Eigenvalues Extraction Sums of Squared Loadings Rotation Sums of Squared Loadings


The number of factors to extract is guided by the size of the eigenvalues, as it was in principal components analysis. But, not all of the variance can be accounted for in the variables.


4.382 33.705 33.705 3.832 29.477 29.477 2.687

2.046 15.736 49.440 1.498 11.526 41.003 2.643

.953 7.328 56.769

Factor1

2

3

Total % of Variance Cumulative % Total % of Variance Cumulative % Total


To the extent there is random error in the measures, the eigenvalues for factor analysis will be smaller than the corresponding eigenvalues in principal components analysis.

Remember, this is because PCA assumes no error of measurement, whereas FA assumes there is measurement error.


33.705 3.832 29.477 29.477 2.687 20.670 20.670

49.440 1.498 11.526 41.003 2.643 20.333 41.003

56.769

Cumulative % Total % of Variance Cumulative % Total % of Variance Cumulative %


The location of the factors might be rotated to a position that allows easier interpretation. This will shift the variance, but preserve the total amount accounted for.

Scree Plot

Factor Number

13121110987654321

Eig

enva

lue

5

4

3

2

1

0

Two factors appear to be sufficient. The attenuated eigenvalues will generally tell the same story.

Factor Matrixa

.541 -.296

.594 -.514

.539 -.483

.653 -.403

.379 .320

.566 .346

.312 .194

.368 .350

.672 -.160

.637 9.973E-02

.516 .209

.472 .414

.654 .343

Cigarettes

Beer

Wine

Liquor

Cocaine

Tranquilizers


Heroin

Marijuana

Hashish

Inhalants

Hallucinogenics

Amphetamines

1 2

Factor


2 factors extracted. 6 iterations required.a.

Component Matrix

.582 -.400 -5.81E-02

.598 -.567 .127

.555 -.561 .215

.665 -.465 5.158E-02

.436 .412 5.078E-02

.613 .370 -.167

.369 .271 .710

.422 .451 .143

.710 -.234 -.231

.688 7.239E-02 -.346

.578 .242 .306

.520 .471 -.107

.688 .332 -.227

CIGS

BEER

WINE

LIQUOR

COCAINE

TRANQS

DRUGSTR

HEROIN

MARIJUAN

HASHISH

INHALE

HALLUCIN

AMPHETS

1 2 3

Component



The loadings will be reduced in FA compared to PCA because not all of the variance in X is due to common factor variance.

PCA FA

Rotated Factor Matrixa

.593 .168

.784 4.961E-02

.723 3.232E-02

.749 .170

4.598E-02 .494

.162 .643

8.689E-02 .357

1.704E-02 .507

.592 .356

.385 .517

.222 .511

4.697E-02 .626

.226 .703

Cigarettes

Beer

Wine

Liquor

Cocaine

Tranquilizers


Heroin

Marijuana

Hashish

Inhalants

Hallucinogenics

Amphetamines

1 2

Factor

Extraction Method: Principal Axis Factoring. Rotation Method: Varimax with Kaiser Normalization.

Rotation converged in 3 iterations.a.

Rotating the factors to simple structure makes the interpretation easier. The first factor appears to be minor “recreational” drug use. The second factor appears to be major “abusive” drug use.

A sample of 303 MBA students were asked to evaluate different makes of cars using 16 different adjectives rated on a 5-point agreement scale (1 = strongly disagree, 5 = strongly agree):

“This car is an exciting car.”

Exciting

Dependable

Luxurious

Outdoorsy

Powerful

Stylish

Comfortable

Rugged

Fun

Safe

Performance

Family

Versatile

Sports

Status

Practical

New Example: FA

Descriptive Statistics

3.21 1.218 294

3.65 1.020 294

3.48 1.107 294

2.68 1.398 294

3.60 1.016 294

3.44 1.237 294

3.70 .851 294

2.53 1.329 294

3.48 1.263 294

3.75 .979 294

3.44 1.181 294

3.46 1.286 294

3.08 1.093 294

3.03 1.318 294

3.57 1.183 294

3.35 1.106 294

Exciting

Dependable

Luxurious

Outdoorsy

Powerful

Stylish

Comfortable

Rugged

Fun

Safe

Performance

Family

Versatile

Sports

Status

Practical

Mean Std. Deviation Analysis N

Are all 16 individual ratings required to understand product evaluation, or, is there a simpler measurement model?

Communalities

.773 .784

.414 .437

.644 .672

.687 .747

.595 .600

.767 .778

.520 .550

.701 .753

.761 .763

.539 .596

.678 .651

.720 .778

.529 .537

.671 .690

.734 .751

.581 .571

Exciting

Dependable

Luxurious

Outdoorsy

Powerful

Stylish

Comfortable

Rugged

Fun

Safe

Performance

Family

Versatile

Sports

Status

Practical

Initial Extraction


The analysis must begin with some idea of the proportion of variance in each variable that can be attributed to the common factors. The most common initial estimate is the squared multiple correlation between a given measure and all the remaining measures.

Scree Plot

Component Number

16151413121110987654321

Eig

enva

lue

7

6

5

4

3

2

1

0

Three common factors appear to underlie the 16 evaluative ratings.


5.927 37.042 37.042 5.644 35.275 35.275 5.632 35.198 35.198

3.187 19.920 56.961 2.791 17.446 52.721 2.624 16.403 51.601

2.542 15.887 72.849 2.222 13.885 66.606 2.401 15.005 66.606

.648 4.052 76.901

.589 3.684 80.585

.477 2.983 83.568

.406 2.539 86.107

.369 2.307 88.414

.337 2.107 90.521

.296 1.852 92.373

.277 1.729 94.102

.242 1.510 95.613

.205 1.282 96.895

.185 1.154 98.049

.169 1.054 99.103

.144 .897 100.000

Factor1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

Total % of Variance Cumulative % Total % of Variance Cumulative % Total % of Variance Cumulative %



The three common factors account for two-thirds of the common factor variance.

Factor Matrixa

.863 9.283E-03 .199

.123 .512 -.400

.636 .244 -.457

4.746E-02 .439 .743

.695 .290 .178

.873 9.610E-02 -8.02E-02

.148 .617 -.385

5.676E-02 .486 .716

.860 6.511E-02 .140

-7.07E-02 .671 -.376

.772 6.326E-02 -.227

-.675 .565 -5.54E-02

-.237 .634 .281

.691 2.867E-03 .461

.829 .109 -.229

-.439 .613 -5.37E-02

Exciting

Dependable

Luxurious

Outdoorsy

Powerful

Stylish

Comfortable

Rugged

Fun

Safe

Performance

Family

Versatile

Sports

Status

Practical

1 2 3

Factor



Without rotation, the interpretation of the factors is not readily apparent, especially the second and third factors.


.864 -.142 .132

.150 .641 -5.82E-02

.644 .425 -.276

8.497E-02 -4.57E-02 .859

.715 .111 .276

.876 8.355E-02 -5.42E-02

.182 .719 1.137E-02

9.692E-02 7.389E-03 .862

.864 -6.28E-02 .113

-3.23E-02 .769 5.788E-02

.772 .142 -.190

-.637 .533 .295

-.192 .384 .594

.695 -.284 .355

.831 .178 -.170

-.399 .560 .312

Exciting

Dependable

Luxurious

Outdoorsy

Powerful

Stylish

Comfortable

Rugged

Fun

Safe

Performance

Family

Versatile

Sports

Status

Practical

1 2 3

Factor



The varimax rotation makes the interpretation a bit easier. What would you name these factors?

Factor 1?

Factor 2?

Factor 3?

It is rare for the different rotational criteria to produce different results


.861 -.142 .147

.152 .640 -6.41E-02

.649 .421 -.272

7.177E-02 -3.48E-02 .860

.711 .114 .285

.877 8.168E-02 -4.19E-02

.183 .719 4.946E-03

8.375E-02 1.832E-02 .863

.862 -6.25E-02 .127

-3.20E-02 .770 4.752E-02

.775 .138 -.180

-.641 .537 .279

-.201 .392 .586

.689 -.280 .369

.834 .175 -.159

-.403 .565 .299

Exciting

Dependable

Luxurious

Outdoorsy

Powerful

Stylish

Comfortable

Rugged

Fun

Safe

Performance

Family

Versatile

Sports

Status

Practical

1 2 3

Factor

Extraction Method: Principal Axis Factoring. Rotation Method: Quartimax with Kaiser Normalization.



.864 -.142 .132

.150 .641 -5.82E-02

.644 .425 -.276

8.497E-02 -4.57E-02 .859

.715 .111 .276

.876 8.355E-02 -5.42E-02

.182 .719 1.137E-02

9.692E-02 7.389E-03 .862

.864 -6.28E-02 .113

-3.23E-02 .769 5.788E-02

.772 .142 -.190

-.637 .533 .295

-.192 .384 .594

.695 -.284 .355

.831 .178 -.170

-.399 .560 .312

Exciting

Dependable

Luxurious

Outdoorsy

Powerful

Stylish

Comfortable

Rugged

Fun

Safe

Performance

Family

Versatile

Sports

Status

Practical

1 2 3

Factor




.866 -.141 .120

.148 .642 -5.50E-02

.640 .429 -.281

9.554E-02 -5.22E-02 .857

.718 .111 .268

.875 8.611E-02 -6.42E-02

.180 .719 1.477E-02

.107 9.192E-04 .861

.865 -6.15E-02 .102

-3.34E-02 .769 6.427E-02

.769 .145 -.198

-.635 .529 .307

-.186 .379 .599

.700 -.285 .344

.829 .181 -.178

-.397 .557 .321

Exciting

Dependable

Luxurious

Outdoorsy

Powerful

Stylish

Comfortable

Rugged

Fun

Safe

Performance

Family

Versatile

Sports

Status

Practical

1 2 3

Factor

Extraction Method: Principal Axis Factoring. Rotation Method: Equamax with Kaiser Normalization.


• Advocates of factor analysis often claim that it is inappropriate to apply principal components procedures in the search for meaning or latent constructs.

• But, does it really matter all that much?

• To the extent that the communalities for all variables are high the two procedures should give very similar results.

• When commonalities are very low factor analysis results may depart from principal components.

PCA vs. FA

Communalities

1.000 .806

1.000 .572

1.000 .728

1.000 .807

1.000 .669

1.000 .798

1.000 .652

1.000 .808

1.000 .790

1.000 .686

1.000 .703

1.000 .794

1.000 .659

1.000 .750

1.000 .778

1.000 .656

Exciting

Dependable

Luxurious

Outdoorsy

Powerful

Stylish

Comfortable

Rugged

Fun

Safe

Performance

Family

Versatile

Sports

Status

Practical

Initial Extraction


Communalities

.773 .784

.414 .437

.644 .672

.687 .747

.595 .600

.767 .778

.520 .550

.701 .753

.761 .763

.539 .596

.678 .651

.720 .778

.529 .537

.671 .690

.734 .751

.581 .571

Exciting

Dependable

Luxurious

Outdoorsy

Powerful

Stylish

Comfortable

Rugged

Fun

Safe

Performance

Family

Versatile

Sports

Status

Practical

Initial Extraction


PCA FA

Component Matrixa

.873 -5.03E-03 .209

.134 .609 -.427

.658 .285 -.463

4.710E-02 .394 .806

.728 .296 .228

.885 .101 -6.72E-02

.157 .697 -.375

5.653E-02 .443 .780

.873 5.610E-02 .154

-7.26E-02 .744 -.357

.801 8.172E-02 -.233

-.684 .572 -9.99E-03

-.251 .671 .382

.710 -2.92E-02 .494

.845 .124 -.221

-.462 .665 -2.53E-03

Exciting

Dependable

Luxurious

Outdoorsy

Powerful

Stylish

Comfortable

Rugged

Fun

Safe

Performance

Family

Versatile

Sports

Status

Practical

1 2 3

Component



Factor Matrixa

.863 9.283E-03 .199

.123 .512 -.400

.636 .244 -.457

4.746E-02 .439 .743

.695 .290 .178

.873 9.610E-02 -8.02E-02

.148 .617 -.385

5.676E-02 .486 .716

.860 6.511E-02 .140

-7.07E-02 .671 -.376

.772 6.326E-02 -.227

-.675 .565 -5.54E-02

-.237 .634 .281

.691 2.867E-03 .461

.829 .109 -.229

-.439 .613 -5.37E-02

Exciting

Dependable

Luxurious

Outdoorsy

Powerful

Stylish

Comfortable

Rugged

Fun

Safe

Performance

Family

Versatile

Sports

Status

Practical

1 2 3

Factor



PCA FA

Rotated Component Matrixa

.876 -.143 .133

.159 .736 -7.42E-02

.661 .452 -.294

9.146E-02 -6.30E-02 .892

.750 .114 .306

.887 8.631E-02 -5.43E-02

.189 .785 1.353E-02

.103 -8.27E-03 .893

.879 -6.26E-02 .116

-3.71E-02 .825 6.459E-02

.798 .156 -.203

-.648 .526 .311

-.201 .400 .678

.720 -.300 .376

.845 .185 -.174

-.421 .595 .353

Exciting

Dependable

Luxurious

Outdoorsy

Powerful

Stylish

Comfortable

Rugged

Fun

Safe

Performance

Family

Versatile

Sports

Status

Practical

1 2 3

Component

Extraction Method: Principal Component Analysis. Rotation Method: Varimax with Kaiser Normalization.



.864 -.142 .132

.150 .641 -5.82E-02

.644 .425 -.276

8.497E-02 -4.57E-02 .859

.715 .111 .276

.876 8.355E-02 -5.42E-02

.182 .719 1.137E-02

9.692E-02 7.389E-03 .862

.864 -6.28E-02 .113

-3.23E-02 .769 5.788E-02

.772 .142 -.190

-.637 .533 .295

-.192 .384 .594

.695 -.284 .355

.831 .178 -.170

-.399 .560 .312

Exciting

Dependable

Luxurious

Outdoorsy

Powerful

Stylish

Comfortable

Rugged

Fun

Safe

Performance

Family

Versatile

Sports

Status

Practical

1 2 3

Factor



PCA FA

• Factor analysis offers a more realistic model of measurement than principal components analysis by admitting the presence of random and systematic error.

• Another way to make the model more realistic is to relax the restriction that factors be orthogonal. Allowing oblique factors has two potential benefits:

• It allows the model to better match the actual data

• It allows the possibility of higher order factors

FA: Oblique Rotation

A sample of 303 MBA students were asked to evaluate different makes of cars using 16 different adjectives rated on a 5-point agreement scale (1 = strongly disagree, 5 = strongly agree):

“This car is an exciting car.”

Exciting

Dependable

Luxurious

Outdoorsy

Powerful

Stylish

Comfortable

Rugged

Fun

Safe

Performance

Family

Versatile

Sports

Status

Practical

FA Example, Oblique Rotation


.864 -.142 .132

.150 .641 -5.82E-02

.644 .425 -.276

8.497E-02 -4.57E-02 .859

.715 .111 .276

.876 8.355E-02 -5.42E-02

.182 .719 1.137E-02

9.692E-02 7.389E-03 .862

.864 -6.28E-02 .113

-3.23E-02 .769 5.788E-02

.772 .142 -.190

-.637 .533 .295

-.192 .384 .594

.695 -.284 .355

.831 .178 -.170

-.399 .560 .312

Exciting

Dependable

Luxurious

Outdoorsy

Powerful

Stylish

Comfortable

Rugged

Fun

Safe

Performance

Family

Versatile

Sports

Status

Practical

1 2 3

Factor



• Oblique rotation relaxes the requirement that factors be independent. This requires the addition of a new matrix—the factor pattern matrix.

• With orthogonal rotation, because the factors are independent, the weights are simply correlations---just the elements of the factor loading (structure) matrix.

• When the factors are not orthogonal, the correlations are not the only weights considered (structure matrix) and a separate matrix containing partial weights is necessary (pattern matrix).

Because the factor pattern matrix represents the unique contribution of each factor to the reconstruction of any variable in X, it provides a better basis for judging simple structure.

Available techniques for achieving simple structure for oblique rotation (e.g., promax, direct oblimin) confront an additional problem---the specification for the amount of correlation among the factors. Unlike orthogonal rotation in which these correlations are fixed at 0, in oblique rotation, simple structure can be sought for any correlations among the factors.

Hypothetical data (N = 500) were created for individuals completing a 12-section test of mental abilities. All variables are in standard form.

Descriptive Statistics

-.0076 .97648 500

-.0038 1.06328 500

-.0148 .99894 500

-.0878 1.08634 500

.0106 .97891 500

-.0286 .96150 500

-.0151 .98903 500

-.0063 .96384 500

-.0168 .93611 500

.0008 .97184 500

-.0046 .98339 500

-.0629 1.01653 500

Grammar

ParagraphComprehension

Vocabulary

Sentence Completion

Geometry

Algebra

Numerical Puzzles

Series Completion

Practical Problem Solving

Symbol Manipulation

Analytical Ability

Formal Logic

Mean Std. Deviation Analysis N


.856

1943.111

66

.000


Approx. Chi-Square

df

Sig.


The correlation matrix is not an identity matrix:

Scree Plot

Factor Number

121110987654321

Eig

enva

lue

5

4

3

2

1

0

The scree test clearly shows the presence of three factors:

Communalities

.428 .535

.410 .518

.387 .471

.426 .547

.367 .473

.375 .475

.357 .483

.399 .508

.397 .482

.382 .498

.409 .525

.417 .525

Grammar


Vocabulary

Sentence Completion

Geometry

Algebra

Numerical Puzzles

Series Completion


Symbol Manipulation

Analytical Ability

Formal Logic

Initial Extraction


On average, the three factors extracted can account for about half of the variance in the individual subtests.


4.216 35.137 35.137 3.722 31.013 31.013 2.751

1.717 14.308 49.445 1.209 10.072 41.085 2.566

1.594 13.287 62.733 1.110 9.247 50.332 2.820

.618 5.150 67.883

.566 4.720 72.603

.541 4.512 77.115

.535 4.461 81.576

.470 3.918 85.494

.462 3.850 89.343

.450 3.751 93.094

.436 3.633 96.727

.393 3.273 100.000

Factor1

2

3

4

5

6

7

8

9

10

11

12

Total % of Variance Cumulative % Total % of Variance Cumulative % Total

Initial Eigenvalues Extraction Sums of Squared Loadings RotationSums ofSquaredLoadings

a


When factors are correlated, sums of squared loadings cannot be added to obtain a total variance.a.

Factor analysis accounts for less variance than principal components and rotation shifts the variance accounted for by the factors.

Why does the sum of the squared rotated loadings not equal the sum of the squared unrotated loadings?

Factor Matrixa

.592 -.165 -.397

.590 -8.05E-02 -.404

.570 -.162 -.347

.579 -.135 -.440

.498 .469 6.766E-02

.554 .403 7.396E-02

.466 .511 6.591E-02

.551 .434 .127

.568 -.263 .300

.525 -.287 .374

.583 -.267 .336

.590 -.275 .319

Grammar


Vocabulary

Sentence Completion

Geometry

Algebra

Numerical Puzzles

Series Completion


Symbol Manipulation

Analytical Ability

Formal Logic

1 2 3

Factor



The initial extraction . . .

Structure Matrix

.343 .278 -.731

.305 .333 -.718

.349 .270 -.685

.302 .284 -.739

.242 .687 -.276

.315 .685 -.330

.199 .692 -.243

.325 .710 -.292

.693 .278 -.342

.704 .238 -.271

.724 .290 -.335

.724 .287 -.352

Grammar


Vocabulary

Sentence Completion

Geometry

Algebra

Numerical Puzzles

Series Completion


Symbol Manipulation

Analytical Ability

Formal Logic

1 2 3

Factor

Extraction Method: Principal Axis Factoring. Rotation Method: Oblimin with Kaiser Normalization.

Pattern Matrixa

2.531E-02 -2.44E-02 -.730

-3.52E-02 6.073E-02 -.709

5.689E-02 -1.89E-02 -.667

-3.50E-02 -7.43E-03 -.758

-2.10E-02 .692 -7.24E-03

5.106E-02 .647 -4.76E-02

-6.35E-02 .723 1.929E-02

7.498E-02 .689 1.862E-02

.673 1.029E-02 -3.65E-02

.733 -1.64E-02 5.129E-02

.713 1.881E-02 -8.38E-03

.706 9.512E-03 -3.13E-02

Grammar


Vocabulary

Sentence Completion

Geometry

Algebra

Numerical Puzzles

Series Completion


Symbol Manipulation

Analytical Ability

Formal Logic

1 2 3

Factor



Oblique rotation is much clearer in the pattern matrix than in the structure matrix:

Factor Correlation Matrix

1.000 .376 -.448

.376 1.000 -.402

-.448 -.402 1.000

Factor1

2

3

1 2 3


The correlations among the factors suggest the presence of a higher order factor:

Why are some of the correlations negative when one would expect all mental abilities to be positively correlated?

Pattern Matrixa

.731 2.489E-02 -2.67E-02

.710 -3.66E-02 5.939E-02

.668 5.664E-02 -2.12E-02

.759 -3.59E-02 -9.24E-03

4.785E-03 -2.72E-02 .696

4.504E-02 4.560E-02 .649

-2.17E-02 -7.01E-02 .727

-2.15E-02 6.933E-02 .692

3.360E-02 .676 4.904E-03

-5.45E-02 .737 -2.22E-02

5.235E-03 .717 1.322E-02

2.829E-02 .710 3.868E-03

Grammar


Vocabulary

Sentence Completion

Geometry

Algebra

Numerical Puzzles

Series Completion


Symbol Manipulation

Analytical Ability

Formal Logic

1 2 3

Factor

Extraction Method: Principal Axis Factoring. Rotation Method: Promax with Kaiser Normalization.


Pattern Matrixa

2.531E-02 -2.44E-02 -.730

-3.52E-02 6.073E-02 -.709

5.689E-02 -1.89E-02 -.667

-3.50E-02 -7.43E-03 -.758

-2.10E-02 .692 -7.24E-03

5.106E-02 .647 -4.76E-02

-6.35E-02 .723 1.929E-02

7.498E-02 .689 1.862E-02

.673 1.029E-02 -3.65E-02

.733 -1.64E-02 5.129E-02

.713 1.881E-02 -8.38E-03

.706 9.512E-03 -3.13E-02

Grammar


Vocabulary

Sentence Completion

Geometry

Algebra

Numerical Puzzles

Series Completion


Symbol Manipulation

Analytical Ability

Formal Logic

1 2 3

Factor



An alternative oblique rotation---Promax---provides much the same answer:

The order of the factors may vary, and one gets reflected, but the essential interpretation is the same.


1.000 .455 .410

.455 1.000 .392

.410 .392 1.000

Factor1

2

3

1 2 3

Extraction Method: Principal Axis Factoring. Rotation Method: Promax with Kaiser Normalization.


1.000 .376 -.448

.376 1.000 -.402

-.448 -.402 1.000

Factor1

2

3

1 2 3


The correlations among the factors are similar for the two procedures.

Exploratory factor analytic methods are sometimes used as a crude way to confirm hypotheses about the latent structure underlying a data set. As a first pass, these methods do just fine. But, more powerful confirmatory factor analytic procedures exist that can better address questions about data that are strongly informed by theory.

Documents

Exploratory Factor Analysis. Principal components analysis seeks linear combinations that best capture the variation in the original variables. Factor