30 MAY 2013 2012 COPYRIGHT SAS INSTITUTE 1

Statistical Text Mining

and Semantics

Torulf Mollestad, Ph.D. SAS Institute Nordic

Syntax, Semantics and Pragmatics

Syntax, Semantics and Pragmatics

Data Mining

Text Mining

Three documents

Six terms

Text Mining

dock river boat borrow credit cash

Doc 1 6 5 2 0 0 1

Doc 2 5 3 4 0 1 0

Doc 3 1 1 0 4 2 7

PC1: Banking/loan

PC2: Boat life

Text Mining

dock river boat borrow credit cash

Doc 1 6 5 2 0 0 1

Doc 2 5 3 4 0 1 0

Doc 3 1 1 0 4 2 7

Text Mining

Astronaut

Space

Rocket

Cosmonaut

Space travel

I got a flu so i couldn’t feed my swine…

I got swine flu…

Text Mining

I got sick and took my aspirin

I took my aspirin and got sick

I didn’t take my aspirin, and got sick

Text Mining

Text Mining

Public Security

Predictive Asset Maintenance

Events over time

Gas flow J_43_FI_81164

high!

low!

unstable!

Rules and usable patterns

Rules generated, e.g.:

IF

J_43_XI_21091VF_v_gt95p and

J_43_LIC_00419SP_j_gt95p

THEN

low_water_closed(0.5, 0.7)

(within 0.5hour, probability 0.7)

The operator: Supervision and Diagnostics

Syntax, Semantics and Pragmatics

…Statistics…