18
What is bigdata inference? Carey E. Priebe Department of Applied Mathematics & Statistics Johns Hopkins University October 30, 2012 DARPA Big Data Colloquium 2012

DARPA Big Data Colloquium 2012 - Department of Applied ...priebe/.FILES/cep-xdata-keynote.pdf · What is bigdata inference? Carey E. Priebe Department of Applied Mathematics & Statistics

Embed Size (px)

Citation preview

Page 1: DARPA Big Data Colloquium 2012 - Department of Applied ...priebe/.FILES/cep-xdata-keynote.pdf · What is bigdata inference? Carey E. Priebe Department of Applied Mathematics & Statistics

What is bigdata inference?

Carey E. Priebe Department of Applied Mathematics & Statistics

Johns Hopkins University

October 30, 2012

DARPA Big Data Colloquium 2012

Page 2: DARPA Big Data Colloquium 2012 - Department of Applied ...priebe/.FILES/cep-xdata-keynote.pdf · What is bigdata inference? Carey E. Priebe Department of Applied Mathematics & Statistics

Kronecker Quote

“The wealth of your practical experience with sane and interesting problems

will give to mathematics a new direction and a new impetus.”

– Leopold Kronecker to Hermann von Helmholtz –

Page 3: DARPA Big Data Colloquium 2012 - Department of Applied ...priebe/.FILES/cep-xdata-keynote.pdf · What is bigdata inference? Carey E. Priebe Department of Applied Mathematics & Statistics

What is mathematics?

“The wealth of your practical experience with sane and interesting problems

will give to mathematics a new direction and a new impetus.”

Page 4: DARPA Big Data Colloquium 2012 - Department of Applied ...priebe/.FILES/cep-xdata-keynote.pdf · What is bigdata inference? Carey E. Priebe Department of Applied Mathematics & Statistics

What is mathematics?

“The wealth of your practical experience with sane and interesting problems

will give to mathematics a new direction and a new impetus.”

With appologies toRichard Courant

andHerbert Robbins

Page 5: DARPA Big Data Colloquium 2012 - Department of Applied ...priebe/.FILES/cep-xdata-keynote.pdf · What is bigdata inference? Carey E. Priebe Department of Applied Mathematics & Statistics

What is mathematics?

“The wealth of your practical experience with sane and interesting problems

will give to mathematics a new direction and a new impetus.”

Mreality

Page 6: DARPA Big Data Colloquium 2012 - Department of Applied ...priebe/.FILES/cep-xdata-keynote.pdf · What is bigdata inference? Carey E. Priebe Department of Applied Mathematics & Statistics

What is mathematics?

“The wealth of your practical experience with sane and interesting problems

will give to mathematics a new direction and a new impetus.”

Mreality

Mmathematics

Page 7: DARPA Big Data Colloquium 2012 - Department of Applied ...priebe/.FILES/cep-xdata-keynote.pdf · What is bigdata inference? Carey E. Priebe Department of Applied Mathematics & Statistics

What is mathematics?

“The wealth of your practical experience with sane and interesting problems

will give to mathematics a new direction and a new impetus.”

Applied

^Mreality

Mmathematics

Page 8: DARPA Big Data Colloquium 2012 - Department of Applied ...priebe/.FILES/cep-xdata-keynote.pdf · What is bigdata inference? Carey E. Priebe Department of Applied Mathematics & Statistics

What is mathematics?

“The wealth of your practical experience with sane and interesting problems

will give to mathematics a new direction and a new impetus.”

Applied

^Mreality

Mmathematics

Page 9: DARPA Big Data Colloquium 2012 - Department of Applied ...priebe/.FILES/cep-xdata-keynote.pdf · What is bigdata inference? Carey E. Priebe Department of Applied Mathematics & Statistics

sensors bigdata

bigdataprocessing

bigdatainference

decisions /decision makers

collection management /sensor deployment

(MMDDDDS)

universe

Page 10: DARPA Big Data Colloquium 2012 - Department of Applied ...priebe/.FILES/cep-xdata-keynote.pdf · What is bigdata inference? Carey E. Priebe Department of Applied Mathematics & Statistics

sensors bigdata

bigdataprocessing

bigdatainference

decisions /decision makers

collection management /sensor deployment

DARPAXDATA

universe

Page 11: DARPA Big Data Colloquium 2012 - Department of Applied ...priebe/.FILES/cep-xdata-keynote.pdf · What is bigdata inference? Carey E. Priebe Department of Applied Mathematics & Statistics

sensors bigdata

bigdataprocessing

bigdatainference

decisions /decision makers

collection management /sensor deployment

universe

Page 12: DARPA Big Data Colloquium 2012 - Department of Applied ...priebe/.FILES/cep-xdata-keynote.pdf · What is bigdata inference? Carey E. Priebe Department of Applied Mathematics & Statistics

Extract {Vi, xi, ti }i∈I

time

h:

Fusion and Inference from Multiple and Massive Disparate Distributed

Dynamic Data Sets

Generate Time Series of Attributed Graphs

Page 13: DARPA Big Data Colloquium 2012 - Department of Applied ...priebe/.FILES/cep-xdata-keynote.pdf · What is bigdata inference? Carey E. Priebe Department of Applied Mathematics & Statistics

time

h:

Fusion and Inference from Multiple and Massive Disparate Distributed

Dynamic Data Sets

�Generate Time Series of Attributed Graphs

⌅1 ⇥ · · ·⇥ ⌅K

Extract {Vi, xi, ti }i∈I

Page 14: DARPA Big Data Colloquium 2012 - Department of Applied ...priebe/.FILES/cep-xdata-keynote.pdf · What is bigdata inference? Carey E. Priebe Department of Applied Mathematics & Statistics

h: �

Fusion and Inference from Multiple and Massive Disparate Distributed

Dynamic Data Sets

⌅1 ⇥ · · ·⇥ ⌅K

Extract {Vi, xi, ti }i∈I

professorstudent

time

Page 15: DARPA Big Data Colloquium 2012 - Department of Applied ...priebe/.FILES/cep-xdata-keynote.pdf · What is bigdata inference? Carey E. Priebe Department of Applied Mathematics & Statistics

Anomaly Detection in Time Series of Attributed Graphs

h: �

Fusion and Inference from Multiple and Massive Disparate Distributed

Dynamic Data Sets

time

⌅1 ⇥ · · ·⇥ ⌅K

Extract {Vi, xi, ti }i∈I

Page 16: DARPA Big Data Colloquium 2012 - Department of Applied ...priebe/.FILES/cep-xdata-keynote.pdf · What is bigdata inference? Carey E. Priebe Department of Applied Mathematics & Statistics

< > - +

The curse of dimensionality

g!

M(d)

!gn(d)

g!(d)

L(!gn(d)) "#d#$

12

L(g!(d)) "#d#$ 0

d0

12

L(!gn(d))

doptn

October 2003 – p.5/35

The Curse of Dimensionality

Dennis M. Healy

Page 17: DARPA Big Data Colloquium 2012 - Department of Applied ...priebe/.FILES/cep-xdata-keynote.pdf · What is bigdata inference? Carey E. Priebe Department of Applied Mathematics & Statistics

synapsedetection

computer vision tracking of axon & dendrite to

neurons associated with synapse

graphconstruction

volume synapses neurons graph

Connectome Example

Page 18: DARPA Big Data Colloquium 2012 - Department of Applied ...priebe/.FILES/cep-xdata-keynote.pdf · What is bigdata inference? Carey E. Priebe Department of Applied Mathematics & Statistics

Kronecker Quote

“The wealth of your practical experience with sane and interesting problems

will give to mathematics a new direction and a new impetus.”

– Leopold Kronecker to Hermann von Helmholtz –