PowerPoint Presentation

How to Feed Good Datato Hungry, Heartless Algorithms

The Mind of a Data ScientistBSc in Cognitive ScienceDiploma in Communication Disorders Hearing Instrument SpecialistMI (Critical Information Studies)Co-authored paper on Queer Theory in Self Surveillance on Bio-Hacking & Health Informatics Mediation of self through technology, through self-surveillance & data

For Example.

A Tour of Terms

Big Data

6

2012-02-03 18:35:34 SampleClass6 [INFO] everything normal for id 5777258512012-02-03 18:35:34 SampleClass4 [FATAL] system problem at id 19912812542012-02-03 18:35:34 SampleClass3 [DEBUG] detail for id 13048076562012-02-03 18:35:34 SampleClass3 [WARN] missing id 4233408952012-02-03 18:35:34 SampleClass5 [TRACE] verbose detail for id 20826549782012-02-03 18:35:34 SampleClass0 [ERROR] incorrect id 18864385132012-02-03 18:35:34 SampleClass9 [TRACE] verbose detail for id 4386342092012-02-03 18:35:34 SampleClass8 [DEBUG] detail for id 20741213102012-02-03 18:35:34 SampleClass0 [TRACE] verbose detail for id 15055825082012-02-03 18:35:34 SampleClass0 [TRACE] verbose detail for id 19038544372012-02-03 18:35:34 SampleClass7 [DEBUG] detail for id 9158531412012-02-03 18:35:34 SampleClass3 [TRACE] verbose detail for id 3031324012012-02-03 18:35:34 SampleClass6 [TRACE] verbose detail for id 1519143692012-02-03 18:35:34 SampleClass2 [DEBUG] detail for id 1465277422012-02-03 18:35:34 SampleClass5 [TRACE] verbose detail for id 11472756562012-02-03 18:35:34 SampleClass0 [TRACE] verbose detail for id 16393225872012-02-03 18:35:34 SampleClass0 [DEBUG] detail for id 19474484652012-02-03 18:35:34 SampleClass8 [INFO] everything normal for id 1462538492012-02-03 18:35:34 SampleClass9 [TRACE] verbose detail for id 16089573782012-02-03 18:35:34 SampleClass0 [TRACE] verbose detail for id 11838082802012-02-03 18:35:34 SampleClass4 [DEBUG] detail for id 13955857662012-02-03 18:35:34 SampleClass4 [TRACE] verbose detail for id 1559830962012-02-03 18:35:34 SampleClass5 [TRACE] verbose detail for id 16328849482012-02-03 18:35:34 SampleClass2 [DEBUG] detail for id 1106822372012-02-03 18:35:34 SampleClass3 [TRACE] verbose detail for id 17397867362012-02-03 18:35:34 SampleClass5 [TRACE] verbose detail for id 7923091182012-02-03 18:35:34 SampleClass1 [DEBUG] detail for id 11428023772012-02-03 18:35:34 SampleClass2 [TRACE] verbose detail for id 15590250432012-02-03 18:35:34 SampleClass7 [TRACE] verbose detail for id 407523152012-02-03 18:35:34 SampleClass8 [DEBUG] detail for id 4997376442012-02-03 18:35:34 SampleClass6 [TRACE] verbose detail for id 4003421532012-02-03 18:35:34 SampleClass4 [INFO] everything normal for id 18684905432012-02-03 18:35:34 SampleClass2 [DEBUG] detail for id 20598124482012-02-03 18:35:34 SampleClass5 [TRACE] verbose detail for id 7664237922012-02-03 18:35:34 SampleClass3 [TRACE] verbose detail for id 6716742052012-02-03 18:35:34 SampleClass3 [DEBUG] detail for id 21462636242012-02-03 18:35:34 SampleClass1 [TRACE] verbose detail for id 6194723732012-02-03 18:35:34 SampleClass5 [TRACE] verbose detail for id 13914298712012-02-03 18:35:34 SampleClass1 [DEBUG] detail for id 15229686902012-02-03 18:35:34 SampleClass1 [TRACE] verbose detail for id 1732167082012-02-03 18:35:34 SampleClass5 [TRACE] verbose detail for id 10898642222012-02-03 18:35:34 SampleClass0 [DEBUG] detail for id 7784599262012-02-03 18:35:34 SampleClass2 [TRACE] verbose detail for id 7751581072012-02-03 18:35:34 SampleClass8 [TRACE] verbose detail for id 11431655012012-02-03 18:35:34 SampleClass5 [DEBUG] detail for id 15338731332012-02-03 18:35:34 SampleClass9 [INFO] everything normal for id 19166666272012-02-03 18:35:34 SampleClass7 [TRACE] verbose detail for id 15307261182012-02-03 18:35:34 SampleClass3 [TRACE] verbose detail for id 382432572012-02-03 18:35:34 SampleClass7 [DEBUG] detail for id 12689395922012-02-03 18:35:34 SampleClass0 [TRACE] verbose detail for id 3098151982012-02-03 18:35:34 SampleClass3 [TRACE] verbose detail for id 4683333592012-02-03 18:35:34 SampleClass8 [DEBUG] detail for id 18259505242012-02-03 18:35:34 SampleClass1 [TRACE] verbose detail for id 14379661132012-02-03 18:35:34 SampleClass9 [TRACE] verbose detail for id 2241289312012-02-03 18:35:34 SampleClass2 [DEBUG] detail for id 15360414812012-02-03 18:35:34 SampleClass6 [TRACE] verbose detail for id 5657822172012-02-03 18:35:34 SampleClass0 [DEBUG] detail for id 11739211592012-02-03 18:35:34 SampleClass0 [TRACE] verbose detail for id 12709491412012-02-03 18:35:34 SampleClass8 [TRACE] verbose detail for id 20130599902012-02-03 18:35:34 SampleClass5 [INFO] everything normal for id 7395876262012-02-03 18:35:34 SampleClass3 [DEBUG] detail for id 5521301532012-02-03 18:35:34 SampleClass7 [TRACE] verbose detail for id 11002322572012-02-03 18:35:34 SampleClass5 [TRACE] verbose detail for id 13956244162012-02-03 18:35:34 SampleClass9 [DEBUG] detail for id 3421637452012-02-03 18:35:34 SampleClass3 [TRACE] verbose detail for id 6175615882012-02-03 18:35:34 SampleClass7 [TRACE] verbose detail for id 13109517872012-02-03 18:35:34 SampleClass6 [DEBUG] detail for id 20027401532012-02-03 18:35:34 SampleClass0 [TRACE] verbose detail for id 15575754162012-02-03 18:35:34 SampleClass9 [TRACE] verbose detail for id 19832506812012-02-03 18:35:34 SampleClass6 [DEBUG] detail for id 11732500862012-02-03 18:35:34 SampleClass9 [TRACE] verbose detail for id 17980666352012-02-03 18:35:34 SampleClass0 [TRACE] verbose detail for id 9319395522012-02-03 18:35:34 SampleClass6 [DEBUG] detail for id 1343204412012-02-03 18:35:34 SampleClass4 [TRACE] verbose detail for id 4224280242012-02-03 18:35:34 SampleClass7 [INFO] everything normal for id 20508323962012-02-03 18:35:34 SampleClass2 [TRACE] verbose detail for id 4354119142012-02-03 18:35:34 SampleClass2 [DEBUG] detail for id 13751532882012-02-03 18:35:34 SampleClass7 [TRACE] verbose detail for id 12857190392012-02-03 18:35:34 SampleClass5 [TRACE] verbose detail for id 1454028842012-02-03 18:35:34 SampleClass6 [DEBUG] detail for id 15646583462012-02-03 18:35:34 SampleClass8 [TRACE] verbose detail for id 19329597192012-02-03 18:35:34 SampleClass7 [DEBUG] detail for id 8476237952012-02-03 18:35:34 SampleClass0 [TRACE] verbose detail for id 5596178792012-02-03 18:35:34 SampleClass4 [TRACE] verbose detail for id 7656270692012-02-03 18:35:34 SampleClass6 [DEBUG] detail for id 12240923202012-02-03 18:35:34 SampleClass5 [TRACE] verbose detail for id 12587726572012-02-03 18:35:34 SampleClass9 [TRACE] verbose detail for id 11138420962012-02-03 18:35:34 SampleClass7 [INFO] everything normal for id 7816239642012-02-03 18:35:34 SampleClass3 [DEBUG] detail for id 14266031652012-02-03 18:35:34 SampleClass7 [TRACE] verbose detail for id 19125287622012-02-03 18:35:34 SampleClass9 [TRACE] verbose detail for id 16161640552012-02-03 18:35:34 SampleClass3 [DEBUG] detail for id 3623820782012-02-03 18:35:34 SampleClass2 [TRACE] verbose detail for id 4570294892012-02-03 18:35:34 SampleClass3 [TRACE] verbose detail for id 2183934622012-02-03 18:35:34 SampleClass0 [DEBUG] detail for id 6458487042012-02-03 18:35:34 SampleClass4 [TRACE] verbose detail for id 3714343952012-02-03 18:35:34 SampleClass8 [TRACE] verbose detail for id 6863643172012-02-03 18:35:34 SampleClass6 [DEBUG] detail for id 10303684972012-02-03 18:35:34 SampleClass4 [TRACE] verbose detail for id 4113977302012-02-03 18:35:34 SampleClass4 [TRACE] verbose detail for id 5914460312012-02-03 18:35:34 SampleClass8 [DEBUG] detail for id 17947738772012-02-03 18:35:34 SampleClass2 [TRACE] verbose detail for id 10899709472012-02-03 18:35:34 SampleClass3 [INFO] everything normal for id 5473139342012-02-03 18:35:34 SampleClass9 [DEBUG] detail for id 165982312012-02-03 18:35:34 SampleClass3 [TRACE] verbose detail for id 3608638852012-02-03 18:35:34 SampleClass1 [TRACE] verbose detail for id 7083692572012-02-03 18:35:34 SampleClass1 [DEBUG] detail for id 616747982012-02-03 18:35:34 SampleClass6 [TRACE] verbose detail for id 7725514552012-02-03 18:35:34 SampleClass0 [TRACE] verbose detail for id 2109068542012-02-03 18:35:34 SampleClass3 [DEBUG] detail for id 520233982012-02-03 18:35:34 SampleClass2 [TRACE] verbose detail for id 14866608782012-02-03 18:35:34 SampleClass2 [TRACE] verbose detail for id 11584506612012-02-03 18:35:34 SampleClass0 [DEBUG] detail for id 11909779322012-02-03 18:35:34 SampleClass2 [TRACE] verbose detail for id 11917573492012-02-03 18:35:34 SampleClass5 [TRACE] verbose detail for id 15567514712012-02-03 18:35:34 SampleClass0 [INFO] everything normal for id 20448923372012-02-03 18:35:34 SampleClass1 [DEBUG] detail for id 21302933352012-02-03 18:35:34 SampleClass9 [TRACE] verbose detail for id 20638265252012-02-03 18:35:34 SampleClass4 [TRACE] verbose detail for id 9883043292012-02-03 18:35:34 SampleClass7 [DEBUG] detail for id 3725876352012-02-03 18:35:34 SampleClass8 [TRACE] verbose detail for id 7575096582012-02-03 18:35:34 SampleClass6 [TRACE] verbose detail for id 11891641692012-02-03 18:35:34 SampleClass4 [DEBUG] detail for id 16206994492012-02-03 18:35:34 SampleClass5 [TRACE] verbose detail for id 14061486142012-02-03 18:35:34 SampleClass2 [TRACE] verbose detail for id 9076855292012-02-03 18:35:34 SampleClass7 [DEBUG] detail for id 17404901432012-02-03 18:35:34 SampleClass1 [TRACE] verbose detail for id 7018895732012-02-03 18:35:34 SampleClass2 [TRACE] verbose detail for id 73960322012-02-03 18:35:34 SampleClass5 [DEBUG] detail for id 7918406602012-02-03 18:35:34 SampleClass8 [INFO] everything normal for id 18578852622012-02-03 18:35:34 SampleClass4 [TRACE] verbose detail for id 7641826142012-02-03 18:35:34 SampleClass0 [DEBUG] detail for id 1768055227

1 microsecond sample log file for MS HD Insights

Big DataWhen there is enough data that it must be mediated to even be loosely understood.

http://crimsonhints.deviantart.com/

https://www.ljll.math.upmc.fr/perronnet/NLSE/NLSEimpl01.png

http://infolab.stanford.edu/~backrub/google.htmlPR(A) = (1-d) + d (PR(T1)/C(T1) + ... + PR(Tn)/C(Tn))

Data Science

Data ScienceThe actual application of the scientific method to business questions.

Scientific MethodA way to approach research so you dont just make shit up.

Data Science:What it Looks Like in Creative Tech

Work you do

Way it is measured

The way you are measured doing it

Infographics

Visuals processed 60,000x faster than textTypical recall for text averages 20%So far, few show any data science results but this will change soon!

Metrics

All digital content, including website performance, social platforms etcDeciding what to measure is tricky So far, we only occasionally use really cool data science stuff- but its growing fast

Tracking

IoT & Quantified Self in the WorkplacePartly hype, partly already here, partly inevitable What does a performance review look like in five years?

Data Science:Why You Should Care

VocabularyVisibilityValues

The Hype

the corporate equation

http://sculptwerks.deviantart.com/

Three Fallacies

http://www.kids-pages.com/folders/colpages/Fairy-Tales/magic-wand.gif

Input Algorithm Output

Input Algorithm Output

Are they really heartless?

Credit: Jonty Wareinghttps://www.flickr.com/photos/electrichamster/

http://news.discovery.com/earth/global-warming/11-billion-people-climate-change-131122.htm

Bad Data

Data Will Always Be:IncompleteMessyHard To Understand

How It Goes Wrong

(Think instant magic when you hear data science)

(Ignore all sci-fi, ever)

(Question nothing)

(Forget all about statistics)

(Dont even ask a question)

(Make sure you have no idea what you have)

(make sure you never wonder about what you actually need)

Believe all hypeRefuse to consider that operating facts might be wrongBias your data all to hell and dont even realize itForget to ask why youre even doing thisDont understand what you haveDont know what you needDont take the time to figure it out

Steps to Making it Work

Ask questionsTake a serious look at how you talk about it nowTake a serious look at the way you measure itLook at the data you have nowFigure out what it means (not in this alone)Decide if youre missing stuff you needGet that stuffDo science!! (hypothesis, method, test, discuss, repeat)SHAREPresent & evangelize

Complexity is a Design ChallengeMetaphors used to discuss data science (needle in a haystack, mining for gold) prevent this from being a collaborative exploration by a whack of cool people, in a room to discuss a problem together.Instead, we think of this as a very solitary, linear, (painful, boring, difficult, awful) process.Which it isnt. Its really cool. But we need everyone to come to the party to make it an epic party.

Why You Should Care (Reprise)Its the work you doWay it is measuredThe way you are measured doing it