Upload
gladys-sparks
View
213
Download
0
Tags:
Embed Size (px)
Citation preview
Data Visualization
Daniel SilverMarch, 2014
[number of slides courtesy of Stan Matwin]
The KDD Process
Selection and Preprocessing
Data Mining
Interpretation and Evaluation
Data Consolidation
Knowledge
p(x)=0.02
DataWarehouse
Data Sources
Patterns & Models
Prepared Data
ConsolidatedData
The KDD Process
The Architecture of a KDD System
Graphical User Interface
DataConsolidation
Selectionand
Preprocessing
DataMining
Interpretation, Visualization
and Evaluation
Warehouse KnowledgeData Sources
Big Data, Saint John, NB 4
Data Visualization
• The greatest value of a picture is when it forces us to notice what we never expected to see – J. Tukey
• Visual analytics can lead to discoveries that neither a computer nor a human could make alone – J. Fekete
• Let your data talk to you – J. Stasko
Big Data, Saint John, NB 5
What is visualization?
• Graphical representation of data and relationships within data
• Why?– right (imagery) and left (analytical) brain
hemisphere• What makes a good visualization?– Informative – Aesthetically pleasing– Often, the right abstraction of the data
Big Data, Saint John, NB 6
Windmap
• http://hint.fm/wind/ [Martin Wattenberg]– Captures the “Big Picture” from the data– Data from the National Digital Forecast Database– Exhibited in MOMA as graphical art
Big Data, Saint John, NB 7
Visualization as a tool for…
• Data exploration• Communication of results
Big Data, Saint John, NB 8
How best to visualize?
Art is good in conveying
complex concepts
Big Data, Saint John, NB 9
Advertising uses great visualizations…
Big Data, Saint John, NB 10
But is it that easy …
• … art conveys emotions, mental states…
• Visualizing data is different, but still…– A good visualization should require no explanation– Allows us to see the data “in a new light”
Big Data, Saint John, NB 11
Geographic misconceptions…
• The True Size of Africa [Kai Kruse]• http://flowingdata.com/2010/10/18/true-size-
of-africa/
Big Data, Saint John, NB 12
Conveying Complex Dynamics
• Projecting on the time axis– Using colour, thickness, etc. to combine and
overlay data
Big Data, Saint John, NB 13
[Minard 1896!]
Big Data, Saint John, NB 14
Email “mountain” [F. Viergas] -making sense of an email archive
Big Data, Saint John, NB 15
Visual exploration process [Yau, Data Points, 2013]
Big Data, Saint John, NB 16
Exploring data
• ACOA database of 28,000 projects• Contains – Client’s address, postal code– Amount of funding– Project description– Program type– ….
Big Data, Saint John, NB 17
ACOA data
Project Number Client Name Client Address Client City Client Postal Code Project Description Project Location Project Geographic Code Program Type Assistance Type ACOA Assistance
197595 (ISIS) Immigrant Settlement and Integration Services Society Halifax, B3L 4P1 Halifax B3L 4P1 IBDS funding for 2010 - 2011 HALIFAX 1209034
Business Development Program
Non-Repayable Contribution 149,832.00
198632 (ISIS) Immigrant Settlement and Integration Services Society Halifax, B3L 4P1 Halifax B3L 4P1Immigrant entrepreneur and business succession integration
ATLANTIC REGION 1400000
BDP - Atlantic Population Strategy
Non-Repayable Contribution 14,040.00
200029 (ISIS) Immigrant Settlement and Integration Services Society Halifax, B3L 4P1 Halifax B3L 4P1The Immigrant Women Entrepreneurial Project (IWEP)
ATLANTIC REGION 1400000
BDP - Atlantic Population Strategy
Non-Repayable Contribution 79,607.00
157116 * Quidi Vidi Rennies River Development FoundationSt. John's, A1B 2Z2 St. John's A1B 2Z2 Environmental Education Expansion ST. JOHN'S 1001519 Partnerships
Non-Repayable Contribution 35,000.00
154136 * Burin Peninsula Community Business Development Corporation Inc.Marystown, A0K 2M0 Marystown A0K 2M0 implement a youth business program MARYSTOWN 1002024
ACOA - Action Program (AAP) Grant 6,800.00
154135 * Burin Peninsula Community Business Development Corporation Inc.Marystown, A0K 2M0 Marystown A0K 2M0 Youth Internship Program - Phase 3 MARYSTOWN 1002024 Community Futures
Non-Repayable Contribution 15,001.00
113504 042691 N. B. LtéePetit Rocher Nord, E8J 2K7 Petit Rocher Nord E8J 2K7
To acquire equipment & make changes to the ventilation,water PETIT-ROCHER 1315014
ACOA - Action Program (AAP)
Unconditionally Repayable Contribution 9,520.00
185785 047100 N.B. Inc.Bouctouche, E4S 2J2 Bouctouche E4S 2J2 Lean Analysis BOUCTOUCHE 1308005
AIP - ISDI - SME Technology Advancement
Non-Repayable Contribution 22,413.00
203111 051996 N.B. Ltd.Mundleville, E4W 2N5 Mundleville E4W 2N5
Building expansion and purchase of equipment for winery RICHIBUCTO 1308018
Business Development Program
Unconditionally Repayable Contribution 74,618.00
167460 052412 N-B LtéeTracadie-Sheila, E1X 1G6 Tracadie-Sheila E1X 1G6
Expansion - pool, washrooms, electrical, road upgrades.... SAUMAREZ 1315001
Business Development Program
Unconditionally Repayable Contribution 18,400.00
167466 052412 N-B LtéeTracadie-Sheila, E1X 1G6 Tracadie-Sheila E1X 1G6
Chalet/Campground expansion - adding activities SAUMAREZ 1315001
Business Development Program
Unconditionally Repayable Contribution 210,000.00
183896 053247 N.B. LtéeSt-Quentin, E8A 2H9 St-Quentin E8A 2H9 Construction of potato storage SAINT-QUENTIN 1314022
Business Development Program
Unconditionally Repayable Contribution 183,390.00
168364 054000 NB Inc.Beresford, E0B 1H0 Beresford E0B 1H0 Purchase 2 blueberry harvestors BERESFORD 1315015 Agriculture
Non-Repayable Contribution 182
169969 056393 N.B. Ltd.Bathurst, E2A 4Z2 Bathurst E2A 4Z2 Estab. of Beach Cleaning Service BATHURST 1315011
ACOA - Action Program (AAP)
Non-Repayable Contribution 38,729.00
Big Data, Saint John, NB 18
Geo mapping
• We map the Postal code of each ACOA project to geographic coordinates using the PCCF Canada Post database
Big Data, Saint John, NB 19
ACOA Data [Tableau]
Big Data, Saint John, NB 20
ACOA Program Types
• Business Development Program
• Productivity and Business Skills (PBS)
• ACOA Action Program (AAP)
• Other
Big Data, Saint John, NB 21
ACOA data – geolocated by project type
Big Data, Saint John, NB 22
Visualization and text
WordcloudwithR
Big Data, Saint John, NB 23
Beyond wordcloud
• How to characterize at a high level what the project is all about?
• Find, by analyzing all project descriptions, high level “clusters” (“topics”) of descriptions can be identified
Big Data, Saint John, NB 24
5 LDA topics for ACOA projectsTopic 1 Topic 2 Topic 3 Topic 4 Topic 5
conference equipment equipment activities program
development development activities existing development
existing plan plan equipment business
facility facility business new system
equipment business host host new
system expansion centre development equipment
plan operation expansion training market
study new new operation study
activities activities market business tourism
project centre project support international
event equipment-add equipment-new training-support study+tourism
Big Data, Saint John, NB 25
So where is the future? - challenges
• Visualization of stream/dynamic data (animation?)
• Visualization of very large graphs• Visualization of multi-dimensional data (see
OECD life quality demo)• Visualization for mobile devices (limited
screen sizes, limited power)• Evaluation of visualization