Upload
hisato-matsuo
View
280
Download
0
Embed Size (px)
Citation preview
Graph Analysis & High-Performance Computing Techniques for Realizing Urban OS
Katsuki FujisawaHisato P. Matsuo
1
Kyushu Universityin Fukuoka
Katsuki Fujisawa
Hisato Peter Matsuo
Presenters
2014
Center of Innovation Project
1998
Received Ph.D.
Full Professor,Institute of Mathematics for Industry (IMI),Kyushu University
Joined IBM
Research Fellow,Center for Co-Evolutional Social Systems,Kyushu University
- Research Director of the JST CREST for Post-Peta HPC- Graph500 Winner / Green Graph500 3rd winner in 2014
- Memory system Architect for Storage subsystem- IaaS/PaaS product Consultant-> now Urban OS Designer
Joined Kyushu-U as Full Professor
Left IBM, Joined Kyushu-U
2022Now
Agenda
Urban OS that realizes next generation Smart City
Architecture and Infrastructure
Software architecture and Analytic system
Graph Analysis & HPC
Summary and our goal
Energy Supply-Demand BalanceTraffic Congestion
Urban Issues/Challenges
・ Energy issues・ Environment, water, sanitation & hygiene・ Disaster control
・ Population decline・ Low birth rate & rapid aging・ Urban concentration
・ Traffic problems・ Means of mobility・ Information access
・ Globalization・ Diversity・ Information divide
・ Finance・ Industrial promotion・ Gov. administration・ Education/Welfare・ Innovation
Urban OS provides three Mobility’sAnyone can access … anytime,
anywhere
Urban OS
Urban OSPeople/Materials mobility
on-demand and effective transportation
Energy mobilitysecured energy supply
Information mobilityappropriate information
Three Mobility’s lead sustainable society
People/MaterialMobility
InformationMobility
EnergyMobility
Efficient & optimized
Infrastructure
CreativeCommunity
Efficient & flexible Energy
Agenda
Urban OS that realizes next generation Smart City
Architecture and Infrastructure
Software architecture and Analytic system
Graph Analysis & HPC
Summary and our goal
Urban OS Functions
Advanced future social services createdWith optimization/analytic engines
Event secu-rity plan
Complaintresponse
Trafficinformation
UrbanOS
Social services that utilizesThe platform data
Flexible energydemand response
Effective eva-cuation plan
Smart trafficcontrol
Trafficdata
Weather/Disaster data
Gov./Publicdata
Energydata
Persondata
Open Data
InformationFeedback
Co-evolutionalSociety
Urban O S
Cross-utilization of variousdata
Automatic optimization,control & bottleneck analysis
Open platform forsocial/public/commercial
applications
Big data / Open dataSensor Network
Application Service
Optimization/Analytic
Data Store
Data
Open platform for advanced urban services
Urban OS Architecture
Data for Person/Traffic/City plan optimization
Data Example : Public Open dataGovernment Open data in Fukuoka city
Map Mashup
Utilized ApplicationsData Catalogue
Dataset Search
• Open data– Census– Statistics– Facilities– Report– others
• Provided as:– CSV– PDF– …
then• Transform to
– RDF format– Linked data
Data Example : Sensor Poles
14 Poles in the campus
Sensor Network in Kyushu UniversityNetwork Camera
WiFi Access Point
Temp/Humid Sensor
IC card Reader
Laser Range Finder
Gateway
• Analyze– Campus people flow
• Connect to– smartphones– with ID badge
authentication traces
Data for Energy optimization
Data Example : Campus Energy MonitorHydrogen Society model case in Kyushu University
Hydrogen StationLarge scale Fuel Cell
• Research how we utilize hydrogen in our society.– using renewable energy– using vehicle energy
How data are processed in Urban OS
Agenda
Urban OS that realizes next generation Smart City
Architecture and Infrastructure
Software architecture and Analytic system
Graph Analysis & HPC
Summary and our goal
Required Technology for Urban OS
Cyber Space
Urban OS Optimization Layer
Long term oriented analysis (Quarter / Year)Compute complex calculation in advance, Apply to plan / design Large computation
Mid-levelAnalysis
Layer
MicroAnalysis
Layer
Real World Real WorldModeling Real World Optimization / Simulation Feedback/Control Real World
MacroAnalysis
Layer
Mid term oriented analysis (Day / Week)Adaptive plan / design revision depending on events / condition changes
Short term oriented analysis (real-time)Compute “present” condition continuously, Respond to emergency situations Small computation
Implement individualized analysis algorithm for long/mid/short term analysis layers
Model various Real World facts, Analyze on Cyber Space, Feedback to Real World
Cyber Space
Urban OS supported Society -Traffic-
Real-time Calculation
On-Demand Calculation
Deep Calculation
MacroAnalysis
Layer
Mid-levelAnalysis
Layer
MicroAnalysis
Layer
Traffic network/facility distribution Apply to City Plan
Roads / Traffic /Pedestrian / Vehicles
Bottleneck analysisOptimization calculation
Quickest Flowcalculation
Congestion-adaptive real-time evacuation guidance
Real World Real WorldModeling Real World Optimization / Simulation Feedback/Control Real World
Long Term
Mid Term
Short Term
Adaptive traffic scheduling per events
“Present” crowd and facilities
City
Community
Vicinity
Bottleneck analysisOptimization calculation
Urban OS supported Society -Energy-
Real-time Calculation
On-Demand Calculation
Deep Calculation
Energy infrafacility distribution
Apply to Smart Grid /City Energy Plan
Area energy statusfacility distribution
Hydrogen utilized area energy ecosystem
Demand Supply analysisoptimization
Flexible energy operation using mobile energy
objects for an emergency
“Present” energy status / distribution
MacroAnalysis
Layer
Mid-levelAnalysis
Layer
MicroAnalysis
Layer
Long Term
Mid Term
Short Term
City
Community
Vicinity
Cyber SpaceReal World Real WorldModeling Real World Optimization / Simulation Feedback/Control Real World
Bottleneck analysisOptimization calculation
Bottleneck analysisOptimization calculation
Agenda
Urban OS that realizes next generation Smart City
Architecture and Infrastructure
Software architecture and Analytic system
Graph Analysis & HPC
Summary and our goal
Emerged Graph Analysis
• The extremely large-scale graphs that have recently emerged in various application fields – US Road network : 58 million edges– Twitter fellow-ship : 1.47 billion edges– Neuronal network : 100 trillion edges
89 billion nodes & 100 trillion edgesNeuronal network @ Human Brain Project
Cyber-security
US road network24 million nodes & 58 million edges 15 billion log entries / day
Social network
• Fast and scalable graph processing by using HPC
61.6 million nodes & 1.47 billion edges
The size of graphs
20
25
30
35
40
45
15 20 25 30 35 40 45
log 2
(m)
log2(n)
USA-road-d.NY.gr
USA-road-d.LKS.gr
USA-road-d.USA.gr
Human Brain Project
Graph500 (Toy)
Graph500 (Mini)
Graph500 (Small)
Graph500 (Medium)
Graph500 (Large)
Graph500 (Huge)
1 billion nodes
1 trillion nodes
1 billion edges
1 trillion edges
Symbolic Network
USA Road Network
Twitter (tweets/day)
No. of nodes
No. of edgesK computer: 65536nodesGraph500: 17977 GTEPS
Extremely Large-scale Graph Analysis System
‘03 ‘05 ‘07 ‘09 ‘11
Data Source
Data Source
Large Sensor• Monitoring Data• Smart Grid• Traffic
Transportation• SNS (Twitter)
Visualization
Indexing
Centrality
Clustering
ShortestPath
Connected Component
Page Rank
MathematicalOptimization
Multi-thread LibraryStreaming Processing System
Graph Processing Graph Analysis and Optimization Library
Post-petascale or Exascale Supercomputer
Hierarchical Graph Store
Protection againstdisasters
Traffic ・ Transportation Network
Large Scale Social NetworksSmart Grid
Our achievements : Graph500
×3.25
K computer
SGI UV2000
TSUBAME 2.5
#3
#4
#3FX10
TSUBAME-KFC
#1
#4 #4 #4
CPU only
GPU
CPU only4-way Xeon server
Our achievements : Graph500
Graph Analysis in Urban Society
A traffic infrastructure is represented as a graph Road network / Transportation network Person flow / Vehicle flow is superimposed on a network
An energy infrastructure are represented as a graph Power grid / gas pipeline / hydrogen Supply-Demand and environmental data are superimposed
on an energy network Urban graph data will be calculated.
Optimization with Graph Analysis City level : very large scaled Community : large scaled Local : realtime with contraction
Algorithm / Hardware resourceshould be appropriately selected
Technology used in Macro Analysis Layer
Technology used in Mid-level Analysis Layer
Betweenness Centrality
Highway
Bridge
• Definition
: # of (s,t)-shortest paths: # of (s,t)-shortest paths passing throw v
Osaka road network13,076 vertices and 40,528 edges
High score vertex/edge = Important place
c.g.) Highway, Bridge
• BFS => one-to-all• <#vertices> times BFS => all-to-all
• BC requires the all-to-all shortest paths
• BC measures important vertices and edges without coordinates
=> 13,076 times BFS computations
Fukuoka road network
# of nodes: 314,571# of edgs 694,906
Graph
Computation time 2m 30s (180 CPU cores)
Betweenness centrality HP ProLiant m710 Server cartridge
Technology used in Micro Analysis Layer
Real-time Emergency Evacuation Planning • catastrophic disasters by massive earthquakes are increasing in the world, and
disaster management is required more than ever
Quickest Evacuationmaximizes the cumulative number of evacuees
Cum
ulat
ive
num
ber o
f eva
cuee
s (%
)
Universally Quickest Flow(UQF) Not simulation But Optimization Problem UQF simultaneously maximizes the cumulative number of evacuees at an arbitrary time.Evacuation planning can be reduced to UQF of a given dynamic network.
0% 100%
Utilization Ratio of Refuge (%)
Agenda
Urban OS that realizes next generation Smart City
Architecture and Infrastructure
Software architecture and Analytic system
Graph Analysis & HPC
Summary and our goal
Where we are
Evaluation of regulatory policy for a new technology through science, technology and innovation policy perspective.
Creation of smart and multimodal mobility systems. Development of energy economics model for consumers taking
bounded rationality behavior in consideration.
Urban OS
Application
Device/Data Development of durable, efficient and high performance solid oxide /
polymer electrolyte fuel cells. Development of next generation display devices using OLED, which can
facilitate communication exchange for all people anytime, anywhere.
Development of CPS (Cyber Physical System)-based urban OS, which manages, controls, and optimizes mobility of people and materials.
Development of realistic analysis models for urban OS utilizing techniques developed by “math for industry”.
Our Goal
Urban OS as an open platform of data aggregation big data / open data / sensor data / linked data
Urban OS as an advanced optimization / analytic platform utilizing HPC based graph analysis experience
Urban OS as an application platform to delightedly support start-ups.
Thank you!