Upload
dothien
View
218
Download
2
Embed Size (px)
Citation preview
Modeling and implementing multi-agents based
simulations on massively parallel architectures
Emmanuel Hermellin
November 18, 2016
sous la direction de Fabien Michel et Jacques Ferber
Rapporteurs Examinateurs
Vincent Chevrier Olivier Simonin
Alessandro Ricci Jean-Christophe Soulie
Introducing MABS
Used to study complex systems in many research domains as collective
robotics, biology, economy, urban management, etc.
Direct modelling of the individuals and their interactions
1/57
Agent-Based Model (ABM)
• Some autonomous entities called agents
• An environment and its dynamics
• Some agent / agent and agent / environment interactions
2/57
Experimenting with MABS
Virtual labs for various purposes
• Predicting the evolution of a system
• Explaining its mechanisms
• Discovering unexpected phenomena
Natural systems Human systems
e.g. bird flocks e.g. traffic and crowd
3/57
Experimenting with MABS
These simulations may need huge computational resources
Trend in MABS
Large scale and multi-level simulations
[Picault and Mathieu, 2011, Camus et al., 2012]
Major issue: Performances
Computing resources requirements often limit the extent to which an
ABM could be studied
4/57
Experimenting with MABS
These simulations may need huge computational resources
Trend in MABS
Large scale and multi-level simulations
[Picault and Mathieu, 2011, Camus et al., 2012]
Major issue: Performances
Computing resources requirements often limit the extent to which an
ABM could be studied
4/57
Introducting HPC
Intensivecomputation
Climate simulation Astrophysics simulation
e.g. CO2 distribution e.g. gravity interaction
5/57
General-Purpose computing on Graphics Processing Units
GPGPU
What is GPGPU ?
Using the massively parallel architecture of usual PC graphics cards to
do General-Purpose Computing
GPGPU characteristics
Huge speed-up, available on almost all computers
7/57
Graphics Processing Unit performances
Central Processing Unit (CPU) vs Graphics Processing Unit (GPU)
8/57
Using GPGPU in MABS
SugarScape on Steroids [D’Souza et al., 2007]
• First ABM implemented with GPGPU
• Large scale simulation (2 million reactive agents)
• Performance never reached with a CPU
9/57
Spreading of GPGPU (source: Google Scholar)
2002 2004 2006 2008 2010 2012 20140
2,000
4,000
6,000
8,000
Years
Pu
blic
atio
ns
GPGPU and MAS
GPGPU and simulation
GPGPU
10/57
Why is this technology not used ?
GPU programming
1. Based on a specific hardware architecture:
Stream processing
2. Required a different programming approach:
Object-oriented programming (OOP) not available
11/57
Why is this technology not used ?
[Perumalla and Aaby, 2008]
”Execution is two to three orders of magnitude faster with a
GPU [...] but at the cost of decrease in modularity, ease of
programmability and reusability. [...] Effective use of data
parallel execution, in general, seems to require resolution of
modeling and execution challenges”
12/57
Thesis objective:
Study how to use GPGPU without encountering the issues
stated in [Perumalla and Aaby, 2008]:
”[...] decrease in modularity, ease of programmability and
reusability [...]”
12/57
Table of contents
1. GPGPU technology
2. MABS & GPGPU: State of the art
3. The GPU delegation principle and its evolution
4. A design method relying on GPU delegation
5. Conclusion and perspectives
13/57
Stream processing
Consists in executing simultaneously a kernel (a series of computations)
on a data set (the flow - stream)
15/57
GPU programming workflow
1 Copy data
onG
PU
2 Initiate
com
puta
tio
ns
3E
xecutekernels
4C
opy
the
resu
lts
18/57
At the beginning of the use of the GPGPU in MABS
2007 SugarScape on steroids [D’Souza et al., 2007]
2008 First GPU framework [Lysenko and D’Souza, 2008]
- ABGPU [Richmond and Romano, 2008]
-
2009
-
2010
2011
2012
-
2013
2014
2015
-
2016
19/57
At the beginning of the use of the GPGPU in MABS
All-in-GPU
[D’Souza et al., 2007]
• Very promising performance • Very complex technology
• Lack of tools and resources
• Hardly reusable
19/57
Emergence of specialized development environments
2007 SugarScape on steroids [D’Souza et al., 2007]
2008 First GPU framework [Lysenko and D’Souza, 2008]
- ABGPU [Richmond and Romano, 2008]
- First flocking simulation [Passos et al., 2008]
2009 Flame GPU [Richmond et al., 2010]
- First crowd and traffic simulation [Strippgen and Nagel, 2009]
2010
2011 Pedestrian framework [Richmond and Romano, 2011]
2012
-
2013
2014
2015
-
2016
20/57
Emergence of specialized development environments
All-in-GPU
[Erra et al., 2009]
• Improved performance
• Simplified implementation
• Implementation from scratch
• Hardly reusable
20/57
Appearance of the hybrid approaches
2007 SugarScape on steroids [D’Souza et al., 2007]
2008 First GPU framework [Lysenko and D’Souza, 2008]
- ABGPU [Richmond and Romano, 2008]
- First flocking simulation [Passos et al., 2008]
2009 Flame GPU [Richmond et al., 2010]
- First crowd and traffic simulation [Strippgen and Nagel, 2009]
2010 First hybrid simulation [Vigueras et al., 2010]
2011 Pedestrian framework [Richmond and Romano, 2011]
2012 Sworm GPU [Laville et al., 2012]
- TurtleKit GPU [Michel, 2013]
2013 [Pavlov and Müller, 2013]
2014 MCMAS [Laville et al., 2014]
2015 SIMILAR [Abouaissa et al., 2015]
- MASON with multiple GPU [Ho et al., 2015]
2016
21/57
Appearance of the hybrid approaches
Hybrid
[Laville et al., 2012]
• More modular design approach
• Compatibility with other technologies
• Hide the use of GPGPU
• Lack of genericity
21/57
Overview of works treating about GPGPU in MABS
Reference Approach Implementation Agent
[D’Souza et al., 2007] All-in-GPU Graphic functions Reactive
[Lysenko and D’Souza, 2008] All-in-GPU Graphic functions Reactive
[Richmond and Romano, 2008] All-in-GPU Graphic functions Reactive
[Perumalla and Aaby, 2008] All-in-GPU Graphic functions Reactive
[Erra et al., 2009] All-in-GPU CUDA Reactive
[Richmond et al., 2010] All-in-GPU CUDA Reactive and deliberative
[Husselmann and Hawick, 2011] All-in-GPU CUDA Reactive and heterogeneous
[Richmond and Romano, 2011] All-in-GPU CUDA Reactive and deliberative
[Laville et al., 2012] Hybrid C + OpenCL Reactive and deliberative
[Pavlov and Mller, 2013] Hybrid C + CUDA Reactive and deliberative
[Michel, 2013] Hybrid Java + CUDA Reactive and deliberative
[Laville et al., 2014] Hybrid Java + OpenCL Reactive and deliberative
[Ho et al., 2015] Hybrid CUDA Reactive and deliberative
[Shekh et al., 2015] Hybrid C + CUDA and OpenCL Reactive and deliberative
[Hermellin and Michel, JFSMA 2014]
22/57
Most of reviewed solutions address only partially the issues raised
by the use of GPGPU in MABS
23/57
GPU delegation: A different approach
Is part of E4MAS trend [Weyns et al., 2007, Ricci et al., 2011]
”The environment is a first-class abstraction [...] that mediates
both the interaction among agents and the access to resources”
Objectives and characteristics of GPU delegation
• Keep the accessibility of an oriented objects programming interface
• Doesn’t hide the underlying technology
• Rely on transforming agent perception computations into
environmental dynamics
24/57
First statement of the principle [Michel, 2013]
Statement
”Any agent perception computation not involving the agents
state could be translated to an endogeneous dynamic of the
environment, and thus considered as a potential GPU
environment module”
25/57
First application of the principle
Multi-Level Emergence [Michel, 2013]
• Increase performances
• Promote reusability with independant GPU modules
• Made in an ad hoc way
• Hard to define its ability to be applied to other types of models
27/57
The GPU delegation principle pursues similar objectives but its
experiment on MLE model was only a first attempt
27/57
Reynolds’ boids as case study
Flocking simulation
Three simple behaviors which describe how an individual boid maneuvers
based on the positions and velocities its nearby flockmates
R.1 Separation R.2 Alignment R.2 Cohesion
R.3 Speed Adaptation
28/57
Characteristics of our flocking model
The most simple and complete flocking model as possible
[Hermellin and Michel, MABS 2015]
29/57
Applying the GPU delegation principle
Some computations involve agent states but do not modify them
30/57
Evolving the GPU delegation principle
Second statement of the principle
”Any agent perception computation not modifying the agents
state could be translated to an endogeneous dynamic of the
environment, and thus considered as a potential GPU
environment module”
[Hermellin and Michel, MABS 2015]
31/57
Results
10 20 30 40 50 601
1.1
1.2
1.3
1.4
Agents’ density in the environment
Per
form
ance
gain
s
Flocking (Environment 256)
Flocking (Environment 512)
34/57
Conclusion of this first case study
[Hermellin and Michel, MABS 2015]
• Evolution of the principle to make it less restrictive
• Interesting performance results
• Creating a new independent GPU module
35/57
Additional experiments [Hermellin and Michel, AAMAS 2016]
Game of Life Schelling’s Segregation
Fire DLA
36/57
Performance gains with GPU delegation
400 600 800 1,000 1,200 1,400 1,600 1,800 2,000
2
4
6
8
10
12
Size of the environment
Per
form
ance
gain
s
Game of Life
Segregation
Fire
DLA
37/57
Summary of the implementations
Model Transformed computation GPU module
Game of Life number of neighbors in the neighborhood Voisinage
Segregation type of neighbors in the neighborhood Voisinage
Fire heat diffusion Diffusion
Generalized from MLE
DLA closeness of the neighbors Voisinage
38/57
Conclusion of these experimentations
[Perumalla and Aaby, 2008]
• Increase the performances with GPGPU is only done at the expense
of modularity, reusability and accessibility of the developed solutions
With GPU delegation
• Good performances
• Modularity
• Reusability
• Accessibility
39/57
Conclusion of these experimentations
[Perumalla and Aaby, 2008]
• Increase the performances with GPGPU is only done at the expense
of modularity, reusability and accessibility of the developed solutions
With GPU delegation
• Good performances
• Modularity
• Reusability
• Accessibility
39/57
These experiments are good proofs of concept but to make this
approach more robust it is necessary to generalize it
[Hermellin and Michel, CNSC 2016]
39/57
Generalization [Hermellin and Michel, CNSC 2016]
Follow each time the same iterative implementation process
1. Identify the most intensive computations
2. Adapt the data structures
3. Implement GPU delegation
The principle did not need to be further modified to be applied to
these new models
40/57
Generalization [Hermellin and Michel, CNSC 2016]
Follow each time the same iterative implementation process
1. Identify the most intensive computations
2. Adapt the data structures
3. Implement GPU delegation
The principle did not need to be further modified to be applied to
these new models
40/57
Step 3: Reusing GPU modules
GPUFieldPerception [Michel, 2013]
Compute the direction toward neighboring cells
which possess the greatest/small amount of a given
variable
In prey-predator model: The data are the presence
indications
45/57
Step 4: Computations compatibility with GPU architecture
Compatibility criteria
• Can be distributed on the GPU grid
• Can be performed on the whole environment
• Have compatible data structures
46/57
Performance result for the prey-predator model
400 600 800 1000 1200 1400 1600 1800 20002
3
4
Size of the environment
Per
form
ance
gain
s
Prey-predator
48/57
Advantages and limits of this method
Advantages
• Versatile method: Extracted from 8 ABM implementations
• Computations to be transformed are easily identified
• Independent, simple and reusable GPU modules
Limit
• Strongly dependent on characteristics of ABM
49/57
Advantages and limits of this method
Advantages
• Versatile method: Extracted from 8 ABM implementations
• Computations to be transformed are easily identified
• Independent, simple and reusable GPU modules
Limit
• Strongly dependent on characteristics of ABM
49/57
Addressed issues
One major issue with MABS: Performances
• Multi-Agent Based Simulations may require a lot of computing
resources
One solution
• Opt for High Performance Computing though GPGPU
GPU programming
• Based on a specific hardware architecture
• Required a different programming approach
50/57
Contributions
Objective
Study how to use GPGPU without encountering the raised issues
Summary of the contributions
1. Identify main issues when using GPGPU in MABS
[Hermellin and Michel, JFSMA 2014]
2. Experiment and evolve GPU delegation
[Hermellin and Michel, MABS 2015, AAMAS 2016]
3. Generalize the approach
[Hermellin and Michel, CNSC 2016, MABS 2016]
51/57
Conclusion
The design method relying on GPU delegation
• Improves the accessibility of the GPGPU in MABS context
• Defines a generic approach allowing to consider varied ABM
• Promotes the reusability of the created tools and developed code
About the limit of this approach
• Performance results are hard to predict
52/57
Short term perspectives
Address the limits of the method
Hard to estimate with this method the expected performance gains
• Develop a software solution (benchmark) based on existing GPU
modules which evaluate the relevance of applying our method
Widen the scope of the method
Most of our use cases embed discretized environments
• Test the method on more heterogeneous ABM to improve its scope
of applicability
54/57
Short term perspectives
Address the limits of the method
Hard to estimate with this method the expected performance gains
• Develop a software solution (benchmark) based on existing GPU
modules which evaluate the relevance of applying our method
Widen the scope of the method
Most of our use cases embed discretized environments
• Test the method on more heterogeneous ABM to improve its scope
of applicability
54/57
Long term perspectives: Impact on AOSE
GPU delegation: At the beginning
• Part of E4MAS trend
• Transform agent perception computations into environmental
dynamics
Using the method relying on GPU delegation
• It considers all computations present in the model
• Independent from the technological context
56/57
Long term perspectives: Impact on AOSE
Focus on pre-processing data
What impact might this have ?
• On the agent behaviors ? • On how we design a model ?
56/57
References
BIBLIOGRAPHIE
[Hermellin and Michel, 2015] Hermellin, E. and Michel, F. (2015). Délégation GPU des perceptions agents : Applicationaux boids de reynolds. In Vercouter, L. and Picard, G., editors, Environnements socio-techniques - JFSMA 15 - Vingt-troisièmes Journées Francophones sur les Systèmes Multi-Agents, Rennes, France, June 30th-July 1st, 2015, pages 185–194. Cépaduès Éditions. 1
[Hermellin and Michel, 2016a] Hermellin, E. and Michel, F. (2016a). Defining a Methodology Based on GPU Delegationfor Developing MABS using GPGPU. In Antunes, L. and Nardin, L. G., editors, Multi-Agent Based Simulation XVII,International Workshop, MABS 2016, Singapore, May 10, 2016, pages –. à paraître. 1
[Hermellin and Michel, 2016b] Hermellin, E. and Michel, F. (2016b). Expérimentation du principe de délégation GPUpour la simulation multiagent. les boids de reynolds comme cas d’étude. Revue d’Intelligence Artificielle, 30(1-2) :109–132. 1
[Hermellin and Michel, 2016c] Hermellin, E. and Michel, F. (2016c). Gpu delegation : Toward a generic approach fordevelopping mabs using gpu programming. In Proceedings of the 2016 International Conference on AutonomousAgents & Multiagent Systems, AAMAS ’16, pages 1249–1258, Richland, SC. International Foundation for AutonomousAgents and Multiagent Systems. 1
[Hermellin and Michel, 2016d] Hermellin, E. and Michel, F. (2016d). Multi-Agent Based Simulation XVI : Internatio-nal Workshop, MABS 2015, Istanbul, Turkey, May 5, 2015, Revised Selected Papers, volume 9568, chapter GPU Envi-ronmental Delegation of Agent Perceptions : Application to Reynolds’s Boids, pages 71–86. Springer InternationalPublishing. 1
[Hermellin and Michel, 2016e] Hermellin, E. and Michel, F. (2016e). Méthodologie pour la modélisation et l’implémen-tation de simulations multi-agents utilisant le gpgpu. Cépaduès Éditions. à paraître. 1
[Hermellin and Michel, 2016f] Hermellin, E. and Michel, F. (2016f). Overview of case studies on adapting MABS modelsto GPU programming. In Bajo, J., Escalona, M. J., Giroux, S., Hoffa-Dabrowska, P., Julián, V., Novais, P., Pi, N. S., Un-land, R., and Silveira, R. A., editors, Highlights of Practical Applications of Scalable Multi-Agent Systems. The PAAMSCollection - International Workshops of PAAMS 2016, Sevilla, Spain, June 1-3, 2016. Proceedings, volume 616 of Com-munications in Computer and Information Science, pages 125–136. Springer. 1
[Hermellin et al., 2014] Hermellin, E., Michel, F., and Ferber, J. (2014). Systèmes multi-agents et GPGPU : état deslieux et directions pour l’avenir. In Courdier, R. and Jamont, J., editors, Principe de Parcimonie - JFSMA 14 - Vingt-deuxièmes Journées Francophones sur les Systèmes Multi-Agents, Loriol-sur-Drôme, France, Octobre 8-10, 2014, pages97–106. Cépaduès Éditions. 1
[Hermellin et al., 2015] Hermellin, E., Michel, F., and Ferber, J. (2015). État de l’art sur les simulations multi-agents etle GPGPU. Revue d’Intelligence Artificielle, 29(3-4) :425–451. 1
5
References i
Camus, B., Siebert, J., Bourjot, C., and Chevrier, V. (2012).
Modelisation multi-niveaux dans AA4MM.
CoRR, abs/1210.5936.
D’Souza, R. M., Lysenko, M., and Rahmani, K. (2007).
SugarScape on steroids: simulating over a million agents at
interactive rates.
Proceedings of Agent 2007 conference.
Erra, U., Frola, B., Scarano, V., and Couzin, I. (2009).
An Efficient GPU Implementation for Large Scale
Individual-Based Simulation of Collective Behavior.
In High Performance Computational Systems Biology, 2009. HIBI
’09. International Workshop on, pages 51–58.
References ii
Ho, N., Thoai, N., and Wong, W. (2015).
Multi-agent simulation on multiple GPUs.
Simulation Modelling Practice and Theory, 57:118 – 132.
Husselmann, A. V. and Hawick, K. A. (2011).
Simulating Species Interactions and Complex Emergence in
Multiple Flocks of Boids with GPUs.
In International Conference on Parallel and Distributed Computing
and Systems, pages 100–107. IASTED.
Laville, G., Mazouzi, K., Lang, C., Marilleau, N., Herrmann, B., and
Philippe, L. (2014).
MCMAS: A Toolkit to Benefit from Many-Core Architecure in
Agent-Based Simulation.
In an Mey, D., Alexander, M., Bientinesi, P., Cannataro, M., Clauss,
C., Costan, A., Kecskemeti, G., Morin, C., Ricci, L., Sahuquillo, J.,
References iii
Schulz, M., Scarano, V., Scott, S., and Weidendorfer, J., editors,
Euro-Par 2013: Parallel Processing Workshops, volume 8374 of
Lecture Notes in Computer Science, pages 544–554. Springer Berlin
Heidelberg.
Laville, G., Mazouzi, K., Lang, C., Marilleau, N., and Philippe, L.
(2012).
Using GPU for Multi-agent Multi-scale Simulations.
In Distributed Computing and Artificial Intelligence, volume 151 of
Advances in Intelligent and Soft Computing, pages 197–204.
Springer Berlin Heidelberg.
Lysenko, M. and D’Souza, R. M. (2008).
A Framework for Megascale Agent Based Model Simulations
on Graphics Processing Units.
Journal of Artificial Societies and Social Simulation, 11(4):10.
References iv
Michel, F. (2013).
Translating Agent Perception Computations into
Environmental Processes in Multi-Agent-Based Simulations: A
means for Integrating Graphics Processing Unit Programming
within Usual Agent-Based Simulation Platforms.
Systems Research and Behavioral Science, 30(6):703–715.
Pavlov, R. and Mller, J. (2013).
Multi-Agent Systems Meet GPU: Deploying Agent-Based
Architectures on Graphics Processors.
In Camarinha-Matos, L., Tomic, S., and Graa, P., editors,
Technological Innovation for the Internet of Things, volume 394 of
IFIP Advances in Information and Communication Technology, pages
115–122. Springer Berlin Heidelberg.
References v
Perumalla, K. S. and Aaby, B. G. (2008).
Data parallel execution challenges and runtime performance of
agent simulations on GPUs.
Proceedings of the 2008 Spring simulation multiconference, pages
116–123.
Picault, S. and Mathieu, P. (2011).
An interaction-oriented model for multi-scale simulation.
In Proceedings of the Twenty-Second International Joint Conference
on Artificial Intelligence - Volume Volume One, IJCAI’11, pages
332–337. AAAI Press.
Ricci, A., Piunti, M., and Viroli, M. (2011).
Environment programming in multi-agent systems: an
artifact-based perspective.
Autonomous Agents and Multi-Agent Systems, 23(2):158–192.
References vi
Richmond, P. and Romano, D. M. (2008).
Agent based GPU, a real-time 3d simulation and interactive
visualisation framework for massive agent based modelling on
the GPU.
In In Proceedings International Workshop on Super Visualisation
(IWSV08).
Richmond, P. and Romano, D. M. (2011).
A High Performance Framework For Agent Based Pedestrian
Dynamics On GPU Hardware.
European Simulation and Modelling.
Richmond, P., Walker, D., Coakley, S., and Romano, D. M. (2010).
High performance cellular level agent-based simulation with
FLAME for the GPU.
Briefings in bioinformatics, 11(3):334–47.
References vii
Shekh, B., de Doncker, E., and Prieto, D. (2015).
Hybrid multi-threaded simulation of agent-based pandemic
modeling using multiple GPUs.
In Bioinformatics and Biomedicine (BIBM), 2015 IEEE International
Conference on, pages 1478–1485.
Weyns, D., Omicini, A., and Odell, J. (2007).
Environment as a first class abstraction in multiagent systems.
Autonomous Agents and Multi-Agent Systems, 14(1):5–30.