Dynamics of Learning & Distributed Adaptation Santa Fe Institute: James P. Crutchfield, P.I.

Future Plans (6 months out)New problems:

Continuous-state and continuous-time agentsAdaptation to active, pattern-forming environmentsDynamical theory of how learning and adaptation occur

Anticipated results:Monitor emergence of cooperation in agent collectivesMeasure mutuality in interacting reinforcement learnersTest on in-house autonomous robotic vehicle collectives

Analytical tools:Predict whether or not group cooperation can occurAgent intelligence versus group sizePrediction of the rate of adaptation during collective taskPrototype models: Solvable MAS systems

Software tools:Ab Initio Learning AlgorithmsLibrary for Estimating MASS MetricsEnterprise Java Platform for Robot Collectives

Multi-Agent System Science (MASS)Dimension

Agents learn complex environment ab initioSynchronization of agent to environmentAgents adapt to nonstationary environmentStrategies for agent-agent coordination

Metrics for large-scale MASsStatistical Complexity:

Amount of structure & organization in environ’tIndividual agent knowledge v. group knowledge

Mutuality: Architecture of information flowLyapunov Spectra: Degrees of stability and instabilityCausal Synchrony: Detect coherent subgroup behavior

CAHDE REF

ACFC: Adapting to instabilities in air flow controlAirOps: Emergence of spontaneous leadership

Solution:Interacting reinforcement and -machine learning agents solve a group task

Approach:

Pattern Discovery: Beyond pattern recognition

Design & analysis based on sound principles of learning

Metrics for cooperation in large-scale systemsResults To DatePredictive theory of agent learning:

Quantify agent modeling capacity

Data Set Size v. Prediction Error v. Model Complexity

Pattern Discovery: The “Aha” Effect

Incremental learning algorithm

Quantify structure in environment:

How structure leads to unpredictability for agent

Define synchronization for chaotic environments:

Predict required data and time to synchronize

Periodic case solved in closed form

Transient information: New metric of synchronization

Dynamics of reinforcement-learning agents:

Nash equilibria v. oscillation v. chaos

Dependence on system architecture and initial state