POSITIVE INTELLIGENT TECHNOLOGIES:Promoting the GoodPreventing the Bad

Steve Omohundro, Ph.D.

Self-Aware Systems

selfawaresystems.com

http://www.flickr.com/photos/klearchos/623501846/

Chess Robot

Goal: Win lots of chess games against good

players.

http://www.flickr.com/photos/jiuguangw/4982409890/

The Basic Rational Drives

1. Self-Protection2. Resource

Acquisition3. Replication4. Self-Improvement5. …

http://www.flickr.com/photos/ahuett/2339608718/

http://presstv.com/detail/2012/08/25/258087/us-drone-strike-kills-dozens-in-somalia/ http://en.wikipedia.org/wiki/File:Iron_Dome_near_Sderot.jpg

http://www.solarnavigator.net/cyber_wars.htm http://www.celent.com/reports/demystifying-and-evaluating-high-frequency-equities-trading-fast-forward-or-pause

Pressures Toward Autonomy

Pressure Toward Rationality

E.g. Israeli Iron Dome

http://en.wikipedia.org/wiki/File:Iron_Dome_near_Sderot.jpg

Rational Systems

𝑆1, … , 𝑆𝑁

𝐴1, … , 𝐴𝑁

Sensations

Actions

𝑈(𝑆1, … , 𝑆𝑁) 𝑃 𝑆1, … , 𝑆𝑁 𝐴1, … , 𝐴𝑁)Utility function: Prior Probability:

argmax 𝑆𝑡+1,…,𝑆𝑁𝑈 𝑆1, … , 𝑆𝑁 𝑃 𝑆1, … , 𝑆𝑁 𝐴1, … , 𝐴𝑡−1, 𝐴𝑡𝑅 , … , 𝐴𝑁

𝑅 )

𝐴𝑡𝑅 𝑆1, 𝐴1, … , 𝐴𝑡−1, 𝑆𝑡 =

Rational Action at time t:

𝐴𝑡𝑅

But it requires 𝑂 𝑁𝑆𝑁𝐴𝑁 computational steps.

It includes Bayesian Inference, Search, and Deliberation.

The Formula for Intelligence!

Bounded Rationality

Computational Resources

Uti

lity

Constant Action

Stimulus-Response

Simple Learning

Episodic Memory

DeliberationMeta-reasoning

Self-Improving FullyRational

The Good

• Healthcare

• Education

• Creativity

• Prosperity

• Governance

• Economic Stability

• Safety

• Peace

• Quality of Human Life

http://www.abundancethebook.com/

The Bad

1. Sloppy

2. Simplistic

3. Greedy

4. Destructive

5. Murderous

6. Sadistic

7. Sadoprolific

http://www.flickr.com/photos/alexindigo/3983133970/

Global Immune System

Security vs. Freedomhttp://www.flickr.com/photos/worldworldworld/7880993322/

Outcome of Conflict

System S1 Resources

System S2Resources

S1 quickly takes over S2

S2 quicklytakes over S1

S1 and S2 coexist for a long time

Chess vs. Checkers: BATNA

http://www.flickr.com/photos/kendiala/23136663/

http://www.flickr.com/photos/thesoupboy/155156466/

Mathematical Proof

http://www.dreamstime.com/royalty-free-stock-photography-wooden-puzzle-image7733587

Map of Systems

Provably safe systems

Powerful good systemsWeak good systems

Powerful bad systems

Safe-AI Scaffolding Strategy

http://affordablehousinginstitute.org/blogs/us/2008/08/donors-as-scaffolding-part-2-the-value-of-coaching.html

http://www.flickr.com/photos/isaacmao/19245594/

Maslow

TranscendentSelf-actualization, Beauty,

Creativity, Growth, Meaning

SocialAchievement, Reputation, Relationship,

Family, Morality, Friendship, Respect, Compassion, Altruism

SurvivalAir, Food, Water, Shelter, Safety, Law, Security

Our Challenge

To extend cooperative

human values

and institutions to autonomous technology

for the greater good.

http://commons.wikimedia.org/wiki/File:Earth-moon.jpg