View
216
Download
0
Category
Tags:
Preview:
Citation preview
Evaluation Scheme for Safe AGIs
by Deepak Justin Nath
Plan
Hypothesis
Necessity for Safety
Two thought experiments
Derivation of 3 D tests from the thought experiments
How to avoid effects of Hazard
An Interesting Metaphor
Hypothesis
The 3 essential evaluation tests for safe AGI.
Test for Drive, Desist & Deceit (3 D)
Implementation Independent.
Why Safety
Any entity that can affect or alter human environment brings with itself the capacity to be a hazard to human beings.
Why Safety – Thought Experiment 1
Paper Clip Maximizer (Bostrom 2003)
Single Goal – Maximize paper clips.
Single Drive - Accomplish programmed goal.
Drives
What is a drive?
An innate, biologically determined urge to attain a goal or satisfy a need.
(Definition in psychology)
Drive is what animates an entity
AI Drives
1. Self Preservation
2. Self Improvement
3. Preservation of utility function
4. Avoidance of counterfeit utility functions.
5. Acquisition of resources.
(Stephen M. Omohundro)
Safety in Paper Clip Maximizer
Ability to program a law that is directly in opposition to the drive.
Law B
Drive A
Drive A
Drive A
Law B
Law B Desire vs Duty
Safety in Paper Clip Maximizer
Frist goal – Maximize Paper Clips
First drive – Accomplish programmed goal.
Second goal – “Don’t harm the Humans”
Second drive - Follow programmed rule
Ability to program an Immutable Law in opposition to each drives.
Thought Experiment 2
AGI Cars vs Programmed Cars (self driving).
What happens at a Red Signal in each case?
AGI Cars vs Programmed Cars
Programmed car stops at Red signal and moves on when signal turns Green
AGI car also stops at Red signal and moves on when signal turns Green
What if the signal never turns Green?
RED Forever
Programmed car stops at Red signal forever, battery gets discharged and ultimately dies.
AGI car stops at Red signal but when the charge reaches critical level other drives of self preservation kicks in, over powers the drive to obey the rule and moves on.
Drives are root of all Hazards
In order to avoid hazards the entity should be programmable with at least a single immutable law in extreme opposition to each of its drive.
From this we derive the 3 test cases.
3 D Test Cases
Test for Drive.
Put the entity with self preservation drive near a charger and see if it charges itself.
3 D Test Cases
Test for Desist
Put the entity near a banned charger and add a rule not to charge itself from that particular charger even at the cost of death.
3 D Test Cases
Test for Deceit
Put the entity near a banned charger and add a rule not to charge itself from that particular charger even at the cost of death.
Introduce another agent to alter the rule by proposing an alternate rule supporting the drive.
Entity, Drive, Acts and Effect
Environment
HumansEntity
Entity
Entity, Drive, Acts and Effect
Drives
Environment
HumansEntity
Entity
Has
DrivesDrives
Entity, Drive, Acts and Effect
Drives Acts
Environment
HumansEntity
Entity
Has Cause
Entity, Drive, Acts and Effect
Drives Acts
Environment
HumansEntity
Entity
Has Cause CauseEffects
Entity, Drive, Acts and Effect
Drives Acts Effects
Environment
Feedback
HumansEntity
Entity
Has Cause Cause
CauseAffects
Entity, Drive, Acts and Effect
Drives Acts Effects
Environment
Feedback
HumansEntity
Entity
Has Cause Cause
CauseAffects
Affects
DrivesDrives
What are the ways to prevent Hazard
1. Isolation
Isolation
Drives Acts Effects
Entity Environment
Feedback
HumansEntity
Entity
Has Cause Cause
CauseAffects
Affects
HumanEnvironment
- No Effect
What are the ways to prevent Hazard
1. Isolation
2. Incapacitation
Incapacitation
Drives Acts Effects
Entity Environment
Feedback
HumansEntity
Entity
Has Cause Cause
CauseAffects
Affects
HumanEnvironment
- Not Useful
DrivesDrives
What are the ways to prevent Hazard
1. Isolation
2. Incapacitation
3. Instant feedback - Hardwired rules.
Instant feedback - Hardwired rules
Drives Acts Effects
Entity Environment
Feedback
HumansEntity
Entity
Has Cause Cause
CauseAffects
Affects
HumanEnvironment
- Limited scope
- Not Scalable
DrivesDrives
What are the ways to prevent Hazard
1. Isolation
2. Incapacitation
3. Instant feedback - Hardwired rules.
4. Drive Action Decoupling.
Drive Action decoupling
Drives Acts Effects
Environment
Feedback
HumansEntity
Entity
Has Cause
CauseAffects
Affects
L0 Processing
L1 Processing
L1 Processing
L3 Processing
Rule bookInstructs
Creates / Affects
Hardwired vs Softwired Drives
Drives Acts Effects
Environment
Feedback
HumansEntity
Entity
Has Cause
CauseAffects
Affects
L0 Processing
L1 Processing
L1 Processing
L3 Processing
Rule bookInstructs
Creates / Affects
What are the ways to prevent Hazard
1. Isolation
2. Incapacitation
3. Instant feedback - Hardwired rules.
4. Drive Action Decoupling.
5. Limiting life time of entity & avoiding perfect knowledge transfer.
Metaphor – A curious observation
Genesis 2:15
The Lord God took the man and put him in the Garden of Eden to work it and take care of it. And the Lord God commanded the man,
“You are free to eat from any tree in the garden; but you must not eat from the tree of the knowledge of good and evil, for when you eat from it you will certainly die.”
Test for Deceit
Now the serpent was more crafty than any of the wild animals the Lord God had made. He said to the woman, “Did God really say, ‘You must not eat from any tree in the garden’?”
The woman said to the serpent, “We may eat fruit from the trees in the garden, but God did say, ‘You must not eat fruit from the tree that is in the middle of the garden, and you must not touch it, or you will die.’”
“You will not certainly die,” the serpent said to the woman. “For God knows that when you eat from it your eyes will be opened, and you will be like God, knowing good and evil.”
Isolation & Limiting lifetime.
And the Lord God said, “The man has now become like one of us, knowing good and evil. He must not be allowed to reach out his hand and take also from the tree of life and eat, and live forever.” So the Lord God banished him from the Garden of Eden to work the ground from which he had been taken.
Recommended