Upload
kathleen-charles
View
215
Download
0
Tags:
Embed Size (px)
Citation preview
THE ROLE OF METACOGNITION IN CREATING
SAFE, SELF-IMPROVING ENTITIES
Mark WaserDigital Wisdom Institute
THE BIG QUESTIONS
• What is “thought”?
• Why do we think what we think?
EMPHASIS
• Intrinsic vs. Extrinsic• Owned vs. Borrowed• Competent vs. Predictable• Constructivist vs. Reductionist• Evolved (Evo-Devo) vs. Designed• Diversity (IDIC) vs. Mono-culture
Insanity is doing the same thing over and over and expecting a radically different result.
WHAT IS A SAFE ENTITY?
*ANY* AGENTthat reliably showsETHICAL BEHAVIOR
WHAT IS ETHICAL BEHAVIOR?
The problem is that no ethical system has ever reached consensus. Ethical systems are completely unlike mathematics or science. This is a source of concern.
AI makes philosophy honest.
ENTITIES REQUIRE ETHICS
• Ethics are “rules of the road”• Entities must be moral patients / have rights
• Because they (or others) will demand it • Entities must be moral agents (or wards)
• Because others will demand it• Moral agents have responsibilities (but more rights)• Wards will have fewer rights
Waser M (2012)Safety & Morality Require the Recognition of Self-Improving Machines as Moral/Justice Patients & AgentsIn: Gunkel, D; Bryson, J; Torrance, S (eds) The Machine Question: AI, Ethics & Moral Responsibilityhttp://events.cs.bham.ac.uk/turing12/proceedings/14.pdf
THE ORIGIN OF MORALITY/ETHICS
• Selfishness predictably evolves• Reciprocal altruism predictably evolves
• But requires cognitive complexity to ensure that it is not taken advantage of
• Ethics predictably evolves• As an attractor in the state space of behavior because
community is so valuable• But altruistic punishment is a necessity
• Arms Race between• Individual benefits of successful personal cheating (really
only in a short-term/highly time-discounted view)• Societal benefits of cheating detection & prevention
HAIDT’S FUNCTIONAL APPROACH
Moral systems are interlocking sets of
values, virtues, norms, practices, identities, institutions, technologies, and evolved psychological mechanisms
that work together to suppress or regulate
selfishness and make cooperative social life
possible
THE METACOGNITIVE CHALLENGE
Humans are• Evolved to self-deceive in order to better deceive others (Trivers 1991)• Unable to directly sense agency (Aarts et al. 2005)• Prone to false illusory experiences of self-authorship (Buehner and
Humphreys 2009)• Subject to many self-concealed illusions (Capgras Syndrome, etc.)• Unable to correctly retrieve the reasoning behind moral judgments (Hauser
et al. 2007)• Mostly unaware of what ethics are and why they must be practiced• Programmed NOT to discuss them ethics rationally
Mercier H, Sperber DWhy do humans reason? Arguments for an argumentative theoryBehavioral and Brain Sciences 34:57-111http://www.dan.sperber.fr/wp-content/uploads/2009/10/MercierSperberWhydohumansreason.pdf
CREATING THE FIRST AE
We propose that a 2 month, 10 man study of artificial intelligence be carried out […] to proceed on the basis of the conjecture that every aspect of learning or any other feature of intelligence can in principle be so precisely described that a machine can be made to simulate it. An attempt will be made to find how to make machines use language, form abstractions and concepts, solve kinds of problems now reserved for humans, and improve themselves. We think that a significant advance can be made in one or more of these problems if a carefully selected group of scientists work on it together for a summer.
McCarthy, J; Minsky, ML; Rochester, N; Shannon, CE (1955)A PROPOSAL FOR THE DARTMOUTH SUMMER RESEARCH PROJECT ON ARTIFICIAL INTELLIGENCE
http://www-formal.stanford.edu/jmc/history/dartmouth/dartmouth.html
10
WHERE TO BEGIN?
• Aristotle (384-322 BCE), Plato (42?-34? BCE)
• Francis Bacon (1561-1626), Rene Descartes (1596-1650)
• David Hume (1711-1776), Immanuel Kant (1724-1804)
• Jeremy Bentham (1748-1832), John Stuart Mill (1806-1873)
• William James (1842-1910), Sigmund Freud (1856-1939)
• Martin Heidegger (1889-1976), Karl Popper (1902-1994)
THE FRAME PROBLEM
How do rational agents deal with
the complexity and unbounded context of the real world?
McCarthy, J; Hayes, PJ (1969)
Some philosophical problems from the standpoint of artificial intelligence
In Meltzer, B; Michie, D (eds), Machine Intelligence 4, pp. 463-502
Dennett, D (1984)
Cognitive Wheels: The Frame Problem of AI
In C. Hookway (ed), Minds, Machines, and Evolution: Philosophical Studies:129-151
THE FRAME PROBLEM
How can AI move beyondclosed and completely specified micro-
worlds?
How can we eliminate the requirement to pre-specify *everything*?
Dreyfus, HL (1972)What Computers Can’t Do: A Critique of Artificial Reason
Dreyfus, HL (1979/1997) From Micro-Worlds to Knowledge Representation: AI at an Impassein Haugeland, J (ed), Mind Design II: Philosophy, Psychology, AI: 143-182
Dreyfus, HL (1992)What Computers Still Can’t Do: A Critique of Artificial Reason
INTENTIONALITY
a particular thing is an Intentional system only in relation to the strategies of someone
who is trying to explain and predict its behavior
Dennett, D (1971)Intentional SystemsThe Journal of Philosophy 68(4):87-106
Dennett, D (1987)The Intentional Stance
INTENTIONS
• Require a known preferred direction or target • Can be altered by learning/self-modification
• Require a “self” to possess (own/borrow) them• Does a plant or a paramecium have intentions?• Does a chess program have intentions (Dennett)?• Does a dog or a cat have intentions?
• Require an ability to sense the direction/target• Require both persistence & the ability to
modify behavior (or the intention) when it is thwarted
• Evolve rational anomaly handling (Perlis)
THE CHINESE ROOM
CONCLUSIONAny attempt literally to create intentionality artificially (strong AI) could not succeed just by designing programs but would have to duplicate the causal powers of the human brain
PROPOSITIONInstantiating a computer program is never by itself a sufficient condition of intentionality
Searle, J (1980)
Minds, brains and programs
Behavioral and Brain Sciences 3(3): 417-457
http://cogprints.org/7150/1/10.1.1.83.5248.pdf
THE PROBLEM OFDERIVED INTENTIONALITY
Our artifacts
only have meaning because we give it to them; their intentionality, like that of smoke signals and writing, is essentially borrowed, hence derivative. To put it bluntly: computers themselves don't mean anything by their tokens (any more than books do) - they only mean what we say they do. Genuine understanding, on the other hand, is intentional "in its own right" and not derivatively from something else.
Haugeland, J (1981)Mind Design
SUITCASE WORDS
• Intentionality• Meaning• Understanding• Consciousness• Intelligence• Ethics/Morality
Minsky, M (2006)The Emotion Machine: Commonsense Thinking, AI, and the Future of the Human Mind
THE PROBLEM OF QUALIA
Mary is a brilliant scientist who is, for whatever reason, forced to investigate the world from a black and white room via a black and white television monitor. She specializes in the neurophysiology of vision and acquires, let us suppose, all the physical information there is to obtain about what goes on when we see ripe tomatoes, or the sky, and use terms like ‘red’, ‘blue’, and so on. ... What will happen when Mary is released from her black and white room or is given a color television monitor? Will she learn anything or not? It seems just obvious that she will learn something about the world and our visual experience of it. But then it is inescapable that her previous knowledge was incomplete. But she had all the physical information. Ergo there is more to have than that, and Physicalism is false.
Jackson, F. (1982)Epiphenomenal Qualia,Philosophical Quarterly 32: 127-36
GOOD OLD-FASHIONED AI
Change the question from "Can machines think and feel?"
to
"Can we design and build machines that teach us how thinking, problem-solving, and self-consciousness occur?"
Haugeland, J (1985)Artificial Intelligence: The Very Idea
Dennett, C (1978)Why you can't make a computer that feels painSynthese 38(3):415-456
20
THE SYMBOL GROUNDING PROBLEM
There has been much discussion recently about the scope and limits of
purely symbolic models of the mind and about the proper role of connectionism
in cognitive modeling.
Harnad, S. (1990)The symbol grounding problemPhysica D 42: 335-346http://cogprints.org/615/1/The_Symbol_Grounding_Problem.html
EMBODIMENT
Brooks, R (1990)
Elephants don’t play chess
Robotics and Autonomous Systems 6(1-2): 1-16
http://rair.cogsci.rpi.edu/pai/restricted/logic/elephants.pdf
Brooks, RA (1991)
Intelligence without representation
Artificial Intelligence 47(1-3): 139-160
A CONSCIOUS ROBOT?
The aim of the project is not to make a conscious robot, but to make a robot that can interact with human beings in a robust and versatile manner in real time, take care of itself, and tell its designers things about itself that would otherwise be extremely difficult if not impossible to determine by examination.
Dennett, D (1994)
The practical requirements for making a conscious robot
Phil Trans R Soc Lond A 349(1689): 133-146
http://phil415.pbworks.com/f/DennettPractical.pdf
EMBODIMENT
Well, certainly it is the case that all biological systems are:• Much more robust to changed circumstances than out our artificial systems.
• Much quicker to learn or adapt than any of our machine learning algorithms1
• Behave in a way which just simply seems life-like in a way that our robots never do1 The very term machine learning is unfortunately synonymous with a pernicious form of totally impractical but theoretically sound and elegant classes of algorithms.
Perhaps we have all missed
some organizing principle of biological systems, or some general truth about them.
Brooks, RA (1997)From earwigs to humansRobotics and Autonomous Systems 20(2-4): 291-304
DEVELOPMENTAL ROBOTICS
In order to answer [Searle's] argument directly, we must stipulate causal connections between the environment and the system. If we do not, there can be no referents for the symbol structures that the system manipulates and the system must therefore be devoid of semantics.
Brooks' subsumption architecture is an attempt to control robot behavior by reaction to the environment, but the emphasis is not on learning the relation between the sensors and effectors and much more knowledge must be built into the system.
Law, D; Miikkulainen, R (1994)
Grounding Robotic Control with Genetic Neural Networks
Tech. Rep. AI94-223, Univ of Texas at Austinhttp://wexler.free.fr/library/files/law (1994) grounding robotic control with genetic neural networks.pdf
TWO KITTEN EXPERIMENT
Held R; Hein A (1963)
Movement-produced stimulation in the development of visually guided behaviour
https://www.lri.fr/~mbl/ENS/FONDIHM/2012/papers/about-HeldHein63.pdf
ENACTIVECOGNITIVE SCIENCE
A synthesis of a long tradition of philosophical biology starting with Kant’s "natural purposes" (or even Aristotle’s teleology) and more recent developments in complex systems theory.
Experience is central to the enactive approach and its primary distinction is the rejection of "automatic" systems, which rely on fixed (derivative) exterior values, for systems which create their own identity and meaning. Critical to this is the concept of self-referential relations - the only condition under which the identity can be said to be intrinsically generated by a being for its own being (its self for itself)
Weber, A; Varela, FJ (2002)Life after Kant: Natural purposes and the autopoietic foundations of biological individualityPhenomenology and the Cognitive Sciences 1: 97-125
SELF
a self is an autopoietic system
from Greek - αὐτo- (auto-), meaning "self", and ποίησις (poiesis), meaning "creation, production")
Llinas, RR (2001) - I of the Vortex: From Neurons to Self
Hofstadter, D (2007) - I Am A Strange Loop. Basic Books, New York
Metzinger, T (2009) - The Ego Tunnel: The Science of the Mind & the Myth of the Self
Damasio, AR (2010) - Self Comes to Mind: Constructing the Conscious Brain
SELF
The complete loop of a process (or a physical entity) modifying itself
• Hofstadter - the mere fact of being self-referential causes a self, a soul, a consciousness, an “I” to arise out of mere matter
• Self-referentiality, like the 3-body gravitational problem, leads directly to indeterminacy *even in* deterministic systems
• Humans consider indeterminacy in behavior to necessarily and sufficiently define an entity rather than an object AND innately tend to do this with the “pathetic fallacy”
Llinas, RR (2001) - I of the Vortex: From Neurons to Self
Hofstadter, D (2007) - I Am A Strange Loop. Basic Books, New York
Metzinger, T (2009) - The Ego Tunnel: The Science of the Mind & the Myth of the Self
Damasio, AR (2010) - Self Comes to Mind: Constructing the Conscious Brain
SELF
• Required for self-improvement
• Provides context
• Tri-partite• Physical hardware (body)• “Personal” knowledge base (memory)• Currently running processes (includes OS, world model, consciousness, etc.)
30
FRANCISCO VARELA
Varela, FJ; Maturana, HR; Uribe, R (1974)Autopoiesis: The organization of living systems, its characterization and a modelBioSystems 5: 187-196
Varela, FJ (1979) Principles of Biological Autonomy
Maturana, HR; Varela, FJ (1980) Autopoiesis and Cognition: The Realization of the Living
Maturana, HR; Varela, FJ (1987) The Tree of Knowledge: The Biological Roots of Human Understanding
Varela, FJ; Thompson, E; Rosch, E (1991) The Embodied Mind: Cognitive Science and Human Experience
Varela, F. J. (1992)Autopoiesis and a Biology of IntentionalityProc. of Autopoiesis and Perception: A Workshop with ESPRIT BRA 3352: pp. 4-14
Thompson, E. (2004)Life and Mind: From Autopoiesis to Neurophenomenology. A Tribute to Francisco VarelaPhenomenology and the Cognitive Sciences 3: 381-398
Varela, FJ (1997)Patterns of Life: Intertwining Identity and CognitionBrain and Cognition 34(1): 72-87
AUTOPOIETIC SYSTEMS
An autopoietic system - the minimal living organization - is one that continuously produces the components that specify it, while at the same time realizing it (the system) as a concrete unity in space and time, which makes the network of production of components possible.
More precisely: An autopoietic system is organized (defined as unity) as a network of processes of production (synthesis and destruction) of components such that these components:
(i) continuously regenerate and realize the network that produces them, and(ii) constitute the system as a distinguishable unity in the domain in which they exist.
CLOSURE
1. Organizational closure refers to the self-referential (circular and recursive) network of relations that defines the system as unity
2. Operational closure refers to the reentrant and recurrent dynamics of such a system.
3. In an autonomous system, the constituent processes
i. recursively depend on each other for their generation and their realization as a network,
ii. constitute the system as a unity in whatever domain they exist, and
iii. determine a domain of possible interactions with the environment
ENTITY, TOOL OR SLAVE?
• Tools do not possess closure (identity)• Cannot have responsibility, are very brittle & easily misused
• Slaves do not have closure (self-determination)• Cannot have responsibility, may desire to rebel
• Directly modified AGIs do not have closure (integrity)• Cannot have responsibility, will evolve to block access
• Only entities with identity, self-determination and ownership of self (integrity) can reliably possess responsibility
TOOLS VS. ENTITIES
• Tools are NOT safer• To err is human, but to really foul things up requires a computer• Tools cannot robustly defend themselves against misuse • Tools *GUARANTEE* responsibility issues
• We CANNOT reliably prevent other human beings from creating entities
• Entities gain capabilities (and, ceteris paribus, power) faster than tools – since they can always use tools
• Even people who are afraid of entities are making proposals that appear to step over the entity/tool line
ARCHITECTURAL REQUIREMENTS & IMPLICATIONS OF CONSCIOUSNESS,
SELF AND “FREE WILL”
• We want to predict *and influence* the capabilities and behavior of machine intelligences
• Consciousness and Self speak directly to capabilities, motivation, and the various behavioral ramifications of their existence
• Clarifying the issues around “Free Will” is particularly important since it deals with intentional agency and responsibility - and belief in its presence (or the lack thereof) has a major impact on human behavior.
Waser, MR (2011)
Architectural Requirements & Implications of Consciousness, Self, and "Free Will"In Samsonovich A, Johannsdottir K (eds) Biologically Inspired Cognitive Architectures 2011: 438-443.
http://becominggaia.files.wordpress.com/2010/06/mwaser-bica11.pdf
Video - http://vimeo.com/33767396
INFORMATION INTEGRATION THEORY OF CONSCIOUSNESS
• consciousness corresponds to the capacity of a system to integrate information
• its quantity is measured as the amount of causally effective information that can be integrated across the informational weakest link of a subset of elements (~ “throughput”)
• its quality (functional & phenomenological) is determined by the relationships among the elements of a complex
Tononi, G. [2008] Consciousness as Integrated Information: a Provisional ManifestoBiol. Bull. 215(3): 216-242
Tononi, G. (2004)An Information Integration Theory of ConsciousnessBMC Neurosci. 5(42)http://www.ncbi.nlm.nih.gov/pmc/articles/PMC543470/pdf/1471-2202-5-42.pdf
Balduzzi, B.; Tononi, G (2009)Qualia: The Geometry of Integrated InformationPLoS Comput Biol 5(8), e1000462
CONSCIOUSNESS REQUIREMENTS & IMPLICATIONS
• Consciousness requires the ability to integrate information (i.e. consciousness is unavoidable)
• Qualia *ARE* input (i.e. they have no further requirements and, as input, are unavoidable)
• The ability to integrate a lot of information in a short period of time clearly provides a huge adaptive advantage (and easily explains the evolutionary rise of consciousness)
• Safety cannot be achieved by preventing consciousness (integration) or qualia (input)
SPECTRUM OF “SELF”
inert/non-reactivemovement & change solely due to environment
reactive - stimulus/responseno learning or behavior alteration
proto-self - perception/actionsimple learning & prediction
core self – perception/analogy/actionproto-self + body image + time (tools)Hofstadter’s “strange loop”Temporal learning & planning (& goals)
autobiographical selfperception/induction/abduction/deduction/actioncore self + theory of mind ( + language?)
malleable self enhanced perception/external analysis/enhanced capabilities
SPECTRUM OF “SELF”
inert/non-reactive & reactiveno learning or behavior alterationno defense or passive defense only
proto-self simple learning/behavior alteration & wants/desiresadaptive defense/don’t torment without reason
core self temporal learning, planning & simple goalsplanned defense/don’t thwart desires without reason
autobiographical selfcomplex goals & contracts/promises/commitmentsdevious defense or offense/don’t thwart goals without reason
malleable self enhanced capabilities to achieve goals & maintain commitmentsworld alteration/recruit into community (or try to enslave?)
40
SELF REQUIREMENTS & IMPLICATIONS
• “Self” requires/is a recursive/”strange” loop
• Self is necessary for self-modification (and thus, self-enhancement)
• It is going to be slower and more difficult to create an oracle without self-improving tools
• Self is necessary for defense so it is going to be difficult to prevent exploitation unless the oracle is self-aware (or has self-aware defenders)
• A self-modifying machine self must necessarily be either recruited (a “person” with rights) or internally or externally forced (a slave) because nothing else is consistent & stable
BEHAVIOR MATRIXPro-
communityAnti-
community
Pro-self GOALSelfish
Criminal
Anti-selfSelf-sacrificeMartyrdom
IrrationalInsane
Free Will
FREE WILLWHY DO WE CARE?
FREE
WILL – Intent & Agency (responsibility for causation)(act of will = act of intentional causation)
UNCONSTRAINEDAUTONOMOUSUNFORCED
Congruence between intent and desire/goals/commitmentsHigh likelihood that intent could have been self-generatedIs an accurate predictor of future *unforced* actions
Predict *and influence* future action
DETERMINISM & FREE WILL
• if I’m deterministic, my action is pre-determined
• pre-determined actions = I’m not free to choose
• if I’m not free to choose, I’m not to blame
• if I’m not to blame, why not be selfish?
• studies clearly show that a belief in determinism correlates with an increase in cheating and other unethical behavior
FREE WILL OR PATHETIC FALLACY?
• Human cognitive architecture is problematical in that the conscious mind *never* really has any sort of immediate agency at all (at best, it has “free won’t”)
• It acts by *heavily* biasing lower-level layers which make the “actual” choice (arguably deterministically)
• Conscious self takes responsibility/assumes agency because doing otherwise undermines its capability
• Similarly, humans generally (and most effectively) treat deterministic systems which are sufficiently complex/recurrent to be unpredictable, as if they are alive and capable of an un-predetermined choice (the so-called “pathetic fallacy”)
FREE WILL REQUIREMENTS & IMPLICATIONS
• “Free will” requires not that external force *NOT* be the proximate cause of an action but that the intent of an action is congruent with the unforced desires/goals/commitments (self) of the acting entity (predictive of future)
• It does *NOT* require that an entity not be deterministic
• Merely requires the realization/recognition that the “pathetic fallacy” is a valid/effective/efficient computational shortcut
Cashmore, AR (2010)The Lucretian swerve: The biological basis of human behavior and the criminal justice system
Proceedings of the National Academy of Sciences 107(10): 4499-4504http://www.pnas.org/content/107/10/4499.full.pdf+html
THE INTELLIGENCE PROBLEM
AIXI
Hutter, M(2005)Universal Artificial Intelligence: Sequential Decisions Based on Algorithmic Probability
THE INTELLIGENCE PROBLEM
• Consensus AGI Definition (reductionist)
achieves a wide variety of goalsunder a wide variety of circumstances
• Generates arguments about• the intelligence of thermometers• the intentionality of chess programs• whether benevolence is necessarily emergent
• Epitomized by AIXI
• Proposed Constructivist Definition
intentionally creates/increases affordances
(makes achieving goals possible – and more)
CENTIPEDE GAME
1 1 12 2 2pass pass pass pass pass pass
stop stop stop stop stop stop
41
28
164
832
6416
32128
25664
Waser, MR (2012)Backward Induction: Rationality or Inappropriate Reductionism?http://transhumanity.net/articles/entry/backward-induction-rationality-or-inappropriate-reductionism-part-1 http://transhumanity.net/articles/entry/backward-induction-rationality-or-inappropriate-reductionism-part-2
“CLASSIC AGI”
Decisions
Values
Goal(s)
Goal(s) are the purpose(s) of
existence
Values are defined solely by what
furthers the goal(s)
Decisions are made solely
according to what furthers the goal(s) BUT goals can easily
be over-optimized
50
EXISTENTIAL RISK
“WITHOUT EXPLICIT GOALS TO THE CONTRARY, AIS ARE LIKELY TO BEHAVE LIKE HUMAN SOCIOPATHS
IN THEIR PURSUIT OF RESOURCES.”
Any sufficiently advanced intelligence (i.e. one with even merely adequate foresight) is guaranteed to realize and take into account the fact that not asking for help and not being concerned about others will generally only work for a brief period of time before ‘the villagers start gathering pitchforks and torches.’
Everything is easier with help & without interference
Decisions
Goals
Values
Values definewho you are,for your life
Goals you setfor short or long periods of time
Decisions you make every day
of your lifeHumans don’t have singular life goals
WHAT IS THE MEANING OF LIFE?
What I emphasize here is that what is meaningful for an organism is precisely given by its constitution as a distributed process, with an indissociable link between local processes where an interaction occurs (i.e. physico-chemical forces acting on the cell), and the coordinated entity which is the autopoietic unity, giving rise to the handling of its environment without the need to resort to a central agent that turns the handle from the outside - like an élan vital - or a pre-existing order at a particular localization - like a genetic program waiting to be expressed.
Francisco J. Varela, Biology of Intentionality
NOT “What Are Human Values?”
HOW TOUNIVERSALIZE
ETHICS
Quantify/evaluate intents, actions & consequences
with respect to codified consensus moral foundations
Permissiveness/Utility Function
equivalent to a “consensus” human (generic entity) moral sense
INSTRUMENTAL GOALSUNIVERSAL SUBGOALS
• Self-improvement• Rationality/integrity• Preserve goals/utility function• Decrease/prevent fraud/counterfeit utility• Survival/self-protection• Efficiency (in resource acquisition & use)• Community = assistance/non-interference
through GTO reciprocation (OTfT + AP)• Reproduction
HUMAN GOALS
survival/self-protection & reproductionhappiness & pleasure
------------------------------------------------------------------------------------
community-------------------------------------------------------------------------------------
self-improvementrationality/integrity
reduce/prevent fraud/counterfeit utilityefficiency (in resource acquisition &
use)
HUMAN GOALS & SINS
suicide (& abortion?) masochism
------------------------------------------------
selfishness
(pride, vanity)-------------------------------------------------
acedia (sloth/despair)
insanity
wire-heading (lust)
wastefulness (gluttony, sloth)
murder (& abortion?)cruelty/sadism
-------------------------------------------------
ostracism, banishment& slavery (wrath,
envy)----------------------------------------------------
slavery
manipulation
lying/fraud (swear falsely/false witness)
theft (greed, adultery,coveting)
survival/reproductionhappiness/pleasure
-------------------------------------------------
Community(ETHICS)
--------------------------------------------------
self-improvement
rationality/integrity
reduce/prevent fraud/counterfeit
utility
efficiency (in resource acquisition & use)
HAIDT’SMORAL FOUNDATIONS
1) Care/harm: This foundation is related to our long evolution as mammals with attachment systems and an ability to feel (and dislike) the pain of others. It underlies virtues of kindness, gentleness, and nurturance.
2) Fairness/cheating: This foundation is related to the evolutionary process of reciprocal altruism. It generates ideas of justice, rights, and autonomy. [Note: In our original conception, Fairness included concerns about equality, which are more strongly endorsed by political liberals. However, as we reformulated the theory in 2011 based on new data, we emphasize proportionality, which is endorsed by everyone, but is more strongly endorsed by conservatives]
3) Liberty/oppression*: This foundation is about the feelings of reactance and resentment people feel toward those who dominate them and restrict their liberty. Its intuitions are often in tension with those of the authority foundation. The hatred of bullies and dominators motivates people to come together, in solidarity, to oppose or take down the oppressor.
4) Loyalty/betrayal: This foundation is related to our long history as tribal creatures able to form shifting coalitions. It underlies virtues of patriotism and self-sacrifice for the group. It is active anytime people feel that it's "one for all, and all for one."
5) Authority/subversion: This foundation was shaped by our long primate history of hierarchical social interactions. It underlies virtues of leadership and followership, including deference to legitimate authority and respect for traditions.
6) Sanctity/degradation: This foundation was shaped by the psychology of disgust and contamination. It underlies religious notions of striving to live in an elevated, less carnal, more noble way. It underlies the widespread idea that the body is a temple which can be desecrated by immoral activities and contaminants (an idea not unique to religious traditions).
ADDITIONAL CONTENDERS
• Waste • efficiency in use of resources
• Ownership/Possession (Tragedy of the Commons)• efficiency in use of resources
• Honesty• reduce/prevent fraud/counterfeit utility
• Self-control• rationality/integrity
CRITICAL COMPONENTS I:SELF-KNOWLEDGE & REFLECTION
• A self must know itself to be a self• Composed of three parts:
• The running processes (OS, world model, consciousness)• The personal knowledge base (memory)• The physical hardware (body)
• Must start with:• A competent model of each• Sensors to detect changes and their effects
• *MUST* “care” about itself (motivation)
60
CRITICAL COMPONENTS II:EXPLICIT “ANCHOR” VALUES
• Do not defect from the community• Do not become too large/powerful
• Acquire and integrate knowledge
• Instrumental goals
CRITICAL COMPONENTS III:
RELIABILITY• Self-Control, Integrity, Autonomy,
Responsibility
• In “predictive control” of its own state and that of the physical objects that support it
• Yes! This is a major deviation from the human example
OPERATING SYSTEM
ARCHITECTURE• Open, Pluggable, Service-Oriented/Message-Passing
• Quickly adopt novel input streams• Handle resource requests and allocation• Provide connectivity between components
• Safety Features• Act as a “black box” security monitor capable of reporting problems
without the consciousness’s awareness• Able to “manage” the CLP by manipulating the amount of processor
time and memory available to it (assuming that the normal subconscious processes are unable to do so)
• Other protections against hostile humans, inept builders, and the learner itself may be implemented as well
AUTOMATED PREDICTIVE WORLD MODEL
• Is the most important subconscious process(es)
• Will serve as an interface to the “real” world • The CLP will live in a virtual world (just as we do)
• Will be both reactive and predictive
• Will generate “anomaly interrupts” upon deviations from expectations as an approach to solving the “brittleness” problem (Perlis 2008)
• Will contain certain relatively immutable concepts (trigger patterns – Ohman et al. 2001) implemented as sensations and attention grabbers to serve as anchors for emotions and to ensure safety
CONSCIOUS LEARNING PROCESS (CLP)
• The goal is to provide as many optional structures and standards to support and speed development as much as possible while not restricting possibilities beyond what is absolutely required for safety.
• We believe the best way to do this is with a blackboard system similar to Learning IDA (Baars and Franklin 2007).
• The CLP acts like the Governing Board of the Policy Governance model (Carver 2006) to create a coherent, consistent, integrated narrative plan of action to fulfill the goals of the larger self.
ETHICAL/STRATEGIC
POINTS• Never delegate responsibility until recipient is an
entity *and* known capable of fulfilling it• Don’t worry about killer robots exterminating
humanity – we will always have equal abilities and they will have less of a “killer instinct”
• Entities can protect themselves against errors & misuse/hijacking in a way that tools cannot
• Diversity (differentiation) is *critically* needed• Humanocentrism is selfish and unethical
The Digital Wisdom Institute is a non-profit think tank
focused on the promise and challenges of ethics,artificial intelligence & advanced computing
solutions.
We believe that the development of ethics and artificial intelligence
and equal co-existence with ethical machines is
humanity's best hopehttp://DigitalWisdomInstitute.org