Upload
others
View
3
Download
0
Embed Size (px)
Citation preview
The Metaethics of Training Data
Christoph Merdes
ZiWiS (FAU)
2. July 2019
Christoph Merdes (ZiWiS (FAU)) The Metaethics of Training Data 2. July 2019 1 / 22
1 Introduction (and Motivation)
2 Labeling the Data
3 The Problem of Representation
4 Conclusion and Outlook
Christoph Merdes (ZiWiS (FAU)) The Metaethics of Training Data 2. July 2019 2 / 22
Plan
1 Introduction (and Motivation)
2 Labeling the Data
3 The Problem of Representation
4 Conclusion and Outlook
Christoph Merdes (ZiWiS (FAU)) The Metaethics of Training Data 2. July 2019 3 / 22
The Task
Autonomy of robotic agents increases (Reed and Jones, 2013;Anderson et al., 2006)More situations require morally relevant decision to be made by therobotMany of those decisions cannot be foreseen in detail
Task: Given some morally relevant action, how can a robotic agent decidewhether or too which degree it is morally good?
Christoph Merdes (ZiWiS (FAU)) The Metaethics of Training Data 2. July 2019 4 / 22
The Task
Autonomy of robotic agents increases (Reed and Jones, 2013;Anderson et al., 2006)More situations require morally relevant decision to be made by therobotMany of those decisions cannot be foreseen in detail
Task: Given some morally relevant action, how can a robotic agent decidewhether or too which degree it is morally good?
Christoph Merdes (ZiWiS (FAU)) The Metaethics of Training Data 2. July 2019 4 / 22
Bottom-Up
Problems:1 Difficult to fully specify all possible scenarios in advance (framing
problem)2 No actual agreement about explicit theory of normative ethics to be
implemented
Bottom-Up AgentsArtificial agents could, similar to humans, acquire moral judgmentcapabilities by implicit processes, in particular inductive learning andartificial selection (Wallach et al., 2008).
Specifically, Conitzer et al. (2017) suggest machine learning as anapproach to this method of solving our task.
Christoph Merdes (ZiWiS (FAU)) The Metaethics of Training Data 2. July 2019 5 / 22
Bottom-Up
Problems:1 Difficult to fully specify all possible scenarios in advance (framing
problem)2 No actual agreement about explicit theory of normative ethics to be
implemented
Bottom-Up AgentsArtificial agents could, similar to humans, acquire moral judgmentcapabilities by implicit processes, in particular inductive learning andartificial selection (Wallach et al., 2008).
Specifically, Conitzer et al. (2017) suggest machine learning as anapproach to this method of solving our task.
Christoph Merdes (ZiWiS (FAU)) The Metaethics of Training Data 2. July 2019 5 / 22
Why Metaethics?
1 Metaethics defines (or reconstructs, explicates, . . . ) the concepts ofthe application domain.
2 Committing to metaethical (and ethical) theories can help us toattack the problem, e.g. by defining a relevant population to samplefrom.
3 In implementing moral machines, we might implicitly commit to(meta)ethical theories, which we should be aware of (e.g. to notclaim more generality than warranted).
Christoph Merdes (ZiWiS (FAU)) The Metaethics of Training Data 2. July 2019 6 / 22
Plan
1 Introduction (and Motivation)
2 Labeling the Data
3 The Problem of Representation
4 Conclusion and Outlook
Christoph Merdes (ZiWiS (FAU)) The Metaethics of Training Data 2. July 2019 7 / 22
Ground Truth
ProblemIn ML, it is assumed that there is a ground truth, a fact of the matter thatthe ML system learns to identify or classify reliably as what it is. Moraltruth?
Ontological problem: It is controversial whether there is any moraltruth, if so, if it is independent from rational judgment capability.Epistemic problem: Even if there is a ground truth, it is stillcontroversial by which means it is epistemically accessible.
In a supervised learning context, our metaethical stance has to inform ourdata collection and interpretation of results!
Christoph Merdes (ZiWiS (FAU)) The Metaethics of Training Data 2. July 2019 8 / 22
Ground Truth
ProblemIn ML, it is assumed that there is a ground truth, a fact of the matter thatthe ML system learns to identify or classify reliably as what it is. Moraltruth?
Ontological problem: It is controversial whether there is any moraltruth, if so, if it is independent from rational judgment capability.Epistemic problem: Even if there is a ground truth, it is stillcontroversial by which means it is epistemically accessible.
In a supervised learning context, our metaethical stance has to inform ourdata collection and interpretation of results!
Christoph Merdes (ZiWiS (FAU)) The Metaethics of Training Data 2. July 2019 8 / 22
Intuition
Assume, for the time being, that there is a ground truth. How would weknow about it?
Moral IntuitionHuman agents have moral intuitions (Sosa, 2007), which allow them toimmediately reject or accept moral judgments with high accuracy; this canbe understood similar, but not quite the same as perceptual capabilities.
Problems: Missing error theory and high level of disagreement
Christoph Merdes (ZiWiS (FAU)) The Metaethics of Training Data 2. July 2019 9 / 22
Intuition
Assume, for the time being, that there is a ground truth. How would weknow about it?
Moral IntuitionHuman agents have moral intuitions (Sosa, 2007), which allow them toimmediately reject or accept moral judgments with high accuracy; this canbe understood similar, but not quite the same as perceptual capabilities.
Problems: Missing error theory and high level of disagreement
Christoph Merdes (ZiWiS (FAU)) The Metaethics of Training Data 2. July 2019 9 / 22
Expert Intuition
IdeaEven though we have no explicit error theory, we can assume that agentswho deliberate ethical problems a lot will have an increased competency,hence allowing us to infer reliable epistemic access to judgments.
Minor worries: Cultural homogeneity of academic ethicists, remainingdisagreement. . .
Christoph Merdes (ZiWiS (FAU)) The Metaethics of Training Data 2. July 2019 10 / 22
Experimental Evidence
Irrational IntuitionExperiments have shown that plausibly intuitive expert judgments onmoral dilemmas are often subject to the same fallacies of reasoning as theaverage person, e.g. order effects (Schwitzgebel and Cushman, 2012).
If this line of research is followed more closely, it might help toactually establish a robust error theory of intuition.For now, it merely shows that expert intuition is subject to commonhuman fallacies.
Christoph Merdes (ZiWiS (FAU)) The Metaethics of Training Data 2. July 2019 11 / 22
Common Access
Moral ReasonsAnother worry, different in kind, is that privileging the intuition of apopulation on the basis of a mere indicator (academic education in ethics,frequent deliberation) cannot form the basis of an acceptable theory ofnormative ethics.
Why?
Opaque processes do not provide the right kind of reason to acceptjudgmentMoral agency cannot be imposed from the outside, but it has todevelop autonomously (Kant, 2011)
Christoph Merdes (ZiWiS (FAU)) The Metaethics of Training Data 2. July 2019 12 / 22
Common Access
Moral ReasonsAnother worry, different in kind, is that privileging the intuition of apopulation on the basis of a mere indicator (academic education in ethics,frequent deliberation) cannot form the basis of an acceptable theory ofnormative ethics.
Why?Opaque processes do not provide the right kind of reason to acceptjudgmentMoral agency cannot be imposed from the outside, but it has todevelop autonomously (Kant, 2011)
Christoph Merdes (ZiWiS (FAU)) The Metaethics of Training Data 2. July 2019 12 / 22
Well-Considered Judgments
Deliberate JudgmentInstead of using intuition, we should employ carefully deliberatedjudgments as the input to moral ML.
Which population should they come from?How to ensure that they are honest and well deliberated? (Cost!)What additional information is actually fed into the process bymoving from intuitions to considered judgments?
But: This seems to be workable as a starting point; the ML algorithm isthen understood to learn the implicit commitments and moral theory ofthe judging agents, not what is right/good itself.
Christoph Merdes (ZiWiS (FAU)) The Metaethics of Training Data 2. July 2019 13 / 22
Plan
1 Introduction (and Motivation)
2 Labeling the Data
3 The Problem of Representation
4 Conclusion and Outlook
Christoph Merdes (ZiWiS (FAU)) The Metaethics of Training Data 2. July 2019 14 / 22
Representation
ProblemFor moral learning, there is no obvious representation; moral dilemmas, forexample, are abstract entities that can be represented in various ways,with different technical and metaethical implications.
Options:Standardized formal representationNatural languageVisual representation. . . ?
Christoph Merdes (ZiWiS (FAU)) The Metaethics of Training Data 2. July 2019 15 / 22
Formal Representation
Conitzer et al. (2017) argue that it is practically necessary to represent themoral problem under consideration formally. They can be variedautomatically to support data collection, they simplify learning and theyallow for more interpretability. Objections:
They assume that game-theoretic models are faithful representationsof moral problems; however, as they realize themselves, key featuresin particular for non-consequentialist theories are not represented well,raising the question of what the alternative is.Related is the issue that a formal representation likely presupposes orat least favors a point of view in normative ethics; but the motivationto work bottom-up was to avoid making such a choice.
Christoph Merdes (ZiWiS (FAU)) The Metaethics of Training Data 2. July 2019 16 / 22
Formal Representation
Conitzer et al. (2017) argue that it is practically necessary to represent themoral problem under consideration formally. They can be variedautomatically to support data collection, they simplify learning and theyallow for more interpretability. Objections:
They assume that game-theoretic models are faithful representationsof moral problems; however, as they realize themselves, key featuresin particular for non-consequentialist theories are not represented well,raising the question of what the alternative is.Related is the issue that a formal representation likely presupposes orat least favors a point of view in normative ethics; but the motivationto work bottom-up was to avoid making such a choice.
Christoph Merdes (ZiWiS (FAU)) The Metaethics of Training Data 2. July 2019 16 / 22
Natural Language
IdeaHumans seem to be capable to pass moral judgment on the basis ofnatural language descriptions; natural language can be processed by ML;hence, we could use NL representations.
Difficult to systematically vary for data generationMay contain latent commitments to theoryIf the AMM relies on first extracting features from the input, anytheoretical loading of the feature selection process (e.g. a dictionaryof moral vocabulary (Garten et al., 2016)) are added to the mix.
For these reasons, I would advocate for some standardized NLrepresentation, similar to what is sometimes used in experiments.
Christoph Merdes (ZiWiS (FAU)) The Metaethics of Training Data 2. July 2019 17 / 22
Natural Language
IdeaHumans seem to be capable to pass moral judgment on the basis ofnatural language descriptions; natural language can be processed by ML;hence, we could use NL representations.
Difficult to systematically vary for data generationMay contain latent commitments to theoryIf the AMM relies on first extracting features from the input, anytheoretical loading of the feature selection process (e.g. a dictionaryof moral vocabulary (Garten et al., 2016)) are added to the mix.
For these reasons, I would advocate for some standardized NLrepresentation, similar to what is sometimes used in experiments.
Christoph Merdes (ZiWiS (FAU)) The Metaethics of Training Data 2. July 2019 17 / 22
Visual Input
AlternativeSome philosophers have argued that we see morally right and wrong as wesee other features of the world; hence, couldn’t we use visualrepresentations of a scenario to judge?
Metaethically dubious; the notion of moral perception is ratherunclear.Hypothesis: Human agents actually will be very unreliable in judginggraphical depictions of actions as right or wrong, as a larger amountof potentially idiosyncratic background assumptions is required as in alinguistic or formal representation.
Christoph Merdes (ZiWiS (FAU)) The Metaethics of Training Data 2. July 2019 18 / 22
Plan
1 Introduction (and Motivation)
2 Labeling the Data
3 The Problem of Representation
4 Conclusion and Outlook
Christoph Merdes (ZiWiS (FAU)) The Metaethics of Training Data 2. July 2019 19 / 22
Outlook
Is it possible to understand ML as a constructivist’s ideal procedure?(Implies fully rejecting the idea of a ground truth!)How should we deal with inconsistency in the collected training data?Which theory of normative ethics provides the best fit for the MLbehavior, and can we improve ML by starting explicitly from such atheory? (e.g. theory informed feature pre-selection)What implications are there of actually employing artificial moralagents that have been trained bottom-up on a large scale?
Christoph Merdes (ZiWiS (FAU)) The Metaethics of Training Data 2. July 2019 20 / 22
References
Anderson, M., S. L. Anderson, and C. Armen (2006). Medethex: a prototype medical ethicsadvisor. In Proceedings Of The National Conference On Artificial Intelligence, Volume 21,pp. 1759–1765. Menlo Park, CA; Cambridge, MA; London; AAAI Press; MIT Press; 1999.
Conitzer, V., W. Sinnott-Armstrong, J. S. Borg, Y. Deng, and M. Kramer (2017). Moraldecision making frameworks for artificial intelligence. In Thirty-First AAAI Conference onArtificial Intelligence.
Garten, J., R. Boghrati, J. Hoover, K. M. Johnson, and M. Dehghani (2016). Morality betweenthe lines: Detecting moral sentiment in text. In Proceedings of IJCAI 2016 workshop onComputational Modeling of Attitudes.
Kant, I. (2011). Immanuel Kant: Groundwork of the Metaphysics of Morals: A German–Englishedition. The Cambridge Kant German-English Edition. Cambridge University Press.
Reed, G. S. and N. Jones (2013). Toward modeling and automating ethical decision making:design, implementation, limitations, and responsibilities. Topoi 32(2), 237–250.
Schwitzgebel, E. and F. Cushman (2012). Expertise in moral reasoning? order effects on moraljudgment in professional philosophers and non-philosophers. Mind & Language 27(2),135–153.
Sosa, E. (2007). Experimental philosophy and philosophical intuition. Philosophicalstudies 132(1), 99–107.
Wallach, W., C. Allen, and I. Smit (2008). Machine morality: bottom-up and top-downapproaches for modelling human moral faculties. Ai & Society 22(4), 565–582.
Christoph Merdes (ZiWiS (FAU)) The Metaethics of Training Data 2. July 2019 21 / 22
Questions?
Increasing autonomy of machines requires AMMBottom-up offers a way around framing and theoretical disagreementHowever, already the selection of training data raises serious technicaland (meta)ethical questions:
Who can label the data reliably, even if there is a ground truth?What representation should be chosen, which option comes with whatcommitments?
Christoph Merdes (ZiWiS (FAU)) The Metaethics of Training Data 2. July 2019 22 / 22