Upload
nasnan
View
31
Download
0
Tags:
Embed Size (px)
DESCRIPTION
Review. What is Critical Thinking?. There are two basic decisions to make in life: 1. Decide what to believe: What do I believe? 2. Decide what to do: What do I do?. What Should I Believe?. Is there any evidence to support the claim? - PowerPoint PPT Presentation
Citation preview
Review
What is Critical Thinking?
There are two basic decisions to make in life:
1. Decide what to believe: What do I believe?
2. Decide what to do: What do I do?
What Should I Believe?
Is there any evidence to support the claim?
Is the evidence reliable and trustworthy? How reliable is it? Should you accept it?
Does the evidence actually support the claim?
Is there other evidence you should consider?
What Should I Do?
What outcomes can my choice lead to?
Does the outcome of my decision depend on factors other than what I choose to do?
What is the likelihood that deciding to take a specific action will lead to a specific outcome?
Which outcomes do I most prefer?
CONTEXT
The Importance of Context
When someone provides you with evidence for a truth-claim, you have to ask:
Does this evidence really support the claim?
Is there other relevant evidence I should look for before assessing the claim?
Taking Quotes out of Context
Every quote is taken out of its original context. That’s why it’s a quote and not a reproduction. This is fine if the person providing the quote has also provided enough context that the quote does not mislead you into thinking someone meant something they did not mean.
Numbers, charts, and figures can be taken out of context as well, in misleading ways.
Trends and Truncated Charts
When you see a chart describing what happened in the last 10 years, or last 10 months, or last 10 weeks, you’re seeing the information out of context. What happened before then?
When the y-axis of a chart does not start at zero (a “zero baseline”), you’re seeing what’s called a “truncated”, “torn” or “gee whiz” graph. You’re seeing the information it presents out of context.
Misleading Figures
Absolute numbers can be misleading. India has about 330,000 people in prison. A lot or a little?
Rates can also be misleading. Russia has 615 prisoners for every 100,000 people. A lot or a little?
Even comparisons can be misleading. If country X has twice as many people in prison today than in 1990, is that bad? What if they have twice as many residents than in 1990?
COGNITIVE BIASES
The Clustering Illusion
Here is 20 random coin flips:
XXXXOOXOOXXOOXOOXOOO
That doesn’t look random. But it is. The coin lands the same as the previous toss 10 times and different from the previous toss 9 times.
Confabulation
The human mind sees patterns where there is only randomness. It also freely invents reasons and explanations to “make the world make sense.”When we encounter random data, we see a pattern that isn’t there. And we explain why there should be a pattern. This can make our bad beliefs difficult to abandon.
Regression to the Mean
Whenever two variables are imperfectly correlated, extreme values of one variable tend to be paired with less extreme values of the other.
The regression fallacy involves attributing a causal explanation to what is nothing more than regression to the mean.
Confirmation Bias
Even though evaluating predictions requires looking at both the rates of true positives among predicted positives, and the rates of false positives among predicted negatives, human beings have a tendency to only consider true positives (and to a lesser extent, true negatives) when evaluating predictions or other claims of imperfect correlation.
The Problem of Absent Data
Sometimes it’s not just that we only look for or evaluate the positive evidence, but that there is no negative evidence. This can lead us to think we have very well-confirmed beliefs when we do not.
Bias
Our expectations often influence how we evaluate claims and evidence. We easily accept as true those things that we expect to be true, but are much more skeptical about things that are unexpected.
Disconfirmation Bias
Disconfirmation bias is the tendency to subject evidence against your views to a greater degree of scrutiny than evidence in favor of your views.
It is a double-standard for evidence evaluation.
Conjunction Fallacy
Linda is 31 years old, single, outspoken, and very bright. She majored in philosophy. As a student, she was deeply concerned with issues of discrimination and social justice, and also participated in anti-nuclear demonstrations.Which is more probable?1. Linda is a bank teller.2. Linda is a bank teller and is active in the feminist movement.
Conjunction Fallacy
Always, the probability of two events happening (Linda being a bank teller AND Linda being a feminist) is less than the probability of just one of those events happening (for example, Linda being a bank teller).
The illusion that the opposite is true especially occurs in cases where one event explains the other.
Representativeness
Our (false) judgment that Linda is more likely to be a feminist bank teller than to just be a bank teller is an example of how we judge the truth of claims based on how “representative” they are.The representativeness heuristic is partly responsible for other biases as well: “clusters” are not representative of random sequences, and testing positive for AIDS is not representative of not having AIDS (base rate).
Base Rates
The “base rate” is the percentage of people in the population who have a certain property.
The base rate of terrorists is the percentage of terrorists in the population, the base rate of HIV/AIDS cases is the percentage of people who have HIV/AIDS in the population, etc.
Base Rates
As we have seen, base rates matter. If the base rate of a condition is very low (small percentage of terrorists), then even very accurate tests (100% true positive, 99% true negative) can be useless. .
Base Rate Neglect
The “base rate neglect fallacy” is the fallacy of ignoring the base rate when making a judgment.
For example, if I assumed you were a terrorist, because you tested positive, I would be committing the base rate neglect fallacy. I should assume you’re still probably not a terrorist, because the base rate of terrorists is so low.
LOGIC AND FALLACIES
Arguments
Philosophical argument consists of two parts: the premises and the conclusion.
The premises are the ‘evidence’ that are given in support of the conclusion.
The conclusion is the ‘claim’ that the premises are supposed to support.
Deductive Validity
We say that an argument is deductively valid when it has the following property:
If the premises of the argument are true, then the conclusion of the argument must be true.
A valid argument is “truth-preserving”: the truth of the premises gets passed on to the conclusion.
Inductive Validity
We say that an argument is inductively valid when it has the following property:
If the premises are true, then the conclusion is likely to be true.
An inductive argument probably preserves truth.
Invalidity
An argument that is not valid is called invalid.
Valid: If the premises are true, then the conclusion must be true.
Invalid: The premises can be true while the conclusion is false.
Deductive Logic
The goal of deductive logic is to identify deductively valid argument forms. We can use these as a formal test for validity: if an argument has a certain form, then that argument is deductively valid.There is no formal test for (deductive) invalidity: no way of looking at the form of an argument and telling that the premises do not guarantee the conclusion.
Fallacies
A fallacy is an invalid argument, usually one that might mislead someone into thinking it’s valid.
Straw Man Fallacy
The Straw Man Fallacy is when you misrepresent your opponent, and argue against the misrepresentation, rather than against your opponents claim.
Assuming the Original Conclusion
Assuming the original conclusion involves trying to show that a claim is true by assuming that it is true in the premises. It has the form:
X is true. Why? Because X.
False Equivocation
Equivocation (or “false equivocation”) is when one word is used with two meanings in the same argument, rendering it invalid.
False Dilemma
An argument commits the false dilemma fallacy when it presents two options as the only options, even though there are actually more options.
Fallacy of the Mean
The fallacy of the mean is the assumption that a “middle point” between two views is the right one.
Distribution Fallacy
The distribution fallacy is committed when one assumes that individuals have the properties of groups to which they belong.
Lingnan has an excellent philosophy department.I am a philosopher at Lingnan._________Therefore, I am an excellent philosopher.
Composition Fallacy
The converse of the distribution fallacy is the composition fallacy, assuming that groups have the properties of the individuals that compose them.
For example: “A point doesn’t have any length; lines are made out of points; therefore, a line doesn’t have any length.”
Ecological Fallacy
Here’s an “ecological inference”.
Countries where, on average, people consume more fat have higher rates of breast cancer.
Therefore, there is a correlation between the amount of fat a person consumes and his/ her chances of breast cancer.
Argument from Ignorance
The argument from ignorance goes like this:
“You can’t prove that God doesn’t exist. Therefore God exists.”
It assumes that because there is no argument against a position, that that position must be correct.
Genetic Fallacy
The genetic fallacy seeks to evaluate a claim on the basis of its origin.
So, for example, someone might say, “Eugenics is wrong, because the Nazis began it and did horrible things for its sake.”
Eugenics may be wrong, but the fact that the Nazis began it is irrelevant to this claim.
Appeal to Motive
Sometimes people argue that a certain claim must be false, or an argument invalid, because of the motives of the person making the claim/ argument.
Naturalistic Fallacy
The Naturalistic Fallacy assumes that just because something is natural, it is good, and just because something is not natural it is not good. Some natural things are bad, some non-natural things are good.
THE SCIENTIFIC METHOD
Scientific Method
Last time we discussed the hypothetico-deductive method, which consisted of four steps:1. Formulate a hypothesis2. Generate testable predictions3. Gather data4. Check predictions against observations
Good Theories
A good theory should:• Have predictive power• Explain the relevant phenomenon in terms of
underlying causal mechanisms• Be fruitful• Be simple• Be coherent
Causation
Much of science is concerned with discovering the causal structure of the world.
We want to understand what causes what so we can predict, explain, and control the events around us.
Correlation
Two variables A, B that are not independent are said to be correlated.
A and B are positively correlated when P(A/ B) > P(A). If B happens, A is more likely to happen.
A and B are negatively correlated when P(A/ B) < P(A). If B happens, A is less likely to happen.
Correlation
Other relationships between variables are often called correlation as well.
A and B are positively correlated when increases in A correspond to increases in B.
A and B are negatively correlated when increases in A correspond to decreases in B.
Causation and Correlation
One thing that can lead two variables A and B to be correlated is when A causes B.
For example, if having a cold causes a runny nose, then having a cold is correlated with having a runny nose:
P(cold/ runny nose) > P(cold)
Causation ≠ Correlation
But causation does not imply correlation. If A and B are correlated there are several possibilities:
• A causes B• B causes A• C causes A and C causes B• A and B are only accidentally correlated
Types of Scientific Studies
There are two basic types of scientific studies:• Observational studies look at data in order to
determine whether two variables are correlated.
• Controlled experiments involve two groups: the experimental group, which receives the treatment we’re interested in, and the control group, which does not.
Observational Studies
Importantly, observational studies can only show whether two variables A and B are correlated. They cannot show whether A causes B, or B causes A, or some third cause causes both, or if the correlation is accidental.
Confounding Variables
A controlled experiment lets you “control for” confounding variables. You can make the control group and the experimental group have equal numbers of people from each age group.
Then you know that if more people in your experimental group die, it wasn’t due to their age (the other group had similar ages).
Controlling
In an observational study, there is no way to rule out B causing A rather than A causing B. Does wine reduce the risk of cancer, or does a lowered risk of cancer increase wine consumption?
If experimenters control who gets wine, then we can rule out the hypothesis that in our study, lowered cancer risk causes wine drinking.
Controlling
In an observational study, there is no way to rule out a common cause for two correlated variables A and B.
In an experimental study, the common cause is ruled out, because the experimenter is the one who causes (“controls”) whether people have A or not.
Randomization
Ideally, an experiment controls for as many variables as possible.
To a large extent, this is done by randomly assigning individuals in the study to either the control group or the experimental group. This way, the members of the group are less likely to share features other than chocolate eating.
Maximal Similarity
In general, the control condition and the experimental condition should be as similar as possible, and differ only in the variable being tested.For example, if the experimental group is given the happiness test by a beautiful woman and the control group is tested by a grumpy professor, that might be the real reason for a difference in scores, not the baby pictures.
“Blinds”
In experimental studies, we say that the participants are blind if they do not know which group they are in: the control group or the experimental group. It’s not always possible to have blind participants, but this is considered best practice.Blinding helps control for the placebo effect and subject bias. Non-blinded studies tend to overstate effects.
Blind Experimenters
Ideally, in experiments the researchers are blind to which group subjects are in.
This prevents the experimenter from accidentally indicating to the subjects which group they are in.
It also prevents experimenter bias.
Placebo and Nocebo
‘Placebo’ is Latin for “I will please,” and the placebo effect is when a treatment that doesn’t by itself cause any improvement leads to positive expectations in the patient that cause improvement.
‘Nocebo’ means “I will harm,” and the nocebo effect is when an inactive treatment causes harm, because we believe that it will.
The Placebo Effects
• 4 sugar pills are better than 2!• Pink sugar pills make you more alert than blue
ones.• Saltwater injections improve headaches better
than sugar pills.• Paying more money for the same drugs makes
you feel better.• Placebos in fancy packages work better…
Ethics of Placebos
There are lots of reasons that we should discourage treatment that is no better than a placebo. If the treatment is:• Expensive• Harmful• Deceptive• Used instead of effective alternatives
The Baseline
Scientists often test small groups, and then aggregate (put together) all the data later. When you have a small group of people– for example 20 or 30, there is a high probability that by random chance either the control group or the experimental group will be doing better.
This is called “the baseline.”
Controlling for the Baseline
You can “control for the baseline” by testing how much people improve over the course of the trial, instead of just testing whether they’re doing well or poorly at the end of the trial.
Null Hypothesis
The hypothesis that there is no causal connection between the variables being studied is called the null hypothesis.
Our goal is to reject the null hypothesis when it is false, and to accept it when it is true.
P-Values
One way to characterize the significance of an observed correlation is with a p-value. The p-value is the probability that we would observe our data on the assumption that the null hypothesis is true. p = P(observations/ null hypothesis = true)We say that an experimental result with p < .05 is statistically significant.
Statistical Significance
What does p < .05 mean?
It means that the probability that our experimental results would happen if the null hypothesis is true is less than 5%.
According to the null hypothesis, there is less than a 1 in 20 chance that we would obtain these results.
Importance
Just because the results of an experiment (or observational study) are “statistically significant” does not mean the revealed correlations are important.
The effect size also matters, that is the strength of the correlation.
Effect Size
In class, we considered two measures of effect sizes:If the probability that a non-smoker will get lung cancer is 2% and the probability that a smoker will get lung cancer is 10%, then the relative risk of lung cancer for smokers relative to non-smokers is 10%/2% = 5. The chance of getting lung cancer for smokers is 5 times the chance for non-smokers.
Sample
In statistics, the people who we are studying are called the sample. (Or if I’m studying the outcomes of coin flips, my sample is the coin flips that I’ve looked at. Or if I’m studying penguins, it’s the penguins I’ve studied.)
Our question is then: what sample size is needed for a result that applies to the population?
Representative Samples
The opposite of a biased sample is a representative sample.
A perfectly representative sample is one where if n% of the population is X, then n% of the sample is X, for every X.
For example, if 10% of the population smokes, 10% of the sample smokes.
Random Sampling
One way to get a representative sample is to randomly select people from the population, so that each has a fair and equal chance of ending up in the sample.
For example, when we randomize our experiments, we randomly sample the participants to obtain our experimental group. (Ideally our participants are randomly sampled from the population at large.)
Sample Size Determination
There are two questions we need to ask:
How many people do I need to study to obtain statistically significant results?
How big should my sample be to accurately estimate effect sizes in the population at large?
Sample Size Determination
If we want to know how many people to look at, we should determine:
1. What level of confidence we want2. How big we want our confidence interval to
be.
Common Choices
Common choices for these numbers are:
1. We want to be 95% confident of our estimation.
2. We want our confidence interval to be 6% wide (e.g. between 42% and 48%).
One Calculation
As we calculated in class, you need about 1,000 people to be 95% sure that the vote counts you estimate from the sample are within 6% of the actual voting behavior of the population.
Things to Note
This doesn’t mean that studies with less than a thousand people can’t tell us anything—
What they tell us will just be either less confident than 95% or have greater error bars than 6%.
If a confidence interval of 20% is fine, you only need 100 people.
The Problem of Multiple Comparisons
The problem of multiple comparisons is the fact that if you look at a data set, you will find lots of correlations completely by random chance.
Thus science prohibits “finding hypotheses in the data.” We can’t look at some data and see if there are any correlations. We have to pick a correlation we’re looking for in advance, and see if it exists in new data.
Why New Data Is Important
It would be really unlikely if (i) I propose a correlation(ii) I test it against some new data(iii) The new data confirm the correlation(iv) All of that was just an accident
Compare this to the fact that it is really likely to find random correlations in the data.
Meta-Analysis
A meta-analysis is an analysis of analyses.
In clearer terms, it is a study that looks at lots of different experiments that have been conducted on the same problem, and tries to “put together” all of the findings.
Cherry Picking
Cherry picking is another name for “the fallacy of incomplete evidence,” and it’s related to confirmation bias and selection bias.
If I want to prove that a treatment works and I pick only those studies that are positive and ignore lots and lots of negative studies, then I’m “cherry picking” my studies.
“Systematic Review”
In the “bad old days” (before the 1980s) review articles were unsystematic, meaning that people writing the reviews included some studies that were relevant, but not all. This resulted in cherry picking: later it was shown that systematic meta-analyses often had the opposite conclusions of unsystematic reviews. A systematic review looks at all the relevant studies, not just some.
The File Drawer Problem
A second and more difficult problem for literature reviews is that the published literature is a biased sample of all the studies that have been done. As we saw last time, there is publication bias against negative results: people tend to publish positive findings, but to leave negative findings unpublished, sitting in the “file drawers” of their offices (metaphorically).
Authority
That’s impossible: we all need to trust other people for knowledge about lots of things.
I’m not a climate scientist, I don’t have stations that collect and analyze data about global temperatures, so I can’t determine myself whether global warming is happening. I have to trust an authority to tell me if it is.
Appeal to Authority
Sometimes you’ll hear about the “fallacy of appealing to authority.” But basing your views off on appeals to authority is not always fallacious. Your authority needs to be:
• An expert, or better, an agreeing group of experts, on the subject in question.
• A proven truth-teller on the subject.
Credentials
One way society has of distinguishing genuine authorities from other people is via credentials.
A person’s authority can be certified by their having certain degrees, like a PhD or MD (doctor of medicine). A person’s ideas can have their authority certified by being published in a top peer-reviewed scientific journal.
Problems with Newspapers
Editorials are often uninformed opinions.Most reported-on scientific studies are false.Most newspaper stories are not written by reporters. Many are written by private companies for commercial purposes.News creators (government, scientists) often require news sources to only give their side.He-said she-said reporting often results in a false equivalence between opposing views.
DECISION THEORY