Introduction to the Course - hosung.weebly.comhosung.weebly.com/uploads/1/7/9/6/17964019/... · 3/3/2018 · South Korea Email: [email protected] ... Ì High school basic mathematics

1/ 28

About the CourseIntroduction

Wrap-Up

Introduction to the Course

Hosung Sohn

Graduate School of Public AdministrationSeoul National University

Ph.D. Level Course

Lecture Slide 0

March 3, 2018 (Saturday)

Hosung Sohn (Lecture Slide 0) Policy Evaluation Research (Spring 2018)

2/ 28


Wrap-Up

Table of Contents

1 About the Course

2 Introduction

3 Wrap-Up


3/ 28


Wrap-Up

Instructor InformationCourse Information

Who Am I?

Name: Hosung Sohn

What am I doing?: I am an assistant professor in the School of PublicService at Chung-Ang University

Where am I from?: South Korea

Email: [email protected]

Office: I don’t have my office at SNU

Office hours: by appoint (via email)

Class webpage: http://hosung.weebly.com/


4/ 28


Wrap-Up


What Is This Class?

A PhD-level course in Policy Evaluation Research

Introduces Ph.D. students in Public Administration and Policy to the logicand a set of statistical and research designs that are required for conductinghigh-quality empirical research on topics in program evaluation

The course will be dealing mainly with “quantitative” methods, as opposeto qualitative methods

The course contents will be invaluable for those who will be doingdata-oriented work

Ultimate goals:

ä Help students distinguish “good” empirical research from empiricalresearch that are not that good

ä Help students use data, in their own setting, to strengthen theirarguments—in compelling ways!


5/ 28


Wrap-Up


Prerequisites

Prerequisites for the class:

Ê An introductory-level knowledge in statistics

Ë An introductory-level knowledge in econometrics

Ì High school basic mathematics

=⇒ We will be reviewing these materials in class!

Statistical software: recommend using STATA, not SPSS

=⇒ We won’t be learning the software in class, but I may give yousome guides on STATA codes for many of the methods we learn, if timepermits


6/ 28


Wrap-Up


Course Materials

Textbook 1 (not required): Mostly Harmless Econometrics (by Angristand Pischke)

Textbook 2 (not required): Mastering Metrics (by Angrist and Pischke)

Lecture slides

Want to rigorously study program evaluation methods? Let me know...


7/ 28


Wrap-Up


Grading

Do you care?

Class attendance and participation (10%)

Referee work: I will assign two manuscripts for you to review (10%)

Research proposals (10%)

In-class exams:

T Midterm (30%)

T Final (40%)


8/ 28


Wrap-Up


Make Up Work

Make up work can be granted under two circumstances:

ä Religious observance

=⇒ Please notify me before the end of the second week of classes

ä Emergency

=⇒ Please provide me with documents to prove your emergency

What is emergency?

Acceptable examples: sick (needs medical proof), death of your families,etc.

Unacceptable examples: I am not in a mood to attend class, yourgirl(boy) friend is sick, etc.


9/ 28


Wrap-Up


Policy on English

No languages other than English are allowed during class

Of course, you are more than welcome to speak in Korean before orafter class

When you really want to ask questions during class, but having hardtime articulating the question in English, just tell me that you want toask in Korean

English should be used when you submit your research proposals,referee reviews, and answers to midterm and final exams

=⇒ Don’t worry about the sophistication of your English grammar.This class is not intended to test your English skills. BUT! Try yourbest to convey your point!


10/ 28


Wrap-Up


Policy on Laptops, Tablets, and Other Devices

In principle, I am happy to accommodate your use of technology duringclass:

ä It is OK to check KAKAO Talk, e-mail, Facebook, etc. once in a whileas long as it does not distract your fellow students!

ä It is NOT OK to watch movies or play video games!

ä And you are more than welcome to use your laptop with the keyskin ontop of the keyboard


11/ 28


Wrap-Up


Policy on Electronic Communication

It is “extremely” important that you check announcements posted onour class webpage:

ä An important announcement, if any, will be posted on this webpage!

ä Send me emails if you need to make an appointment, need helpregarding make up work, meeting deadlines, etc.

ä But please refrain from asking class-related materials via email! Rather,please come by my office.

=⇒ A bad example of an email inquiry: Can you tell me why Equation(A) becomes Equation (B) on page XXX?


12/ 28


Wrap-Up


Regarding Lecture Slides

The lecture slides are written by me, specifically for this class

Why lecture slides, instead of a textbook?

ä Textbooks often contain lots of unnecessary stuff

ä Reading textbooks alone may sometimes mislead students to considerless important topic as very important. And lecture slides will helpprevent students from making such mistakes

ä It has been my experience that a single source is not enough to conveythe necessary topics adequately

Lecture slides are derived heavily from many resources: please do notshare these slides to other websites, etc.


13/ 28


Wrap-Up


Regarding Lecture Slides

The lecture slide for each topic will be posted on class webpage, roughlya week before the schedule

For the exams, I am not going to ask you to solve problems on topicsthat are not covered in my lecture slides

I am always looking to make my slides better—I would be very muchappreciated if you could give me feedback on my slides (e.g., identifyingtypos)

=⇒ If you think my slides are terrible, just tell me why it is terrible. Ipromise not to hold it against you!


14/ 28


Wrap-Up


Calendar and Important Dates

Important due dates and schedules are provided in our syllabus

I will post the revised syllabus on my webpage


15/ 28


Wrap-Up


Regarding the Research Proposal

Purpose: ideally, I hope the research proposal you submit becomes aresearch paper that can be published in an academic journal

Topic: you can choose any topic of your choice

À e.g.) Your dissertation

Á e.g.) Your thesis

Á e.g.) The work you are doing with other colleagues

=⇒ BUT!! You should be able to apply quantitative programevaluation methods learned in our class to the topic of your choice


16/ 28


Wrap-Up



What must be included in your research proposal:

À Purpose of your research

O What is your hypothesis?O What is the benefit of testing your hypothesis?

Á Previous literature

O What are the limitations of previous literature that justify your research?

Â Empirical methods

O How are your going to test your hypothesis quantitatively in compellingways?

Ã Data

O Which data will you be using to test your hypothesis?


17/ 28


Wrap-Up



Format:

À A4 4 pages (single-spaced) in MS-Word

Á Due date: final exam date


18/ 28


Wrap-Up


Regarding the Referee Reports

What must be included in your referee reports:

À Purpose and conclusion of the research paper

O What is the purpose of the research paper?O What is the conclusion of the research paper?

Á Do you consider the validity of the research paper to be strong such thatit can be published in a good academic journal?

O If not, please provide reasons for your arguments

Â How should the authors revise their work in order to make the papermore compelling?


19/ 28


Wrap-Up


Regarding the Referee Reports

Format:

À A4 2 pages (single-spaced) in MS-Word for each referee report

Á Due date: final exam date


20/ 28


Wrap-Up


Questions

Any questions?


21/ 28


Wrap-Up

“What”, “Why”, “How,” and “Who” of Policy Evaluation MethodsOverview of the Course Topics

1. What is Policy Evaluation?

The policy evaluation method is the science of answering the followingquestion:

“How do we know whether a policy is having its intendedimpact?”

Example: Does income subsidy promote welfare of low-income families?

Example: Does getting a degree at SNU raises the probability of gettinga good job?

Example: Does nursery allowances raises fertility?


22/ 28


Wrap-Up


2. Why Should We Learn Policy Evaluation Methods?

To get a job, you need publications!

Many academic journals in public administration and policy publishpapers on policy evaluation:

ä Journal of Policy Analysis and Management

ä Journal of Public Administration Research and Theory

Invaluable for your post-PhD career!

=⇒ Many works involve quantitative policy evaluation

Many research papers are becoming quantitatively rigorous nowadays

=⇒ A firm grasp of policy evaluation skills, therefore, would helpseparate “sense” from “nonsense” in the era of quantitative research!


23/ 28


Wrap-Up


2. Why Should We Learn Policy Evaluation Methods?

Example: we would like to know whether providing an unemploymentallowance increases the chance of re-employment

ä Method: compare the re-employment rate between those who receivedthe allowance and those who did not receive the allowance?

=⇒ Does this comparison allow us to answer the question at hand?

Example: A newspaper reports the finding from an academic journalthat high school students who received private tutoring did better inCSAT than those who did not receive the tutoring

ä Do you think students who received private tutoring did better in CSATbecause of this tutoring?

=⇒ Depends on how this research was conducted!


24/ 28


Wrap-Up


3. How to Study Policy Evaluation Methods?

Two ways to study policy evaluation methods:

=⇒ Our approach: the latter!Hosung Sohn (Lecture Slide 0) Policy Evaluation Research (Spring 2018)

25/ 28


Wrap-Up


4. Who Should Study Policy Evaluation Methods?

Students whose major is public administration, public policy, and othersocial science-related fields

Those who want to use data in their research

Those who want to assess whether a data-driven research is credible ornot

Those who don’t want to be fooled by “method-guys”


26/ 28


Wrap-Up


5. All in All...

The ideas of policy evaluation methods—like those of othersubjects—took a long time to discover and systematize

In my opinion, it’s not an easy subject that you can master in onesemester. You will definitely need to go through a series of trial anderror

Attitudes you need: persistence, practice, practice, and practice!

=⇒ The gain will be worth the pain


27/ 28


Wrap-Up


Brief Overview of the Course Topics

Review of probability and statistics

Ordinary least squares and agnostic regression

Introduction to causality and research design

Value of randomized experiments

Selection on observables designs

Selection on unobservables designs

The problem of statistical inference

=⇒ Note that I may end up skipping or including some new topicstoward the end of the course depending on the pace of our class


28/ 28


Wrap-Up

Wrap Up

My name is Hosung—you can just call me Hosung

Download the lecture slides from my webpage

Please do not worry—excessively—about your grade: I am NOT here to“evaluate” you; rather I am here to “help” you learn something useful

Mark the important dates

Have fun learning new program evaluation methods!


Documents

Introduction to the Course - hosung.weebly.comhosung.weebly.com/uploads/1/7/9/6/17964019/... · 3/3/2018 · South Korea Email: [email protected] ... Ì High school basic mathematics