i190 Spring 2014: Information and Communications Technology

for Development (ICTD) in Practice

University of California Berkeley, School of Information

LECTURE 22: 21 Apr 2014

Instructor: San Ng (www.sanng.com)

Class Website: i190spring2014.sanng.com

i190 Framework

Conceptual

Week1: Introduction to Course

W2: What is Development?

W3: What is ICTD

W4: Who Does What in Practice?

Mapping the ICTD

Landscape

i190

ICTD in

Practice:

Core Skills

Technical

(eApplications)

W5: Overarching Issues of

eApplications

W6: Infrastructure, Telecenters,

Agriculture,

W7: Revisiting Agriculture,,

W8 : e-Health, Education

W9: eGovernance

Microfinance

Management

W10: Break

W11: Intro to Project Management

Planning and Assessment

W12: Initiating: Design, Scheduling,

Budgeting, HR

W13: Implementation

W14: Monitoring and Evaluation/

Next Cycle

W15: Final Projects &

Wrap Up

Introduction to Project Management

Planning

Initiation

Implementation

Monitoring & Evaluation

Next Phase? Transformation?

Implementation- Best Practices

Case: ITC e-Choupal: What are the needs/problems

that this ICTD project is trying to address?

Implementation- Best Practices

Case: ITC e-Choupal

What made

implementation

successful?

•Trust: choice

•Meets Needs: Clear

Value

•Appropriate Tech:

Simplicity of

technology, new and

old tech

•Local structures and

systems

•Incremental Roll out

•Mission-based

Implementation-IT requirements/User Design

Case: ITC e-Choupal

Implementation-IT requirements/User Design

Case: ITC e-Choupal

Implementation- Complex Environments

Case: Competing for Development (A)

•If you were Ghazialam, would you go ahead with the

$65,000 investment?

•What are the key tradeoffs? What would an ‘ideal’

outcome look like?

Implementation- Complex Environments

Case: Competing for Development (B1-6)

6 groups

Role play exercise: Make the case within your role for

the $65,000 investment

Introduction to Project Management

Planning

Initiation

Implementation

Monitoring & Evaluation

Next Phase? Transformation?

Different Types of Evaluation and Performance Measurement

Program Level Organization Level Community and

Societal level

Wikipedia

Case

Evaluation Purpose

Measuring program effectiveness

Determining if a program meets its objectives

TYPES

*baselinesFormative

*ongoing

* feedback

* changing

the program

Summative

* look at final outcomes

*impacts

* cut or keep

OTHER PURPOSES

* compliance

* legitimacy

* certification

* lessons learned

* check for unintended consequences

* benchmarking

* more money

* white wash and eye wash p

* kill a project

* political attack

* new opportunities

* protection and self interest

* melt down indicators

EVALUATION CHECK LIST

WHEN IS EVALUATION WORTH DOING?

* Who Wants This and What Decision Do They Want to Make?

(lessons learned)

* Are the Impediments Manageable?

(resources, objectives, agreement, special issues)

* Is there Political Support?

(general support)

THE REGULAR COMPONENTS OF THE EVALUATION PROCESS

* Purpose and Objectives

* Indicators

* Design

* Data and Utilization

* Problems

EVALUATION ACTOR MAPPING

BOARD

OR

LEGISLATORS

OTHER

STAFF

OR DEPTS.

TOP MGMT

PROGRAM

PARTICIPANTS

PROGRAM STAFF

PRESS OR

COMMUNITYDONORS

MIDDLE

MGMT.

OUTSIDERS

FOR

TEACHING

PURPOSES

RANGE OF INDICATORS

FEELINGS DO YOU TRUST THESE

INPUTS PEOPLE/

PROCESS involvement/coordination

OUTCOMES

(INTERMEDIATE) INCREASED INCOME

(FINAL) X LEVEL OF CONTANIMATION

EFFICIENCY AND PRODUCTIVITY lbs. Of fish/$ $/lbs. Of fish

SPAN AND SCOPE OF COVERAGE % target population served

SATISFACTION customer satisfaction/ commercial

IMPACTS (sustainable) measurable change/ broader/

(PROGRAM CAUSED OUTCOME) longer term

* HOW WOULD YOU ASSURE THAT YOUR

RESULTS WERE VALID AND RELIABLE?

RELIABLITY-- DO YOU GET THE SAME RESULT

TIME AFTER TIME.

VALIDITY-- UNBIASED COMPARED TO A STANDARD

What is the purpose of a research design?

* tailored to each problem

* to answer very specific questions

* weigh benefits and costs and resources

* can allow cross comparison

* was it the program that made the difference?

Using Evaluation Results-- Style Differences

Academic Style

* Slow

* Scientific

Method

* Clear Objectives

* Careful Study

* Written Communication

* Precision

* Academic Reference Group

Managerial Style

* Pressure to Decide

* Many Simultaneous

+Fragmented Tasks

* Competing Objectives

* Action

* Verbal Communication

* Incomplete Date

* Managerial Reference Group

P

R

O

B

L

E

M

S

* TIME * VERY RIGOROUS * IRRELEVANT

* FORMAT * NOT RIGOROUS * COMMUNICATION

DESIGN TYPES

PRE EXPERIMENTAL (PE)

* Goals verses Performance

* Before and After 0 X 0

QUASI EXPERIMENTAL (QE)

* Time Series 01 02 03 04

* Non Equivalent Control Group

01 X 02

03 04

* Multiple Time Series

01 02 X 03 04

05 06 07 08

EXPERIMENTAL

* Two Group Pre and Post Test

01 X 02

R

03 04

* Post Test Only

X 01

R

02

* Solomon Four

01 X 02

03 04

X 05

06

R

Some of the More Common Methods

* Balanced Score Cards and Other Overall General Assessments

* Goals vs. Performance and also Cost and Efficiency

* Outcome Assessment

* Benchmarking

* Best Practice

* Rapid Assessment Tools

(quicker and dirtier rather than deeper)

* More based on sampling than 100% study

DATA COLLECTION METHODS

* General Statistical Analysis

* Cost Benefit/ Rates of Return

* Simulations

* Content Analysis

* Record Reviews

* Unobtrusive Measures

* Group Observation

* Surveys and Testing

* Personal Interviews

* Participation Observation

* Case Studies

LESS

INTRUSIVE

MORE

INTRUSIVE

THE ETERNAL

TRIANGLE

Precision

Cost Complexity

Measureable

Indicators

Types of Data

needed

Data Collection

Methods/Frequency for

M&E

Overriding Goal

Objectives (at

least 4)

Instructions: Everyone was pleasantly shocked by these successful results.

However, the founder Jimbo Wales intuitively knows that the number of

articles per se does not measure Wikipedia's success completely, especially

since Wikipedia began with a completely different set of goals/activities and

became 'successful' only organically. He wants to hire you to determine a

sound methodology to evaluate Wikipedia's success. He wants you to design

a Logical Framework for Wikipedia (based on what we already learned in

class), with indicators he can measure to determine Wikipedia progress and

success. He has given us a sample template that we will discuss and

brainstorm in class: