Anthony Chow, Ph.D. Assistant Professor Department of Library and Information Studies

Educational InformaticsDesigning performance-based measurement systems for

rapid response learning environments

Anthony Chow, Ph.D. Assistant ProfessorDepartment of Library and Information StudiesThe University of North Carolina at Greensboro

Ingrid Guerra-Lopez, Ph.D. Associate ProfessorDirector, Institute for Learning & Performance ImprovementWayne State University

Mark Whitlock, MBAChief Executive OfficerThe Central Educational CenterDirector, Institute for Learning & Performance ImprovementWayne State University

04/2

2/20

23

• Literature• Problem• Method• Results• Implications

Overview

2 of 21

04/2

2/20

23

Educational Informatics

Educational informatics is:• Intersection of three disciplines:

• teaching and learning, information science, and information communication technologies (ICTs) (Ford, 2004; Haythornthwaite, 2006; Sheffield University, 2011)

• “The study of the application of digital technologies and techniques to the use and communication of information in learning and education” (Levy, Ford, Foster, Madden, Miller, Baptista Nunes, McPherson, & Webber, 2003, p. 299)

• Educational informatics parallels the use of informatics in other fields• medical informatics, bioinformatics, and health informatics

3 of 21

04/2

2/20

23

Educational Informatics meets information science and systems design• Educational informatics and information science

• The word “informatics” means the science of information• The word “analytics” means the logical organization and analysis

of information• Educational informatics within a systems framework:

• The science of using and analyzing educational information in a logical fashion.

• Chow (2008) defined it by uniting computing, information science, and systems design in an education setting as:

• “how information technology is used to collect, organize, use, and disseminate information to support and help improve overall performance of the educational system” (Chow, 2008, p. 51)

4 of 21

04/2

2/20

23

The Big Questions?• How is our school doing?• What does success look like?• How do we measure it?• How do we know if our systemic change effort is working?

•What data do we need to have by when to answer these questions?

5 of 21

04/2

2/20

23

Educational Informatics and Organizational Performance

• How are ICTs being used to collect data and how are organizations able to translate and use this data?

• Information they can use to continuously improve current and future decisions and performance in a real-time, dash board fashion?

Mega

Macro

Micro

6 of 21

04/2

2/20

23

Impact Evaluation Process (IEP)

• Guerra-Lopez’s Impact Evaluation Process (2007; 2012) provides the tactical steps in which to identify and build a customized organizational SEI model.

FOUNDATION

Continually Improve

Value added

to Clients & Society

Value added

to Clients & Society

IMPACT

Recommendations for Improvement

Data Analysis

Data Collection Methods

Data Sources

Measurable Indicators

Decisions & Objectives

Stakeholders & Expectations

7 of 21

04/2

2/20

23

The SEI model

The SEI model is characterized by four primary characteristics: 1. System indicators2. Real-time data3. Automation, and 4. Performance improvement focus (primarily gap identification

and causal analysis).

8 of 21

04/2

2/20

23

CEC Overview• Harless’ “Grand Conspiracy” – basing a school on systems

thinking which starts with user needs and end accomplishments

• Central Educational Center – a join venture of societal stakeholders

• Charter School • Designed using ADDIE• Embraces innovation and the concept of educational

informatics and real-time data

9 of 21

04/2

2/20

23

Systemic Educational Informatics (SEI) Model

The Mega level represents the societal level and requires that an organization measures its intended value-added impact on society itself (Of what value are we to society?) to ensure proper alignment.

10 of 21

04/2

2/20

23

Systemic Educational Informatics (SEI) Model

The Macro level involves the medium term goals of the organization and stakeholders, the more immediate or direct impact that benefits the organization itself (What does success mean and look like for our organization?).

11 of 21

04/2

2/20

23

Systemic Educational Informatics (SEI) Model

The Micro level involves internal building-block results of the organization (What short-term products have to be accomplished by teams and individual organizational members?) (Kaufman, 2006; 2011).

12 of 21

04/2

2/20

23

CEC’sSEI Model

13 of 21

04/2

2/20

23

CEC’s Mega Level

14 of 21

04/2

2/20

23

CEC’s Macro Level

15 of 21

04/2

2/20

23

CEC’s Micro Level

16 of 21

04/2

2/20

23

SEI Model – Data view

DatabaseData

Data

Data Data

Data

Google Doc

CEOMega

Macro

Micro

17 of 21

04/2

2/20

23

CEC SEI Model Dashboard• Here is the initial design

18 of 21

04/2

2/20

23

CEC – SEI Model• Educational Informatics

• Information about CEC, a systemic change effort• Scientific application of continuous evaluation called for in ADDIE

process• Data leads to information, which leads to knowledge; collectively

HPT• SEI models can accompany all systemic change efforts

• Real-time formative and summative evaluation

19 of 21

04/2

2/20

23

Next Steps

1. Refining Model (add input activities and data layers)2. Establishing targets3. Populating data 4. Pilot the model5. Scalability

20 of 21

04/2

2/20

23

Resources• Joe Harless, “The Eden Conspiracy”• Anthony Chow, “Systems Thinking and 21st Century Education”• Mark Whitlock – mark.whitlock@cowetaschools.net• Anthony Chow – aschow@uncg.edu

21 of 21

04/2

2/20

23

Q&A

Thank You!

22 of 21