Improving the ways that students make evidence-based decisions

Dr Manisha ThakkarDr Jeanne Young Kirby

MELTed SA Core Group

KnowledgeIntuitive

Rational

Empirical

Authoritative

Traditional Inquiry based

Teaching and learning

Evidence based decision making

• STEM based disciplines• Creative performance based

disciplines

• Demands for research skills

Core of Inquiry based teaching and learning

NCCMT, 2016

Models of Engaged Learning and Teaching

Problem Solving

School Education

Critical Thinking

Researcher Skills

Work Skills

Clinical Reflection

Your ContextThe different adaptations of the RSD that comprise the MELT

Your MELT

• Develop • Implement • Identify and validate the impact

Stage 1: Pathology and clinical

science MELT

Implementing MELT at Endeavour College of Natural Health

• Develop • Institute wide implementation

Stage 2: Endeavour MELT

Pathology and clinical science MELT:Clinical Management Pentagon (CMP)

Observe and Ask

Organise, Manage & Communicate

When in doubt, go to the centre…

• Embark and clarify = Observe and ask• Find and generate = Find and Generate• Analyse and synthesize =Analyse and Interpret• Evaluate and reflect = Investigate and Reflect• Organise and manage = Synthesize and apply• Communicate and apply = Organise, Manage and

Communicate

RSD facets and Clinical Management Skills

Approach

• Reframing the marking of an existing assessment• Developing a diagnostic assessment • Modifying and revising the emphasis of other existing

resources

“Curriculum redesign”-A three step approach:

Marking rubrics in Pathology and clinical science

CRITERIA MARKSAppropriate differential diagnoses 0 1 2 3Illustrate and analyse the role of symptomatology in process of differential diagnosis

0 1 2 3 4 5 6

Conceptualise how mechanism of disease pathology contributes to the loss of normal functions and the clinical presentations.

0 1 2 3 4 5

Demonstrate the selection, application and interpretation of correct diagnostic testing.

0 1 2 3 4

Discuss appropriate management for the patient 0 1 2 3 4Appropriate referencing and bibliography 0 1 2Presentation and word count 0 1

• Improvement in students’ research and analytical skills

Outcome measure

• Pre and Post exposure Student survey • Willison, Lievre and Lee, (2010)• 14 Likert scale questions• Self assessment of research skills

• Lecturers survey• Improvement in student’s abilities

• Student interview after 1 year • General research skills and attitudes

Tools

Outcome measures and tools

Data analysis: Student survey

Data analysis: Student SurveyUnpaired T test Paired T test

Questions

Mean Likert Score

Pre-Post (n1=57, n2=62)

Diff1

(Post-pre) Effect Size

Mean Likert Score

Pre-Post(n=40)

Diff1 (Post-pre) Effect size

My general research skills are good 3.86-3.97 0.11 3.83-4.08 0.25*Small-

medium

My Research Skills in pathology and clinical science are good 3.54-3.90 0.36** Medium 3.60-4.05 0.45**Medium

-large

I am able to ask clear, researchable questions in pathology and clinical science 3.65-3.94 0.26* Small-

medium 3.78-4.05 0.28* MediumI can devise procedures in pathology and clinical science to generate relevant and high quality information 3.32-3.73 0.41** Large 3.45-3.90 0.45** Large

I can effectively evaluate the credibility of sources of information in pathology and clinical science 3.81-4.16 0.35** Medium 3.95-4.18 0.23**

Small-medium

I can organise information from multiple sources effectively in pathology and clinical science 3.77-4.11 0.34** Medium 3.75-4.13 0.38** Medium

I am able to analyse information from different sources effectively in pathology and clinical science 3.74-4.06 0.32** Medium 3.85-4.10 0.25*

Small-medium

I can clearly communicate in writing what I understand from my research in pathology and clinical science 3.70-3.85 0.15

3.675-3.975 0.3*

Small-medium

What is next?

• Lecturers survey• Students interview after 1 year • Concise the marking rubrics

Stage 1: Pathology and Clinical Science MELT

• Institute wide development and application• Training academic leaders and Subject coordinators

Stage 2: Endeavour MELT

RSD facets and Endeavour’s graduate attributes

Endeavour’s Graduate Attribute

In-depth knowledge and skills Effective communication Critical judgment Ethical and social behaviour Innovation and creativity

RSD Facets

Embark and Clarify Understand how other disciplines relate to the field/s of study

Define and analyse problems Appreciation of the philosophical and social contextsAppreciation of cultural diversity

Identify problems Generate ideas

Find & Generate Generate comprehensive and well-founded knowledge in the field/s of study

Collect information Find information from a range of sources

Knowledge of ethics and ethical standardsKnowledge of cultural diversity

Create solutions

Evaluate & Reflect Review the knowledge and skills learnt

Evaluate opinionsReflect critically on the justifications

Adapt innovatively to changing environments

Organise & Manage Consolidate the knowledge and skills learnt

Organise information Engage effectively and appropriately with information and communication technologies

Work and learn independently

Analyse & Synthesise Extend the knowledge and skills learnt

Analyse information Apply critical reasoning to issues

Be self-directed and engage in lifelong learning

Communicate & Apply ethically

Apply the knowledge and skills learnt

Convey ideas clearly, fluently, in both written and spoken formsSelect and use the appropriate level, style and means of communicationCommunicate confidently and effectively

ff l h h

Make decisions based on justifications and reflections

Application of ethics and ethical standards,demonstrate social and civic responsibility

Improve current practices

https://www.endeavour.edu.au/about/graduate-attributes

Endeavour’s MELT

Knowledge

Organise and Communicate

Observe and Ask

Find and Generate

Analyse and Interpret

Investigate and Reflect

Manage and Apply

Org

anise

and

Com

mun

icat

e

Knowledge

The Flower of Knowledge

Dr Manisha ThakkarBiosciences

Emerging MELT

• First Year STEM: 700+ cohort (39 Courses/Degrees)• Evidence Based Decision Making (EBDM)• Authentic Research Experience

CONTEXT:

Find & Generate

RESEARCH PROJECT MELT

Embark & Clarify

When in doubt, go to the centre…

What is my purpose?

What do I need?

Dr Jeanne Young KirbyBiological Sciences

•FORMATIVE – In Class EBDM scenarios•SUMMATIVE – Authentic Research Experience

APPLICATIONS:

• STUDENT• STAFF

EVALUTION – PRE and POST PERCEPTION SURVEYS

Emerging MELTDr Jeanne Young KirbyBiological Sciences

FACET RESEARCH PROJECT CRITERIAA. Embark & Clarify What is my purpose?

• AIMS• HYPOTHESIS • CONTEXT/JUSTIFICATION

B. Find and GenerateWhat do I need?

• BACKGROUND –PEER REVIEW LITERATURE; ANNOTATED BIBLIOGRAPHY

• EXPERIMEMTAL DESIGN – VARIABLES, SAMPLE SIZE

• STUDY SITES/ORGANISMS/TYPES OF SAMPLES• METHODS: SAMPLE COLLECTION AND

ANALYSIS• METHODS: DATA ANALYSIS

C. Evaluate & ReflectWhat should I trust?

• GAPS IN KNOWLEDGE• CONFLICTS IN LITERATURE• RELEVANCE OF RESEARCH TO

TOPIC_CONTEXT/JUSTIFICATION

• EVOLUTIONARY SIGNIFICANCE• RESULTS: ALIGNMENT WITH PRIOR

KNOWLEDGE• SELF AND PEER ASSESSMENT

D. Organise & ManageHow do I arrange? • COMMUNICATION CONTRACT

• OH&S, ETHICS• TIMELINE

• LABORATORY OR FIELD TRIP • ACCESSING EQUIPMENT• DATA RECORD: FIELD, LAB, SAMPLE ANALYSIS

E. Analyse & SynthesiseWhat does it mean?

• DATA ANALYSIS (GRAPHICAL AND STATISTICAL)• INTERPRETATION

F. Communicate & ApplyHow do we relate?

• GROUP FORMATION AND BRAINSTORMING• RESEARCH PROPOSAL DEVELOPMENT AND

FEEDBACK RESPONSE

• POSTER CRITERIA AND FORMAT: DRAFT & FINAL

• SELF & PEER ASSESSMENT

What is my purpose?

When in doubt, read your palm

DELT: Digitally Engaged Learning & Teaching

Dr John WillisonSchool of EducationEmerging MELT

Emerging MELTDr David P Wilson

Emerging MELT Jason HomeMusic Teacher

Mind map MELT

Emerging MELT

Project knowledge building and developing interconnectivity in a work integrated learning course

Roopa HowardSchool of Education

ReferencesA Handbook for Research Skill Development and Assessment in the Curriculum, 2009, Australian Learning and Teaching Council Ltd, http://www.adelaide.edu.au/rsd/docs/rsd_Handbook_Dec09.pdf

National Collaborating Centre for Methods and Tools, 2016, Building capacity for evidence-informed public health, http://www.nccmt.ca/professional-development/eiph/define

Optimising Problem Solving (OPS) pentagon, 2014, designed by Mechanical Engineering Communications Tutors, University of Adelaide, Australia.

Willison, J., & O'Regan, K. (2007). Commonly known, commonly not known, totally unknown: a framework for students becoming researchers. Higher Education Research & Development, 26(4), 393-409.

Willison, J, Lievre, KL, and Lee, I, 2010, Making Research Skill Development Explicit in Coursework, Australian Learning and Teaching Council Ltd, https://www.adelaide.edu.au/rsd/evidence/study/RSD_in_coursework_with_appendices.pdf

MELTed SA GroupDr Manisha Thakkar

Dr Jeanne Young Kirby

Dr John Willison

Dr David Wilson

Jason Home

Roopa Howard

Thank YouQ & A