Upload
others
View
4
Download
0
Embed Size (px)
Citation preview
© 2021 NACADA: The Global Community for Academic Advising
The contents of all material in this presentation are copyrighted by NACADA: The Global Community for Academic Advising, unless otherwise indicated. Copyright is not claimed as to any part of an original work prepared by a U.S. or state government officer or employee as part of that person's official duties. All rights are reserved by NACADA, and content may not be reproduced, downloaded, disseminated, published, or transferred in any form or by any means, except with the prior written permission of NACADA, or as indicated below. Members of NACADA may download pages or other content for their own use, consistent with the mission and purpose of NACADA. However, no part of such content may be otherwise or subsequently be reproduced, downloaded, disseminated, published, or transferred, in any form or by any means, except with the prior written permission of, and with express attribution to NACADA. Copyright infringement is a violation of federal law and is subject to criminal and civil penalties. NACADA and NACADA: The Global Community for Academic Advising are service marks of the NACADA: The Global Community for Academic Advising
IMPROVING ENGAGEMENT IN REMOTE LEARNING ENVIRONMENTS
FACILITATING TRANSITION INTO HIGHER EDUCATION
JOHN WYATT, UNIVERSITY COLLEGE DUBLINDR. MAURICE KINSELLA, UNIVERSITY COLLEGE DUBLIN
OUTLINE• UCD & UCD LEAP
• DESIGN & IMPLEMENTATION
• COVID-19
• VLE DESIGN CHANGES
• KEY FINDINGS
• LESSONS LEARNED & FUTURE
UCD VET MEDICINE: THE SCHOOL
• UCD Vet Teaching Hospital open 24/7/365
• Top 25 QS World Subject Ranking
• AMVA, EAEVE, VCI accredited
• Requirements from UCD & accreditors
UCD VET MEDICINE: THE STUDENTS
• Approx. 300 1st year students
• 33% International Students (23% UCD Avg.)
• Classroom & practical learning components
• Student Adviser for support
UCD LEAP: SUPPORT DELIVERY ISSUES• Disengagement only apparent post-exams
• Difficult re-engaging students
• Existing supports under-used
• Negative impact on wellbeing
• Retention issues
• Social integration issues
UCD LEAP: CHANGES NEEDED
• Real-time engagement info sources
• Support interventions linked to data
• More immediate support for better outcomes
• Signposting both generic and tailored supports
• UCD Live Engagement & Attendance Project
• Bluetooth attendance data smartphone app
• At-risk students contacted
• Underpinned by Self-Determination Theory
• Self-populated
INITIAL DESIGN
2019• Student Feedback
2020• Student & SA Feedback
2021• Student & Research Team Feedback
LEAP DESIGN
AttendanceData
Reporting
Intervention
Progression
IMPLEMENTATION: INITIAL ROLLOUT
• More attendance data visibility
• Real-time interventions commenced
• Preliminary findings confirmed relationship
• Setup issues (accuracy & timetabling)
• Embedding issues (student & staff buy-in)
• High-attendance support gap
IMPLEMENTATION: FEEDBACK & CHANGES
• “Trusted Persons” format
• Light touch first intervention
• Stage 0 creation
• VLE identified as key engagement source
COVID-19
“My learning is nothing like it was
and I have never felt worse about my
performance”
“It’s a lot harder to engage in such a clinical program
remotely”
Classes(1162)
Students (70x avg)
80,000+ data points
lost
“My appreciation for the teaching staff has grown significantly for the supports and work
they put in for us”
1. Login Frequency
2. Quality of interaction
VLE DESIGN: CRITERIA
VLE DESIGN: PROGRAMME VIEW
VLE DESIGN: STUDENT LOG EXAMPLE
KEY FINDINGS: VLE DATA
N: MVB1 (94), MVB2 (89), GE1 (52), VNUR1 (44)
KEY FINDINGS: VLE DATA
N: MVB1 (94), MVB2 (89), GE1 (52), VNUR1 (44)
KEY FINDINGS: VLE DATA
N: MVB1 (94), MVB2 (89), GE1 (52), VNUR1 (44)
KEY FINDINGS: VLE DATA
N: MVB1 (94), MVB2 (89), GE1 (52), VNUR1 (44)
VLE DESIGN: STUDENT LOG EXAMPLE
Of students who failed modules, 54.5% were flagged, 45.5% were unflagged
Flag Info Autumn Spring
Total Flags 95 161
Unique Students flagged 37 43
Avg Flags per student 2.57 3.74
Avg Flags by week 7.92 13.42
KEY FINDINGS: VLE DATA
3.3
3.4
3.5
3.6
3.7
3.8
3.9
4
4.1
-6 -4 -2 0 2 4 6
SEAtS Usage & GPA VET10060
0.00%
10.00%
20.00%
30.00%
40.00%
50.00%
60.00%
70.00%
80.00%
90.00%
0 100 200 300 400
VLE Topic AccessAccess %
TopicsSEAtS Usage
GPA
KEY FINDINGS: ASSESSMENT DATA2020/21 Assessment Component Type 2019/20 Assessment Component Type
RESEARCH AND FEEDBACKSite:School of Veterinary Medicine, University College Dublin
Participants:Students: 2018:n=13 2019: n=18; 2020: Interviews n=14; 2021: n=21 SAs: 2021: n=10
Methodology:Mixed-method approach
Instruments:i.Questionnaire – Writtenii.Qualitative Interview – Phone and Written
Analysis: Reflexive Thematic Analysis(Braun & Clarke, 2014; Clarke & Braun, 2018)
• F2F instruction is missed
• Student Advisers seen as vital
• Support for early flagging
KEY FINDINGS: 2020 STUDENT FEEDBACK
“Professors are very available for help and
questions”
“Physical attendance is important so they
can explain fully what they mean”
“I would not be here today without them”“Really helped with my personal growth”
“Helps you try to solve the problem”
“If its not helping every person but it is helping one person,
you like that”
“There’s that 1% that you maybe need to
keep an eye so reaching out is nice”
• F2F instruction is still missed
• Student Advisers still seen as vital
• Challenges of online learning
KEY FINDINGS: 2021 STUDENT FEEDBACK
““(Advisor Name) is a great help”“She is amazing and so helpful”
“Great to know that there is a readily available advisor always there for you”
“Online learning makes my studies
seem more like chores”
“I don’t feel like a student in university without any practical
work”
“Lack of organization of lecture content”
“Bombarded with work”
“The balance of college work and personal time has
been lost”
• F2F support is still needed
• Student Advisers foster engagement
• Case for blended approach
KEY FINDINGS: 2021 ADVISER FEEDBACK
“Supporting students who may feel disconnected”
“Key element of role is supporting student integration to third level”
“Difficult to support students remotely, in
particular when students are upset”
“My student cohort are finding remote learning difficult”
“Online space has a place going into the
next iteration of student services”
“Tasks can be completed at a
distance but some face to face contact
is desired”
LESSONS LEARNED
•Scalability•Actionable intervention
data.•Accurate, but limitations
(ie: asynchronous downloading).
•Off-site architecture needed.•Low construction and maintenance costs.
•Address VLE Module ‘Siloing’.•Exists within UCD’s digital
infrastructure.•Ready integration into
stakeholder practice.
•VLEs capacity to foster multi-dimensional engagement.•Ongoing role of on-site student engagement.
Conceptual Operational
TechnicalEconomic
RECOMMENDATIONS: KEY INSIGHTS
• VLE data can enable Advisers to facilitate interventions
• Digital and in-person supports are interlinked
• Try to capture relative, not absolute engagement
• Remote learning has changed support delivery
RECOMMENDATIONS: FUTURE ACTIVITY
• Continue assessing VLE engagement model
• Implement ‘blended’ engagement monitoring tools
• Disseminate academic & internal lessons learned
• Identify value-add activity areas for continuation
CONTACT US• [email protected]
REFERENCES1. Akçapınar, G., Mohammad, N. H., Majumdar, R., Flanagan, B., & Ogata, H. (2019). Developing an early-warning system for spotting at-risk students by using eBook interaction logs. Smart
Learning Environments, 6(1), 1-15. doi:http://dx.doi.org.ucd.idm.oclc.org/10.1186/s40561-019-0083-4 2. Alves, P., L, M., & Morais, C. (2017). The Influence of Virtual Learning Environments in Students' Performance. Universal Journal of Educational Research, 5, 517-527.
doi:10.13189/ujer.2017.0503253. Alves, P., Miranda, L. & Morais, C. (2016). Learning styles and access to virtual learning environments in the academic performance, Academic Conferences International Limited, Kidmore
End, pp. 25.4. Ayala, J. C., & Manzano, G. (2018). Academic performance of first-year university students: the influence of resilience and engagement. Higher Education Research & Development, 37(7),
1321-1335. doi:10.1080/07294360.2018.15022585. Black, A. E., & Deci, E. L. (2000). The effects of instructors' autonomy support and students' autonomous motivation on learning organic chemistry: A self-determination theory perspective.
Science Education, 84(6), 740–756. 6. Brooks, C., Thompson, C., Teasley, S. (2015) A time series interaction analysis method for building predictive models of learners using log data. In: Proceedings of the Fifth International
Conference on Learning Analytics and Knowledge, pp. 126–135. ACM, New York (2015a)7. Brophy, J. E. (1988). On motivating students. In D. Berliner & B. Rosenshine (Eds.), Talks to teachers (pp. 201–45). New York: Random House.8. Chipchase, L., et al. (2017). ‘Conceptualising and Measuring Student Disengagement in Higher Education: A Synthesis of the Literature’. International Journal of Higher Education. 6(2), 31-
42. 9. Cook, David & Artino, Anthony. (2016). Motivation to learn: an overview of contemporary theories. Medical Education. 50. 997-1014. 10.1111/medu.13074. 10. Cooper, M., Ferguson, R., & Wolff, A. (2016). What can analytics contribute to accessibility in e-learning systems and to disabled students' learning? Paper presented at the 99-103.
doi:10.1145/2883851.288394611. Credé, M., et al. (2010). Class Attendance in College: A Meta-Analytic Review of the Relationship of Class Attendance with Grades and Student Characteristics. Review of Educational
Research. 80(2), 272-295.12. Fuller, M. B., Wilson, M.A., & Tobin, R. M. (2011) The national survey of student engagement as a predictor of undergraduate GPA: a cross-sectional and longitudinal
examination, Assessment & Evaluation in Higher Education, 36:6, 735-748, DOI: 10.1080/02602938.2010.48879113. Gardner, J., & Brooks, C. (2018;2017;). Student success prediction in MOOCs. User Modeling and User-Adapted Interaction, 28(2), 127-203. doi:10.1007/s11257-018-9203-z14. Gasevic, D., Dawson, S., Rogers, T., & Gasevic, D. (2016). Learning analytics should not promote one size fits all: The effects of instructional conditions in predicting academic success.
Internet and Higher Education, 28,68–84.15. Hlosta, M., Zdrahal, Z., & Zendulka, J. (2017). Ouroboros: Early identification of at-risk students without models based on legacy data. Paper presented at the 6-15.
doi:10.1145/3027385.302744916. Jayaprakash, S. M., Moody, E. W., Laurıa, E. J., Regan, J. R., and Baron. J. D. Early alert of academically at-risk students: An open source analytics initiative. Journal of Learning Analytics,
1(1), 6-47.17. Kinsella, M., Nestor, N., Rackard, S., Last, J. & Wyatt, J. Using Attendance Analytics as a Motivational Tool for First-year University Students: The Live Engagement and Attendance Project
(LEAP). The European Conference on Education 2020 (ECE2020), 16th-19th July 2020 University of London. IAFOR.
18. Kinsella, M., Wyatt, J. & Nestor, N. 2021c. Responding to Acute Changes in Higher-education Engagement: Insights from UCD ‘Live Engagement and Attendance Project’. Ireland International Conference on Education (IICE 2021). Dun Laoghaire, Ireland.
19. Kelly, Anna M., and Padden, Lisa (2018). Toolkit for Inclusive Higher Education Institutions: From Vision to Practice. Dublin: UCD Access & Lifelong Learning https://www.ucd.ie/all/t4media/0274_UCD_TOOLKIT_1118_ONLINE_LR.pdf
20. Morra, T., & Reynolds, J. (2010). Universal Design for Learning: Application for Technology-Enhanced Learning. Inquiry: The Journal of the Virginia Community Colleges, 15 (1). Retrieved from https://commons.vccs.edu/inquiry/vol15/iss1/5
21. Macfadyen, L. P., & Dawson, S. (2010). Mining LMS data to develop an “early warning system” for educators: A proof of concept. Computers & Education, 54(2), 588–599. http://dx.doi.org/10.1016/j.compedu.2009.09.008.
22. Mubarak, A. A., Cao, H., & Zhang, W. (2020). Prediction of students’ early dropout based on their interaction logs in online learning environment. Interactive Learning Environments, , 1-20. doi:10.1080/10494820.2020.1727529
23. Nistor, N. & Neubauer, K., From participation to dropout: quantitative participation patterns in online university courses. Computers & Education, 55, 2, (2010), 663-672.24. National Centre for Learning Disabilities (2020). Promise and Peril: Examining the Role of Ed Tech for Students With Disabilities https://www.ncld.org/wp-
content/uploads/2020/07/Promise-and-Peril-Examining-the-Role-of-Ed-Tech-for-Students-with-Disabilities.07212020.pdf25. Nik Nurul Hafzan, Mat Yaacob, Safaai, D., Asiah, M., Mohd Saberi, M., & Siti Syuhaida, S. (2019). Review on predictive modelling techniques for identifying students at risk in university
environment. MATEC Web of Conferences, 255, 3002. doi:10.1051/matecconf/20192550300226. Ormond Simpson (2013) Student retention in distance education: are we failing our students?, Open Learning: The Journal of Open, Distance and e-Learning, 28:2, 105-
119, DOI: 10.1080/02680513.2013.84736327. QS. 2020a. The Impact of the Coronavirus on Global Higher Education. QS. Accessed at: http://info.qs.com/rs/335-VIN-535/images/The-Impact-of-the-Coronavirus-on-Global-Higher-
Education.pdf28. Sclater, N. 2013. Learning Analytics: The Current State of Play in UK Higher and Further Education. JISC29. Simpson, S. (2013). ‘Student retention in distance education: are we failing our students?’ Open Learning: The Journal of Open, Distance and e-Learning. 28(2), 105-119.30. UCD Agile: Mapping our Student’s Experience of UCD – Executive Summary: https://www.ucd.ie/agile/whatagiledoes/mappingourstudentsexperience/thestudentperspective/31. UCD Index Survey 2019: https://www.ucd.ie/itservices/t4media/UCD%20INDEx%202019%20Student%20Survey%20Quantitative%20Analysis%20.pdf32. Valle, A., Núñez, J. C., Cabanach, R. G., González-Pienda, J. A., Rodríguez, S., Rosário, P., et al. (2009). Academic goals and learning quality in higher education students. The Spanish
Journal of Psychology, 12(1), 96-105.33. Webber, Karen L., et al. Does Involvement Really Matter? Indicators of College Student Success and Satisfaction. Journal of College Student Development, vol. 54 no. 6, 2013, p. 591-
611. Project MUSE, doi:10.1353/csd.2013.0090.34. Wolff, A., Zdrahal, Z., Nikolov, A. & Pantucek, M. (2013) Improving retention: predicting at-risk students by analysing clicking behaviour in a virtual learning environment, LAK13, Leuven,
Belgium.
REFERENCES
THANK YOU!