1

How We Taught and What We Learned: Teaching and Learning at SCU in Spring, 2020

August 7, 2020

The Collaborative for Teaching Innovation (A partnership of Faculty Development,

Academic Technology, and the Office of Assessment)

Executive Summary

SCU faculty and students began Spring Quarter in unexpectedly novel and stressful circumstances as the community had to make a sudden shift to remote teaching and learning because of the COVID-19 pandemic. Surveys completed in late May and June by 396 faculty and 1,421 undergraduate students provide us with a detailed account of the adaptations made by faculty to their courses and instructional methods, as well as undergraduate students’ learning experiences during the quarter. Through their responses, we learn what approaches faculty and students perceived as working well (and not so well) in teaching and learning. As a community of teachers and learners, we can carry these insights forward.

From faculty, we learned o Most faculty started the quarter with little previous online teaching experience; 82% spent somewhat

or much more time on teaching than in previous quarters o About three-quarters of faculty met synchronously with students, using Zoom for lectures or

discussions. o Although synchronous sessions worked fairly well for most, there were some challenges, especially

with discussions and larger classes o About 70% of faculty created pre-recorded material—mostly creating their own videos, but also using

pre-recorded materials created by other sources o Some instructors adjusted their instructional practices, assessments, and communication to better fit

the affordances of the online modalities and to better meet students’ needs during the pandemic o For future teaching online, most faculty would opt for a hybrid model (approximately 60% synchronous

and 40% asynchronous) o As the quarter ended, faculty felt much more prepared to teach online (just 33% felt unprepared or

only somewhat prepared to teach online compared to 66% at the start of the quarter)

From students, we learned o Students experienced considerable stress during the quarter due the disruption of their typical living

and learning situations, and being isolated from their peers o 58% had considerable or great concerns about the level of learning they would be able to achieve; an

additional 26% had moderate concerns o Students valued faculty contact and social presence through synchronous sessions, pre-recorded

videos, and communications o Many experienced frustrations with course organization, access to recorded class materials, and some

of the assessments—especially timed or close-book exams o Students developed new insights about benefits of some online instructional approaches, including

pre-recorded lectures to help them learn and review o Most struggled with course workload, finding it too heavy; at the same time, many identified their own

challenges with focus, organization, and motivation o Most students did not feel they had sufficient opportunities to interact with their classmates and they

deeply missed this aspect of their course experience

2

3

Table of Contents

1. The Context for Spring Quarter 2. Overview of the Surveys and the Respondents 3. Instructional Methods in Courses

3.1. Synchronous Instruction 3.2 Asynchronous Instruction

4. Assessments 4.1 Faculty Use of Low-Stakes Assessments 4.2 Faculty Use of High-Stakes Assessments 4.3 Students’ Perceptions of Assessments 4.4 Spotlight on Camino Discussions 4.5 Spotlight on Group Projects 4.6 Spotlight on Workload and Assignments

5. Using Camino 6. Communication Between Faculty and Students 7. Student-Student Interaction 8. We Keep Learning

4

HOW WE TAUGHT AND WHAT WE LEARNED August 7, 2020

1. The Context for Spring Quarter Just prior to the last week of Winter Quarter, SCU informed the community that students would complete the last week of the quarter and finals remotely. Faculty and students made hasty preparations for how to complete the quarter’s work. SCU subsequently made the decision that all instruction during Spring Quarter would be remote. The start of Spring Quarter was delayed by one and a half weeks to provide additional time for faculty to adapt their courses for remote instruction. Most had not had previous experience in online teaching, and, for many instructors, this work needed to be done not only from home, but also while caretaking children and family since the Bay Area was now sheltering in place. Two surveys, one at the beginning of the quarter and one mid-quarter, identified considerable sources of stress and concern for students.

Snapshot of the Faculty Context Limited Prior Experience

o 73% had not taught a course online before spring quarter o 66% felt unprepared or only somewhat prepared to teach remotely

Significant COVID-19 Related Challenges o Balancing family, household, and work responsibilities o Time management o Adjusting to working remotely

Time Demands o 82% reported spending much more or somewhat more time on teaching than usual o 78% reported spending much less or somewhat less time on scholarly/creative work

Snapshot of the Student Context

As the quarter started students expressed considerable stress and uncertainty about:

o Academics: Maintaining academic performance and managing expectations for learning o Social: Staying connected with peers and faculty o Impact of COVID-19 on physical health of friends, family and self o Having adequate mental health support o Access to Academic and Learning Support Services o Access to basic resources (income, adequate technology)

As the quarter progressed, students expressed additional concerns about:

o Challenges with technology (unreliable, competing demands at home) o Finding quiet spaces to attend class and study at home o Difficult home dynamics o Disruptions to study routines that helped them manage their work o Course workload o The impact of social isolation and absence of social support o Stress and anxiety leading to difficulty focusing and working productively

5

2. Overview of the Surveys and the Respondents

Two surveys were administered at the end of the quarter to better understand the teaching and learning experiences of faculty and students.

The Faculty Survey The faculty survey was developed jointly by the Collaborative for Teaching Innovation and the Faculty Senate. Between May 28 and June 7, all SCU faculty were invited to complete a survey about their learning experiences in spring quarter, 2020. A total of 396 faculty provided completed surveys by the deadline.

Faculty Respondents School/College College of Arts and Sciences: 62% Leavey School of Business: 16% School of Engineering: 11% School of Education and Counseling Psychology: 6% School of Law: 3% Jesuit School of Theology: 1%

Appointment Type Tenured/Tenure-track: 48% Lecturer/senior lecturer: 24% AYAL: 17% QAL/SAL: 6% Other: 4%

The Student Survey

During the last week on instruction, all 5,370 undergraduate students enrolled in spring quarter were invited to complete a survey about their learning experiences in Spring Quarter, 2020. A total of 1,421 students completed surveys by the deadline, representing students across all schools and all academic years.

Student Respondents School/College College of Arts and Sciences: 56% Leavey School of Business: 26% School of Engineering: 18%

Gender Woman: 60% Man: 40% Non-binary: <1% (n=7)

Responses by Race and Ethnicity (total greater than 100% due to multiple responses) African-American/Black: 4.3% (n=54) Asian/Asian-American: 25.5% (n=318) Hispanic/LatinX: 17.7% (n=221) Middle Eastern: 3.3% (n=41) Native American/Alaskan: .9% (n=11) Native Hawaiian/Pacific Islander: 1.7% (n=21) Non-Hispanic White: 63.9% (n=798) South/Southeast Asian: 6.8% (n=85)

Year in School First-year 32% Second year: 28% Third year: 25% Fourth year: 15%

First-Generation Status Neither parent/guardian attended college: 8% One or both parents/guardians attended, but no degree: 5% One or both parents/guardians have a 4-year degree: 87%

6

3. Instructional Methods in Courses Most instructors drew upon a variety of instructional methods in Spring 2020, including synchronous lectures and discussions and pre-recorded lectures made by themselves and others. Figure 1 shows the percentage of the faculty who used various instructional methods, and of these, how many felt they worked well. Figure 1. Percent of faculty reporting use and effectiveness of instructional methods

3.1 Synchronous Instruction

Benefits of Synchronous: Most Similar to In-person Classes

Faculty Students “All my students told me they not only appreciated the human interaction being live together, but also felt they learned most effectively being live together instead of listening to recordings. We connected every single time we met—we were able to get to know each other so well and talk about our experiences, and lectures felt personal and interactive. I say this from the feedback I received from them weekly. The students also said that they really appreciated us keeping our lecture and lab schedule exactly as it would have been in-class. They depended on the reliability and routine and said that it made their lives easier and less stressful.”

“Synchronous classes with an attempt to create a similar lecture/discussion/test analysis model that I use in face to face classes. Most students seem to need to be 'in class' and to have the focused attention of a faculty member.”

“I really enjoyed having the structure of a weekly Zoom class! I think being with other students and the professor in real time approximates the classroom experience as best as it can and worked quite nicely.” “Use of Zoom makes classes feel more connected to teacher and other students.” “Zoom classes should be at least in part devoted to either lecture time or time to work on assignments/readings. It is much easier to ask questions and gain understanding of concepts when you are working through the material with the professor in real time.”

73

66

54

71

60

22

16

67

58

59

72

66

70

65

0 10 20 30 40 50 60 70 80

Synchronous lectures via Zoom

Synchronous discussions via Zoom

Zoom breakout rooms

Content videos created by faculty

Content videos created by others

Audio recordings by instructor

Audio recordings by others

Used Worked well

7

Most faculty opted for synchronous sessions using Zoom as the primary teaching modality: o 58% of the faculty used most or all of their scheduled class times for Zoom sessions; o 32% used Zoom some of their scheduled class times for Zoom sessions; o only 10% did not use Zoom for their courses.

Faculty felt that synchronous sessions over Zoom met students’ needs for contact with them and their peers, and provided an effective way to respond to students’ questions about material. Many instructors reported feeling more prepared to adapt their instruction for synchronous sessions using Zoom in the brief time they had available to adapt their courses for the quarter. Most students also found synchronous instruction to be a critical element in their course experience. Synchronous class sessions in Zoom were the closest thing to a typical class experience, allowing students to connect with others, ask questions, and interact with faculty. Many students reported that Zoom class sessions helped to keep them on track with their studies and learning. Synchronous Sessions for Content Instruction: Strategies and Challenges Many instructors felt synchronous sessions via Zoom worked effectively for their discipline and to engage students. With the screen sharing function on Zoom, faculty shared instructional materials, such as slides, images or graphs, or video clips. Others incorporated other tools such as using an iPad or a document camera for writing formulas or working problems to simulate the kind of work that would have been done on a whiteboard in class. Although many instructors found the synchronous sessions worked better than they expected, a number noted challenges in “reading the room” and knowing whether they were keeping students’ attention. This was more difficult with larger classes and in the cases where students had their cameras off. Some instructors described adapting to these conditions by shortening their class period or limiting the total number of synchronous meetings. Another approach was to shift the focus of the class time to focus more on previously solicited student-generated inputs, including students’ questions, their identification of “murkiest points” or other observations, before going onto new material. Although students valued the synchronous sessions, a substantial number (31%) did not find them useful in their learning (59% did). One clear limitation was when recordings of Zoom class sessions were not readily available. By not having timely access to class recordings, they experienced a less equitable and effective learning environment.

o Students described being unable to attend class in part or in its entirety due to technological problems or limited bandwidth with an entire family using the Internet at the same time.

o They also experienced challenges with time zone differences, family issues, or distractions in their home study environment making it hard to concentrate during all Zoom sessions.

o They also referred to learning benefits of being able to review class recordings to better understand the material and prepare for exams.

A second limitation stemmed from the nature of the modality itself and how the experience of learning through a screen differs from learning in person. From the students’ perspective, this necessitated adjustments by faculty—shortening lengthy sessions, providing relevant work beforehand to prepare them to get the most out of class, breaking up the class into smaller sections, and not necessarily using Zoom for every single class.

8

Students’ Advice about Synchronous Class Sessions: Record and Adjust

“[P]lease require professors to upload a recording of our class by a certain date, some of my professors …. would wait multiple days and frequently over a week to upload a recording of our zoom meeting which is obviously extremely frustrating.”

“I think recording live Zoom lectures is the best option. The live component facilitated class engagement, but having the recorded lectures allowed students to go back and absorb crucial concepts in class. The nature of online classes makes it harder to concentrate, so you need to be able to re-watch things to fully understand the material.”

“I think recording all our zoom meetings would be very helpful. Most of my professors did this but some didn't. It was really great when studying for exams to go back to previous lectures or discussions from class. Another thing was, most of my professors were really accommodating to international students with our time difference and allowed us to watch over the zoom meetings after class but some professors weren't ... [then] I wasn't able to focus on the lectures or contribute well to class discussions.”

“Don't expect a Zoom lecture to be an actual lecture—the fact of the matter is it isn't the same and we as students don't retain nearly as much information.” “Not too many or too long of Zoom classes or lectures. Talk about expectations with students over Zoom.” “Professors need to be engaging with the students and provide lots of examples for the material. It is already difficult to look at a screen all day, but learning at a fast pace is no help.”

Synchronous Class Discussions: Strategies and Challenges Two-thirds of faculty reported integrating discussions into their synchronous classes. Somewhat fewer (54%) used the break out room feature of Zoom to facilitate smaller group discussions among students. Synchronous class discussions in Zoom enabled faculty to further recreate part of the character of in-person teaching.

Strategies for Managing Synchronous Class Discussions

Faculty Students “Breaking the class into smaller (12-15 students) Zoom discussion sections following my video lectures created the most connection between me and students, as well as achieving the most discussion feel to the class.” “Break out rooms in Zoom were the gold mine! My weekly synchronous class meetings start with a whole group meeting and Q&A, then break outs to discuss a particular prompt, then back together to share out and discuss.”

“Engage the class with open-ended questions and discussions. Breakout rooms were really helpful to discuss material and interact with classmates virtually.”

“For discussion-based classes, it would be useful to have a more organized way for students to volunteer to participate. Additionally, using break out rooms would be useful to help increase student-to-student interactions, which all of my classes were missing.”

9

“I believe that the most helpful strategies were the class discussions via Zoom, which followed individual short assignments based on readings/viewings. I think they were effective because they were collaborative, engaged students in group and class activities, gave everyone a chance to share comments, do mini-presentations, bring to class points that were difficult to comprehend, and also concerns that students had.”

“Breakout rooms are very useful in Zoom as long as their purpose is directed, as is encouraging cameras to be on when possible.”

“[B]reak out rooms with a recurring group of students is very helpful.”

The greatest challenge for faculty with whole-class synchronous class discussions involved managing the dynamics of a large group discussion on a platform like Zoom, especially when students turned their cameras off. Some instructors opted to hold scheduled discussions by Zoom with a subset of the students on alternate dates to manage the challenges of holding discussions with larger groups. The solution for others was to use break out rooms to have students engage with one-another. Faculty reported that asking students to produce a clear outcome from the break out room discussions was a useful way to help students focus—this might include sharing out an image, slide, or Google doc summarizing the main points of the small group discussion. Students reported that whole-class discussions (especially of larger classes) on Zoom worked best when faculty managed the class environment by asking students to raise their hands, use the chat function to guide discussion (possibly including a student to moderate the chat), and when they could see that faculty felt comfortable with the interface. Students offered clear advice about the need for faculty guidance about participation norms. In break out rooms, students asked for specific prompts to guide discussion and a meaningful way of reporting the group’s discussion.

3.2 Asynchronous Instruction Pre-recorded videos were a second commonly used instructional method in Spring Quarter. Most of the recordings used were shorter lectures in length and were created by faculty, but some instructors also incorporated content-specific video recordings produced by others. Fewer faculty made or used audio recordings, but the overwhelming number who did felt they were successful (see Figure 1).

Why Asynchronous Instruction Worked Faculty Students

“Recorded lectures, broken into smaller chunks of time, for ease of reference [worked best]. I believe this allowed the students easy access to go back and review specific topics.”

“It was not possible for me to present new material effectively during a Zoom session. I opted instead to substitute one hour of our three-hour class session with pre-recorded videos of PowerPoint presentations, supplying students with one question in advance for each 15-minute video and then quizzing them at the start of class

“Pre-recorded lectures are great because students can pause when confused, slow the video down to better understand the concept or re-watch it entirely for a refresher. However, the only disadvantage to this is that students may not watch the videos and fall behind. .... Creating periodic assignments/assessments (ideally short quizzes) should incentivize the student to keep up.”

“… I appreciated when my classes that were more data driven (i.e. OMIS & FNCE) were pre-

10

on the content. It helped them focus when watching the videos and identify what was most important, and what they were responsible for. It also provided a baseline from which I could see what they understood and what they did not, so I could focus on the material they were struggling with.”

“I flipped my class. I have been toying with the idea of flipping it for years, but the need for asynchronous lectures gave me the nudge I needed to do it. Student outcomes are FAR better on tests this quarter despite the stress/issues with pandemic.”

“The asynchronous video presentations allowed students the flexibility they need to arrange learning around time zone and other constraints.”

recorded since discussion wasn’t a necessary part of the class and I could do the work on my own time.” “…For classes relating to art, such as painting and dance, it is useful to have professors record demonstrations and tutorials so that students can learn on their own time and that way, it is more clear to view the tutorial and learn rather than learn through Zoom meetings.” “Pre-recorded lectures are a great study tool for students because we can play them back at accelerated speeds, pause to take notes at a comfortable pace, and rewind for material that confused us. Although we can't ask questions in real time, this incentivizes us to attend Zoom office hours, or dig for the answer on our own, which is the best way to absorb knowledge compared to blind note taking.”

Many instructors recognized the flexibility that pre-recorded materials afforded students in planning their study time, in accommodating students from different time zones, and recognized their instructional value for pacing learning and review. Pre-recording their lectures also allowed instructors to use tools that they felt were more appropriate for student learning in their disciplines, such as a white board, demonstrations, etc., and that would be more difficult to manage over Zoom. Relatively few courses were taught entirely asynchronously; many instructors used pre-recorded material in conjunction with in-person Zoom sessions and reported that this was effective. Developing systems for accountability for student viewing was critical in a flipped classroom model. Giving a quiz or assignment linked to a pre-recorded lecture helped ensure students had viewed it prior to class and this led to more productive in-class sessions. Some faculty discovered the value of various digital tools to insert “pauses” for questions or guided reflection in their pre-recorded lectures.

Over two-thirds of students agreed that pre-recorded content, especially when that material was prepared by faculty, was useful in their learning. Reviewing pre-recorded instructional content was particularly helpful when the faculty delivered technical, procedural, or complex material.

Qualities of instructional video content students found most helpful Students were asked about the qualities of pre-recorded videos most helpful in their learning.

o Most preferred a video of 11-20 minutes in length o More students preferred seeing pre-recorded lectures that contain both slides and video of the

instructor, rather than a narrated slide deck without the instructor present o Nearly 1,000 students affirmed that it was helpful to be able to review pre-recorded lectures,

with some recommending that these be used even in to face-to-face instruction

11

Figure 2. Number of student responses to helpful qualities in pre-recorded lectures

4. Assessments Faculty used a variety of low-stakes assessments and high stakes assessments in their instruction, often adapting them to the digital environment. Both faculty and students described what they perceived as working well, in addition to the challenges they encountered with assignments and exams in the spring.

4.1 Faculty Use of Low-Stakes Assessments Faculty included an array of informal or low-stakes (smaller) assessments to both gauge and promote student learning.

o Many of the methods used were tried and true, such as short written summaries of student learning, Camino quizzes, and surveys designed by the instructor to check in with students about their course learning experiences.

o The chat function, non-verbal features, and polls in Zoom, were brought in as a way to get quick feedback on student learning and to promote active engagement in the class sessions with fairly helpful results, according to faculty.

Figure 3. Percent of instructors reporting using informal/low-stakes assessments and their efficacy

526

714

416

955

345

804

0 200 400 600 800 1000 1200

Short pre-recorded lectures (less that 10 minutes)

Mid-length pre-recorded lectures (11-20 minutes)

Full-length pre-recorded lectures (more than 20 minutes)

Being able to view pre-recorded lectures more than once

Pre-recorded lectures consisting only of narrated slides (novideo of instructor)

Pre-recorded lectures that included both visuals and videoof instructor

70 69

50 49 49

33

76

46 4254

65

40

0

20

40

60

80

Short writtensummaries

Zoom chat Zoom non-verbalfeatures

Check-in surveys Camino quizzes Zoom polls

Used Worked Well

12

4.2 Faculty Use of High-Stakes Assessments Faculty also used a variety of high-stakes assessments including multimedia assignments, research papers, exams, presentations, and group work. Figure 4 displays which assessment methods were used and how effective faculty found them. It also shows instructors’ use of features within Camino to support these assessments. Figure 4. Percent of faculty reporting using high stakes assessments and their efficacy

Some instructors reported modifying their assessments to better fit the change in instructional modality and/or because they felt the new assessments better addressed students’ needs and stress levels during the pandemic. Exams: Greater use of open book/notes than closed book/notes

o More faculty (50%) opted for open book/open notes exams, compared to closed book/open notes exams (30%), noting that these were a better fit for the online environment.

o Few instructors reported using the Lockdown Browser or Monitor feature; some commented that these features undercut expectations of the Academic Integrity Pledge, and urged that we treat students as responsible adults.

Assignments: Blend of “traditional” research papers and multimedia assignments

o 70% of faculty asked students to complete multimedia assignments making use of the technologies on which faculty and students were now relying; 68% assigned research papers and reports.

o About half of the faculty (49%) used group assignments and student presentations in the Spring as well. In each case, two-thirds of instructors (66%) felt they worked well.

o Camino supported instructors’ assessments, especially with Speed Grader which proved to be a welcome discovery for some faculty. Rubrics were not widely used, but when used, faculty

reported they worked well. o Some faculty recognized the need to provide greater clarity in assignment instructions for the

online environment and noted this as an area for improvement in future courses, possibly making a video to explain the assignment in more detail.

75 70 68

50 49 4839

32 30

9 6

7767

7765 67 66 70 65

5850

44

0102030405060708090

Used Worked Well

13

4.3 Students’ Perceptions of Assessments Assessments are key piece of course expectations for students. Two-thirds of students reported that course expectations were clear in most or all of their courses, but just 54% reported being clear how they would be evaluated. In terms of their role in supporting learning, assignments—in comparison with tests—proved to be most helpful to students in most or all of their classes. Figure 5. Percent of students reporting that assessments were helpful to their learning

None of my classes

Some of my classes

Most or all of my classes

“Short quizzes were mostly helpful to my learning” 17% 43% 40%

“Exams were mostly helpful to my learning” 18% 46% 36%

“Assignments were mostly helpful to my learning” 3% 42% 55%

Quizzes and Exams: The Challenges Frustrations with test-taking conditions were the number one source of students’ concerns about tests. Monitoring strategies were experienced as invasive and created a source of stress beyond the exam itself. Students felt anxiety while taking tests due to disruptions because of not having access to a private study space. There were concerns about technology failing mid-test and that timed tests were often harder to complete in the designated period in an online environment. Many students recommend that faculty design alternative assessments like take-home exams, projects, or assignments, rather than tests, that work better for remote, online learning.

Students’ Challenges in Taking Quizzes and Tests

“In one of my classes I had Camino quizzes that seemed to be very hard to complete in the given time. I assume that my teacher made the time limits so that we wouldn't have time to cheat… This was really frustrating for me as I assume I would have been given a lot more time to finish the quiz if the class had met in person....”

“…I personally feel uncomfortable using software that use my webcam to watch me take tests & with a family at home it’s hard to guarantee it will be silent in the background or they won’t walk by.”

“With a full family sheltering in place, our internet speed, has been slow. Because of this, our internet has frozen and kicked me out throughout classes, (and) giving me anxiety every time I have to take a test in fear that my internet may fail mid-testing.” “Make tests take-home and don't force students to have their face camera on. Screen anxiety is a real thing because you are forced to look at your own face the entire time and it makes it really hard to focus on the task, especially in tests…”

14

“Consider the time zone issue and maybe give multiple time slots for quizzes/tests as I had to start exams at 9:30 at night more than once.”

“If you teach a MWF class, please no longer post exams and quizzes on Tuesdays and Thursdays--I don't even know how this was allowed. I was required to skip lectures and labs for other classes to take these exams and quizzes.”

“One of my teachers allowed open-book, open-note, open-internet tests, and I think that was a great policy. It's more representative of the "real world", it reduces cheating, and students could learn things while taking the tests...”

4.4 Spotlight on Camino Discussions

Nearly two-thirds (64%) of instructors developed asynchronous discussion assignments in Camino and about half of those thought they worked well. Camino discussions were addressed quite extensively by faculty and students in their comments and received mixed reviews.

Comments about Camino Discussions

Faculty Students “I was very impressed with my students' responses to the Camino discussion boards (which I based on the readings). I would guess that higher percentage of students did the readings in my online class vs in-person.”

“Discussion threads [worked well] because students could reflect more deeply on their responses and interact without the social anxiety of being in a classroom or of taking up too much time. I'll incorporate them into future in-person classes now.”

“The students just don't like doing them, it feels like busy work and they have a ton of them to do for every class, …they're rarely particularly good or interesting or even unique (they all start to sound the same after a while), except when the students relate things to their own lives and I get to see a little more personality in the posts.”

“The Camino discussion boards ended up not working well. Students did not relate comments back to course materials and sometimes put comments that were incorrect and spread misinformation as others agreed and continued to comment. Not being able to provide immediate feedback during the discussion and redirect it as I would in class proved to be very problematic.”

“… the use of discussions in Camino was extremely helpful in making sure we kept to the readings and helped with synthesizing ideas and discussing things we might not have understood about them.”

Depending on the course, Camino discussions are helpful and allow us to communicate with our classmates. “In my opinion, Camino discussions are just short writing assignments with a facade of socialization [sic], whereas Zoom discussions are better for understanding each other’s perspectives…”

“Camino discussions feel weird because it feels like everyone is trying to write formal comments when the classroom discussion would be much more informal.”

“Don’t use Camino discussions with comment quotas too much if you really want students to think deeply about the material.”

“I don’t particularly feel as if I gained any knowledge (other than from the instructors) by working on Camino discussions because everyone simply writes a response, comments on two other discussions and leaves it at that.”

15

When discussions worked well, faculty commented on students’ critical engagement with course ideas and the positive impact of this type of preparation on discussions in the synchronous Zoom classroom. Other faculty found discussions less useful in sparking student engagement, noting repetition among posts, the lack of deep engagement in the topic and in other students’ posts, and at times, the perpetuation of incorrect information.

Just 38% of students found Camino discussions to be useful for their learning; 45% claimed they were not useful. Their comments highlight problems with purpose, frequency, and in the logistics of completing the task.

o Some students found value when Camino discussions were framed less as a way of encouraging “interaction” among students and more of a useful way to keep up with the course readings and engage in critical reflection

o In some classes, students felt they were used too often, or that with all the other assigned work, completing them became a heavy burden

o Some students experienced time zone pressure in meeting the requirement for responding to other students’ posts by a given time if the deadline for a response was too close to the deadline for submission

4.5 Spotlight on Group Projects Nearly half (49%) of instructors assigned group projects, with two-thirds of them reporting they worked well.

Comments about Group Projects

Faculty Students

“Students together define a problem related to the course material, analyze it, research related information and arguments, and create their own Zoom presentations as well as written reports…students seemed to come most alive when engaged with … those projects.”

“Group projects: the students were yearning for connection with each other and actually put more work into these than into other assignments in the class.”

“Dividing even my smallest class into smaller teams for both collaborative projects and discussion groups created more intimate connections among students and allowed me to focus my feedback and other engagement. … [s]etting up groups at the beginning of the course to engage in low-stakes discussions that built to more complex collaborative work went well.”

“I tried really hard to create a "community of learners" even though we were remote learning.

“…group work is definitely less effective in a virtual course, since it is much harder to organize and coordinate.”

“Group projects are already terrible and having them online was one of the worst ideas ever. Do not make students do online group projects, no one is team building, people are ignoring the one person working.”

“Multiple times this quarter, my group mates just didn't do group work, and I was left doing entire group projects by myself.”

“It was incredibly difficult to coordinate with peer review or group projects in classes where you taught yourself [no synchronous Zoom].”

“I … think faculty should assign groups if they use group projects as it's hard for students to connect with and meet other students in their classes if they were to choose a group.”

16

To that end, I created a group project for the last assignment and I also required them to work in groups on their Problem Sets.”

In their comments, faculty often addressed their motivation for assigning group work as a way to give students an opportunity to learn together.

However, similar to Camino discussions, a greater number of students experienced their group projects as not useful to their learning (45%), than found them useful (38%). Additionally, three-quarters of students reported that completing group projects was more difficult when learning virtually in Spring than in prior quarters. Students expressed the types of frustrations about group projects that instructors often hear in in-person classes, including challenges with coordinating teammates’ schedules and inequitable workload distributions. Many students added that all of these become even more difficult online, when students are dispersed over different time zones, without consistently reliable Internet, and without having much of an opportunity to get to know their classmates before launching into the project. Students in certain disciplines mentioned the additional challenges of not having access to the tools they would have in a face-to-face setting to allow them to work through complex materials. Student comments suggest that if group projects are assigned, instructors should design them judiciously and provide students with strategies that will lead them to be successful, including:

o frequent check-ins with faculty by Zoom o preassigning team members rather than asking student to pick team members themselves o considering lower-stakes ways for students to work in groups other than on a big project

4.6 Spotlight on Workload and Assignments

One of the most common types of feedback provided by students about assignments pertained to their role in the overall course workload.

Students’ Perceptions of Workload

“There is no need to make courses harder … I felt as though a few of my professors were assigning more work just because it was online and they thought we wouldn't learn as much as in the classroom. If every teacher has this mentality, you can see how that can lead to a very overwhelming and stressful experience even in the middle of a pandemic and one's own life struggles with learning at home.”

“I am an engineering student in my third year and used to a heavy workload, but this quarter was by far one of the heaviest I have endured. I worked on schoolwork every single day of the week. I have not left my house for the entire 10 weeks as it has consumed my life. This quarter was heavier in assignments/having to re-watch lectures and online assignments in order to not fall behind. Quarters already move at a rapid pace, this one was as if someone taped a brick to the gas pedal.”

Less than half of students (45%) found the workload to be appropriate in most or all of their classes; another 50% said the level was appropriate in some of their classes. Students speculated that perhaps faculty overcompensated because they wanted to make up for the missing one and a half weeks of instruction, or were concerned about how the change of instructional modality would affect student learning, or that faculty believed that students had a lot more time on their hands because they were

17

staying home because of COVID. Although students seemed to prefer assignments over exams (especially heavily weighted ones), there seemed to be a tipping point.

Further, many students wrote of their frustration with the difficulty they had locating information about assignments, including due dates, and understanding the assignment’s purpose, instructions, or how it would be evaluated. This contributed to stress, a feeling of being overwhelmed, and to a sense of unfairness. They recommended sharing assignments earlier to allow more time for completion, being realistic about how much time assignments will take to complete, and ensuring that assignments are meaningful, relevant to what’s being actually taught, and connected to the real world. Students also discussed the importance of using Camino intentionally and accurately to help them manage their work.

Students acknowledged they had more difficulties focusing, staying organized, and finding motivation during the quarter, and this had an impact on how effectively they worked. Survey data show that maintaining important aspects of their wellbeing was indeed much harder in Spring: students reported that it was more difficult to maintain physical wellbeing, stress management, mental wellbeing, connections with friends and contact with campus groups. Over three-quarters (78%) told us that it was more difficult to put as much effort into their courses as they are used to doing.

5. Using Camino

Many instructors recognized that Camino is an essential platform for students that guides instruction and learning. It is also a medium by which faculty can communicate and reinforce course learning objectives in synchronous and asynchronous class sessions. Students offered many observations about faculty use of Camino, commenting on strategies they found helpful and others they found frustrating.

Use of Camino to Guide Instruction

Faculty Students

“I found breaking the course into modules and creating a "module roadmap" with all of the necessary links to readings, submission pages on Camino, etc. to be very helpful.”

“[My use of] Camino had positive feedback from students that they found it very helpful that I made my expectations and deadlines extremely clear to students through clearly labeled weekly modules and an announcement on Sunday night laying out and linking to the assignments for the week.”

“1) Created a weekly module and a cadence each week so students knew what to expect. Every Monday night a discussion thread was due. I reviewed Tuesday AM to tweak our afternoon Zoom session and answer their questions. Then a quiz was due later that week. Every week had one DT and 1 quiz. They were all lowish points that quickly added up as the weeks went on. 2)

“Creating week-by-week modules in Camino through which students have access to a checklist of “things to-do by ___date” is immensely helpful. Also, making it a priority to check in with students at the end of the week and having flexibility when the professor’s expectations were too high/demanding is conducive to learning.”

“Having Camino organized to reflect relevant due dates, as well as what is an assignment vs what a just a reading is helpful. Taking the time to organize Camino and update your syllabus to reflect online learning instead of just telling students as you go what is accurate versus what is just leftovers from the in-person version of the class will help students tremendously organize their quarter and understand what is happening in the class”

“There should be a designated place for everything that's intuitive and straightforward.

18

For the quizzes, they were hand written responses only, that students could either print out my quiz and hand write on the quiz and upload back OR they could write the answers directly in their notebooks and photograph and upload notebook pages (so didn't require a printer).”

“While I designed my course so that it could be taken asynchronously (with pre-recorded lecturers and learning/assignment modules on Camino), I also offered drop in sessions during our scheduled class times. At least once a week, I also scheduled a live synchronous discussion or workshop. I found that these sessions enabled me to provide support to students and also foster community. I posted regular announcements on Camino encouraging students to attend these sessions, reminding them about assignments due, and sharing course-related resources. In meetings, students mentioned that they found this structure to be helpful and easy to follow.”

Modules should be heavily utilized because, in the online structure, it's difficult to keep track of weeks and lecture numbers. Modules, with dates, should be used to organize assignments/recorded lectures/course materials for each week, lecture, or topic as needed.”

“Simplifying the Camino page and posting announcements that are also emailed out to students is definitely key in engaging students. Having clear, simple instructions and reaching out to students to check in with them is beneficial. I think that recording the lectures and separate pre-recorded lectures for walking through assignments or problems is helpful for students to refer back to multiple times.”

“It helps when you make an intro video or at least audio recording of the layout of the week, since we are meeting less often in some classes.”

Student comments emphasize that an organized, easily navigable, accurate Camino course site is a pre-requisite for a successful course experience. About two-thirds of students (68%) found the organization of material in their Camino courses clear, and this gave them the foundation to be successful. Those that didn’t find Camino materials to be well-organized reported they spent a lot of time looking for relevant course information and at times missed deadlines.

19

6. Communication Between Faculty and Students

Comments about communication between faculty and students

Faculty Students

“I emphasized a couple of things: the importance of my frequent communications (often twice-weekly) indicating where students independently working 'should' be in the scheme of things. I also emphasized that *classroom* nature of our asynchronous, small group discussions...”

“Prompt feedback! I tried to answer student emails as soon as possible. By hearing from students regarding assignments, learning objectives I could gauge what potential modifications or additional support was necessary.”

“… scheduled 4 periods of voluntary 1x1 conferences. In weeks when 1x1 conferences were not scheduled, I offered voluntary class meetings. Regular (weekly) opportunities for virtual face-to-face connection have been constructive.”

“… I also found a texting app for teachers that masks users’ numbers, really helpful. I could send quick reminders/announcements to the whole class (usually with ‘check Camino for details’ because of character limits). But more usefully, it allows students to text me quick questions through the app…”

“Acknowledging that students were facing unique challenges this quarter and being flexible about deadlines, helped because students needed to know that I cared about them as whole human beings, not just assignment producers. Focusing on my essential learning outcomes for students and paring away anything that didn't help students achieve those outcomes helped because students appreciate efficient teaching.”

[Most helpful]…“Being in constant communication via email, Zoom, Camino Announcements, Camino discussions, teaching assistant Zoom office hours.”

“[Professor] made great use of the announcements feature via Camino to communicate with my peers and I. She always showed concern for our well-being before jumping into course material which made my classmates and I feel valid in our feelings and valued by her as a professor…”

“…During class, we had weekly check-ins to discuss how we were doing with everything going on in the world. She understood that the stressors surrounding coronavirus and racial injustice shown all over T.V. affected our ability to focus on class, and she did everything in her power to let us know that she was here for us during these times.”

“I had two professors who made effort to schedule one-on-one Zoom meetings throughout the quarter and it was really nice to talk to professors individually.…Both firmly acknowledged hardships associated with current events this quarter and were very accommodating with school work and reducing workload or just being open to talk about what was going on.”

“I felt that bias in classes was not addressed using the online format and that can be really detrimental for students of color. It may be harder for an instructor to shut down a racist comment via zoom, but they could at least make an effort to. And it shouldn't be the job of students to correct insensitive discussion posts on Camino either.”

“The lack of sympathy and support I have received from several of my professors regarding my current situation—no exceptions, alternative assignments, or quiz/exam accommodations, given my time difference and mental health issues—has caused me to seriously question my self-worth as a student and person.”

20

The three primary modes for faculty communication with students were email, office hours over Zoom, and Camino announcements. Of these, faculty judged email and Camino announcements to work the best.

Many instructors reported challenges with getting students to show up to virtual office hours and seemed to have the best luck when they made dropping-in to office hours mandatory—even if it was just for a 10-minute check-in (using Camino Calendar to offer slots to meet), rather than hosting open office hours. Additionally, some faculty described how they approached communication and faculty-student interaction in the spring in a very intentional way, recognizing how important it was for frequent and responsive communication because of the pandemic and the virtual learning environment.

Sixty-five percent of students agreed that they had sufficient opportunities to connect with their instructors in most or all of their courses in the spring, and 65% reported that their instructors seemed concerned with their well-being. Many students wrote about positive experiences interacting with faculty in the spring emphasizing both how important frequency and tenor of the communication were to them. However, some students described disappointing experiences with instructors because of a lack of availability, expressed concern with student well-being, and flexibility in accommodating students’ challenges. Student comments address the important role communication with faculty plays in their learning.

o Clear communication is essential for students to understand what is expected of them so they can do their best work.

o In the context of the stress of the pandemic, faculty communication signaled an openness to engage with students and understand the kinds of familial, economic, and mental health challenges they were experiencing while being at home.

o Faculty communication can help foster or erode students’ sense of belonging and confidence, so it is critical to a successful learning environment for all students and as we have learned at SCU, particularly for students of color, who are less likely to experience a welcoming classroom environment).

7. Student-Student Interaction Fostering interaction among students and building a sense of community in the class was the most elusive component of the class experience in spring, and the one that students reporting missing the most. Instructors used a variety of methods to promote interaction among students, although acknowledged not all achieved that goal. Some faculty were able to encourage community building by forming small groups of students that stayed together all quarter and participated in small group discussions on Camino or in break out rooms during synchronous class time, and even in small group discussions outside of class time. Some faculty set up virtual office hours where students could ask questions, or simply just work alongside the instructor and other students in community. Faculty acknowledged that some of these methods fell short of their intended goal, especially office hours.

21

Figure 6. Percent of faculty using methods to promote student-to-student interaction

Only in some of their courses did students feel they had sufficient opportunities to communicate and interact with classmates, and in even fewer did they feel a sense of community in their classes. Figure 7. Percent of students experiencing peer-to-peer interaction and community

Class interactions and community None of my

courses Some of my

courses

Most or all of my

courses

“I had sufficient opportunities to communicate and interact with my classmates”

17% 61% 22%

“I felt a sense of community in my classes” 35% 53% 12%

Students had a few suggestions for how to encourage interaction, but seemed to recognize that this was indeed very challenging.

Students’ Observations about Student-Student Interaction

“I didn't have any experiences of classes that were especially unengaging ... The only thing I did not like was not having any sort of interaction with my classmates—some kind of (required) discussion space was important to have.”

“You could consider telling us to show our faces so we see the other classmates, which would increase our sense of belonging.”

“Student to student interaction is CRITICAL. Facilitate it at every step if your courses have challenging material. Those courses without a support structure of fellow students were made monumentally more difficult. Please do not expect pre-recorded lectures to work well for all your students, they certainly do not work for me. It is simply not how I learn best.”

“Have more opportunities for classmates to interact with each other.”

“Make the classes more discussion-based, more collaboration between students, less lecture-based.”

78 7364

56 49 45

26

5742

61 5943

0

20

40

60

80

100

Zoom officehours/drop-in

Synchronousdiscussions via

Zoom

Asynchronousdiscussions via

Camino

Zoom breakoutrooms

Group projects Student self-introduction

videos

Used Worked Well

22

“Faculty should receive training on how to adapt online classes to be as engaging as they would in the classroom. It felt like most of my instructors gave up on building a sense of community for the class and having us learn through discussion/work with peers.”

“[Helpful is] putting your class into ‘study groups’ so we can get to know at least 4-6 other students that can help us get through the quarter.”

8. We Keep Learning Faculty and students learned a lot about teaching and learning online in Spring 2020, and what they would do differently were they to engage in virtual courses again.

Faculty, in the midst of an overwhelming teaching workload, experimented with many different technologies and pedagogies, and gained considerable experience. The percentage of faculty who said they felt unprepared or only somewhat prepared to teach online dropped from 66% at the start of the quarter to just 33% by the end of the quarter.

Most faculty stated they would opt to use a blend of synchronous and asynchronous approaches in teaching virtually again, anticipating a 60/40 split of synchronous and asynchronous. Students also identified how hybrid models benefit their learning for many (but not necessarily all) of their courses. Their preferences are grounded in an awareness of the affordances of different modalities, their particular living-study conditions, and understanding of their personal learning processes for different types of subject matter.

When asked for recommendations to guide instruction in online courses in the future, some faculty noted that this needs to begin with rethinking the course itself: how to divide material into smaller modules; identifying what work students can do independently and what needs to be reviewed and discussed together; what assessments are best suited to the learning objectives; and to address creating community more deliberately.

Both faculty and students described the value of including an array of virtual methods including synchronous classes over Zoom to support face-to-face contact, short pre-recorded lectures for certain concepts or material so they can be reviewed, polls or other online tools that allow for more interactivity, drawing upon Camino in a variety of ways—peer review, discussion threads, to provide audio or video feedback, communication, and accountability, and use of “old tech” (e.g., the telephone) to supplement or allow a break from the “new tech.”

Reports of student learning and student satisfaction with remote instruction on our campus dovetails with other research identifying the conditions for more effective online teaching and learning. In a national study of undergraduate students done in Spring, 2020 when most institutions were mid-semester when the pivot to online took place, Means and Neisler (2020)1 found that courses that incorporated a greater number of eight established instructional practices for online teaching and learning received higher levels of student satisfaction. These factors included:

1 Means, B., and Neisler, J., with Langer Research Associates. (2020). Suddenly Online: A National Survey of

Undergraduates During the COVID-19 Pandemic. San Mateo, CA: Digital Promise.

23

o Assignments that ask students to express what they have learned and what they still need to learn

o Breaking up class activities into shorter pieces than in an in-person course o Frequent quizzes or other assessments o Live sessions in which students can ask questions and participate in discussions o Meeting in “breakout groups” during a live class o Personal messages to individual students about how they are doing in the course or to make

sure they can access course materials o Using real world examples to illustrate course content o Work on group projects separately from the course meetings

The proportion of students who reported being very or somewhat satisfied for courses employing 0-2 of the recommended online instructional practices was 43 percent compared to 61 percent for courses using 3-5 of the practices, and 74 percent for courses using 6-8 of the practices. At SCU, many of these same practices were identified by faculty and students as “what worked best.” More generally, our findings affirm the relevance of seven evidence-based principles for undergraduate teaching (Chickering and Gamson, 1987)2 and their extension to hybrid and online teaching.

1. Encourage contact between students and faculty 2. Develop reciprocity and cooperation among students 3. Encourage active learning 4. Give prompt feedback 5. Emphasize time on task 6. Communicate high expectations 7. Respect diverse talents and ways of learning

The learning continues. In summer, 2020, over 300 faculty members are engaged in SCU training experiences--gaining more expertise in pedagogical approaches and use of technology that will directly benefit their teaching. Before Fall Quarter, students too will have an opportunity to complete a multi-part online course co-developed by SCU students, faculty, external consultants and staff that will allow them to build the areas of growth they identified in the survey, and identify additional opportunities to develop as independent, self-directed student-scholars.

2 Chickering, A. W., & Gamson, Z. F. (1987, March). Seven principles for good practice in undergraduate education. AAHE Bulletin, 39(7), 3 – 7.