Work in Progress: Suitability of Spatial Visualization

Paper ID #33761

Work in Progress: Suitability of Spatial Visualization Training forRemote Learning

Dr. Lelli Van Den Einde, University of California, San Diego

Van Den Einde is a Teaching Professor in Structural Engineering at UC San Diego and the President ofeGrove Education, Inc. She incorporates education innovations into courses (Peer Instruction, Project-based learning), prepares next generation faculty, advises student organizations, hears cases of academicmisconduct, is responsible for ABET, and is committed to fostering a supportive environment for di-verse students. Her research focuses on engagement strategies for large classrooms and developing K-16curriculum in earthquake engineering and spatial visualization.

Prof. Nathan Delson, University of California at San Diego

Nathan Delson is a Teaching Professor at the University of California at San Diego. His research inter-ests include robotics, biomedical devices, and engineering education. He teaches introductory design,mechanics, mechatronics, capstone design, medical devices, and product design & entrepreneurship. Hisinterests in design education include increasing student motivation, teamwork, hands-on projects, andintegration of theory into design projects. In 1999 he co-founded Coactive Drive Corporation (currentlyGeneral Vibration), a company that provides force feedback solutions. In 2016 Nate co-founded eGroveEducation, Inc. an educational software company focused on teaching sketching and spatial visualizationskills.

Elizabeth Rose Cowan, eGrove Education

Elizabeth Cowan is a User Experience Researcher and Designer for eGrove Education, Inc., which de-velops Spatial Visualization Training software. Her degree is in Cognitive Science and Human-ComputerInteraction from the University of California San Diego.

c©American Society for Engineering Education, 2021

WIP: Suitability of Spatial Visualization Training for Remote

Learning

This work in progress (WIP) describes the impact that a spatial visualization training app had on

learning outcomes and engagement during the COVID-19 pandemic. When the pandemic hit,

classes with computer aided design (CAD) instruction had two challenges to overcome as people

transitioned to remote or online learning. One challenge was that the remote nature of the

instruction made it difficult for teachers to provide feedback to students as quickly as they did in

a classroom setting thereby maintaining student engagement. The second challenge was the lack

of student access to computers or technology necessary to run the CAD packages. To address

these challenges, a number of middle schools, high schools and post-secondary schools adopted

for the first time the use of Spatial Vis, a freehand sketching and spatial visualization app that ran

on smartphones, tablets, and Chromebooks. Since Spatial Vis automatically grades students’

sketches and provides immediate feedback, it has the potential to increase student engagement

during remote instruction and prepare them for CAD instruction. For many of these newly

adopted courses, as well as for courses that had used the app prior to the pandemic, the use of the

app resulted in an increased amount of spatial visualization instruction to compensate for the

technical constraints and reduced capabilities to teach CAD. Due to the rush to switch to remote

instruction, the courses that adopted the Spatial Vis app were not part of a controlled study.

However, the digital nature of the app allows for the quantification of data such as how many

assignments the students completed in each module and quantification of student persistence in

the app. Furthermore, instructor feedback provides qualitative information regarding learning

outcomes. In this study, student performance was evaluated in two introductory CAD courses at

San Diego State University and University of California, San Diego, both of which used the app

prior to COVID-19 and during the pandemic. In both classes, pre and post testing was done with

a standardized test to measure spatial visualization ability. Comparisons were made of app

performance and test performance before COVID-19 and during COVID-19. Conclusions were

drawn regarding the suitability of the Spatial Vis app for remote learning.

1. Introduction

The COVID-19 pandemic resulted in educational institutions across the world pivoting quickly

to online learning. Initially, teachers had to adapt to emergency remote teaching and were

overwhelmed with the additional work required to develop their remote classes [1]. One survey

conducted to evaluate the successes and challenges teachers faced as they moved to virtual

instruction in spring 2020 showed that although teachers transitioned immediately to virtual

instruction, they were not properly trained or given enough time to effectively redesign their

pedagogy. Teachers were inundated with information and resources to help with virtual

instruction and overwhelmed by how quickly they had to transition [2]. Some courses, especially

in the area of career and technical education (CTE) or other science, technology, engineering,

and mathematics courses such as Computer Aided Design (CAD) that normally have hands-on or

laboratory learning were not simple to convert to a virtual environment [3]. During the initial

move to remote instruction, these courses were challenged with providing access to high-end

equipment or computing resources requiring high-speed internet, and they no longer could

benefit from in-person, close-up demonstrations of proper technique or the opportunity for

teachers to ascertain students’ proficiency of the material in real time [4]. There were concerns

that some students might not be receiving quality instruction in these courses, which would

negatively impact their college and career readiness upon graduation [5] or their fundamental

CAD skills in support of higher education engineering degrees.

To address these challenges, eGrove Education, Inc. [6] offered their Spatial Vis app to teachers

for free when the pandemic hit. The authors of this paper are affiliated with eGrove Education,

Inc. and have developed the Spatial Vis app. Middle Schools, high schools and higher education

institutions jumped at the opportunity to use the Spatial Vis app for the first time. Spatial Vis

teaches freehand sketching on smartphones, tablets, and Chromebooks to improve spatial skills.

Studies have shown that spatial visualization training can increase grades in introductory science,

technology, engineering, and math (STEM) courses [7], and can improve computer aided design

(CAD) and design skills. The app had the potential to increase student engagement during

remote instruction since it automatically grades students’ sketches and provides immediate

feedback. Many classes had to compensate for the technical constraints and reduced capabilities

to teach their traditional hands-on curriculum. For example, some teachers that normally teach

hand sketching had to require students to scan or take photographs of their sketches and submit

them to the teacher for grading, which would take several days to provide feedback to the

students. Therefore, the use of the Spatial Vis app in many of these newly adopted courses in

spring and fall 2020 resulted in an increased amount of spatial visualization instruction for the

students.

Because the move to remote instruction occurred quickly, the courses that adopted the Spatial

Vis app during the pandemic were not part of a controlled study. However, through instructor

and student feedback, qualitative information regarding learning outcomes was established and

the impact that the Spatial Vis app had on learning outcomes and engagement during the

COVID-19 pandemic was ascertained. These results are described in [8] which concluded that

despite rapid adoption in these middle school, high school and higher education institutions, the

Spatial Vis app provided a resource that was not only effective in the classroom but worked

seamlessly outside of the classroom and provided flexibility to teachers.

In order to provide quantitative measurement of the impact that the pandemic had on spatial

visualization training, two higher education courses that had used the Spatial Vis app prior to the

pandemic and also during the pandemic were analyzed. This paper particularly focuses on two

freshman engineering CAD courses taught at two different institutions, San Diego State

University and the University of California, San Diego. A primary objective for both of these

courses is to teach students spatial visualization and CAD skills. One major difference between

these two courses is that when instruction was in-person, the UC San Diego course taught CAD

through physical design-build projects, while the other focused on purely computer-based design

assignments. In both classes the Spatial Vis app was used as supplemental material to other

course content.

This paper describes the quantitative analysis conducted on these two CAD courses for the year

prior to the pandemic and during the pandemic to evaluate the Spatial Vis app’s suitability not

only for in the classroom but also for remote instruction. Additionally, feedback from teacher

interviews is included to provide instructor perspectives.

2. Spatial Vis Sketching App

Spatial visualization has been shown to be a fundamental skill for student success. A spatial

visualization app was developed by eGrove Education, Inc. [9] to make use of the prevalence of

touchscreen devices in education and to make spatial visualization training more engaging and

easier to teach. The app teaches freehand sketching to improve spatial skills for students in pre-

engineering, manufacturing, and CAD courses. The app was motivated by Sorby’s finding that

the “importance of sketching in developing 3-D spatial skills cannot be understated” [7].

Sketching provides an added benefit beyond spatial visualization, and has been correlated to

communication, teamwork, and creativity [10]. During the study, the app ran on iOS and

Android phones and tablets, and Chromebooks (as of this publication the app can also run from

any web connected computer). Students sketch isometric and orthographic assignments, and the

app provides automatic grading and personalized feedback on student work making it especially

well-suited for remote learning as well as in-class instruction. Studies with the Spatial Vis app

have demonstrated its efficacy when used in class or as homework ([11] through [16]).

Figure 1. Spatial Vis™ App

2.1. Features Suitable for Remote Instruction

2.1.1. Automatic Grading Algorithm with Hint and Peek Capabilities

The Spatial Vis app provides a sketching window and an assignment window, each with a

reference dot so students know the position in the sketching window they need to draw their

solution for the grading algorithm to correctly score their submission (Fig. 2a). The grading

algorithm is initiated when the user selects the submit button at the bottom of the screen. If the

solution is correct, the student can move on to the next assignment. If incorrect, a pop-up

window is produced by the grading algorithm that gives the student immediate feedback by

providing them with a circular progress indicator showing them how much of their sketch is

correct and how much is incorrect (Fig. 2b). The student has the option to retry the assignment as

many times as they would like without losing any credit. If the student would like more help,

they can select the Hint button or take a Peek at the solution [8].

The Hint tells the user which parts of their drawing are correct by highlighting them in green

(Fig. 2c). If the Hint shows that most of their submission is highlighted in green, then the student

is close to the solution. However, if the Hint shows that most of their submission is not green,

then they know they need to rethink the problem, ask for help, or possibly use a Peek.

The Peek highlights the correct parts of the drawing in green, incorrect parts in red, and missing

lines in blue (Fig. 2d). Additionally, gamification was added to the app to encourage persistence.

Students receive three stars for each assignment they solve correctly without any assistance

regardless of how many times they attempt the sketch. If a Hint is used, then they are awarded

two stars for the problem and if a Peek is used then students earn only one star when they

successfully complete the sketch (Fig. 3) [8]. The motivation for gamification is based on a study

that showed that students who tried to solve a problem on their own before asking for a Peek at

the solution had increased gains in spatial visualization ability. The average number of stars per

assignment that a student receives is one indication of their persistence in working to solve a

problem on their own.

Figure 2. Orthographic Sketch Assignment with Different Feedback Options

Figure 3. Star Reward System

2.1.2. Personalized Feedback (Mini Hints)

During in-person instruction, a student struggling on an assignment will often receive a small

hint from an expert teacher looking over their shoulder. There is value for creating hints that

provide only a small amount of help. This small hint will keep the student productively working

on the assignment while reducing frustration, but still elicits effective struggle to solve a difficult

assignment. To try to replicate a teacher looking over the students’ shoulder, the Spatial Vis app

provides Mini Hints to students as “free hints” without the loss of stars. The grading algorithm

identifies the appropriate hint based on specific known errors of incorrect sketches. If these free

mini hints are not sufficient, a student will still have the option of looking at the original Hint

and Peek options that were shown in Fig. 2 but will lose stars. An ideal mini hint will move a

student forward in the solution, but still require them to actively engage and struggle to solve an

assignment [17]. For remote instruction, mini hints could potentially take upon increased

pedagogical importance.

The most common mini hint is a progress indicator indicating the percentage of the lines in the

student’s sketch that are correct, incorrect, and/or missing. This was previously shown in Fig. 2b.

The progress indicator helps quantify to the student how much of the various categories of lines

are present giving the student a sense of how close they are to finishing the sketch. However, it

does not provide specific graphical information as to what part of the sketch is correct or what

the actual solution is. If a student sees that their initial lines are correct, then they will know that

they are on the right track and feel encouraged to continue. Alternatively, a student may see early

on that much of their approach is incorrect which may encourage them to start over.

Another type of mini-hint provided by the grading algorithm is specific textual messages such as

“Hidden Line Incorrect”, “You may be missing a line”, “You may have an extra line”, or “Close!

Draw more carefully” (Fig. 4). Other hints that are anticipated in the future include “Shape is

drawn correctly but in the wrong location”, “The object is rotated incorrectly”, “Error in right

side (or top, front, etc.) view”, etc. The purpose of a mini hint is to provide some guidance to the

student without showing them the solution.

Figure 4. Textual Mini Hints Provided to Students Based on Errors in Submitted Sketch

2.1.3. 2.1.3 Other Features Within the App

To facilitate instruction, eGrove Education also has created Lecture Slides and Videos that

introduce the basic concepts necessary to solve the assignments in each lesson. These slides and

videos are provided on the eGrove Education website [9] or can be directly accessed by the

students within the app on the first page of each lesson.

Additionally, test questions are provided as the last three questions of each lesson. These

assessments were chosen to be of moderate difficulty, and the Hint and Peek features are

disabled. These test questions try to incentivize students to learn the material in each lesson

rather than relying on the Hint and Peek features because students are aware that they will have

to eventually complete three assignments without help features before they can move on to the

next chapter.

Furthermore, a challenge with remote learning is ensuring all students have equitable access to

the Internet. The Spatial Vis app can work in offline mode if access to the Internet is a problem.

Essentially, grading of assignments can occur locally and progress in the app is saved until the

app can connect with the internet and sync student progress with eGrove Education’s servers.

3. Evaluation Methods

This evaluation takes place in two college engineering CAD courses in San Diego, CA. The first

course denoted as Class A (CIVE 121 Computer Graphics for the Built Environment) was taught

at San Diego State University. It was on a semester system and was approximately 5-6 weeks

into the semester when the pandemic hit. The other course denoted as Class B (SE 3 Graphical

Communication for Engineering Design) was taught at University of California, San Diego. It

started the quarter remotely. Students were given a standardized spatial skills test, the Purdue

Spatial Visualization Test: Rotations (PSVT: R) [18], before doing lessons in the app and

participants were assigned all nine lessons as part of their course. The lessons, total number of

assignments, and number of required assignments in each lesson, as well as the aggregate

average time for required assignments are summarized in Table 1. The assignments in each

lesson increased in difficulty. Both courses required the same Spatial Vis assignments: assigning

all of the odd-numbered problems in the app and the three test questions in which the assignment

reverts back to medium level difficulty and the hint and peek features are disabled. However, to

encourage persistence, students were tasked to complete these required problems without losing

any stars. If they needed to take a hint or peek and subsequently lose stars, they were able to

make up the lost stars by doing extra even-numbered problems. After completion, a post-test,

PSVT:R, was administered. Class A assigned one app module per week while Class B assigned

three modules per week and completed the app in the first three weeks of the quarter.

Table 1. Spatial Vis Lessons and Assignments

Lesson Total Number of

Assignments in

Lesson

Number of

Required

Assignments in

Lesson

Average Time for

Required

Assignments

Lesson 0: Introduction 6 6 3m 48s

Lesson 1: 2D Rotations 30 16 13m 48s

Lesson 2: Iso Cubes 35 19 57m 12s

Lesson 3: Ortho Cubes 33 18 53m 24s

Lesson 4: 2D to 3D 29 16 53m 42s

Lesson 5: Slopes and Curves 31 17 47m 48s

Lesson 6: Flat Patterns 31 17 38m 0s

Lesson 7: Rotations about 1 Axis 30 16 37m 30s

Lesson 8: Rotations about 2 Axes 30 16 44m 12s

Lesson 9: Assembly 20 11 21m 48s

Because the move to online was immediate, a controlled study regarding the impact that Spatial

Vis had on these institutions and students was not possible. However, qualitative information

regarding learning outcomes was established through instructor feedback. Both courses had

similar learning outcomes such as understanding engineering design methodology and

demonstrating the role of graphics in the engineering design process, organizing and delivering

effective verbal, written and graphical communication, and applying relevant sketching, 2D and

3D techniques and skills using modern engineering tools. Developing spatial visualization and

reasoning skills, related directly to the Spatial Vis app, was also one of the primary learning

outcomes for both courses.

Additionally, the digital nature of the app allows for the quantification of data such as how many

assignments the students completed in each module as well their performance on all questions.

These data points were used to establish the impact that the Spatial Vis app had on learning

outcomes and engagement during the COVID-19 pandemic to demonstrate its suitability not only

in the classroom but also for remote instruction.

3.1. Participants

San Diego State University is a large public institution with more than 30,000 undergraduate

students that has a very high racial diversity. 66 percent of students are minorities or people of

color (BIPOC). While diversity information for Class A was not readily available it was assumed

consistent with the Civil Engineering graduating class of 2018-2019, which consisted of 28.5%

White, 26.3% Hispanic or Latino, 22.6% Non-resident, 13.1% Asian, 8% Other races, and 1.5%

Black or African American [19]. During the Spring 2019 semester, there were 255 enrolled in

Class A of which 208 were participants in this analysis because they completed the pre and post

PSVT:R test. In Spring 2020 when COVID-19 hit, 195 of the 247 enrolled students were

included as part of the analysis. There were no hands-on building projects in this course.

Class B is a Structural Engineering Graphics course at the University of California, San Diego,

which is also a large public institution with more than 30,000 undergraduate students that has

very good racial diversity. 59% of students are minorities or people of color. Again, the diversity

statistics specifically for Class B were unavailable but was fairly consistent with the university

breakdown consisting of 33.8% Asian, 20.1% Hispanic or Latino, 19.4% White, 18.4% Non-

resident, 1.6% Black or African American, 0.4% Other and 6.2% ethnicity unknown [20].

During the Spring 2019 quarter, 129 students were enrolled in the course with 89 being included

in this analysis because they completed both the pre- and post- PSVT:R test, while in Spring

2020, 92 out of 105 enrolled students were included in the analysis. While the learning

objectives for Class B were similar, in the 2019 offering, it also included a team-based hands-on

project where students designed in CAD and did rapid prototyping using 3D printers and 2D

laser cutters to build a Rube Goldberg machine. In 2020, because the course was remote, the

team project was software based rather than the prior hands-on rapid prototyping project.

3.2. Instruments

Students’ performance was assessed using the Purdue Spatial Visualization Test: Rotations

(PSVT:R) [18] to measure student proficiency on spatial visualization skills. The PSVT:R is a

timed 30 multiple choice question assessment where students identify the correct object

orientation based on a particular rotation. Participants had 20 minutes to complete the PSVT:R

using an online learning management system (Canvas for both courses). The same online test

was used both pre-COVID-19 and during the pandemic. In both cases Spatial Vis lessons were

then assigned as part of the students’ regular schoolwork. After completion of the Spatial Vis

app, the PSVT:R was administered again as a post-test. For Class A the pre- and post-tests were

administered during Week 2 and Week 15 of the semester while Class B administered the pre-

and post-tests during Week 1 and Week 10 of the quarter. Both instructors were interviewed to

provide their personal opinion on the development of their student’s spatial visualization skills

and the impact that the app had during the pandemic.

3.3. Analysis

The average scores on the pre- and post- PSVT:R test were analyzed for Class A and Class B.

Students who scored low on their pretest were of specific interest. In this analysis the students

whose pre-test was < 70% are categorized as "At-Risk" for low graduation rates in future STEM

undergraduate programs due to low Spatial Visualization skills [7].

The average gains in PSVT:R scores do not tell the complete story, especially within the “At-

Risk” group. It has been shown in previous studies that learning gains between the pre- and post-

tests are correlated to students’ persistence [12], which is measured by how often a student

completes an assignment without using a hint or peek. Therefore, to assess persistence, the

average stars that students earned on assignments within the app were also evaluated. Persistent

students who try an assignment as many times as needed to get to the solution without taking

hints or peeks will have average stars close to 3.

Another metric to assess the effectiveness of the spatial visualization training is the percentage of

students who scored less than 70% on the PSVT:R but then raised their PSVT:R to above this

threshold. This metric captures gains in spatial visualization skills among students that need it

the most, since the 70% threshold corresponds to the skill level that is correlated to higher

graduation rates in STEM. This metric can be seen as an indicator of the percentage of students

who moved out of the "At-Risk" category for low graduation rates due to low spatial

visualization skills, and these large gains in PSVT:R scores can be used to characterize

effectiveness of spatial visualization training [12].

For both classes, the analysis separated the “At-Risk” students and evaluated pre- and post-

PSVT:R average scores, percent gains on the PSVT:R, average number of assignments

completed in the App, average number of test assignments (test assignments have Hint and Peek

disabled) completed in the app, average stars earned on assignments, and percent of students who

rose out of the “At-Risk” category. These results are summarized in the next section.

3.4. Results

The results of the analyses are provided in Tables 2 through 5. Table 2 shows the performance of

all of the students in Class A, while Table 3 summarizes the performance of the “At-Risk”

students in Class A. Similarly, Table 4 shows the performance of all of the students in Class B

while Table 5 shows the results for only the “At-Risk” students. For Class A, in 2019 there were

123 out of 208 students in the “At-Risk” category representing 59% of the students having low

spatial visualization skills, and 112 out of 195 students in 2020 representing 57%. For Class B,

37 out of 89 students in 2019 (42%) and 39 out of 92 students in 2020 (42%) were categorized in

the low spatial skills category.

Table 2. Overall Performance Results for All Students in Class A

Class A: Complete Class Spring 2019 (Before

COVID, n=208)

Spring 2020 (During

COVID, n=195)

Average Pre-PSVT:R 62.2% 62.2%

Average Post-PSVT:R 74.3% 68.6%

Average PSVT:R Gain 12.1% 6.4%

Average No. of Assignments Completed* 203.5 232.6

Average No. of Test Assignments Completed* 18.5 23.7

Average Stars 2.80 2.77

Correlation Between No. of Assignments Completed and PSVT:R gain r=-0.055, p=0.434 r=-0.129, p=0.072

Correlation Between Test Assignments Completed and PSVT:R gain r=-0.079, p=0.259 r=-0.037, p=0.604

Correlation Between Persistence Indicator (Ave. Stars) and PSVT:R gain r=-0.074, p=0.291 r=-0.053, p=0.466

Table 3. Overall Performance Results for “At-Risk” Students in Class A

Class A: At-Risk Students (Pre-PSVT:R < 70%) Spring 2019 (Before

COVID, n=123)

Spring 2020 (During

COVID, n=112)







Percent of Student Who Rose to Be Out of Risk 48.8% 35.7%

Correlation Between No. of Assignments Completed and PSVT:R gain r=0.129, p=0.157 r=0.200, p=0.034*

Correlation Between Test Assignments Completed and PSVT:R gain r=0.192, p=0.033* r=0.117, p=0.221

Correlation Between Persistence Indicator (Ave. Stars) and PSVT:R gain r=0.165, p=0.069 r=0.137, p=0.149

* Statistically Significant when p < 0.05

Table 4. Overall Performance Results for All Students in Class B

Class B: Complete Class Spring 2019 (Before

COVID, n=89)

Spring 2020 (During

COVID, n=92)







Correlation Between No. of Assignments Completed and PSVT:R gain r=-0.047, p=0.665 r=0.152, p=0.148

Correlation Between Test Assignments Completed and PSVT:R gain r=-0.062, p=0.566 r=0.128, p=0.224

Correlation Between Persistence Indicator (Ave. Stars) and PSVT:R gain r=-0.227, p=0.032* r=0.162,p= 0.122


Table 5: Overall Performance Results for “At-Risk” Students in Class B

Class A: At-Risk Students (Pre-PSVT:R < 70%) Spring 2019 (Before

COVID, n=37)

Spring 2020 (During

COVID, n=39)







Percent of Student Who Rose to Be Out of Risk 78.4% 69.2%

Correlation Between No. of Assignments Completed and PSVT:R gain r=-0.205, p=0.224 r=0.385, p=0.015*

Correlation Between Test Assignments Completed and PSVT:R gain r=0.114, p=0.502 r=0.133, p=0.418

Correlation Between Persistence Indicator (Ave. Stars) and PSVT:R gain r=-0.084, p=0.621 r=0.504, p=0.001*


A key observation is that both before COVID-19 and during COVID-19 students increased their

spatial visualization ability after using the app. However, under the remote learning environment

there were some reductions in gains. In Class A, across all students who participated in the study,

the PSVT:R gains dropped from 12.1% in 2019 to 6.4% in 2020. For the At-Risk students the

gains dropped from 18.9% in 2019 to 11.3% in 2020. While these drops are not desirable, it was

still a positive indication that under the COVID-19 environments students still increased their

spatial visualization scores. In the At-Risk group the gains were still at 2/3 of the pre COVID-19

class.

For Class B, COVID-19 had only a very slight negative effect. Over the whole class, the

PSVT:R score gains dropped from 13.4% to 12.9%. For the At-Risk students the gains dropped

from 22.3% to 22.2%. The reason for why Class B performed higher than Class A is unknown,

but it is interesting to note that the persistence indicator of average stars was higher in Class B

than Class A. For example, the average stars for the At-Risk students during COVID-19 for

Class B was 2.91 while for Class A it was 2.71. This result should not detract that in both

classes, students had significant gains in spatial visualization ability.

Correlations were performed between gains in the PSVT:R scores and the average stars, number

of assignments completed, and number of test assignments completed. Those which were

statistically significant p<0.05 are indicated with an asterix in Tables 2 through 5. Plots in

Figures 5 and 6 show the correlations.

a) Class A b) Class B

Figure 5. Change in PSVT:R Score vs. Persistence Indicator for “At-Risk” Students

a) Class A b) Class B

Figure 6. Change in PSVT:R Score vs. Number of Assignments “At-Risk” Students

3.5. Instructor Feedback

The instructors (Class B instructor is an author on this paper) had already used the Spatial Vis

App prior to the pandemic and already knew that the app provided the ability for students to

draw by hand without being in the classroom, but nevertheless had very positive feedback about

the app’s usefulness for remote learning. Instructor for Class A said, “When we went online,

students couldn’t access the engineering graphics software that we used for the course without

access to our computer labs. But Spatial Vis worked as well online as in person. I didn’t have to

change anything and was able to seamlessly transition with Spatial Vis.”

The two instructors indicated that their implementation was not that different when their course

transitioned to online. Instructor for Class A said, “It was nice to have something that students

already had access to remotely. The only thing I did was make Spatial Vis a larger amount of

their grade since I was unsure about how to do online testing of CAD software”.

4. Conclusions

Spatial visualization training and specifically freehand sketching isometric and orthographic

shapes has been shown to increase graduation rates in engineering for students with low spatial

visualization skills. However, the ability to teach hand sketching during the pandemic and

remote instruction can be difficult. Accordingly, the potential for spatial visualization training in

which students can work independently is attractive. The Spatial Vis app was designed to meet

this need as it allows for freehand sketches to be graded automatically and for hints to be

provided when needed. The Spatial Vis app provides a resource that is not only effective in the

classroom but works seamlessly outside of the classroom providing flexibility to teachers.

The digital nature of the app allows for the quantification of data such as how many assignments

the students completed in each module as well their performance on test questions. This

quantifiable data was used to establish the impact that the Spatial Vis app had on learning

outcomes and engagement during the COVID-19 pandemic.

It is understood that the act of learning and practicing 3D CAD skills alone can improve spatial

visualization skills even without specific sketching training using the Spatial Vis app. However,

a Fall 2017 trial was conducted at the University of San Diego in which the Spatial Vis app was

assigned as homework for an experimental section (n=23), while a second section (n=22) was

used as a control. Among the low-performing students, those who used the app showed

remarkable progress. 8 of 13 (62%) raised their test scores above 70% compared to just 2 of 14

(14%) in the control group. While this study needs to be duplicated with a larger number of

students, this represents a 341% increase in the number of students who moved out of an “At-

Risk” category due to low spatial visualization skills [11] after using the Spatial Vis app and

demonstrates the efficacy of the app in increasing PSVT:R scores.

Also, it is not entirely clear why the COVID-19 classes (Spring 2020) resulted in slightly lower

scores than those of the pre-COVID-19 classes (Spring 2019). It could be attributed to low

engagement and challenges with struggling students when courses went to emergency remote

instruction. But the results pre-COVID-19 and during COVID-19 which essentially represent the

difference between face-to-face and remote learning were not significantly different.

Since the pandemic hit, eGrove Education has made significant improvements to the Spatial Vis

app and their teacher resources to support remote and in-person instruction. The teacher interface

was improved and now allows instructors to track student persistence (whether they are hinting

and peeking too often) and intervene as necessary.

Additionally, the experience of moving to remote instruction during the pandemic resulted in an

increased interest by middle schools and a much wider range of classes that use the Spatial Vis

app. Rather than developing a separate curriculum for the younger ages or for different types of

courses, teachers are now provided with the capability to customize the assignments their

students do within the app and take advantage of the scaffolding built into the system to assign

problems of relevant difficulty and applicability to their courses. Students can identify within the

app which assignments are required by their teachers and which are optional, and grade reports

for the teachers correlate directly to the required assignments.

5. Acknowledgements & Disclosure

Nathan Delson and Lelli Van Den Einde have equity interest in eGrove Education, Inc., a

company that may potentially benefit from the research results. The terms of this arrangement

have been reviewed and approved by the University of California, San Diego in accordance with

its conflict of interest policies. In addition, a Small Business Innovation Research (SBIR) grant

was awarded to eGrove Education, Inc., by the NSF (Award # 1648534), that also supported the

research effort of this publication.

References

[1] E. Nordmann, C. Horlin, J. Hutchison, J.A. Murray, L. Robson, M.K. Seery, J. MacKay,

"Ten simple rules for supporting a temporary online pivot in higher education”, PLOS

Computational Biology, October 1, 2020, Retrieved from

https://doi.org/10.1371/journal.pcbi.1008242.

[2] D. Schaffhauser, “Educators Feeling Stressed, Anxious, Overwhelmed and Capable”, The

Journal: Transforming Education, June 6, 2020, Retrieved from

https://thejournal.com/articles/2020/06/02/survey-teachers-feeling-stressed-anxious-

overwhelmed-and-capable.aspx.

[3] C. Cahill, S. Jackson, N. Summerall, K. Harruna, “Helping Career and Technical Education

Programs Meet this Moment”, JFF, September 1, 2020, Retrieved from:

https://www.jff.org/what-we-do/impact-stories/pathways-to-prosperity-network/helping-

cte-programs-meet-moment/

[4] E. Richmond, “Shop Class, Over Zoom: How one career and technical high school is going

remote”, The Atlantic, Retrieved from:

https://www.theatlantic.com/education/archive/2020/05/remote-career-technical-school-

COVID-19-pandemic/611047/

[5] S. Blackburn, “How schools can improve CTE during COVID-19 Online learning could

have increased the equity gap for students enrolled in career technical education courses”,

District Administration, Retrieved from: https://districtadministration.com/how-schools-

navigate-cte-online-cte-access-during-COVID-19/

[6] Spatial Vis App, www.egrove.education

[7] S. Sorby, “Educational research in developing 3‐D spatial skills for engineering students”.

International Journal of Science Education. Vol 31(3), pp 459-80, Feb. 2009.

[8] L. Van Den Einde, N. Delson, L. Cowan, “Sketching App to Teach Spatial Visualization

Skills Suitable for Remote and In-Person Instruction”, Proceedings of INTED 2021, virtual

conference, March 8-9, 2021.

[9] N. Delson, L. Van Den Einde, E. Cowan, J. Tara “eGrove Education.” [Online] Available

www.egrove.education.

[10] J. Power, J. Buckley, and N. Seery. “Visualizing Success: Investigating the Relationship

between Ability and Self-Efficacy in the Domain of Visual Processing”. 70th ASEE

Engineering Design Graphics Division Midyear Conference, Embry-Riddle Aeronautical

University, FL, January, 2016.

https://districtadministration.com/how-schools-navigate-cte-online-cte-access-during-covid-19/




http://www.egrove.education/

[11] G. Hoople, E. Cowan, L. Van Den Einde, J. Tara, and N. Delson. “Teaching Spatial

Visualization: A Controlled Trial of a Touchscreen App Implemented as Homework”,

Conference proceedings for Frontiers in Education: Fostering Education through Diversity,

October 3-6. San Jose, CA. 2018.

[12] N. Delson and L. Van Den Einde, “Tracking student engagement with a touchscreen app

for Spatial Visualization Training and freehand sketching,” 122nd ASEE Annual

Conference & Exposition proceedings, Seattle, WA, 2015.

[13] E. Cowan, N. Delson, R. Mihelich, and L. Van Den Einde, “Improvement in Freehand

Sketching Application for Spatial Visualization Training,” in Conference on Pen and

Touch Technology in Education, Illinois, 2017.

[14] L. Van Den Einde, N. Delson, E. Cowan, and D. Yang, “Increasing Student Persistence In

A Sketching App For Spatial Visualization Training,” ICERI2017 Proceedings, pp. 5373–

5381, October 2017.

[15] N. Delson, and L. Van Den Einde, “Sketching, Assessment, and Persistence in Spatial

Visualization Training on a Touchscreen”, in 124th ASEE Annual Conference &

Exposition proceedings, Salt Lake City, UT, 2018. 10.18260/1-2--30968.

[16] L. Van Den Einde, N. Delson, E. Cowan, “Freehand Sketching on Smartphones for

Teaching Spatial Visualization”, in 126th ASEE Annual Conference & Exposition

proceedings, Tampa, FL, 2019. 10.18260/1-2--32859

[17] N. Delson, L. Van Den Einde, E. Cowan, B. Mihelich, “Mini-Hints for Improved Spatial

Visualization Training”, in 126th ASEE Annual Conference & Exposition proceedings,

Tampa, FL, 2019, 10.18260/1-2--33112.

[18] R. Guay, (1977). “Purdue Spatial Visualization Test - Visualization of Rotations”. W.

Lafayette, IN. Purdue Research Foundation.

[19] Civil Engineering at the San Diego State University, College Factual, Accessed April 29,

2021, https://www.collegefactual.com/colleges/san-diego-state-university/academic-

life/academic-majors/engineering/civil-engineering/index.html

[20] University of California, San Diego Undergraduate Ethnic Diversity Breakdown, College

Factual, Accessed April 29, 2021, https://www.collegefactual.com/colleges/university-of-

california-san-diego/student-life/diversity/chart-ethnic-diversity.html

[21] J. Ottway, “Integrating CAD and CAM for Design-build Projects”, in 124th ASEE Annual

Conference & Exposition proceedings, Salt Lake City, UT, 2018. Retrieved from:

https://www.asee.org/public/conferences/106/papers/22132/view

https://www.collegefactual.com/colleges/san-diego-state-university/academic-life/academic-majors/engineering/civil-engineering/index.html

https://www.collegefactual.com/colleges/san-diego-state-university/academic-life/academic-majors/engineering/civil-engineering/index.html

https://www.collegefactual.com/colleges/university-of-california-san-diego/student-life/diversity/chart-ethnic-diversity.html

https://www.collegefactual.com/colleges/university-of-california-san-diego/student-life/diversity/chart-ethnic-diversity.html

https://www.asee.org/public/conferences/106/papers/22132/view

Documents

Work in Progress: Suitability of Spatial Visualization