Course Evaluation Project

7/31/2019 Course Evaluation Project

1/20

Evaluation of St. Francis Xavier Junior High Laptop Program within the Language ArtsCurriculum

By: Lauren PintoEDTECH 505-4172

Submitted to Dr. Thompson on July 25, 2012

EVALUATION OF SFX LAPTOP PROGRAM

1


2/20

Table of Contents

Executive Summary 3

Purposes of Evaluation 3Background Information 4

Evaluation Method 6

Results 7

Discussion 9

Project Cost 10

Appendices 11


2


3/20

Executive Summary

! This is an evaluation of the St. Francis Xavier School junior high school laptop program.The program was instituted during the 2011-2012 school year with the hopes of aiding thestudents while marketing the school as a technology school. The laptops were placed in Ms.

Lauren Pintos classroom, and were used for a variety of tasks, activities, projects, andassessments.

Prior to the laptops being instituted, a formal evaluation plan was not created. As a result,objectives were determined by the technology planning committee in hindsight, and thoseobjectives dealt with: depth of student work, proficiency with GoogleDocs and GoogleDrawing,laptop usage for classroom purposes, easement of day-to-day tasks, reduction of paper use, andalleviating the student attrition rate.

The evaluation was conducted between June 4, 2012 and July 25, 2012. During that time,the evaluator developed a student survey which she posted online for one week and a half. All

junior high students were invited to respond, but only eighteen completed the form. Ms. Pinto

also reflected on using the laptops, and her responses were compared to the students tounderstand how the laptops worked within her classroom.

The data revealed that the laptop program was mostly effective; several of the goals weremet in full while others give St. Francis Xavier School room to grow. The goals which werewholly achieved are: usage of laptops, paper reduction, and easement of day-to-day tasks. Thosegoals which offer room for improvements are: depth of student work, proficiency withGoogleDocs and GoogleDrawing, and alleviating the attrition rate. As a result, the evaluator hasrecommended several strategies to the St. Francis Xavier stakeholders to improve their laptop

program.

Purposes of Evaluation

General PurposeWhen the St. Francis Xavier junior high school laptop program was first instituted, a

formal plan for implementation and evaluation was not created. As a result, it is necessary thatthe school uses this evaluation process to understand its investment so as to see how the devicesimpacted student work. Also, as the school moves forward and attempts to brand itself as atechnology school, it is imperative to determine if this program is helping to meet thisobjective.

Central Questions to be AnsweredDoes student work show definitions, examples, reflection and/or application?Do students report that their work shows definitions, examples, reflection and/or application?Can students work collaboratively and proficiently with GoogleDocs and Google Drawing?Do students report working collaboratively and proficiently with GoogleDocs andGoogleDrawing?Are the laptops used for note-taking, revision, creation of assignments, and assessment?


3


4/20

What do the students report using the laptops for?Are day-to-day tasks eased (e.g., note-taking, revision)?Is paper use reduced and/or saved?Has the laptop program recruited new students and kept old students?Do the students report the laptops being a big factor in staying at the school?

Impact of EvaluationOf the stakeholders, the one who stands to benefit most from this evaluation is the

classroom teacher. This is the case because she is the one who works most closely with thedevices and the students. Also, the devices are meant to support the junior highs constructivistapproach to the curriculum, so this evaluation will let her know whether or not this is happening

by determining how the students are using the laptops.

Background Information

Program OriginSince St. Francis Xavier Schools founding in 1956, the school has built a reputation

within the Burbank, California community for providing an outstanding Catholic education. For many years, the school turned away students on the basis that their classes were full and couldnot accommodate the extra interest. Since the 2007 recession, however, the school has founditself in the position of losing students to the public sector or to schools which offer differentresources or extra-curricular activities. As a result, St. Francis Xavier purchased andimplemented educational technology. Starting in the 2008-2009 school year, the school

purchased SmartBoards for each classroom, and since then, has purchased additional hardware towork in conjunction with the SmartBoards. By doing so, St. Francis Xavier attempted to brand

themselves as the technology school since other schools in the area (both public and parochial)lacked this type of hardware.However, the purchase of these devices did not seem to make a difference with

enrollment, so in an effort to market themselves further, the school began to look into laptops for 1:1 student use. The school hoped that capitalizing on the ubiquity of technology would draw innew students while simultaneously helping the school keep their reputation for providing aquality education. So, funds were allocated for the purchase of a classroom set of laptops. St.Francis Xavier has one class per grade level (K-8), with the schools junior high program (grades6-8) rotating the three grades among three teachers. Because this rotation provided access to themost amount of students, the technology planning committee chose the junior high program andMs. Lauren Pinto, the junior high language arts teacher, to use the computers. Thirty-eightlaptops along with accompanying software and hardware were purchased in July 2011 for approximately $25,000, and were implemented for the 2011-2012 school year.

Program GoalsIn an effort to determine and understand the programs goals, the evaluator spoke with St.

Francis Xaviers technology planning committee, which is made up of the schools accounts


4


5/20

manager, vice principal/fifth grade teacher, and Ms. Pinto. Together, the technology planningcommittee identified the following six objectives:

1. Student work shows definitions, examples, reflection and/or application2. Students work collaboratively and proficiently with GoogleDocs and Google Drawing3. Laptops are used for note-taking, revision, creation of assignments, and assessment

4. Day-to-day tasks are eased (e.g., note-taking, revision)5. Paper use is reduced and/or saved6. Laptop program recruits new students and keeps old students

Previous ProgramsWhile St. Francis Xavier has purchased educational technology in the past, the school had

no prior experience in purchasing and/or implementing a 1:1 laptop program. Yet, in purchasingand using other educational technology, the school learned that student devices need to drawupon prior experience (whether a device is used at home or at school), small enough to behandled on student desks, easy to use, able to be kept in a centralized place, and interactive.

People Involved in ProgramThe following is a list and description of all those involved in the junior high laptop

program at St. Francis Xavier School. Mrs. Rita Recker- St. Francis Xavier Accounts Manager. Rita Recker has beenworking at St. Francis Xavier for fifteen years. She is in charge of allocating funds for the schooland the parish; this includes but is not limited to: parish activities, textbook budgeting, classroom

budgeting, building repairs, office supplies, and office machinery repairs. For the schools laptop program, Mrs. Recker was in charge of budgeting and purchasing thirty-eight computer units, acharging cart, and a single license of Smart ClassroomSuite software.

Mrs. Teresa Smith- Fifth Grade Teacher and Vice Principal. Teresa Smith has beenworking as the St. Francis Xavier fifth-grade teacher for thirteen years. Ten years into her time atthe school, she had the privilege of being appointed Vice Principal. As part of the planningcommittee, Mrs. Smith provided both an administrative and teacher perspective.

Ms. Lauren Pinto- Junior High Language Arts Teacher. Ms. Pinto has been workingat St. Francis Xavier for five years. She is comfortable working with technology, and because sheis pursuing a graduate degree in educational technology, she was the perfect teacher to use thelaptops with the junior high students. However, Ms. Pinto is not an expert with computers, andwhen issues arose, it was her responsibility to seek help and repair any issues. Junior High Students. Ninety individuals comprise the junior high student population.The students come from various socio-economic statuses (ranging from blue collar to affluent)and have various technical abilities (from competent to advanced).

Characteristics of the Program Devices. Thirty-eight student laptops were purchased, with three students across the threegrades sharing one device. This resulted in thirty-four laptops which were used by students andfour extra devices for incidentals. All laptops are clamshell-tablets with a ruggedized shell; theycould be used as a traditional laptop or rotate to become a touch-screen tablet. Each unit has a


5


6/20

webcam, enabled wifi, two USB ports, and came equipped with: Windows 7, Microsoft Word(Starter Edition), Microsoft Excel (Starter Edition), and ArtRage 2 (drawing software). With the

purchase of Smarts ClassroomSuite, all laptops are installed with a host of Smart productswhich include: Notebook (interactive lesson software), Sync (remote desktop classroommanagement software for teacher use), and Response (assessment software). The battery life for

each unit is approximately six hours, resulting in the need to charge the computers mid-day. Inorder to do this, a charging cart was purchased. This cart simultaneously charges thirty-sixlaptops, and at the close of each school day, the laptops are securely stored within it. Internet Infrastructure. St. Francis Xavier has a wireless Internet connection whichallow the laptops to be online. The wireless access point is housed approximately thirty feetaway from Ms. Pintos classroom in a room separated by cement walls. Unfortunately, theInternet is not always reliable, resulting in a sometimes poor Internet connection which went onto cause problems for students working online. SmartSync offered the ability to filter certain webcontent, but Ms. Pinto reported not using it often. Thus, issues surrounding websiteappropriateness arose.

Accounts. Each student was given their own user account on their assigned device under the main administrative account. Students were given permission to customize their desktop

pictures, mouse pointer, and screen savers to reflect their personality. No student was given permission to enter anothers account. Google. Each student was given a free Google for Education account. This granted thestudents access to GMail and GoogleDocs while giving Ms. Pinto administrative control over theaccounts. Ms. Pinto had her students use these Google accounts to create presentations, andmany of them chose to use GoogleDocs as their word processing program. Edmodo. Ms. Pinto employed the use of Edmodo.com, a private microblogging site builtfor educational purposes. Each student created an Edmodo account and joined their classmates in

an online community for their grade level. Ms. Pinto used this site to distribute handouts andassessments as well as collect assignments. Students were given a non-curricular group which letthem communicate with each other about non-school topics.

Evaluation Method

ParticipantsThe participants in this evaluation came from junior high population of ninety students.

Because this evaluation occurred over summer vacation between the 2011-2012 and 2012-2013school years, a judgement sample was taken. The survey was distributed via Edmodo.com, andthose students who visited the site between June 22, 2012 and July 2, 2012 were encouraged totake the survey. A total of eighteen students completed the survey, six of whom just completedtheir sixth-grade year and twelve of whom just completed their seventh-grade year.

Procedures Due to summer vacation, the laptop program was not in use during the time which thisevaluation took place. A survey was sent out to participants asking them to reflect on their experience using the laptops. Using Edmodo.com, student work from those who answered the


6


7/20

survey was analyzed for definitions, examples, reflection, and/or application; the amount of work submitted via Edmodo was also tabulated to determine an approximation as to how much paper was saved.

Data Sources

Data was collected regarding the objectives of the program: how the students used thelaptops, students skill level with GoogleDocs, ease of day-to-day tasks, whether paper wassaved, and whether the laptops impact student return rate. General comments and informationwas amassed as well, providing the evaluator with more information about what other skills wereacquired by working with laptops. These comments are intended to provide the evaluator with aholistic understanding of the program, aside from looking at the stated objectives.

A permanent link to the survey administered to students as well as a permanent link to theresults can be found in Appendix A.

Results

Student Work Two assignments completed by sixth graders andtwo assignments completed by seventh graders wereanalyzed. Sixth-grade Assignments. The sixth-gradeassignments which were analyzed were: Mr. Stick PresentsConjunctions and Interjections slides. Both assignmentscontain either definitions, examples, and/or application of aconcept. Samples of student work can be found in

Appendix B. Seventh-grade Assignments. The seventh-gradeassignments which were analyzed are: School Newspaper Article and Thank You Cards. Both assignments containeither definitions, examples, and/or application of aconcept. Samples of student work can be found inAppendix C. Student Responses. Students were asked if the laptops allowed them to add definitions,examples, reflection, and/or application to their assignments. Seventeen participants (94%) statedthat they were able to add definitions to their work, sixteen (88%) stated that they could makesomething based on what they learned, thirteen (72%) said that they could add an example, andnine (50%) stated that they were able to think about what they learned (Figure 1.1).

Student Use of LaptopsAll participants reported using the laptops for note-taking, presentation, and quizzes/test.

Between fifteen and seventeen students stated that the laptops were also used for: drafting,editing, and revising assignments; adding pictures to notes or assignments; drawing through


7

Finish this statement, "When used as part of class, the laptops let me..."

Figure 1.1


8/20

either ArtRage 2 or GoogleDrawing; playing games which were part of class; and web searchingthat was part of class.

Student Skill Level with GoogleDocsStudents were given a list of eight skills related to

GoogleDocs such as creating a new file, sharing the Docwith others, adding a picture, and changing the font size.They were asked to rate each skill, and then count how manyskills they are good or great at. Of the eighteenresponses, thirteen students (72%) state that they arecompetent at six or more skills (Figure 1.2).

Day-to-Day Tasks! All participants stated that the laptops eased day-to-day tasks, citing, among other

reasons, that they were able to type notes and save them digitally, look up information faster, andstart assignments at school.

Saving of PaperAll participants stated that paper was saved because notes, assignments, and assessments

were mostly done on the computers. Forty-eight total tests and 105 total assignments were turnedin via Edmodo.com across all three grade levels. Each test and assignment was minimum one

page of paper, which means that at least 18,090 sheets of paper were saved.

Attrition Rate

Students were asked two questions which were meant to uncover how well the laptopsworked as a marketing tool. Seventy-eight percent stated that the laptops are not the main reasonfor returning to the school (Figure 1.3), and the participants were split in half as to whether thelaptops would make them want to stay if they were to leave the school (Figure 1.4).


8

Using the skills above, how many of them did you rate as good or great?

Figure 1.2

If you are coming back next year, are thelaptops the main reason for staying?

Figure 1.3

If you are not coming back next year, arethe laptops the main reason that would make you want to stay?

Figure 1.4


9/20

Discussion! The purpose of evaluating the laptop program was to determine its effectiveness againstthe six stated objectives identified by the technology planning committee. The data andevaluation show that the program was mostly effective. Further discussion follows below.

Objective 1: Student work shows definitions, examples, reflection and/or application Student work consistently contains definitions, examples, reflection, and/or application,and students report that their works has these elements. For this reason, this objective is deemedas mostly effective. Reflection is an element that is missing from a lot of student work, so it isrecommended that Ms. Pinto add more ways for students to reflect in their work. This can comein the form of a writing prompt, reflection question, or progress points wherein students look attheir changes throughout the course of the year.

Objective 2: Students work collaboratively and proficiently with GoogleDocs and GoogleDrawing

! Both students and Ms. Pinto report that most are able to perform the basic functions of GoogleDocs and GoogleDrawing. For this reason, this objective is deemed as effective. It isrecommended that Ms. Pinto spends more time at the start of the year acquainting the studentswith the functions of the GoogleDocs and GoogleDrawing; this can be done via whole-group or small-group mini-lessons.

Objective 3: Laptops are used for note-taking, revision, creation of assignments, andassessment! Both students and Ms. Pinto reported that laptops were used in the capacities listedabove, however, some students did report using the laptops for recreational purposes like

online shopping, Google searching, or games not related to class. For this reason, this objectiveis deemed as effective. It is suggested that Ms. Pinto continues to use the laptops as she has been,while making sure to use the Smart Sync remote desktop software to better monitor and managestudents who are not on task.

Objective 4: Day-to-day tasks are eased (e.g., note-taking, revision)! The laptops allowed students to keep better track of their work, access their work athome, submit assignments outside of school hours, finish work quicker, and streamline materials.For this reason, this objective is deemed at highly effective. It is suggested that Ms. Pintocontinues to use the laptops in this way.

Objective 5: Paper use is reduced and/or savedAt least 18,090 sheets of paper were used, and students state that they used paper for only

selected assignments within Ms. Pintos class. As a result, this objective is deemed at highlyeffective. It is recommended that Ms. Pinto continues to digitize assignments, and strive towardmaking the classroom completely paperless. Objective 6: Laptop program recruits new students and keeps old students


9


10/20

The results show that the laptops did not provide a significant incentive for staying at theschool. Moreover, the school only welcomed two new students to the junior high for the2012-2013 school year while losing nine. It is because of this that this objective is deemedineffective. It is recommended that the school not attempt to bolster the laptops (or other educational technology) as the sole reason for having new families attend the school, but rather

highlight technology as a learning addition to the methods the teachers use.

Project CostThe budget below is inclusive of all services rendered and costs related to the evaluation

of the St. Francis Xavier laptop program. The evaluator salary is the largest value, while theothers are reflective of the day-to-day necessities for completing this evaluation.

The evaluation took thirty days, and during that time, the evaluator spoke withstakeholders, developed and distributed the student survey, collected survey results, analyzeddata, and wrote the evaluation report. Because speaking with stakeholders took only three days,talk and text time was kept to a minimum. However, since the surveys were distributed and

collected via GoogleForms, Internet access was required for the full time. Lastly, since thisevaluation took place over the schools summer vacation, the evaluator was able to speak withstakeholders, collect and analyze data, and write the evaluation report from from home duringthe thirty days; no cost related to travel was accrued.

Personnel (Includes per diem)

Name Salary Cost

Evaluator (Lauren Pinto) 30 days @ $500 $15,000

Communication

Area Number of Days Total Cost

Talk and Text 3 days @ $100/month $10

Internet 30 days @ $75/month $75

Travel

Reason

Speaking with Rita Recker and Teresa Smith

None $0

Data collection and analysis None $0

Writing of evaluation report None $0

TOTAL: $15,085


10


11/20

Appendices


11


12/20

Appendix A: Student Survey and Responses

Student SurveyThe student surveys can be found at the following link: http://tinyurl.com/88p8hdz

Student ResponsesThe student responses can be found at the following link: http://tinyurl.com/6r3o4s5


12
http://tinyurl.com/6r3o4s5http://tinyurl.com/6r3o4s5http://tinyurl.com/88p8hdzhttp://tinyurl.com/88p8hdz


13/20

Appendix B: Sixth-Grade Student Work

Mr. Stick Presents: Conjunctions


13


14/20

Interjection Slides


14


15/20


15


16/20

Appendix C: Seventh-Grade Student Work

School Newspaper


16


17/20


17


18/20


18


19/20

Thank You Cards


19


20/20


20

Documents

Course Evaluation Project