Embed Size (px)
Citation preview
NECC 2007 - MP356
Creating Web-Based Online Surveys
2
Jeff Sun [email protected]
www.sun-associates.com/necc2007
You? Who’s working on projects now? Existing survey work?
3
Objectives
To understand the role of data collection in an overall evaluation process
To understand the role of surveys as one tool in an evaluation data collection process
To become initially familiar with tools for creating online surveys
Others?
4
Why Evaluate?
To fulfill program requirementsProject sustainabilityGeneration of new and improved project
ideas
5
By Definition, Evaluation…
Is both formative and summative Helps clarify project goals, processes, products Should be tied to indicators of success written for your
project’s goals Is not a “test” or simply a checklist of completed
activities Qualitatively, are you achieving your goals? What adjustments can be made to your project to
realize greater success?
6
Data Collection vs. Evaluation
Evaluation is more than data collectionEvaluation is about…
Creating questions Creating indicators Collecting data Analyzing and using data
Data collection occurs within the context of a broader evaluation effort
7
An Iterative Process
Evaluation breaks your vision down into increasingly observable and measurable pieces.
8
Tools and Techniques
Surveys, Focus Groups and Interviews Measuring satisfaction and determining attitudes
ObservationsArtifact AnalysisTriangulation with data from administrators and
staff Do other groups confirm that teachers are being
served?
9
Surveys
Online vs. Paper Is there sufficient connectivity? Often works best if people complete the
instruments all at the same time Same goes for paper surveys
Online surveys provide immediate dataSpreadsheets which can be exported to a
variety of different programs for analysis
10
Surveys
Online VIVED Profiler LoTi Zoomerang SurveyMonkey.com
11
Make Your Own!
www.sun-associates.com/surveyws/surveys.html www.sun-associates.com/necc2007/monday/y2survey.html
Based on a CGI script on your web serverOutputs to a text file, readable by ExcelWorks with yes/no, choose from a list, and free text
input (no branching)
12
The Survey Script
www.bignosebird.com/carchive/bnbform.shtml
You need access to your WWW server’s directory structure If you do not have this, you should use some of the
commercial survey services discussed earlier
You need FTP access to your WWW serverThe bnbform.cgi script goes in your /cgi directoryThe readme.txt file contains all the installation info
necessary
13
Once Installed
Surveys are HTML forms that make use of the POST command <FORM METHOD="POST"
ACTION="/cgi-bin/bnbform.cgi" ENCTYPE="x-www-form-urlencoded">
Let’s examine an example… www.sun-associates.com/necc2007/monday/y2survey.html
14
For Today’s Exercise
Create a folder called <yourinit>testsite on your desktop
In that folder, create a folder called wwwIn the /www folder, create a folder called
<yourinit>test
15
www.sun-associates.com/necc2007/monday/y2survey.html
Save this as “sample.html” as Page SourcePlace that file in your <yourinit>test folder
16
Start Dreamweaver /Site/New SiteLocal Root Folder = <yourinit>testsiteHttp address = www.sun-associates.comRemote Info
Access = FTP Host = ftp.business.earthlink.net Host Dir = leave blank Login = wfp44289%sun-associates.com Pswd = neccmp356
17
Creating Forms
Dreamweaver (or another HTML authoring tool) is recommended
It’s best to start with a completed survey form and then modify That way, you get all of the correct <form>
tags and hidden files You can use the previous example as your
template (just Save As… to save the html) Let’s do that now
18
Things to Do
(in “Design” view) Delete text between first line on the page and the text just before the SUBMIT button
Create new form elements in the resulting space Pull Down List Text Area Radio Button Pair (yes/no)
Modify the hidden elements (in “Code” view) at the end of the source code
<P><!-- SCRIPT CONFIGURATION SECTION --><INPUT TYPE="hidden"NAME="data_order" VALUE="district,other_dist,…8i"><INPUT TYPE="hidden" NAME="outputfile" VALUE="neccwsy2.log"> <INPUT TYPE="hidden" NAME="form_id" VALUE="SAMPLE Y2 Expectations Survey"> <INPUT TYPE="hidden" NAME="ok_url" VALUE="http://www.sun-associates.com/thanks.html"><INPUT TYPE="hidden" NAME="not_ok_url" VALUE="http://www.sun-associates.com/sorry.html"><!-- END OF SCRIPT CONFIGURATION SECTION -->
19
Survey Considerations
There actually is a science involved in survey creation
Issues of validity both in questions and instruments
Sample size Attaining a sufficiently large sample to test
validity Piloting
20
Survey Tips
Keep them short (under 10 minutes)Avoid huge long checklistsAllow for text commentsSupport anonymity
But allow for categorical identifications -- school, job function, grade, etc.
21
Coordinate and support survey administration Avoid the “mailbox stuffer” Work with building leaders Provide clear deadlines
22
Three Big Points
Surveys alone mean nothing TRIANGULATE!
100% response rate is virtually impossible On the other hand, nearly 100% is very
possible if you follow our tips!
Share the data No one wants to fill in forms for no purpose
23
Data Dissemination
Compile the reportDetermine how to share the report
School committee presentation Press releases Community meetings
24
10 Tips for Data Collection
Ground your data collection within a larger evaluation framework Know why you’re asking what you’re asking
Not all data is quantitativeCast a wide net
It’s all about stakeholders
Dig deep Try to collect the data that can’t easily be observed or
counted
25
Use confirming sources Triangulate! Surveys alone do nothing.
Have multiple writers Stakeholders and different perspectives
Think before you collect Choose questions carefully and with regard to
what you really expect to find
26
Set (reasonable) expectations for participation Time and effort
Forget about mailbox surveys Usually waste more time than their value
Report back Don’t be a data collection black hole!
27
More Information
[email protected] 978-251-1600 ext. 204
www.sun-associates.com/evaluationwww.sun-associates.com/necc2007www.edtechevaluation.com
