View
223
Download
2
Category
Tags:
Preview:
Citation preview
Hua Ai, Xiaojun Peng, Rosa Ren, Nan ZhouHua Ai, Xiaojun Peng, Rosa Ren, Nan Zhou
http://dream.sims.berkeley.edu/4hmwk/
Overview
Introduction Introduction Problem StatementProblem Statement GoalsGoals
SystemSystem ProjectProject
Research and AnalysisResearch and Analysis Needs AssessmentNeeds Assessment Competitive SystemsCompetitive Systems
DesignDesign Pilot ImplementationPilot Implementation
PrototypesPrototypes DemoDemo
Future WorkFuture Work AcknowledgementAcknowledgement
myHomework@SIMS
For: For:
About:About:
What: What:
SIMS professors and studentsSIMS professors and students
Assignments and related informationAssignments and related information
A solution that enablesA solution that enables Effective document managementEffective document management
Effective information useEffective information use
What’s the Problem? We have no standard way to manage assignments for all We have no standard way to manage assignments for all
courses. courses. Every professor does things a little differently Every professor does things a little differently Various submission and feedback methods can be confusing, each Various submission and feedback methods can be confusing, each
with their own advantages and disadvantages with their own advantages and disadvantages IT has to support a variety of ways to do electronic submission, IT has to support a variety of ways to do electronic submission,
extra time is spent customizing and managing solutions for extra time is spent customizing and managing solutions for individual coursesindividual courses
Assignment related information not used toAssignment related information not used to enable better communication between professors and studentsenable better communication between professors and students allow efficient workload management by professors and studentsallow efficient workload management by professors and students
Practice Inconsistent with SIMS’s Mission Practice Inconsistent with SIMS’s Mission Advance information management and the use of information Advance information management and the use of information
technology to enable more efficient flow of information, better technology to enable more efficient flow of information, better communication and cooperation among individuals and institutions communication and cooperation among individuals and institutions
System Goals
Provide consistent mechanism for all SIMS coursesProvide consistent mechanism for all SIMS courses manage assignments and related informationmanage assignments and related information encourage better use of the documents and their metadataencourage better use of the documents and their metadata
Document ManagementDocument Management Information UseInformation Use
ProfessorsProfessors -Easily obtain and manage submissions by students & groups
-Assignment planning
-Easily grade & give feedback
-Track student performance
StudentsStudents -Simplify submission process
-Share assignments
-Archive previous work
-Better plan & manage work
-Receive grade & feedback
-Track overall performance
Project GoalsLimited by time, we focused our effort onLimited by time, we focused our effort on Research and AnalysisResearch and Analysis
Understand the Understand the current “system”current “system” and its and its problem areasproblem areas Thoroughly understand the Thoroughly understand the needs needs of each user group of each user group Evaluate Evaluate existingexisting systemssystems
DesignDesign Develop high level design of a system that would Develop high level design of a system that would
address the problemsaddress the problems Pilot implementationPilot implementation
Develop pilot interface and databaseDevelop pilot interface and database
Needs Assessment Overview
Purpose:Purpose: Understand overall process and context of issuesUnderstand overall process and context of issues Understand specific user and organizational needsUnderstand specific user and organizational needs
MethodologyMethodology
Stake
Holders/Users
Professors / Professors / TA’sTA’s
StudentsStudents SIMS & SIMS & Campus ITCampus IT
Method Semi-structured Semi-structured InterviewsInterviews
Structured Structured SurveySurvey
Open-ended Open-ended InterviewsInterviews
Participants 7 professors7 professors
2 TA’s2 TA’s
35 students35 students KevinKevin
ETS staffETS staff
Overall Context
On averageOn average Professors teach 1-2 courses (including seminars) each Professors teach 1-2 courses (including seminars) each
termterm Students are enrolled in 2-3 SIMS courses each termStudents are enrolled in 2-3 SIMS courses each term Many courses require 6-10 assignments and 3-4 course Many courses require 6-10 assignments and 3-4 course
project related submissionsproject related submissions Almost all students have submitted assignment Almost all students have submitted assignment
electronically, especially using email with attachmentselectronically, especially using email with attachments 57% experienced problems w/ electronic submission57% experienced problems w/ electronic submission Few professors and students have used a “Learning Few professors and students have used a “Learning
Management System”Management System”
Professors’ Context Most teach at least 5 years and many 15+ Most teach at least 5 years and many 15+ A lot of course preparation before term startsA lot of course preparation before term starts
Scope of courseScope of course Material: review books and articles, do readers…Material: review books and articles, do readers… Course schedule: topics, readings assignments, projectsCourse schedule: topics, readings assignments, projects
Real goal is to help students learnReal goal is to help students learn Submission management not a pleasant but necessary task (among others)Submission management not a pleasant but necessary task (among others) Most have tried everything: paper, email, web pages, directoriesMost have tried everything: paper, email, web pages, directories
Grades are evil but necessaryGrades are evil but necessary Really represent individuals and their overall effort, progress? Really represent individuals and their overall effort, progress? Same grade for different students really mean the same thing?Same grade for different students really mean the same thing? Grades have their places and not always the most importantGrades have their places and not always the most important
Professors’ Concerns
Balance of needs when designing assignmentsBalance of needs when designing assignments Purpose of specific assignments Purpose of specific assignments Kinds of equipment, tools and material requiredKinds of equipment, tools and material required Amount of time involvedAmount of time involved Flexible submission methodsFlexible submission methods Appropriate formatAppropriate format Manageable artifactsManageable artifacts
Group situations most complicatedGroup situations most complicated Track progress by groupTrack progress by group Feedback for group and individual membersFeedback for group and individual members
Using student workUsing student work permission is biggest concern and difficult to managepermission is biggest concern and difficult to manage
Student Preferences
35 responses35 responses 15 115 1stst, 15 2, 15 2ndnd, 5 others, 5 others 16 female, 17 male16 female, 17 male Almost all used some form Almost all used some form
electronic submission electronic submission 57% (20) experienced 57% (20) experienced
problems w/ e-submitproblems w/ e-submit Many preference questions Many preference questions
1-7 Likert scale1-7 Likert scale 1 least prefer, 7 most1 least prefer, 7 most
ElectronicElectronic submission 5.7submission 5.7 grade 5.3grade 5.3 feedback 4.3feedback 4.3
More concerned w/ receiving More concerned w/ receiving notification when professor notification when professor plan to share (5.5) than plan to share (5.5) than giving actual permission giving actual permission (5.0)(5.0)
51% (18) archive 2-5 yrs, 51% (18) archive 2-5 yrs, 22% (8) indefinitely22% (8) indefinitely
Willingness prepare files for Willingness prepare files for long term archive = 4.8long term archive = 4.8
IT Concerns
Current support provided by SIMS IT Current support provided by SIMS IT Create course homework email addressCreate course homework email address Create and manage share directories for courses, project groupsCreate and manage share directories for courses, project groups Set up various services to support websites, databases, many special Set up various services to support websites, databases, many special
software supportsoftware support Recently custom upload web interface for Document EngineeringRecently custom upload web interface for Document Engineering
IT concernsIT concerns Authentication, security of database and filesAuthentication, security of database and files Storage of documentsStorage of documents Long term archive issuesLong term archive issues
ETS StaffETS Staff Majority of support helping professors become familiar w/ the Majority of support helping professors become familiar w/ the
supported systems, setting up accounts, handholding alone the waysupported systems, setting up accounts, handholding alone the way
Competitive Systems
SystemSystem StrengthsStrengths Not suitable for SIMS because:Not suitable for SIMS because:
Web-CTWeb-CT Currently supported by Currently supported by Berkeley (ETS) Berkeley (ETS)
Free for each professorFree for each professor
Stove pipe model: handle assignment Stove pipe model: handle assignment spec, collection, grading isolately. spec, collection, grading isolately.
Developed elsewhere, no plug in for Developed elsewhere, no plug in for extension. (e.g., e-grade)extension. (e.g., e-grade)
Lack support for planning or archivingLack support for planning or archiving
BlackBoard BlackBoard (CourseInfo)(CourseInfo)
Currently supported by Currently supported by Berkeley (ETS)Berkeley (ETS)
Free for each professorFree for each professor
Many features for course management, yet Many features for course management, yet each part too simple. each part too simple.
Assignment management does not meet Assignment management does not meet SIMS and not easy to use enough.SIMS and not easy to use enough.
WebcrossingWebcrossing Few professors are Few professors are experimenting experimenting
Focuses on discussion forum and Focuses on discussion forum and online live lectures.online live lectures.
MonsterTrack & MonsterTrack & NSF Fast Track NSF Fast Track Proposal SystemProposal System
Both document Both document management system management system used by manyused by many
Different application and audienceDifferent application and audience
Limited types of documentsLimited types of documents
Used for schedulingUsed for scheduling interviewinterview
Halftime
Introduction Introduction MotivationMotivation GoalsGoals
SystemSystem ProjectProject
Research and AnalysisResearch and Analysis Needs assessmentNeeds assessment Competitive systemsCompetitive systems
DesignDesign Pilot ImplementationPilot Implementation
PrototypesPrototypes DemoDemo
Future WorkFuture Work AcknowledgementAcknowledgement
Design & Pilot Implementation
JSP
Web Interface
Architecture
Application(dynamic processing)
JDBC API
Databasemysql
Interface Prototype
Implementation
Some Implementation
From ER to Data Load-up
Interface Prototype PurposePurpose
Iteratively develop a Iteratively develop a potential interface with potential interface with the involvement of the involvement of users from early onusers from early on
MethodologyMethodology
ProfessorsProfessors StudentsStudents
Round 1Round 1 3 participants3 participants 3 participants3 participants
Round 2Round 2 2 participants2 participants 3 participants3 participants
Prototyping Usability Testing
Interface Prototypes – Tasks
Professor Professor Plan and specify submission detailsPlan and specify submission details View, grade assignments, and give feedback View, grade assignments, and give feedback Calculate final gradeCalculate final grade
StudentStudent Submit assignmentsSubmit assignments Receive grade and feedbackReceive grade and feedback Adjust permission setting Adjust permission setting
Interface Prototype - Issues TerminologyTerminology
assignment, project, homework, submission, deliverable assignment, project, homework, submission, deliverable Sequential Process ModelingSequential Process Modeling
Assignment specification and submissionAssignment specification and submission PlanningPlanning
Force professors to enter one assignment at a time without overall Force professors to enter one assignment at a time without overall context context
Homepage does not provide helpful overviews for all coursesHomepage does not provide helpful overviews for all courses Group ManagementGroup Management
Who set up and change, how to keep track and reuseWho set up and change, how to keep track and reuse Final GradesFinal Grades
Handle by system or defer to Excel?Handle by system or defer to Excel?
Future Work More management functions More management functions
Group management: Feedback from fellow group membersGroup management: Feedback from fellow group members Security: Access rights for TAsSecurity: Access rights for TAs Sophisticated grades manipulation Sophisticated grades manipulation
More information-use functionsMore information-use functions Archiving / Reusing (content management)Archiving / Reusing (content management) Planning and ReportingPlanning and Reporting
MoreMore implementationimplementation supportingsupporting functions by underlying JSP and database so SIMS has functions by underlying JSP and database so SIMS has
a real system!a real system!
SIMS “Learning Management System”SIMS “Learning Management System” course content (website) managementcourse content (website) management
Why We Choose This Project
Problem can be easily constrained so we can arrive at a Problem can be easily constrained so we can arrive at a potential solution with the available resourcespotential solution with the available resources
Chance for group members to use various skills and Chance for group members to use various skills and training gained at SIMS training gained at SIMS
Easy access to all user groupsEasy access to all user groups Professors (chance to get to know them better and get advice for Professors (chance to get to know them better and get advice for
our project!)our project!) Cool IT staffCool IT staff Awesome SIMS students – (easily bribed by food :)Awesome SIMS students – (easily bribed by food :)
Most importantly, we wanted a chance to contribute to Most importantly, we wanted a chance to contribute to SIMS by helping to address a real SIMS problem SIMS by helping to address a real SIMS problem
Lessons Learned
Understanding user needs is iterative, iterative, iterative Understanding user needs is iterative, iterative, iterative Project focused on front end issues should have more than Project focused on front end issues should have more than
1 person with experience1 person with experience Vision is important when too immerged in tasks and Vision is important when too immerged in tasks and
detailsdetails Need another term to implementNeed another term to implement SIMS students like rice puddingSIMS students like rice pudding Frozen burritos and a big bottle of hot sauce can go a long Frozen burritos and a big bottle of hot sauce can go a long
wayway Spouses and boyfriends make great tech support but need Spouses and boyfriends make great tech support but need
to prevent them from interrupting our offsite meetingsto prevent them from interrupting our offsite meetings
Thank You! Professors: Professors:
Ray LarsonRay Larson Marti HearstMarti Hearst David Messerschmitt David Messerschmitt Bob GlushkoBob Glushko John ChuangJohn Chuang Peter LymanPeter Lyman Nancy Van HouseNancy Van House
SIMS IT: SIMS IT: Kevin HeardKevin Heard
ETS Staff: ETS Staff: Fred BeshearsFred Beshears Howie LanHowie Lan Charmaine FalcaoCharmaine Falcao
MonsterTrak: MonsterTrak: Scott HolcombScott Holcomb
Students:Students: All Survey RespondentsAll Survey Respondents Moryma AydelottMoryma Aydelott Steve KafkaSteve Kafka Mary ThompsonMary Thompson Thuan NguyenThuan Nguyen Luis VillafanaLuis Villafana Linda HarjonoLinda Harjono Mike KimMike Kim Craig RixfordCraig Rixford Calvin SmithCalvin Smith Ramona MartinezRamona Martinez Linda DuffyLinda Duffy Peter RoesslerPeter Roessler
Recommended