Embed Size (px)
Citation preview
We Need Your HelpWhat we need you to do for us:
• If we build it, will you use it?• Be willing to test-drive the user interface and provide feedback.• Help seed the database with assessments and data from your
course(s)
What is the FIRST Database?Faculty Institutes for Reforming Science Teaching (FIRST) has engaged faculty from over 50 institutions in professional development focused on active, inquiry-based teaching designed to improve student learning.
We are constructing a database to support research on undergraduate STEM education. The database will support storing, searching and analyzing assessment data from undergraduate STEM courses. This database will facilitate both data-driven instructional decision making (Figure 1) and research in science education.
FIRST Assessment Database:Your Role
Database FunctionalityWe are using a large variety of assessment data collected at Michigan State University and from colleges and universities across the nation to design the FIRST Assessment Database. Faculty from all STEM disciplines will input and retrieve data from the database to explore questions about effective teaching and learning in undergraduate education (Figure 2).
This project will facilitate cross-institutional studies using assessment data from large numbers of students and classes. The database is the bridge between teaching and research that enables faculty to become both expert users of and contributors to the scholarship of scientific teaching.
http://first.ecoinformatics.org
Your Research QuestionDid teaching students to create, interpret and critique models result in better student understanding of evolution?
Figure 1. Two models of instructional design.
Metadata - data about dataTo promote STEM education research, you tag assessment items with metadata, such as Discipline-specific concepts. In biology, for example, we plan to use the National Biological Information Infrastructure (NBII) Biocomplexity Thesaurus (Figure 5).
Additional metadata tags will include standard psychometrics such as difficulty and discrimination, Bloom’s Taxonomy of Educational Objectives, Professional society tags and whether an assessment item is copyrighted (as in a textbook question or a published concept inventory).
Metadata collected by the FIRST database will also encompass information about courses and institutions (Figure 6). These data will facilitate longitudinal studies in addition to comparisons among courses across institutions.
Figure 5: Example of concepts retrieved from the NBII Biocomplexity Thesaurus (http://thesaurus.nbii.gov).
Figure 3. Data sources from a faculty member, including: (a) output from course management software, (b) excel spreadsheets of student exam, homework and in-class assignments, (c) clicker data, (d) student open-ended responses and (e) exams.
Your Data SourcesYou upload to the FIRST Database all assessments and related data from your introductory biology course (Figure 3), including assessment questions and grading rubrics. You tag each assessment item with both a concept category (see Figure 5) and an instructional strategy (i.e., modeling, pair-share, JiTT, etc). You have the option to tag assessment in several additional ways (see the discussion of Metadata, next column), or you may bypass this task at this time.
Security, Intellectual Property, IRBInstitutional review boards (IRB) are unique to each college and university. This reality demands a flexible database that is both secure and protective of student data. The FIRST Assessment Database will support two levels of data access - restricted (available only to the faculty member who uploaded the data) and public (available to all registered database users).
Many assessment items are copyrighted - as in textbook questions or published concept inventories. Faculty members using the database must be aware of and take responsibility for obtaining appropriate permissions for copyrighted or otherwise protected material. Without such permissions, an assessment item cannot be part of the public database. For more information on copyright issues and intellectual property rights, please see our website.
Student responses to the FIRST Assessment Database are de-identified by a hashing function that ensures student anonymity. To promote longitudinal studies, however, the database will link students across courses at an institution.
A faculty member may chose or be directed by their institution’s IRB to remove identifying information about themselves and the institution.
This material is based upon work supported by the National Science Foundation (NSF) under award 0618501. Any opinions, findings and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the NSF.Figure 2. Overview of the FIRST Assessment Database.
Figure 4. Potential data output from the FIRST database.
Data AnalysisTo analyze whether teaching students to make models improved their learning of evolution, you query your current course and two past courses for data on evolution and modeling. The FIRST database returns a single spreadsheet file with the data from each course (Figure 4). You import the data into your favorite statistical analysis software.
As a correlated question, your are interested in comparing your students’ modeling abilities with those of students from other institutions. Your again search the FIRST database for courses at other colleges or universities using model-based education to teach evolution. You limit your query to include only large introductory biology courses at institutions similar in size to yours. You then limit the returned data to include only assessments related to modeling in evolution. The FIRST database again returns a spreadsheet that you can import into statistical software for analysis.
Based on your analysis, model-based learning led to significant gains in student learning of evolution in your course and courses at other universities. These results prompt you to redesign additional portions of your course to include modeling.
Institution type/sizeCourse format (lab, lecture, discussion, etc)Course sizeTargeted students (majors, non-majors, lower or upper level)Course Syllabus
Type of assessment (e.g., in-class, open book, exam)Proportion of final gradeBloom’s level of understandingConcept category
Course-level Metadata
Assessment-level Metadata
Figure 6: Examples of course- and assessment-level metadata captured by the FIRST Assessment Database.
Data-driven instructional design
Analyze student learning outcomes
Analysis of student data drives course modification
Design course
Teach course
Assess student outcomes
Content-driven instructional design
Discipline-based knowledge (textbooks)
For more information, please contact Diane Ebert-May ([email protected]) or Mark Urban-Lurain ([email protected]).
Design course
Teach course
Assess student outcomes
Discipline-based knowledge (textbooks)
Formative feedback
Selection
EvolutionGenetic driftGroup selectionHost selectionKin selectionNatural selectionAdaptations (biological)EcophenesFluctuating asymmetryGeneticsMutationPhylogenySelective mediaSpeciation
CompetitionDiversity indicesDominant speciesEnvironmental effectsFitnessGenetic loadGeneticsSurvival value
