ITEST Management Information System (MIS) – Annual Survey

Community members Educators Guidance counselors Parents/caregivers Youth Other (please specify) ____________________ The project did not serve any participants during this time period

Answer if project served educators, youth, or parents/caregivers during this time period.

1b. Use this table to record the number of participants served by your ITEST project from September 1, 2016 to August 31, 2017. Write "0" if no participants were served in a particular category.

Participants Served Total number

Total number of EDUCATORS served Total number of YOUTH served DIRECTLY through project activities [This number should not include those youth that were taught by participating educators. You will report those youth below.] Total number of YOUTH taught by participating educators rather than served directly through project activities [This number should not include any of the youth served directly through project activities that you reported above] Total number of PARENTS / CAREGIVERS served

ITEST Management Information System (MIS) – Annual Survey (Fall 2017)

PARTICIPANTS

1. Indicate the type(s) of the participants that your project served from September 1, 2016 toAugust 31, 2017. (check all that apply)


Answer if project served youth directly during this time period.

1c In the previous section, you indicated that (#) YOUTH were served DIRECTLY through project activities from September 1, 2016 to August 31, 2017. Please now record the demographics for this population. Use this table to record the number of YOUTH served DIRECTLY in project activities belonging to each category. Write "0" if no YOUTH were served DIRECTLY in a particular category.

[NOTE: You will notice that the race/ethnicity categories below may be different from how data is collected in your project. This is because the MIS collects project-level data and therefore combines some of the categories.]

Youth served directly through project activities

Total number Girls Boys American Indian or Alaska Native Asian Black or African American (not of Hispanic origin) Hispanic or Latino Native Hawaiian or Other Pacific Islander White or Caucasian (not of Hispanic origin) More than one race reported Youth eligible for free or reduced price lunch English language learners Youth with disabilities


PARTNERS

2. Indicate the type(s) of individuals (other than project staff) who provided support to your project, whether paid or unpaid, from September 1, 2016 to August 31, 2017. (check all that apply) IT/STEM professionals Professional association staff Tribal elders Undergraduate or graduate students University faculty or staff Other (please specify) ____________________ This project was not supported by any additional individuals during this time period

3. Indicate what type(s) of partner institution(s) worked with your project from September 1, 2016 to August 31, 2017. (check all that apply) Business/Industry College/University (four-year) Community College (two-year) Community-based organization Government laboratory Informal science education organization Not-for-profit organization Research institution School (any grade, PK-12 school) Other (please specify) ____________________ The project did not work with partner institutions during this time period (skip to the Project

Implementation section)

Answer if partners are four-year or two-year colleges/universities

3b Indicate if any of your college/university partners are minority serving institutions. (check all that apply) Hispanic serving institution Historically Black college/university Tribal college/university Other minority serving (please specify) ____________________ The project does not work with a minority serving institution I do not know (please provide the name of the college/university so that STELAR may

determine if it is a minority serving institution) ____________________


4. Indicate how your partner institution(s) participated in your project from September 1, 2016to August 31, 2017. (check all that apply). Assisted with curriculum development Assisted with project planning or design Assisted with recruitment Contributed to dissemination Participated in delivery of program Provided employment to ITEST participants (part/full time) Provided external evaluation Provided field trip sites, shadowships Provided funding Provided incentives for participation (e.g., prizes, meals, events) Provided internships/externships Provided mentors Provided real world problems for students to solve Provided scientists/engineers/technologists Provided tools/equipment/facilities Served as advisors to the ITEST project Volunteered in classroom/team teaching of lessons Other (please specify) ____________________ The partner institution(s) did not contribute to the project during this time period

5. Indicate how often your partner institution(s) were involved in project activitiesfrom September 1, 2016 to August 31, 2017. (check all that apply) Weekly Monthly Quarterly Annually

6. How have your partner institution(s) added value to your project? Please provide at leastone specific example.

Student experiences with emerging technologies: What learning experiences involvingemerging technologies effectively enable diverse populations of students gain familiarity andrelevant competencies with these technologies, and what factors influence the outcomes ofthe learning experiences?

Motivation and preparedness to pursue STEM careers: What factors and key experienceseffectively promote awareness of STEM careers, motivation to pursue a STEM career, andpersistence in undertaking education pathways to those careers, particularly amongstudents from underrepresented populations?

Instructional and curricular innovations: What culturally-responsive instructional andcurricular practices and models used by teachers enhance student understanding of andinterest in STEM occupations, and what factors influence the outcomes of the practices andmodels?

Partnerships with business and industry: In what roles and in what ways do business andindustry workforce members: (1) motivate students from diverse underrepresentedpopulations to become aware of, interested in, and prepared for careers in the STEMworkforce, and (2) support teachers in efforts to promote STEM workforce awareness andinterest?

Partnerships with communities: In what roles and in what ways do parents, mentors,caregivers, and other community members motivate students to become aware of,interested in, and prepared for STEM careers?

Partnerships with school policy leaders: What strategies to engage principals, guidancecounselors, and other school system administrative leaders effectively promote student andteacher adoption and effective use of practices and technologies that support STEM learningand career awareness, and what factors influence the outcomes of those strategies?

Partnerships with career technical education: How do STEM education teachers andspecialists effectively collaborate with Career Technical Education specialists to design andimplement innovative learning experiences and learning environments from early gradesthrough high school to foster awareness of, interests in, and persistence in pursuing STEMoccupations?

The project does not address any of these questions.

7. 2017 ITEST solicitation required ITEST proposals to address one or more of the followingquestions. Although your project may not have been funded through that particular solicitation,NSF is interested in knowing how many projects are addressing the current questions. Indicatewhich question(s), if any, your project addresses. (check all that apply)


PROJECT IMPLEMENTATION

ITEST Management Information System (MIS) - Annual Survey (Fall 2017)

8. The 2017 ITEST solicitation mentioned the following three important project goals with

respect to youth. Although your project may not have been funded through that particular

solicitation, NSF is interested in knowing how many projects are addressing the current goals.

Indicate which goal(s), if any, your project addresses. (check all that apply)

Increase student awareness of STEM occupations Motivate students to pursue appropriate education pathways to STEM occupations Develop disciplinary-based knowledge and practices, or promote critical thinking,

reasoning skills, or communication skills needed for entering STEM workforce sectors The project does not address any of these goals

9. Indicate which technology tools were used with or by the participants you served from September 1, 2016 to August 31, 2017. (check all that apply)

Communication tools (e.g., email, messaging, blogs, video conferencing) Data analysis/computation tools Electronics/robotics tools Engineering/design tools Game development tools Geospatial technologies (GIS/GPS/RS) Imaging technologies Mobile devices (e.g., tablet, smart phone) Multi-media authoring tools Online learning modules or courses Programming tools Simulations Social media Virtual reality Visualization/computer modeling tools Wearable technologies Other (please specify) ____________________ The project did not use any technology tools with or by participants during this time period

10. What contributed to your project’s success from September 1, 2016 to August 31, 2017?

11. What challenges did your project encounter from September 1, 2016 to August 31, 2017, and how has your project addressed those challenges?


RESEARCH AND EVALUATION

12. The Common Guidelines for Education Research and Development[https://www.nsf.gov/pubs/2013/nsf13126/nsf13126.pdf], published by the Institute of Education Sciences, U.S. Department of Education and the National Science Foundation, describes six types of research. Starting with the 2014 ITEST Solicitation, ITEST projects are required to include a research component. However, projects were not required to follow the Common Guidelines. Indicate which type of research from the Common Guidelines, if any, your project conducts. (check all that apply)

If your project conducts research, but not one of those outlined in the Common Guidelines, please select Other and briefly describe the research your project conducts.

Foundational researchProvides the fundamental knowledge that may contribute to improved learning and other relevant education outcomes. Studies of this type seek to test, develop, or refine theories of teaching or learning and may develop innovations in methodologies and/or technologies that will influence and inform research and development in different contexts.

Early-stage or exploratory research Examines relationships among important constructs in education and learning to establish logical connections that may form the basis for future interventions or strategies to improve education outcomes. These connections are usually correlational rather than causal.

Design and development

Develops solutions to achieve a goal related to education or learning, such as improving student engagement or mastery of a set of skills. Research projects of this type draw on existing theory and evidence to design and iteratively develop interventions or strategies, including testing individual components to provide feedback in the development process. These projects may include pilot tests of fully developed interventions to determine whether they achieve their intended outcomes under various conditions. Results from these studies could lead to additional work to better understand the foundational theory behind the results or could indicate that the intervention or strategy is sufficiently promising to warrant more-advanced testing.

Efficacy studyAllows for testing of a strategy or intervention under “ideal” circumstances, including with a higher level of support or developer involvement than would be the case under normal circumstances. Efficacy Research studies may choose to limit the investigation to a single population of interest.

Effectiveness studyExamines effectiveness of a strategy or intervention under circumstances that would typically prevail in the target context. The importance of “typical” circumstances means that there should not be more substantial developer support than in normal implementation, and there should not be substantial developer involvement in the evaluation of the strategy or intervention.

Scale-up studyExamines effectiveness in a wide range of populations, contexts, and circumstances, without substantial developer involvement in implementation or evaluation. As with Effectiveness Research, Scale-up Research should be carried out with no more developer involvement than what would be expected under typical implementation.

Other (please specify) ____________________ The project does not conduct research


13. What research question(s) are you addressing in your project?[Write “n/a” if your project does not have any research questions]

14. What evaluation question(s) are you addressing in your project?[Write “n/a” if your project does not have any evaluation questions]

15. Did your project’s research and/or evaluation study design include a comparison group from September 1, 2016 to August 31, 2017? Yes No

16. Did your project’s research and/or evaluation study design include pre- and post-tests from September 1, 2016 to August 31, 2017? Yes No

https://www.nsf.gov/pubs/2013/nsf13126/nsf13126.pdf


17. Indicate the data collection method(s) your project used in its research and/or evaluation from September 1, 2016 to August 31, 2017. (check all that apply)

Artifacts produced by participants (e.g., class plans, in-class projects, homework) Embedded assessments

Focus groups

Interviews

Observations

Performance-based assessments

Self-reports (journals)

Surveys Videos

Web-based data (blogging, emails, posts)

Records of assessments or activities, not as part of the ITEST project (e.g., school grades, state test scores, classes, extracurriculars, major, etc.)

Other (please specify) ____________________

18. Indicate the type of instrumentation your project used to answer the research and/or evaluation questions from September 1, 2016 to August 31, 2017. (check all that apply)

Note: A validated instrument has undergone statistical testing for reliability (the ability of the instrument to produce consistent results) and validity (the ability of the instrument to produce true results)

Externally developed and validated instrument(s) in their original form

Internally developed and validated instrument(s)

Externally developed and validated instrument(s) modified to fit the needs of the project

Internally developed instrument(s), not validated


Answer if project used a validated instrument.18b. Please list the names as well as citations and/or URLs of the validated instrument(s) your project used from September 1, 2016 to August 31, 2017.

Note: A validated instrument is an instrument that has undergone statistical testing for reliability (the ability of the instrument to produce consistent results) and validity (the ability of the instrument to produce true results).

The validated instrument(s) you provide will be listed in the ITEST Instrument Database on the STELAR website (http://stelar.edc.org/publications/itest-instrument-database). The ITEST Instrument Database was developed to “encourage the use of common instruments to support improved coordination and understanding of program impacts” (2015 NSF ITEST Solicitation). Please provide enough information about the instrument so that STELAR staff can locate a copy of it and information about it.

Once the instrument is listed on the STELAR website we encourage you to add feedback about your experience with the instrument to share with others who may consider using it.

You can see what instruments (if any) your project has contributed to the database by visiting your STELAR project profile and clicking on the "View Related Content" button.

Answer if 34_Yes is selected

20b. What data do you intend to collect from participants after the ITEST funding is over?

19. Please provide a brief summary of your project findings from September 1, 2016 to August 31, 2017. These could include answers to the research and/or evaluation questions, if applicable.

20. Do you intend to collect any additional data from participants after the ITEST funding is over? Yes No

http://stelar.edc.org/publications/itest-instrument-database


DISSEMINATION

21. Indicate in what dissemination strategy(ies) your project engaged from September 1, 2016 to August 31, 2017. Select ONLY those activities in which your project engaged, not those planned for the future. (check all that apply) Branding your project Communicating through mass media (e.g., newspaper, TV, video, radio) Cultivating dissemination partners or champions Developing products [NOTE: You will be asked about these products below] Hosting meetings/topical convenings Hosting or presenting in podcasts or webinars Networking Organizing workshops Presenting at conferences Using social media (e.g., blogs, twitter, Facebook, LinkedIn) Other (please specify) ____________________ The project did not engage in any dissemination strategies during this time period (skip to

the Additional Information section)

Answer if project presented at a conference

21b Indicate at which conference(s) your project’s work was presented from September 1, 2016 to August 31, 2017. Select ONLY those conferences in which a presentation was given, not those planned for the future. (check all that apply) ACM Special Interest Group on Computer Science Education (SIGCSE) American Educational Research Association (AERA) American Evaluation Association (AEA) American Indian Science and Engineering Society (AISES) American Society for Engineering Education (ASEE) Association for Science Teacher Education (ASTE) Association of Science-Technology Centers (ASTC) Computer Science Teachers' Association (CSTA) International Conference on Computer-Supported Collaborative Learning (CSCL) International Council of the Learning Sciences (ICLS) International Society for Technology in Education (ISTE) International Technology and Engineering Educators Association (ITEEA) League for Innovation Science, Technology, Engineering and Math (STEMtech) National Afterschool Association (NAA) National Association for Research in Teaching (NARST) National Education Association (NEA) National Science Teachers Association (NSTA) SACNAS: Advancing Hispanics/Chicanos & Native Americans in Science Serious Games Society for Information Technology and Teacher Education (SITE) Other (please specify) ____________________


Answer if project presented at a conference

21c Did one or more of the conference presentations from September 1, 2016 to August 31, 2017 involve collaboration with other ITEST projects in some way? Yes No

22. Indicate what product(s) were developed for or by your project from September 1, 2016 to August 31, 2017. (check all that apply) Apps for phone or tablet Books/book chapters Curriculum/instructional materials Games/simulations Implementation models Instruments (e.g., to assess interest, engagement, persistence, motivation, skills, knowledge

of dispositions, etc.) Newsletters Peer-reviewed journal articles Podcasts or webinars Professional development materials/teacher training Program models Reports or white papers Research findings Software Technology designs/Prototypes Textbooks Theoretical constructs Videos/Multimedia Virtual environments Websites Other (please specify) ____________________ The project did not develop any products during this time period

23. Indicate the target audience(s) of your project's dissemination strategy(ies) and/or products from September 1, 2016 to August 31, 2017. (check all that apply) Educators General Public IT/STEM Industry professionals Parents Policymakers Researchers/Evaluators Youth Other (please specify) ____________________


24. What have you found to be the most effective dissemination strategy or product for reaching your intended audience(s) from September 1, 2016 to August 31, 2017? How did you know it was effective? Please provide at least one specific example, and include the strategy and/or product and its intended audience in your response.

25. Please enter the citation for articles, conference or journal papers, curriculum materials, or news pieces produced for or by your project from September 1, 2016 to August 31, 2017 (if any). Please also include accompanying URLs (if any).

The purpose of this MIS question is to allow STELAR to help disseminate your project’s work and news. STELAR adds the items you provide here to the resources section of the STELAR website (http://stelar.edc.org/resources), promotes them when relevant in the thematic highlights (http://stelar.edc.org/stelar-materials), and shares them on social media. Let us help you share your important work! Please provide enough information so that STELAR staff can locate the items(s) online. If you would like to share a PDF of the item(s) please note that in the box below and a STELAR staff member will contact you. Remember, these resources will be on the STELAR website and open to the public. Please only enter the citations of finished products that you wish to distribute.

STELAR staff would be happy to upload your share-able products to the website at any time. If you do not have a finished project at this time, feel free to email us ([email protected]) when you do and a STELAR staff member will help you upload it.


ADDITIONAL INFORMATION

Yes (a STELAR staff member will contact you) No

Yes No

Answer if project has applied or will apply for a no-cost extension

30b. What is the proposed end date for your project including the no-cost extension?

31. We are interested in knowing whether you have been able to provide us with all the relevant information about your project in this format. This MIS report contains all relevant information about the project from September 1, 2016

to August 31, 2017. We have important results, outcomes, or other information about the project that we were

not able to share in this format. (Please share that information in your response to thequestion below).

Before you submit your MIS, please be sure that you have answered all questions in each section. Even if sections listed in the table of contents have a check mark, there may be remaining unanswered questions within that section. Once you have checked your responses to all questions, please click "Submit MIS" below. Upon submission you will receive an email with all of your responses and a link to a url from which you can save the responses as a pdf file. Once again, thank you for completing this year's MIS. Your participation contributes to a broader understanding of the value of the ITEST program for the youth and educators it serves.

26. Enter your project’s website (if applicable).

27. Enter your project’s Facebook page, twitter handle, and/or other social media outlets (if applicable).

28. Are you interested in sharing photos or videos from your project?

29. What is the current end date of your project?

30. Have you applied / Do you expect to apply for a no-cost extension?

32. If there is anything else you would like us to know about your project that you have not had the opportunity to note, please use this box to do so.

33. Approximately how long (in minutes) did it take you to complete this survey?

Documents

ITEST Management Information System (MIS) – Annual Survey