21-Tech Formative Evaluation Phase II Brief

21-Tech Formative Evaluation Phase II BriefCECILIA GARIBAY & JANE SCHAEFER

Table of ContentsPhase II Study Overview 3

Results: 8Engagement 9Group Dynamics 12Switch between PMT & Physical Exhibit 14 Questions and Challenges 16 Enjoyment 17Emotional Response 18Impact on the Visitor Experience 20Likelihood of Searching out Facilitators with Tablets 23

Conclusions 25

References 28

Appendices 30

Garibay Group | 21-Tech | Phase II Findings | Spring 2012 2

Phase II Study Overview 21-Tech, an IMLS-funded project, is focused on developing strategies to increase exhibit facilitators’ capacity to use personal mobile technologies (PMT) to engage visitor learning at their museums. A major aspect of the project focuses on testing strategies for using PMTs in facilitation. This brief presents the main findings from Phase II testing.

Garibay Group | 21-Tech | Phase II Findings | Spring 2012

Staff from the four partner museums of this project (Children’s Museum of Houston, New York Hall of Science, Oregon Museum of Science and Industry, and Sciencenter) conducted Phase II Testing in December 2011 and January 2012. Phase II was intended to build on findings from Phase I testing.

Phase I backgroundPhase I testing, exploratory in nature, was designed to give staff the opportunity to pursue a range of questions about how facilitators might use PMTs to positively engage visitors with various exhibits.

21-Tech partner museum staff prototyped facilitation strategies with PMTs in one or more exhibits using a range of content (applications, games, videos, photo-graphs, etc.) with visitors. Partners made their own choices about what to try and in

which exhibits to test PMTs. (See Phase I evaluation presentation document [Garibay & Schaefer, 2011] for more detail.)

Phase II testingWhereas Phase I was intentionally broad and exploratory, the second phase aimed to narrow the focus based on earlier findings. The specific goals of this phase were to:

• Narrow the testing questions.

• Test facilitation strategies with visitors at common types of exhibit experiences.

• Test a facilitation experience at more than one partner site.

• Test and inventory App categories.

Partner institutions paired up based on the availability of similar exhibits at their institutions (see “Exhibit Themes and Test Sites” table). These institution pairs

Phase II Evaluation Questions

• What is the nature of the facilitator-visitor interaction?

• In what ways do exhibits benefit from PMT-based facilitation?

• In what ways does PMT-based facilitation impact the quality of visitor interactions?

3

Phase II Study Overview, cont’d.


worked together to develop and test afacilitation strategy at the identified exhibits at both sites. Test site one developed, tested, and revised the facilitation strategy. Test site two then tested the strategy and recommended further revisions based on their findings. This method allowed the partners to test facilitation at four exhibits in an iterative fashion. (See Appendix A and B for more information about the exhibits and PMT apps used by each institution.)

Garibay Group worked with the project team to develop testing protocols and instruments. Partner staff collected data.

Typically, one staff member facilitated the interactions at the exhibit using an iPad while a second staff person observed and collected data.

Throughout Phase II, the 21-Tech team met monthly by phone to share progress, discuss findings, and reflect on the process.

MethodsThe partners collected observational data (Bobbie, 1998) using a standardized instrument that captured both qualitative and quantitative data about visitors’ interactions with the PMT, the exhibit, and facilitators; engagement within the group; and responses to

Exhibit themeTest site 1

(Developer)

Number of observations at

Test site 1 Test site 2

Number of observations at

Test site 2Number of

observations

Colored Shadows Sciencenter 15 NYSCI 8 23

Microscopy OMSI 29 CMH 8 37

Pitch NYSCI 8 Sciencenter 8 16

Structures CMH 8 OMSI 10 18

Exhibit Themes and Test Sites

PMT-facilitated interaction. The four partner institutions documented 94 such interactions.

Visitor feedback was also collected via surveys. After an interaction, facilitators asked one adult in each visitor group to complete the brief, written survey. Fifty visitor surveys were collected (a 52% response rate).

4

Phase II Study Overview, cont’d.


Data analysisGaribay Group conducted an independent analysis of Phase II testing data.

Quantitative data were analyzed and summary statistics calculated, Data were cross-tabulated on several variables to identify major differences between respondent characteristics (e.g., age, gender). Differences are reported as appropriate. Also note that percentages in results are rounded up and, therefore, in some cases, may not add up to 100.

For qualitative data, data were coded using thematic analysis. In thematic analysis, the research identifies patterns (themes) within the data. A theme “captures something important about the data in relation to the research question and represents some level of patterned response or meaning within the data set.” (Braun and Clarke, 2006)

LimitationsThe goal of the qualitative data was to capture rich descriptions and stories of the interactions in order to deepen our

understanding of PMT-facilitated interactions. In analyzing the data, however, we found that the descriptions of interactions and visitor behavior were often brief and missing details and therefore did not easily lend themselves to deep levels of analysis.

Other limitations to the study included the low response rate to the visitor survey and the even lower response rate to its one open-ended question. The low response rate seemed to be due, in part, to not consistently asking each participating visitor group to complete the survey.

Additionally, in an ideal situation, surveys would have been orally administered, which would have increased the number of responses to open-ended questions. The testing situation, however, did not allow observers to collect survey data; they were typically occupied with writing up observation notes and it was not appropriate for facilitators to orally administer feedback surveys on their own.

The diagram above was developed in Phase I to illustrate factors affecting the visitor-facilitator interaction. Phase II data collection focused on understanding the interaction from the visitor’s perspective.

5

Phase II Study Overview: Visitor Profile65% of interactions came with groups of two (33%) or three (32%) visitors. Of visitor group members, 86 were adult females and 52 were adult males. Visitor groups included children from ages 1-17, with the majority of children in the 4- to 8-year-old range.


1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 170

2

4

6

8

10

12

14

16

18

20

4 46 5

7

13

86 6

4 31 2 1

32

1110

6

8 12

3 7

1 4

6

31 1

Ages of Children

Male (N=78)

Female (N=70)

N=148

6

Phase II Study Overview: Visitor Profile, cont’d.It is likely that many respondents were comfortable with iPads or smart phone apps. More than half of visitors surveyed reported owning a smart phone, while over a third owned an iPad or other tablet.


Smart Phone (iPhone,

Blackberry, Android)

iPad or other tablet

iPod Touch or similar device

I don't own one of these mobile

devices

Other0%

10%

20%

30%

40%

50%

60%

70%61%

35%

16%

27%

0%

Devices Owned by Participants(could choose more than one)

N=49

7

Results


EngagementVisitor groups engaged highly with the physical exhibit, the PMT, and the facilitator during their interactions. There was also evidence of interest and engagement in visitor-facilitator interactions. During 73% of interactions, at least one person was classified as intently engaged. During 14% of interactions, at least one visitor was classified as someone who “didn’t want to leave.”


Doing Exhibit

Doing PMT

Listening

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

93%

85%

77%

Engagement Classification:Interactions in Which at Least One Person Was Observed Engaging in Listed Behavior

N=94

Average 5.5Mode 5

Minimum 1Maximum 21

Length of interactions (in minutes)

9

Engagement, cont’d.Overall, children took a more active role in the visitor-facilitator interactions than adults. This included both engaging with the exhibit and interacting with the PMT. When it came to listening during facilitated interactions, however, children and adults took part fairly equally.


Behavior Adults Children

Doing the exhibit(N=165)

32% 68%

Doing the PMT(N=132)

30% 70%

Listening(N=171)

47% 53%

Children: Gender and AgeWe disaggregated data for children by gender to look for differences. Overall, we found no significant behavioral differences during interactions.

• An equal percentage of male and female children (76%) engaged with the physical exhibits during interactions.

• A slightly higher percentage of male children engaged with the PMT (65%) than female children (60%).

• When the ages of children who engaged the PMT were charted, the distribution followed the overall age distribution of child visitors and did not indicate that any one age was more likely to engage with the PMTs.

10

Engagement, cont’d.


1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 170

2

4

6

8

10

12

14

16

0

23

8

12

1514 14

5

7

4

2

4

1 10

1

Children: PMT-Engagement Age Distribution

N=132

11

Group DynamicsInteractions were almost evenly divided between those led by adult visitors, those led by children, and those led by facilitators. (Nearly one-third of the interactions were missing descriptive data; therefore, further analysis was not possible.)


0%

5%

10%

15%

20%

25%

30%

35%

40% 36%34%

12%

34%

0%

Primary Type of Group Interaction(can be more than one type)

N=79

12

In 15 cases, the interaction fit two categories (e.g., started facilitator-led but switched to child-led). These were removed from the data set.

Group Dynamics: Effect of Group Size In addition to noting behaviors that indicated engagement during visitor-facilitator interactions, observers also saw group members who appeared to not be engaged during the interaction. While at least one visitor who did not engage was present in 27% of all interactions, these visitors represented only 15% of total visitors.


Group size affected the chance that at least one person in the group might not engage. Almost all interactions in which someone was not engaged involved visitor groups of three or more.

Adults were slightly more likely than children to be disengaged. 57% of visitors categorized in observation data as “not engaged” were adults.

No significant differences emerged between groups based on age or gender. Descriptions of visitor groups with adults who were not engaged indicated that they

Mom was off to the side monitoring a younger child. The two boys were fully engaged with the Giant Arch and the iPad.

One boy was watching [the facilitator] talk about bridges, but [the facilitator] could tell that he was losing interest as he watched his peer build. [The facilitator] decided to switch to the timer so that he could time them build[ing]. [The boys became] Much more excited…began building quickly and the parents began to help hold the parts of the arch.

Man watched from the side while observing the two smaller children, but then joined in to use the microscope later.

The three-year-old male child was doing his own thing. The 13-year-old and seven-year-old male children actively listened and participated.

were often watching smaller children in the space or stood to the side observing his or her children interacting with the facilitator, PMT, or exhibit. The percentages for child males and child females were slightly lower than adults at 12% and 14% respectively.

13

Switch Between PMT and Physical ExhibitIn the majority of interactions, groups were observed switching between the PMT experience and the physical exhibit with minimal difficulty.


In 12 instances, the switch between virtual and physical experience was classified into more than one category. These data have been removed from this analysis because the categories are mutually exclusive.

Seamlessly Fairly Smoothly With Some Dif-ficulty

Stayed Primarily with PMT

Stayed Primarily with Exhibit

0%5%

10%15%20%25%30%35%40%45%

40%

32%

6% 6%

16%

Switch Between PMT and Physical Exhibit

N=82

14

Switch Between PMT and Physical by Exhibit, cont’d.The Colored Shadows exhibit had the highest percentage of switches classified as “seamless.”


StructuresColored

Shadows Microscopy Pitch

Seamlessly 17% 70% 38% 25%

Fairly Smoothly 44% 26% 21% 50%

Some Difficulty 17% 0% 3% 8%

Stayed Primarily with PMT 6% 0% 14% 0%

Stayed Primarily with Exhibit 17% 4% 24% 17%

Switch Between PMT and Exhibit

15

The app used on the PMT allowed visitors to mix colors in much the same way as the physical exhibit, so this may have accounted for the seamlessness of the switch. The Structures-themed exhibits had the highest percentage of switches categorized as “some difficulty.” Based on descriptive data, this may have been because of the physical nature of the exhibit activity and because the PMT was used only to show photos.

Questions and ChallengesIn 51% of interactions, someone in the visitor group asked a question. Both children (37) and adults (29) asked questions in these groups. In contrast, far fewer interactions (10%) involved visitors posing challenges to their groups. Only eight adults and three children posed challenges during interactions.


QuestionsWhen coded, questions asked by visitors fell into two major categories: those related to the content of the exhibit and those related to the PMT itself. The following are example of questions related to the PMT:

Can you move [the lights in the exhibit] with that [the iPad]?

Is this an app I can get?

The following are examples of questions related to the content of the given exhibit:

Why can’t we get it [the arch] to stand?

If I punch out the keystone, everything will fall down?

What keeps the disease from infecting white blood cells?

Is that the nucleus?

Why does it [the wind over the tube] sound like that?

ChallengesChallenges during interactions were primarily posed by facilitators (74% of engagements). Descriptive data on challenges posed by visitors was not robust enough for qualitative analysis. Below are a few examples of visitors posing challenges:

The first boy built the arch one time by himself with [the facilitator’s] help and then again with two other boys' help. Then he ran to get his cousin and there they set a challenge to complete the arch as quickly as possible. They kept this up for many builds. The arch was built six times during this interaction...Each time they laughed and had a good time. They worked well together utilizing a teamwork approach to building the arch.

[The man] asked about the read-outs on the Tuner+ app and then prompted kids to watch the notes change and they slid the plunger on the pipes.

It’s important to note that overall, more content-related questions were asked by visitors than PMT-related questions. This suggests that the PMT-facilitated interactions encouraged visitors to think about the exhibits, not just about what they were doing on the PMT.

When data were disaggregated by exhibit, Microscopy and Pitch tended to generate deeper lines of questioning than the other exhibits (i.e., visitors asked simple questions that progressed to more sophisticated ones). Because of the level of documentation of these questions, however, it’s unclear whether deeper questions about these exhibits were due to their content (i.e., blood cells, onion cells, sound), the particular PMT app used, the skill of an individual facilitator, or some other factor.

16

EnjoymentThe majority of survey respondents (80%) rated their experience using the tablet “enjoyable” or “very enjoyable.”


Not at all En-joyable

Somewhat En-joyable

Enjoyable Very Enjoyable0%

10%

20%

30%

40%

50%

6%

14%

38%42%

Enjoyment Ratings

N=50

17

Emotional ResponseMost emotional expressions by visitors during interactions were positive. Smiles and/or laughter were present in over half the interactions. Examples of typical verbal exclamations were “Cool” and “Wow!” Visitors expressing confusion or frustration were observed in only a few interactions.


At least one person Smiled/Laughed

At least one person was Playful

At least one person made a Verbal Exclamation

At least one person was Excited

At least one person was Confused

At least one person was Frustrated

0% 10% 20% 30% 40% 50% 60% 70%

61%

45%

37%

34%

4%

2%

Emotional Expressions(could choose more than one)

N=94

18

The woman and girl were the group members initially interested in “Zoom In” and seeing the iPad images. They were having fun using the microscope on their own clothes, hair, and skin. The woman was definitely more vocal and kept saying, “Ooh, look at this and that.” The man joined in later and then woman and man both stayed at “Zoom In” while the three girls played with the “Stuff Sorter.”

The 9-year-old was very engaged. She smiled upon success with color matches on iPad. Left saying “Cool.”

Emotional Response, cont’d.


Frustration or ConfusionThe two documented cases of frustration involved visitors who had to wait on their group or leave an exhibit with their group when they wanted to stay.

All cases in which confusion was noted (N=4) were not sufficiently documented for in-depth analysis. In one case, though, the PMT app helped a confused visitor to better understand the exhibit.

The six-year-old girl was confused about how the colored shadows were created in the exhibit. The iPad was very helpful in clarifying relationship between how to create cyan, for example, which is different from the blue light in the exhibit.

19

Impact on the Visitor ExperienceThere was evidence that the PMT-based interactions facilitated connections for visitors between the content at hand and prior experiences. During 47% of interactions, at least one visitor was observed making a connection between the exhibit and PMT. During 29% of interactions, at least one visitor was observed making a connection to his or her previous experiences.

Connections between exhibit and PMTIn most cases (N= 38), children made connections between the exhibit and the PMT. In 14 interactions, adults made the connection between the exhibit and PMT. Though descriptions of visitors’ connections were not detailed enough to allow in-depth analysis to determine whether patterns existed in the types of connections, the following are examples of connections visitors made between the physical exhibits and the PMTs:

The 3-year-old realized that the colors on the wall can be made on the iPad.

The [girl] was able to make the connection of the keystone from the picture to the actual Giant Arch and describe why it was able to stand without help.

[The five-year-old boy] could see “dots” (nuclei) in both PMT and the microscope.

[The man] talked about using digital tuners to tune his violin.

Both mom and daughter had previously discussed how black is made using pigments. [They were] interested to see how white is made with light.

[The visitor] connected colds and white blood cells.

2.Connection to school. In a couple of instances, children mentioned that they had “studied [cells] in school” or had used microscopes to look at blood cells in school.

3.Connection to everyday things. This category included connections that visitors made to things they encounter in everyday life. For example, a child compared the Giant Arch to a rainbow. Another compared blood cells to a polka-dotted shirt.

[The eight-year-old girl] said "I think it's [the colored shadows are] because of [the exhibit lights]." …She was then able to recreate the colors of the shadows using the PMT.

After seeing pictures on the iPad and building the arch, the child realized that the keystone holds the entire structure together and that if it is removed, the arch will fall.

Connections to prior experiencesIn 17 interactions, adults made connections to their previous knowledge or experience while in 14 instances, children made this type of connection. Three categories of connections emerged:

1.Connection to previous knowledge. This category included connections visitors could make to their professions, hobbies or previous discussions. For example:


Impact on the Visitor Experience, cont’d.Evidence suggests that in 19% of interactions (18 of 94), the visitor group wished to extend the experience of the PMT-facilitated interaction.


This category was created to capture data related to visitor groups’ interest in extending the experience in specific ways (i.e., asking if the facilitator could provide a PMT-facilitated interaction at a nearby exhibit or asking if the app was available for the visitor to use at home), but the data collected captured broader ways in which visitors “extended” their initial experience. These fell either into those who extended the experience during their museum visit or those expressing a desire to extend the experience post-museum visit.

Extension after the visitThe following are the types of behavior that indicated a visitor wanted to extend experience after museum visit:

• Visitor asked about the app or PMT

• Visitor expressed a desire to use app at home

• Visitor expressed a desire to use app at school

Based on the brief descriptions of the documented evidence for extending the experience, it is difficult to attribute visitors’ desires to extend their experiences to any one factor. Options might include visitors’ engagement with the PMT and apps, visitors’ overall comfort with a facilitator, visitors’ enjoyment of the experience in general, or something else altogether.

Extension during the visitThe following are the types of behavior that suggested a visitor extended the experience during the museum visit:

• Visitor asked about other exhibits

• Visitor engaged with the exhibit multiple times

• Visitor tried another app on the PMT

• Visitor tried a suggested activity at the exhibit

• Visitor engaged with another, nearby exhibit

21

Impact on the Visitor Experience, cont’d.Respondents indicated that their interactions enhanced their overall experience. The majority of respondents (90%) agreed or strongly agreed with the statement “My interaction with the exhibit was enhanced by the use of the tablet.”


Strongly Disagree

Disagree Agree Strongly Agree0%

10%

20%

30%

40%

50%

60%

2%8%

54%

36%

My Interaction with the Exhibit was Enhanced by the Use of the Tablet

N=50

22

Likelihood of Searching Out Facilitators with TabletsOver 80% of respondents said they would be “somewhat” or “very likely” to look for museum facilitators who had tablets with them.


Very Unlikely Unlikely Somewhat Likely

Very Likely0%

10%

20%

30%

40%

50%

6%10%

44%40%

How Likely are you to Look for Museum Facilitators who had Tablets With Them?

N=50

23

Likelihood of Searching Out Facilitators with Tablets, cont’d.


Twenty six respondents who indicated they were “somewhat” or “very likely” to look for museum facilitators who had tablets provided reasons for their ratings. When responses were categorized, three primary reasons emerged. (Note that some respondents gave more than one reason.)

• Learning: Visitors noted that the PMT-facilitated experience helped them better understand the exhibit or that they learned something new (N=11).

• Fun: Respondents felt the PMT-facilitated interactions they had were fun or good experiences for their group (N=8).

• iPad Technology: Parents commented that, in general, their children were attracted to the Tablets (N=9). One parent further noted that they liked that it provided opportunities for children to spend time with the technology as well as with a facilitator.

Three respondents qualified their ratings by noting that they it would depend on the

Seeing the explanation on an app gave a better understanding of what I was experiencing. Very excited about you guys using the iPad!

They really seemed to enhance the knowledge imparted to my kids! Thanks!

If the tablet doesn’t take away from the activity, then it’s more likely I will look for a person with a tablet.

It made viewing the image very easy to zoom in and out and it focused easily!

I think it was very fun and interesting adding this additional tool into the interaction. My child really enjoys playing with my iPhone, so it was nice to see that on the museum floor.

My four-year-old was having a great time learning while sitting still for a whole ten minutes. Thank you.

I think my children benefitted from spending time both with an educator and with the technology.

It was nice to have the opportunity to use the iPad with my children.

age of the children because they did not think the iPad was particularly useful to younger children. On the other hand, one parent at the Colored Shadows exhibit commented that he was not sure the exhibit would have made sense for his children without the iPad because they were young.

Reasons respondents were unlikely or very unlikely to look for facilitators with tabletsOf the respondents who gave “unlikely” or “very unlikely” ratings, the primary reasons were that they saw the iPad as useful in providing particular types of information, but not something that could be used widely (N=2) and that they did not feel the activity was relevant.

The iPad is good for showing very specific things, but isn’t exactly as handy as a pocket book with that information.

The timer doesn’t show anything relevant.

24

Conclusions


Conclusions


Based on Phase II data, the use of PMTs during visitor-facilitator interactions seems to result in a positive visitor experience.

The majority of visitors (80%) rated their experience as “enjoyable” or “very enjoyable” and 90% agreed that the tablet enhanced their experience with the exhibit itself. Observations noted solid positive affective responses. In over half the interactions, at least one visitor in the group smiled or laughed. Expressions of frustration or confusion were not common.

Overall, visitor groups engaged with both the physical exhibits and virtual apps during their interaction with the facilitators. The PMT-facilitated interactions did not seem to distract visitors significantly from interacting with the physical exhibit. A mix of visitors worked with the physical exhibit, using the PMT, and listened during the visitor-facilitator interaction. The percentage of visitors classified as not engaged in interactions was low. Furthermore, there is evidence that children of all ages and both genders participate in these interactions.

There was also evidence that the PMT-based interactions positively impacted visitors’ engagement with the content. In about half the visitor-facilitator interactions, someone in the visitor group asked a question. Most of the questions documen-ted concerned the content of the exhibit. Visitors were also able to make connections between the exhibit and their prior experiences. In surveys, some respondents also commented on their experience, noting that the PMT-based interactions helped them better understand exhibit content. This suggests that visitors are thinking about the exhibit content during these interactions.

Visitors also made connections between the exhibit and the PMT apps in over half the observed interactions. This, in part, suggests that visitors were engaged with the exhibit—not just the PMT—during their interactions. This finding is another indication that facilitation is successfully integrating the PMT with the physical exhibit. Furthermore, in the majority of interactions, the visitor group was able to switch between the physical and virtual experience without much difficulty.

For discussionSome Phase II findings also raise important questions for the 21-Tech team.

Visitor QuestionsThe 21-Tech team outlined visitors asking questions as an important indicator of successful engagement. Just over half (51%) of interactions, had someone in the visitor group ask a question. The team needs to determine whether this percentage is on target or if, ideally, it should be higher. (If so, the team will want to explore strategies for increasing the percentage of interactions in which visitors ask questions.)

Visitor ChallengesHaving visitors pose a challenge was also identified by the 21-Tech team as an indicator of successful engagement. Only 10% of interactions included a visitor who posed a challenge to his or her group. This may be an acceptable percentage, especially given that in 74% of interactions the facilitator posed a

26

Conclusions, cont’d.


challenge, but it suggests that appropriate targets with accompanying strategies may need to be discussed.

Group dynamics82% of the interactions were classified as adult-, “near-peer”, or child-led (vs. facilitator-led) at some point. While this finding suggests that at some level, visitor groups took ownership of the interaction, more qualitative descriptive data would need to be gathered to fully understand the extent to which interactions are facilitator-led vs. visitor-led.

Switch between PMT and physical exhibitWhile overall the switches between PMTs and physical exhibits by visitors were classified as seamless or fairly smooth, of the four exhibit themes, the switches for Structures and Pitch were more likely to be classified as “fairly smoothly” or “with some difficulty.” This finding suggests a number of questions for discussion including:

• Are more physically engaging exhibits, such as Structures, less appropriate for PMT-assisted facilitation?

• Do facilitators need to use more interactive PMT apps instead of just photos?

• How can teams ensure that apps relate closely to the exhibit so that visitors can switch easily between the two?

Connections to previous knowledgeIn 29% of interactions, someone made a

connection to his or her previousknowledge and/or experience. This type ofconnection is an important indicator of

visitors integrating their experiences into their existing knowledge. Because of its importance, it might be worthwhile to determine the extent to which facilitators should attempt to help visitors make more explicit connections to their own personal experiences and existing knowledge.

LearningThere were indications that visitors

perceived that the PMT-based interactions helped them deepen their understanding of the exhibit content. This finding merits further investigation about the process and ways these interactions facilitate deeper experiences with the content.

27

Extension of the experiencePhase II data contained little evidence that visitors desired to extend their experience beyond their single interaction with the facilitator. This finding, however, contrasts with the visitor survey data that suggested that over 80% of respondents were somewhat or very likely to look for facilitators with tablets in the future. This finding merits further investigation.

References


References

Bobbie, E. (1998). The Practice of Social Research. Albany: Wadsworth Publishing.

Braun, V. & Clarke, V. (2006). Using Thematic Analysis in Psychology. In Qualitative Research in Psychology, 3: 77-101.

Garibay, C. and Schaefer, J. (2011) . 21-Tech Formative Evaluation: Phase I Results Presentation. Available at: http://www.21-tech.org/2012/03/formative-evaluation-phase-i-results/

Miles, M. B. & Huber man, A. M. (1994). Qualitative Data Analysis (2nd ed.). London: Sage Publications.


Appendices


Appendix A: Exhibits and Selected Apps Used During Testing

Colored Shadows Exhibit Theme

Colored Shadows exhibition at Sciencenter

Sciencenter facilitators used Color Uncovered and Bobo Explores Light with their Colored Shadows exhibition.

Colored Shadows exhibition at NYSCI

NYSCI facilitators used Kaleidoscope and Color Mix HDwith their Colored Shadows exhibition.


Appendix A: Exhibits and Selected Apps Used During Testing, cont’d.

Microscopy Exhibit Theme

Microscopy Lab at OMSI

OMSI facilitators used Objective and Anatomy 3D with the Microscopy Lab.

Zoom In exhibition at CMH

CMH facilitators used photos with the Zoom In exhibition.



Pitch Exhibit Theme

Pipes of Pan exhibition at NYSCI

NYSCI facilitators used Tuning Fork and Tuner+ with their Pipes of Pan exhibition.

Slap Organ and Pipes of Pan exhibition at Sciencenter

Sciencenter facilitators used Tuner+ and WI Orchestra with their Slap Organ and Pipes of Pan exhibitions.


OMSI facilitators used Largest Bridges and Giant Timer with the Catenary Arch exhibition.


Structures Exhibit Theme

Giant Arch exhibition at CMH

CMH facilitators used Largest Bridges and Giant Timer with the Giant Arch exhibition.

Catenary Arch exhibition at OMSI


Appendix B: PMT Apps Used During Testing

App Name Platform Testing DateTesting

Institution TypeExhibit Theme

Exhibit/ Component

Testing Institution Rating (1 lowest; 5 highest) Reason for Rating

Use in Future

Testing?Bobo Explores Light (Color Mixing page)

is December 2011

SC Book Colored Shadows

Colored Shadows

5 Great connection to exhibit content! The color mixing page in this app ties in perfectly to the colored shadows exhibit content. The app allows to manipulate and control the light sources and to experiment with directly mixing different colored light so that they can then reference the colors on the wall or in the exhibit.

Yes

Color Mix HD is January 2012

NYSCI Tool Colored Shadows

Colored Shadows

5 Strong connection between exhibit and allows for strong transition. It created various challenges. Also, our museum has an exhibit that shows this same concept, but it is situated far from colored shadows, so helped with the next step of the concept.

Yes

Shadows is January 2012


Colored Shadows

3 Helped younger children understand what a shadow was, but was too simple a concept for older children.

No

Shadow Play is January 2012


Colored Shadows

4 Created a challenge from the app that then transitioned to the exhibit.

Yes

Smarty: Find The Shadow 2

is January 2012


Colored Shadows

3 Helped younger children understand what a shadow was, but was too simple a concept for older children.

No

Objective (onion mitosis)

is December 2012

OMSI Tool Microscopy Microscope with onion slide

4.5 Objective is almost perfect. The only thing that would make this app better would be if there were more slides continuously added to it. The ones they have are amazing.

Yes


Appendix B: PMT Apps Used During Testing, cont’d.


App Name Platform Testing Date Testing Institution TypeExhibit Theme

Exhibit/ Component


Use in Future

Testing?Objective (Blood Slide)

is December 2011

OMSI Tool Microscopy Microscope with Blood Slides and

Immunology Books

4.5 Objective is almost perfect. The only thing that would make this app better would be if there were more slides continuously added to it. The ones they have are amazing.

Yes

Anatomy 3D is December 2011

OMSI Reference Microscopy Microscope with Onion

Slide

2 We used this app for the mitosis video on it. You could access that video other places for free. This app was not worth the price, but since we had it we used it. It also didn't remember where you were in the app, and you would have to find your way back to the video every time you accessed it.

No

Wikipedia is December 2011

OMSI Reference Microscopy Microscope with Blood Slides and

Immunology Books

5 We used this once for reference. It was a request for a visitor. People are comfortable with it, and it's a good first source. It's not always accurate, but it's better than it was five years ago; it will only continue to get better.

Yes

Tuner+ is January and February 2012

SC Tool Pitch Pitch 3 There's a bit of delay between recording sound and finding the note, and it doesn't do that well with a lot of ambient noise.

Yes

Tuning Fork is January and February 2012

SC Tool Pitch Pitch 1 Not appropriate for this content. No

WI Orchestra is January and February 2012

SC Tool Pitch Pitch 1 Might be better with additional instruments, but we never found this that useful.

No

Tone Generator

is January 2012 NYSCI Tool Pitch Pipes of Pan 3 Shows concept, but not as interactive. No

Tuner is January 2012 NYSCI Tool Pitch Pipes of Pan 4 It provided a challenge that we could directly relate to exhibit.

Yes

36

Appendix B: PMT Apps Used During Testing, cont’d.


App Name Platform Testing Date Testing Institution TypeExhibit Theme

Exhibit/ Component


Use in Future

Testing?Tuning Fork is January 2012 NYSCI Tool Pitch Pipes of Pan 3 Shows concept, but not very interactive.

Dry way of presenting concept.No

Animal Sounds

is January and February 2012

SC Tool Sound Pitch 4 While this content is a total departure from the exhibit content, the kids love pretending to be different animals and the recording feature is useful. Kids experimented with different sounds and recorded little songs to play back to family and friends.

Yes

Photos is December 2011

CMH Reference Structures Giant Arch 4 The real-life examples and diagrams are helpful when discussing the different parts of an arch and also show different sized arches as well as arches made of many different materials.

Yes

Giant Timer is December 2011

CMH Tool Structures Giant Arch 4 This app works well as a part of a longer facilitation to extend the experience.

Yes

Largest Bridges

is December 2011

CMH Reference Structures Giant Arch 4 Facilitators can use this app to have visitors locate arches in bridges and discuss the different parts to an arch.

Yes

Giant Timer is January and February 2012

OMSI Tool Structures Giant Arch 3 Works well to create "hype" around building the arch. Additionally, can hand the iPad to a visitor to facilitate.

Yes

Bridges is January and February 2012

OMSI Reference Structures Giant Arch 2 Works well in conjunction with Giant Timer. Hard to facilitate during the build. Best for use before or after. Can also be a method for engaging visitor who is the "third wheel" during a building activity.

37

Documents

21-Tech Formative Evaluation Phase II Brief