Idea Garden Tuning and Generalization

Faezeh Bahmani School of Electrical Engineering and Computer Science

Oregon State University Corvallis, U.S.A.

bahmanif@onid.orst.edu

I. INTRODUCTION End-user programmers encounter many barriers as they try

to accomplish their tasks. To help solve this problem, Researchers have developed solutions to help end users overcome some of these barriers, such as arcane language constructs that are difficult to learn, syntax rules that are hard to memorize, and finding suitable example programs that other end-users have written [2]. Approaches developed previously to tackle these issues include programming by demonstration [9], natural programming methodology [10], and mashup environments. In spite of such efforts, empirical experiments show that many end-user programmers still struggle. One possible explanation for this phenomenon is that most approaches try to remove these barriers rather than trying to empower users through increasing end users’ problem-solving skills, creativity, and design thinking. To fill this gap, recently, Cao et al. introduced the Idea Garden approach [1, 2]. Idea Garden is not intended to solve problems automatically on its own or to replace online tutorials. Instead, it supplements online tutorials by suggesting problem solving strategies, programming domain concepts, and patterns based on the users' actual tasks.

II. RESEARCH QUESTIONS Since the Idea Garden approach has been investigated only

in CoScripter, my main research goal is to investigate whether the Idea Garden approach can be generalized across a variety of end-user programming environments and if yes, to what extent , and what adaptions have to be made based on its host environment.

The following sub-questions arise from this main goal.

RQ1 (Barriers): To what extent do end-users’ programming barriers found in previous settings generalize? Cao et al. [4], Ko et al. [7], and Kissinger et al. [6] have studied barriers in end-user programming environments. Although there is some overlap among these sets of barriers, the literature has not reported how the barriers generalize across environments.

RQ2 (Triggers): What is the appropriate trigger for suggesting problem-solving approaches or concepts to end users? Building on the Surprise-Explain-Reward strategy [11], Cao et al. decided on triggers for showing suggestions in CoScripter and implemented them [4] but did not evaluate these triggers. As part of Idea Garden generalization, I would

like to devise a principled, more general approach to triggering Idea Garden suggestions in end-user programming environments, with the goal of presenting information at a time that the user can benefit from it while being minimally disrupted.

RQ3 (Content): How should a suggestion balance “correct” parts against (intentionally) “flawed” parts? Some parts of Idea Garden features are intentionally “flawed” to avoid providing working solutions. Instead, we aim to arouse end users’ curiosity and keep them intellectually engaged. At the same time, the Idea Garden approach’s primary intention is educating end users and this implies that it has to contain correct information. It also has to be relevant to the users’ task according to the “Minimalist Learning Theory” [5]. The challenge is deciding on parts that should become flawed, ways and extent of making them flawed.

RQ4 (Problem-solving strategies): What problem-solving strategies are especially helpful to end-user programmers “stuck” on how to make their programs work? The existing literature talks about several problem-solving strategies such as “Divide and Conquer”, “Sleep on it”, “Working backward”, “Analogy” and “Generalization”. However, in [3] we saw that some users had problems understanding some of the strategies. Thus, further work is needed in order to find strategies of helping end users in each circumstance.

III. RESEARCH APPROACH I am currently working on RQ1. As part of this, I chose a

game programming environment for end users called Gidget [8], a programming game in which the user corrects the faulty code of a broken robot so that it completes its missions (see Fig. 1). I am currently conducting a one-on-one think-aloud study that began this spring. In the study, I have been observing teenagers playing Gidget, collecting audio video recordings and observation notes. I am in the process of qualitative coding using code sets derived from [4, 6, 7].

Studying barriers in Gidget is the first step toward generalizing the Idea Garden approach to this environment. The next step will then be to design features that address these barriers. There are at least two factors that make this step challenging. First, Gidget is a gaming environment and so the design of features must conform to the design practices of gaming environments. For example, our team has started implementing a “telephone call” metaphor as a means of communication between the player and a character who

Identify applicable sponsor/s here. If no sponsors, delete this text box (sponsors).

2013 IEEE Symposium on Visual Languages and Human-Centric Computing

978-1-4799-0369-6/13/$31.00 ©2013 IEEE 175

provides information in the style of the Idea Garden. The telephone call metaphor is a common way of offering help to players in many games.

Second, Gidget’s target population is teenagers which is different from the adult populations originally studied in order to derive the Garden approach. There may be differences in the learning approach and computer problem-solving strategies of these two populations. Consequently, the current Idea Garden approach may need to adjust to age-related end-user programming differences.

IV. HOPED-FOR BENEFITS FROM THE CONSORTIUM I hope to receive feedback, expertise and insight on:

1. Finding appropriate methodologies for investigating the generalization of the Idea Garden approach: The Idea Garden approach was originally investigated in CoScripter. Currently, I am investigating Idea Garden features within Gidget. However, these two investigations may not provide enough evidence for the generalization of the Idea Garden approach.

Resolving this question matters because, if we can feasibly show that the Idea Garden approach can be generalized, we will then be able to create a stand-alone plugin that can be applied to a variety of end-user programming environments with the ultimate goal of seeing fewer instances of end users feeling “stuck,” saying: “I am not sure what to do next.”

2. How to investigate principle for designing Idea Garden triggers, the appropriate “imperfection” of suggestions and the appropriate problem-solving strategies.

If we can identify the proper triggers, proper imperfection, and the proper problem-solving strategies, then we can further

refine the Idea Garden approach, and therefore more effectively help end users.

REFERENCES [1] J. Cao, Helping End-User Programmers Help Themselves – The Idea

Garden Approach. PhD Dissertation, Oregon State University, April 2013.

[2] J. Cao, S. Fleming and M. Burnett, An exploration of design opportunities for ‘gardening’ end-user programmers’ ideas, IEEE VL/HCC, 35–42, 2011.

[3] J. Cao, I. Kwan, F. Bahmani, M. Burnett, J. Jordahl, A. Horvath, S. Fleming, and S. Yang, End-User programmers in Trouble: can the Idea Garden help them to help themselves?, IEEE VL/HCC, 2013.

[4] J. Cao, I. Kwan, R. White, S. Fleming, M. Burnett and C. Scaffidi, Fom barriers to learning in the Idea Garden: An empirical study, IEEE VL/HCC, 59-66, 2012.

[5] J. Carroll, M. Rosson, Paradox of the Active User, in Interfacing Thought: Cognitive Aspects of Human-Computer Interaction, MIT Press , 80-111, 1987.

[6] C. Kissinger, M. Burnett, S. Stumpf, N. Subrahmaniyan, L. Beckwith, S. Yang, and M. B. Rosson, Supporting end user debugging: What do users want to know?, ACM AVI, 135-142, 2006.

[7] A. Ko, B. Myers, and H. Aung, Six learning barriers in end user programming systems, IEEE. VL/HCC, 199-206, 2004.

[8] M. Lee and A. Ko, Personifying programming tool feedback improves novice programmers’ learning, ACM ICER, 109-116, 2011.

[9] H. Lieberman, Your Wish Is My Command: Programming By Example. Morgan Kaufmann. 2001.

[10] J. Pane, B. Myers, More natural programming languages and environments, in End User Development (Henry Lieberman et al., eds.), Springer, 31-50 , 2006.

[11] A. Wilson, M. Burnett, L. Beckwith, O. Granatir, L. Casburn, C. Cook, M. Durham, and G. Rothermel.Harnessing curiosity to increase correctness in end-user programming”, ACM CHI, 305–312, 2003.

Fig. 1. Gidget, a debugging game to teach programming. The player is given a program on the left-hand side to debug. Commands in the program affecthow Gidget interacts with the world’s objects in the center.

176