Understanding and Analysing Novice Programmer Interactions in a Facebook

Programming Group Mercy Maleko, Margaret Hamilton, Daryl D’Souza and Falk Scholer

School of Computer Science & IT RMIT University, PO Box 2476V

Melbourne Australia, 3001

Abstract —This paper presents an analysis of first year programming student (novice programmer) messages posted on a Facebook Programming Group created to support the learning of programming by increasing novice to novice interactions anywhere, anytime. The captured messages are anonymized and analysed with a view to exploring the extent to which Facebook groups have supported the learning of programming, as well as to identify struggles and challenges faced by learners who use the environments. The results of our analysis show that interactions on Facebook groups can lead to social learning and can encourage engagement with learning. The Facebook groups are used effectively by novices to solve significant problems that would not have been discussed in their respective learning management systems. Our findings highlight some interesting problems faced by novice programmers and include useful learning analytics of novice programmer interactions within these groups.

Keywords— Programming and programming languages; Facebook group;computer-mediated communication; human-computer interface.

I. INTRODUCTION As the advances in mobile technologies have made

current university students more mobile, connected and socially oriented, many educators have exploited these traits in order to increase interactions and communication among students that are necessary for their success. We set up a Facebook Programming Group (FPG) aimed to connect novices, increase interactions and provide a familiar environment for learning of programming. Students used the FPG for discussions, chat, brainstorm and upload programming related materials and links for self-learning. Also, from the instructor point of view, the FPG aimed to facilitate easy capture of early novice misconceptions such as variable initialization, loops, conditionals and all other actions that take place behind the scene such as updating loop variables.

Novices could join and access FPG anywhere, anytime, using their mobile devices including smart phones, feature phones, tablets and laptops with access to the Internet. Novices who did not have mobile devices were able to use desktop computers to access the group. Access to the FPG was purely voluntary and novices were invited to join via email and personal invitation during lectures for their programming classes. Names were modified and the FPG was only open to students enrolled in the same course. No assessments were set in the FPG and discussions and chats were not moderated. Novice

discussions and chats created a vast amount of data that needed to be analysed in order to understand whether or not they had “meaningful” interactions within these informal FPGs. Also, as these interactions occur between peers in a social network, social learning is expected to occur, either as a process where programming concepts are learned while interacting (instantly), or as an outcome where programming concepts are learned later, as a result of these interactions.

This paper reports the analysis of messages posted on three FPGs from three different universities with three distinct modes of teaching. While understanding and analysing the messages, we sought to answer the following question:

To what extent do Facebook Programming Group interactions

(a) bring about social learning? (b) encourage novice engagement with the learning of programming?

II. RELATED WORK Previous studies on difficulties that novices face while

learning to program have suggested a range of interventions to help novices in their learning of programming. These interventions aim to increase interactions or promote learning either between novices and via content accessible through visualization software [1], or between novices and automated tutoring systems [2], or between novices and peers via mentoring [3]. Student interactions are necessary and represent a fundamental process for cognitive development and knowledge acquisition [4].

Student-to-student interactions, in particular, have been shown to have a positive influence on student success. According to Swan [5]. These interactions, when they occur between peers in a social networks, may lead to social learning, as, according to Vygotsky [6], learning occurs at the Zone of Proximal Development. The Zone of Proximal Development is the distance between a student’s ability to solve the task independently and their ability to perform a task with peer assistance [7]. With the help of more knowledgeable peers, novices could solve programming tasks or understand programming concepts that they could not otherwise be able to solve or understand by themselves.

2014 International Conference on Teaching and Learning in Computing and Engineering

978-1-4799-3592-5/14 $31.00 © 2014 IEEE

DOI 10.1109/LaTiCE.2014.28

112

It is through assessing the quality of such interactions one may be able to assert whether or not novices have meaningful interactions that might lead to conceptual learning where the programming concepts learned can be applied to solve different programming tasks. Conceptual learning needs collaborative efforts that require supportive dialogue between learners[8] . It is then appropriate to expect that student interactions on the FPG will have the potential to support and facilitate those efforts and encourage programming concept development. Also, student interactions in the FPG might lead to programming knowledge building and sharing, as noted by Aviv et al [9], that social interactions lead to effective knowledge building. However, user acceptance of the medium is one of the determinant factors of effective knowledge building [8]. Several studies have reported positive student experiences when social networking sites were used to support learning [10]. In our previous research results novices agreed that the FPGs had motivated them to interact more with each other and encouraged them to engage more in the learning of programming [11].

In assessing the quality of student interactions, several studies have suggested different methods such as multi- factor metrics [8, 12], while others have specifically focused on areas such as participation [12, 13], the role of the instructor [14, 15], quantity of student posts [8, 16], quality of discussion [17, 18], feedback [19, 20], guidelines posted by instructor [21] and the discussion board prompts [22, 23].

In assessing the quality of interactions in FPGs, the focus on cognitive processing will help to understand the knowledge and processing level demonstrated by the novice messages, as according to Zhu [24], student’s cognitive engagement during online discussions is critical for constructing new understanding and knowledge. This can be accomplished by the use of the revised version of Bloom’s Taxonomy [25].

TABLE 1: THE COGNITIVE PROCESSES OF THE SIX LEVELS OF RBT

Level Categories Cognitive processes in RBT 1 Remembering Recognising and recalling 2 Understanding Interpreting, exemplifying,

classifying, summarising, inferring, comparing and explaining

3 Applying Executing and implementing 4 Analysing Differentiating, organising and

attributing 5 Evaluating Checking and critiquing 6 Creating Generating, Planning, and

Producing

The revised Bloom’s Taxonomy (RBT) describes six levels of student learning within the cognitive dimension, which are remembering, understanding, applying, analysing, evaluating and creating as listed in Table 1. These levels show the order of cognitive processes a student may be engaged in [26].

Using assessment methods such as above, this study specifically assess the quality of messages from FPGs with a focus on cognitive engagement levels, student participations, feedback based on the number of resolved

topics, quality of posts and percentage of code containing posts.

III. METHODOLOGY For this study we examine messages posted by novices

in three different FPGs. Each FPG was accessed by programming novices who, for at least ten weeks, were involved in discussions, chats, brainstorms and uploads of links and materials related to the Java programming course in which they were enrolled. As mentioned earlier, our interest is to understand whether or not interactions within FPGs enhance novice learning of programming, in terms of interactions leading to social learning and increased engagement.

A. Context and Participants We introduced three separate FPGs to programming

novices enrolled in introductory programming courses in three different universities in the second-half of the year 2012. Participation in the study was purely voluntary, and no assessments were carried out on the user posts or interactions. The cohort details are as follows.

Cohort A: RMIT University, Australia, on-campus students studying Programming 1 (Java language), a traditional course offered in face-to-face mode with lectures, tutorials and laboratories. They also had access a Blackboard Discussion Forum and mentoring sessions. For convenience, we have named the Facebook programming group for this cohort as FPG-A.

Cohort B: University of Dar es Salaam, Tanzania, on-campus students studying Programming in Java, a course offered in a blended learning mode with all resources available online, and facilitators delivering tutorials face-to-face. These students used Moodle as their learning management system. The Facebook programming group for this Cohort is named FPG-B.

Cohort C: Open Universities of Australia, fully online students studying introduction to Programming (Java Language), with materials provided in Blackboard and a weekly chat session using Elluminate. The Facebook programming group for this Cohort is named FPG-C.

Cohort A had 150 students enrolled and 110 students joined the FPG-A, representing 73% of the cohort. Cohort B had 120 students enrolled and 101 (84%) students joined the FPG-B, while Cohort C had 350 students enrolled and 295 (84%) students joined the FPG-C. Each cohort was treated independently and separately invited to join their designated FPG.

B. Data Collection The FPGs were introduced to each cohort during the

first week of teaching. Students were invited to join by sending a request to the group administrator. Once accepted, they had to read and sign an online consent form to allow researchers to use posts to be generated for research purposes. After the final examinations, for each cohort, we extracted generated messages for analysis. Messages may be topics or replies. If a student posts an item to initiate a new discussion thread, we denote such a post, as a topic. A topic is therefore the opening message in a thread. Students may post responses to a topic, which we denote as replies. Therefore, a thread has one topic

113

and zero or more replies (Figure 1). Typically, in Facebook, contributors are represented by a photo and a name. We have replaced those by a user number for our analysis, to conceal the actual identities of users.

C. Data Analysis Our analysis focuses on cognitive engagement levels,

student participations, feedback based on the number of resolved topics, quality of posts and code containing posts.

The quantitative analysis of the messages involve counting the number of active users who posted a topic or reply to a topic, or both, the number of topics or threads posted for the entire duration, the number of replies, and the mean number of replies in each FPG.

Figure 1: One example of a topic with three replies labelled as one thread for this research and counted as four messages posted. A:

Beginning of a thread. B: A thread with 4 messages--one topic and three replies.

To identify messages posted that contain computer code, we use “;”, “{” and “}” as keywords. According to Lahtinen et al. [27], some misconceptions that novices experience relate to variable initialization, loops and conditionals. These topics are normally covered in the beginning of the programming course. We extract novice messages up to week 5 where we expect students would have discussed and posted statements reflecting these topics. Using keywords such as “if(”, ”for(”, “switch(” and “while(”, we further identify programming statements contained in the computer codes (Table 2).

To assess cognitive engagement levels we use revised Bloom’s Taxonomy levels (Table 1). In each level, there are known keywords that are used to guide how questions may be formulated in order to engage students in each level, from low to high orders of cognitive processes. These keywords are listed in Appendix A.

Using question words such as “what”, “when”, “how”, “why”, “where”, “when”, “question”, “error”, “help” and “?”, (Table 2), we are able to extract all questions that novices asked one another. From these questions, we extract keywords that novices used to set the questions and group them according to the RBT cognitive levels.

TABLE 2: KEYWORDS USED FOR SEARCHING QUESTIONS AND CODE

Attributes Keywords or symbols

Message with a Question

“what”, “when”, “how”, “why”, “where”, “when”, “question”, “error”, “help”, “debug” and “?”

Message with code “;” or “{” and “}” if-statement “if (” for-loop “for (” switch-statement “switch (” while-statement “while (”

Novices also used words like “thanks”, “cheers”,

“okay”, “ok”, “np”, “no problem” and “thank you”, when a question was responded or for which some helpful guidance was provided as in Figure 2. We use such words as cues for resolutions of questions posted, and subsequently obtain the number of resolved topics, and, in turn, establish the percentages of resolved topics.

IV. RESULTS In this section we present various analytics of learner

behaviours from the data extracted from FPGs.

Overall, a total of 6,081 messages were posted on the three FPGs, comprising 708 topics (threads) and 5,373 replies within these topics. FPG-A had 105 members with 87 (83%) active members, FPG-B had 101 members with 75 (75%) active members and FPG-C had 280 members, 266 (93%) were active. The number of active members exceeding 70% in all the FPGs. FPG-B had novices taught in blended mode, so we expected them to have more active members than those in FPG-A, a face-to-face group. However, novices from this group had reported problems with internet access within the University, and very few had mobile devices with internet access support [11].

The following discussions look at the quantitative assessment of posts made to the FPGs. Table 3 presents the number of topics (new threads), number of replies made over all topics and number of questions over all topics that were asked in each environment.

TABLE 3: QUANTITATIVE INFORMATION OF ATTRIBUTES ON FPGS

FPGs A B C Number of active members 87

(83%) 75 (75%)

266 (93%)

Total number of topics 60 135 513 Total number of replies 500 952 3,921 Total number of questions

(topic or reply) 101 220 579

Total Messages 560 1,087 4,434 Students from FPG-A, FPG-B and FPG-C posted 60,

135 and 513 new threads, respectively. In the FPG-A new threads received 500 replies, an arithmetic mean of 8.3 replies per thread (Table 4). FPG-B had 952 replies, a mean of 7 replies per thread. The new threads for FPG-C received 3,921 replies with a mean of 7.6 replies per thread. The number of replies per thread shows the length of the thread, this means the minimum length of a thread

A

B

114

was 7, across the three environments. The mean message per user is also shown in Table 4.

On average, across the three FPGs, each novice posted a minimum of 6 messages (topic or reply). The median messages per user in FPG-A, FPG-B and FPG-C were 6, 7 and 8 respectively. These medians showed that at least 50% of users in FPG-A, FPG-B and FPG-C had posted 6, 7 and 9 messages respectively. The median replies per topic showed that more than 50% of topic had 5 messages or more in FPG-A, 6 messages or more in FPG-B and FPG-C.

TABLE 4: MESSAGE ANALYTICS.

FPGs A B C Minimum messages /active user 1 1 1 Mean messages per user 6.4 14.5 16.7 Median messages per user 6 7 9 Maximum messages per user 23 59 425 Minimum replies per topic 0 0 0 Mean replies per topic 8.3 7.1 7.6 Median replies per topic 5 6 6 Maximum replies per topic 47 31 50

In terms of user participations, we note from FPG-A,

FPG-B and FPG-C that 49%, 79% and 62% of students participated in posting replies while 28%, 48% and 46% of students participated in posting topics, respectively. Only 25%, 44% and 44% of students from the three FPGs respectively, posted both topics and replies. These results show that students participated more in posting replies that topics (Table 5).

TABLE 5: USER ANALYTICS

FPGs A B C Number of users who posted a topic

24 (28%)

36 (48%)

122 (46%)

Number of users who posted a reply

43 (49%)

59 (79%)

166 (62%)

Number of users who posted a topic and a reply

22 (25%)

33 (44%)

118 (44%)

Number of users who posted code

21 (24%)

33 (44%)

66 (25%)

Some students preferred to post computer code than

explaining the code in words (Figure 2). It seems that when a student was unable to proceed while attempting to compile or run a program, it was easy to post the entire program and ask for help to fix the bug. The percentages of students who posted computer code from FPG-A, FPG-B and FPG-C were 24%, 44% and 25%, respectively.

The percentages of messages posted across the three environments that had code in them were 6.25% for FPG-A, 9.75% for FPG-B and 4.5% for FPG-C. Further analysis of the code reveals the dominance of “if” statements followed by code involving “for” loops, then, “while” loops, and lastly “switch” statements (Table 6), where they appear mostly between week 3 and week 5.

TABLE 6: CODE ANALYTICS

FPGs A B C

Number of messages/posts containing code

35 106 198

Percentage of messages containing code 6.25% 9.7% 4.5%

Number of if-statements 11 98 65 Number of switch-statements 0 8 1 Number of for-loops 5 56 49 Number of while-loops 1 9 13

Figure 2: Example of code posting and resolving the topic by the

replies.

In order to explore how cognitively engaged novices were in the FPGs while answering questions asked by others, we identify the keywords used by novice when they post their questions and group those questions into different levels of RBT. The results across the three environments show that novices asked more questions at the remembering level, followed by the evaluating level and then the creating level Figure 3. Very few questions are found that had keywords matching the keywords present in other levels (understanding, applying and analysing). The RBT cognitive levels are considered as a continuum, this means, a novice performing at a higher level is expected to have performed at lower level of the cognitive hierarchy [28]. Since some keywords in Appendix A occur at different levels, for example, the keyword “choose” is at level 1 as well as at levels 5 and 6. In this case it is reasonable to remove all repeating keywords from lower level and retain the upper most containing levels. In this case for example, the keyword “choose” is counted at level 6.

115

Figure 3: Levels of questions based on Revised Bloom Taxonomy

(RBT)

The results of grouping novice questions based on the RBT keywords after removing the repeating keywords still show that a large percentage of questions come under the remembering level and the remaining questions are at the evaluating and creating levels. However, questions that originally fall in to the applying level (level 3) almost disappear. The remembering level questions suggest that novices were asking questions that appropriately evaluated their comprehension, diagnosed their strengths and weaknesses in programming, and also provided a general review of their programming concepts. Although the questions that fall into higher levels (level 5 and 6) indicate that novices asked questions that encouraged deep and critical thinking, and encouraged further information seeking [29], we are not certain that novices knew that they were asking questions at these levels. Figure 4 shows an example of a question that was categorised at level 5-evaluating. The question was “Could someone please explain how one would do that?”

Figure 4. An example of a question classified at level 6.

Table 7 presents response times (minimum, mean and maximum), that is, the time taken between posting a topic

and receiving the last reply to that topic. It also presents the response times for resolved threads (topics that represent a resolution or a satisfactory outcome towards a resolution was reached within FPG), the mean time for the first response in the resolved thread and the Percentage of resolved topics/threads.

From Table 7, the mean response time are affected by the maximum response times being above 10 days in most cases. This is caused by either having a long unresolved thread, whereby a student each day posts a reply to the thread or a prolonged discussion even after a resolution to a topic has been reached. Also, some students do not login frequently or have set their notifications off, so that when they login, they may decide to reply to an old post, thereby increasing the maximum response time. We also note that some students asked new questions within the thread and, this too increases the response time with respect to the topic.

Nevertheless, it is encouraging to find that the mean time elapsed between posting a topic and receiving the first response for the resolved topics is less than 2 hours (0.73, 1.9, 1.55 hours) respectively. The percentages of resolved threads/topics are more than 50%.

TABLE 7: RESPONSE TIME ANALYTICS

FPGs A B C

Response time with respect to the topic Minimum 10 secs 27 secs 19.5 secs Mean 9 hrs 46 hrs 15 hrs Maximum 18 days 29 days 26 days

Response time for resolved threads Minimum 31 secs 27 secs 26 secs Mean 6 hrs 30 hrs 13 hrs Maximum 3.7 days 13 days 27 days Mean time for the first response for the resolved thread

0.73 hrs 1.9 hrs 1.55 hrs

Percentage of resolved topics

52% 56% 61%

V. DISCUSSION This paper reports the results of analysis of novice

messages posted on FPGs. The aim of these FPGs was to increase interactions among novices so as to enhance their learning of programming. The paper set out to explore the extent to which these informal interactions (i) brought about social learning, and (ii) encouraged novice engagement with the learning of programming. We discuss each of the foregoing points in the following subsections.

A. Social Learning Social learning is learning that occurs between peers

while interacting on social networks or learning that occurs as an outcome of such interactions [30]. We aimed to study whether the student interactions in the FPGs were sufficient to bring about social learning.

According to Kay [8, 31] evidence of social learning in a discussion environment is based on the length of a thread, number of messages posted per user, primary focus

116

of the posted messages whether it aligns with the topic being discussed, and whether a discussion thread was resolved. In a discussion thread, social learning happens after the 5th message [31]. In our FPGs, many threads exceeded 6 messages (see Table 4). Medians of replies per topic were 5, 6 and 6, and the maximum number of replies per topic was 47, 31 and 50 for FPG-A, FPG-B and FPG-C respectively. Also, students posted more replies than topics. This means that novices were following the discussions closely and were willing to answer others’ queries.

The means of messages per user exceeded 6 and the medians were from 6 to 9 across the three FPGs. The percentages of unique users who posted a reply were above 49% across all three FPGs. The percentages of resolved topics across the three FPGs ranged between 52% and 61%. The results of extracting keywords from the questions posted by novices showed that some novices asked questions that encouraged deep and critical thinking. These questions were in levels 5 and 6. More than half of the questions posted from each cohort belonged to the cognitive processing level (level 1), that is appropriate for evaluating comprehension, and for diagnosing strengths and weaknesses in programming (see Figure 3). This provided evidence that novices were not only engaged in “social activity” through social interactions, but they were active and cooperatively trying to understand some programming concepts expressed as computer code (Figure 2) or as normal text message (Figure 5). Hence, according to Kay [8, 31] and Vila and Statkus [32], social learning had occurred in the FPGs.

Figure 5: Example of programming concept posted as normal text.

B. Engagement with Learning of Programming Student engagement can be defined as a combination

of student time on tasks and their willingness to participate in activities [33]. Novice engagement with learning of programming was assessed by looking at the percentages of students who actively participated in FPGs, the number

of messages posted that involved code, the response time, and the percentages of students who posted messages with code. According to Coates [34], student engagement is largely influenced by activities that foster collaboration within the environment and the sense of a learning community.

In this study, novices were voluntarily joining the FPGs for discussions, chats, brainstorming and sometimes they uploaded links related to programming that they felt were worth sharing. There were no specific activities given to novices by instructors, therefore, their engagement was solely influenced by their own will to contribute to or to make use of the community formed.

The fact that 83% members in FPG-A, 75% members in FPG-B and 93% members in FPG-C, were actively interacting through posting topics and replies, provides convincing evidence that novices were engaged with overall learning of programming.

Although in terms of percentages of messages posted across the three FPGs containing computer code were low, as we were searching for keywords such as “;”,“{” and “}” (Table 6), there were messages that expressed computer code in words (Figure 5). Depending on the content of the message, a message with computer code might involve use of cognitive processes more than a message without code, as, before a reply is posted, novices need to understand what the code is about. This meant that, they had to identify some construct in those pieces of code, remember the implementation of the subject area concept, and recall any materials that were explicitly covered during the lecture. Novices had to move through different levels of the cognitive domain before they could reply to a message with code. The high percentages (see Table 5), of users who posted code and who replied to the messages also provides an indication that novices were actively engaging with their learning of programming.

Student participation is an important indicator of engagement and student success [33]. Beer et al used the number of clicks students made to the learning management system as an indication of engagement. In our study, 79% of unique users posted replies in FPG-B, 62% and 49% of unique users posted replies in FPG-C and FPG-A, respectively. Also 44% of unique users posted topic and replies from FPG-B, 44% and 25% of unique users posted topic and replies from FPG-C and FPG-A respectively. And, on average these students each posted 6, 14 and 17 messages in FPG-A, FPG-B and FPG-C respectively. So, according to Beer at al [33], the number of messages students posted to the FPGs was an indicator of their engagement with learning.

The mean response times were also observed as evidence of engagement, especially with the fact that there was no instructor involvement in the discussions or chats. The mean times of the first response for the threads that were resolved ranged from 44 minutes to 2 hours, however, some topics received responses in seconds and medians of the time between two replies were 2.4, 36 and 4.8 minutes for FPG-A, FPG-B and FPG-C respectively. These mean times and medians also indicate how willing and active novices were in the FPGs, and, provide, an indication that a sense of learning community was developing overtime.

117

VI. CONCLUSION This paper has analysed messages posted in the FPGs

for three different groups of programming novices. The aim of these FPGs were to increase novice interactions while learning programming, and to help instructors identify misunderstandings and misconceptions early enough so as to be able to provide the necessary teaching and learning support and to reduce perceived difficulties in learning. FPGs provided social environments, wherein novices could freely and easily interact and upload programming related materials.

The FPGs provided environments for interactions as novices were able to use them anytime, anywhere. Some novices were able to fully utilize the opportunities presented to them. Many discussion threads exceeded 6 messages; the mean posts per user exceeded 6; at least 49% of users from each FPG posted replies to different topics; programming-code related questions were discussed; and at least 50% of all topics from each FPG were resolved. These results present evidence that FPGs brought about social learning, as novices were focusing on activities related to their learning, and were actively interacting in order to learn from each other. Moreover, having topics resolved has showed that interactions in FPGs can and do support novices in their learning of programming.

The results also showed that interactions in FPGs have encouraged engagement with the learning of programming through active participation of novices. Each FPG had at least 75% of novices voluntarily discussing, chatting, brainstorming and sometimes uploading links related to programming whenever they felt it was worth of being shared. As there were no assessed tasks uploaded into the FPGs, novice engagement with learning of programming was mainly influenced by the experiences they had within the FPGs and the sense of contribution to the gradually-forming learning community.

In terms of learning experiences, some questions that novices posted encouraged critical and deep thinking as they fell under levels 5 and 6. This was a result of considering that the RBT cognitive levels are continuum, that all repeating keywords were counted at the upper most levels. Further analysis will be done to group the questions while keeping repeating keywords at the lower levels and compare the findings. Again, as we saw in the results section, the percentages of resolved topics, the percentages of messages related to learning of programming, the mean response times, are all convincing evidence that novices were actively interacting and had meaningful interactions [35]. Novices learning experiences were enhanced with the support of FPGs.

From the analysis of novice messages, the following theoretical implication about FPGs and their support to the learning of programming can be drawn. First, FPGs provide alternative environments that encourage novices to engage with the learning of programming. Second, FPGs provide interactive environments that lead to social learning and interactions that are meaningful for the learning of programming. An educational implication that can be drawn from these results is that FPGs were used effectively by novices to solve significant problems that would not have been discussed in their respective learning management systems, or in other contexts provided for

learning. Moreover, FPGs mirror student brainstorming activities and, could simultaneously, better inform instructors about pedagogical improvements in their pursuit of better teaching and learning strategies and ideas. Hence the FPG can well be used to complement the current learning management systems and provide a learner-learner interaction environment that can further support learning of programming.

As part of our further work, we plan to upload programming exercises and past exam questions to the FPGs so that novices can engage around these exercises. We also plan to qualitatively analyse messages from the FPGs in order unfold the information captured in terms of programming knowledge. These approaches will contribute to the development of a social learning environment, which will integrate and deliver more meaningful learning analytics.

VII. REFERENCES

[1] N. Hashim and S. Salam, "Integration of Visualization Techniques and Completion Strategy to Improve Learning in Computer Programming," 2009, pp. 665-669.

[2] C. J. Butz, et al., "A web-based intelligent tutoring system for computer programming," in IEEE/WIC/ACM International Conference on Web Intelligence (WI’04), 2004, pp. 159-165.

[3] D. D'Souza, et al., "Transforming learning of programming: a mentoring project," in 10th Australasian Computing Education Conference, Wollongong, Australia, 2008.

[4] L. Song and W. S. McNary, "Understanding Students' Online Interactions: Analysis of Discussion Board Postings," Journal of Interactive Online Learning, vol. 10, 2011.

[5] K. Swan, "Building learning communities in online courses: The importance of interaction," Education, Communication & Information, vol. 2, pp. 23-49, 2002.

[6] L. S. Vygotsky, Mind in Society. Cambridge, MA: Harvard University Press, 1978.

[7] A. Kozulin, Vygotsky's educational theory in cultural context: Cambridge Univ Pr, 2003.

[8] R. H. Kay, "Developing a comprehensive metric for assessing discussion board effectiveness," British Journal of Educational Technology, vol. 37, pp. 761-783, 2006:a.

[9] R. Aviv, et al., "Network analysis of knowledge construction in asynchronous learning networks," Journal of Asynchronous Learning Networks, vol. 7, pp. 1-23, 2003.

[10] K. Gray, et al., "Medical students' use of Facebook to support learning: Insights from four case studies," Medical Teacher, vol. 32, pp. 971-976, 2010.

[11] M. Maleko, et al., "Novices’ Perceptions and Experiences of a Mobile Social Learning Environment for Learning of Programming," in Innovation and Technology in Computer Science Education (ITiCSE) Conference, Haifa, Israel, 2012:a.

[12] V. P. Dennen, "Pedagogical lurking: Student engagement in non-posting discussion behavior," Computers in Human Behavior, vol. 24, pp. 1624-1633, 2008.

[13] S. Gulati, "Compulsory participation in online discussions: is this constructivism or normalisation of learning?," Innovations in education and teaching international, vol. 45, pp. 183-192, 2008.

[14] P. J. Shea, et al., "A follow-up investigation of “teaching presence” in the SUNY Learning Network," Journal of Asynchronous Learning Networks, vol. 7, pp. 61-80, 2003.

[15] M. Mazzolini and S. Maddison, "Sage, guide or ghost? The effect of instructor intervention on student participation in online discussion forums," Computers & Education, vol. 40, pp. 237-253, 2003.

[16] D. Nisbet, "Measuring the quantity and quality of online discussion group interaction," Journal of eLiteracy, vol. 1, pp. 122-139, 2004.

118

[17] K. A. Meyer, "The ebb and flow of online discussions: What Bloom can tell us about our students’ conversations," Journal of Asynchronous Learning Networks, vol. 9, pp. 53-63, 2005.

[18] M. D. Roblyer and W. R. Wiencke, "Design and use of a rubric to assess and encourage interactive qualities in distance courses," The American Journal of Distance Education, vol. 17, pp. 77-98, 2003.

[19] A. P. Rovai, "A constructivist approach to online college learning," The Internet and Higher Education, vol. 7, pp. 79-93, 2004.

[20] J. B. Pena-Shaff and C. Nicholls, "Analyzing student interactions and meaning construction in computer bulletin board discussions," Computers & Education, vol. 42, pp. 243-265, 2004.

[21] P. K. Gilbert and N. Dabbagh, "How to structure online discussions for meaningful discourse: A case study," British Journal of Educational Technology, vol. 36, pp. 5-18, 2005.

[22] M. M. Christopher, et al., "Raising the bar: Encouraging high level thinking in online discussion forums," Roeper Review, vol. 26, pp. 166-171, 2004.

[23] K. A. Meyer, "The method (and madness) of evaluating online discussions," Journal of Asynchronous Learning Networks, vol. 10, pp. 83-97, 2006.

[24] E. Zhu, "Interaction and cognitive engagement: An analysis of four asynchronous online discussions," Instructional Science, vol. 34, pp. 451-480, 2006.

[25] L. W. Anderson, et al., A taxonomy for learning, teaching and assessing:a revision of bloom's taxonomy of educational objectives, Eds ed. New York: Addison Wesley Longman, 2001.

[26] M. Crowe and P. Stanford, "Questioning for quality," Educators and Changing Times, vol. 76, pp. 36-42, 2010.

[27] E. Lahtinen, et al., "A Study of the Difficulties of Novice Programmers"," presented at the 10th annual SIGCSE on Innovation and Technology in Computer Science Education, 2005.

[28] U. Fuller, et al., "Developing a computer science-specific learning taxonomy," ACM SIGCSE Bulletin, vol. 39, pp. 152-170, 2007.

[29] D. Tarlinton. (2003, April , 2012). Bloom's Revised Taxonomy. Available: http://www. kurwongbss.eq.edu.au/thinking/Bloom/bloomspres.ppt

[30] M. S. Reed, et al., "What is social learning? ," Ecology and Society vol. 15, 2010.

[31] R. H. Kay, "Developing a Metric for Evaluating Discussion Boards," in World Conference on E-Learning in Corporate, Government, Healthcare, and Higher Education, Chesapeake, VA:AACE, 2004.

[32] R. I. Roig Vila and S. E. Rosales Statkus, "Assessing participation in online discussion forums: a proposal for multidimensional analysis," Pixel-Bit: Revista de medios y educación (Journal of Media and Education), pp. 137-149, 2012.

[33] C. Beer, et al., "Indicators of engagement," in Ascilite Sydney, 2010.

[34] H. Coates, "A model of online and general campus-based student engagement," Assessment & Evaluation in Higher Education, vol. 32, pp. 121-141, 2007.

[35] K. K. Seo, "Utilizing peer moderating in online discussions: Addressing the controversy between teacher moderation and nonmoderation," The American Journal of Distance Education, vol. 21, pp. 21-36, 2007.

Appendix A: Levels of Revised Bloom Taxonomy and associated keywords

Level Description Keywords Level 1: Remembering Exhibit memory of previously

learned material by recalling facts, terms, basic concepts, and answers

Choose, define, find, how, label, list, match, name, omit, recall, relate, select, show, spell, tell, what, when, where, which, who, why

Level 2: Understanding Demonstrate understanding of facts and ideas by organizing, comparing, translating, interpreting, giving descriptions, and stating main ideas.

Classify, compare, contrast, demonstrate, explain, extend, illustrate, infer, interpret, outline, relate, rephrase, show, summarize, translate

Level 3: Applying Solve problems to new situations by applying acquired knowledge, facts, techniques and rules in a different way.

Apply, build, choose, construct, develop, experiment, with, identify, interview, make use of, model, organize, plan, select, solve, utilize

Level 4: Analysing Examine and break information into parts by identifying motives or causes. Make inferences and find evidence to support generalizations.

Analyse, assume, categorize, classify, compare, conclusion, contrast, discover, dissect, distinguish, divide, examine, function, inference, inspect, list, motive, relationships, simplify, survey, take part in, test for, theme

Level 5: Evaluating Present and defend opinions by making judgments about information, validity of ideas, or quality of work based on a set of criteria.

Agree, appraise, assess, award, choose, compare, conclude, criteria, criticize, decide, deduct, defend, determine, disprove, dispute, estimate, evaluate, explain, importance, influence, interpret, judge, justify, mark, measure, opinion, perceive, prioritize, prove, rate, recommend, rule on, select, support, value

Level 6: Creating Compile information together in a different way by combining elements in a new pattern or proposing alternative solutions.

Adapt, build, change, choose, combine, compile, compose, construct, create, delete, design, develop, discuss, elaborate, estimate, formulate, happen, imagine, improve, invent, make up, maximize, minimize, modify, original, originate, plan, predict, propose, solution, solve, suppose, test, theory

Source: EDUPRESS EP 729 – www.edupressinc.com

119