Upload
wellington-pinheiro
View
138
Download
2
Tags:
Embed Size (px)
DESCRIPTION
Presentation of my master degree project in the International Conference on Computer Aided Blended Learning, November 2008.
Citation preview
Introduction Background The solution Conclusions
Programming Learning: a Hierarchical Model BasedDiagnosis Approach
Wellington R. Pinheiro, Leliane N. de Barros, Karina V. Delgado
Department of Computer ScienceInstitute of Mathematics and Statistics - University of Sao Paulo
{wrp,leliane,kvd}@ime.usp.br
November 5th, 2008
ICBL- November 5th, 2008 Slide: 1/22
Introduction Background The solution Conclusions
Contents
1 Introduction
2 Background
3 The solution
4 Conclusions
ICBL- November 5th, 2008 Slide: 2/22
Introduction Background The solution Conclusions
General Concepts
Intelligent Tutoring System for Programming
Intelligent Tutoring System [Wenger, 1987] (ITS) is an e-learningtool that uses Artificial Intelligence techniques.
An ITS for programming should be able to:
propose programming problems based on the student knowledgelevel;
detect possible faults in the student program solution (i.e., todiagnose the student’s program);
communicate the detected faults in such a way that the studentcan learn with this experience;
construct the student model (representation of the student’sknowledge) based on the results of the tutor diagnosis.
ICBL- November 5th, 2008 Slide: 3/22
Introduction Background The solution Conclusions
General Concepts
Intelligent Tutoring System for Programming
Intelligent Tutoring System [Wenger, 1987] (ITS) is an e-learningtool that uses Artificial Intelligence techniques.
An ITS for programming should be able to:
propose programming problems based on the student knowledgelevel;
detect possible faults in the student program solution (i.e., todiagnose the student’s program);
communicate the detected faults in such a way that the studentcan learn with this experience;
construct the student model (representation of the student’sknowledge) based on the results of the tutor diagnosis.
ICBL- November 5th, 2008 Slide: 3/22
Introduction Background The solution Conclusions
General Concepts
Diagnosis of the student’s program
ProPAT [Delgado, 2005]
Features:
Able to detect logical faults in students’ programs.Based on previous work that uses model based diagnosis toperform program debugging (model based software debugging).Only needs the program, the test cases and some informationabout the programming language to detect the faultyinstructions.
Limitation:
The faults are communicated to the student in terms of programlines, which gives little or no information to the student to findout his mistakes or misconceptions.
ICBL- November 5th, 2008 Slide: 4/22
Introduction Background The solution Conclusions
General Concepts
Model Based Diagnosis
Model based diagnosis (MBD) [de Kleer and Williams, 1987][Reiter, 1987]
An Artificial Intelligence technique to find faulty components inphysical devices.
Reasons over models that describe the correct behavior of the system.
Involves three sub-tasks [Benjamins, 1993]: symptom detection,hypothesis generation and hypothesis discrimination.
Model based software debugging (MBSD) [Mayer et al., 2002]
An application of MBD to program debugging.
Basic idea: step by step program simulation that points out theinstructions that justifies the observable discrepancies (i.e., thedifferences between program outputs and expected outputs -symptoms).
Can be used by an ITS to try to understand the difference betweenthe student’s intentions and his program.
ICBL- November 5th, 2008 Slide: 5/22
Introduction Background The solution Conclusions
General Concepts
MBD versus MBSD
Comparison between model based diagnosis and model based softwaredebugging
Model based diagnosis Model based software debugging
ICBL- November 5th, 2008 Slide: 6/22
Introduction Background The solution Conclusions
General Concepts
Model based software debugging
ICBL- November 5th, 2008 Slide: 7/22
Introduction Background The solution Conclusions
General Concepts
MBSD for programming learning: limitations
Pointing out the possible faulty lines of the program can not beinformative enough for a novice to correct his program.
Even for a regular size program, there can be many hypothesesof failure to be communicated (and discriminated) to thestudent, which can make him tired and confused.
A novice student does not have enough knowledge to makepredictions about the program behavior.
ICBL- November 5th, 2008 Slide: 8/22
Introduction Background The solution Conclusions
Goals
Proposal
To extend the MBSD technique with the use of Hierarchical ModelBased Diagnosis (hierarchical diagnosis), so that:
Faults in the student’s program can be found at different levelsof the program abstraction.
Functions, procedures and programming (elementary) patterns,are viewed as abstract components.
The communication with the student can be made in terms ofthese abstract components, i.e., through a high-level languageinstead of program lines.
In the average case, the tutor would communicate a smaller setof hypotheses of failure to the student.
ICBL- November 5th, 2008 Slide: 9/22
Introduction Background The solution Conclusions
HMBD
Hierarchical Model Based Diagnosis (HMBD)
A technique used to increase performance during the diagnosis.
The system is modeled at different levels of abstraction.
Abstractions are used to represent components (structuralabstraction) or the system behavior (behavioral abstraction).
Uses traditional MBD techniques to find the solution.
ICBL- November 5th, 2008 Slide: 10/22
Introduction Background The solution Conclusions
HMBD
Models and abstractions
Example:
ICBL- November 5th, 2008 Slide: 11/22
Introduction Background The solution Conclusions
HMBD
Abstraction tree
Subsystems are supposed to be independent
ICBL- November 5th, 2008 Slide: 12/22
Introduction Background The solution Conclusions
HMBD
Abstract component
Models of an abstract component
An abstract component can be described by:internal structural model, external structural model and behavioral model.
Behavior of an abstract component
An abstract componentAC1 and the aggre-gated components C1,C2 and C3
The behavior of AC1 isassumed as correct
So, the behavior of C1,C2 and C3 must be as-sumed as correct
This behavior can be formally described by the following axiom:
ok(CA) → ok(C1) ∧ ok(C2) ∧ · · · ∧ ok(Cm) (1)
ICBL- November 5th, 2008 Slide: 13/22
Introduction Background The solution Conclusions
HMBD
Top-down Hierarchical Diagnosis
An algorithm to perform hierarchical diagnosis can be described as[Mozetic, 1991]:
1 Verify the observations that can be used at each level ofabstraction.
2 For each level i , from l to 0, where l is the highest level thatdoes not hide all the connections with value, find thehypothesis set with an MBD algorithm. For each abstractcomponent CA not involved in any hypothesis set at level i ,apply the Axiom 1 for the level i − 1.
3 Return the hypothesis set for level 0.
ICBL- November 5th, 2008 Slide: 14/22
Introduction Background The solution Conclusions
HMBD
HMBD example
ICBL- November 5th, 2008 Slide: 15/22
Introduction Background The solution Conclusions
HMBD
HMBD example
ICBL- November 5th, 2008 Slide: 15/22
Introduction Background The solution Conclusions
HMBD
HMBD example
ICBL- November 5th, 2008 Slide: 15/22
Introduction Background The solution Conclusions
HMBD
HMBD example
ICBL- November 5th, 2008 Slide: 15/22
Introduction Background The solution Conclusions
HMBD
HMBD example
ICBL- November 5th, 2008 Slide: 15/22
Introduction Background The solution Conclusions
HMBD
HMBD example
ICBL- November 5th, 2008 Slide: 15/22
Introduction Background The solution Conclusions
HMBD
HMBD example
ICBL- November 5th, 2008 Slide: 15/22
Introduction Background The solution Conclusions
HMBD
HMBD example
ICBL- November 5th, 2008 Slide: 15/22
Introduction Background The solution Conclusions
MBSD+HMBD
Basic idea of MBSD + HMBD
Give the opportunity for the student to understand and correctthe faults in the program at different levels of abstraction.
After the hypothesis set for a specific level has been found, thestudent can choose between:
change his program with an expectation to correct the faults;discriminate the hypotheses at the current level of abstraction(by supplying new observations about the expected behavior ofthe program);try to correct the problem at a more detailed level, by replacingan abstract component by its aggregated components(refinement).
ICBL- November 5th, 2008 Slide: 16/22
Introduction Background The solution Conclusions
MBSD+HMBD
Program example
Programming Problem: Given a set v of integer numbers, calculate the difference
between the sum of even numbers and odd numbers.
1 public int evenMinusOdd(int v[]) {2 int n = v.length;
* 3 int counter = 1;
4 int evenSum = 0;
5 int oddSum = 0;
6
* 7 while (counter < n) {* 8 if (v[counter] % 2 == 0) {* 9 evenSum = evenSum + v[counter];
* 10 } else {* 11 oddSum = oddSum + v[counter];
* 12 }* 13 counter = counter + 1;
* 14 }15 return evenSum - oddSum;
16 }
(*) Lines used to compose a Counting Loop elementary pattern.
ICBL- November 5th, 2008 Slide: 17/22
Introduction Background The solution Conclusions
MBSD+HMBD
Program model
Assignment
counter = 1
Assignment
Assignment
Assignment
n
evenSum
oddSum
n = v.length
evenSum = 0
oddSum = 0
counter
evenSun
oddSum Expr
evenSum - oddSumresult
Assignment
oddSum = 0
Assignment
Assignment
Assignment
Count ingLoop
n
evenSum
oddSum
counter
evenSun
oddSum
n = v.length
evenSum = 0
oddSum = 0
Expr
evenSum - oddSum result
AssignmentoddSum = 0
Level 0
Level 1
Expr counter < n
LOOP
LOOPBODY
counter
v.length
const1
0
const2
0
const3
1
v.length
const1
0
const2
0
ICBL- November 5th, 2008 Slide: 18/22
Introduction Background The solution Conclusions
MBSD+HMBD
Characteristics of an algorithm for MBSD + HMBD
We consider the following items as important characteristics of ouralgorithm to MBSD + HMBD for programming learning:
The student can discriminate the hypotheses of failure atdifferent levels of abstraction.
Abstract components can be refined according to the decisionof the student.
At any level of abstraction, if the student corrects the faults inhis program, the diagnosis process is finished (even if thecurrent level has abstract components).
All observations can be used during the diagnosis process (i.e.,there is no connection with observed value hidden in anabstract component, during the whole diagnosis process).
ICBL- November 5th, 2008 Slide: 19/22
Introduction Background The solution Conclusions
Conclusions and future work
In this presentation, we showed:
A technique of model based diagnosis to locate logical faults inprograms (MBSD) and its limitations when applied to programminglearning.
An extension of the hierarchical diagnosis to detect faults in programsrepresented with different levels of abstraction, considering abstractcomponents, such as functions, procedures and elementary patterns.
A suitable approach to debug the students’ program and communicatethe faults in a high-level language (considering abstractions), that webelieve to be more appropriate to promote learning.
We have planned to:
Finish the implementation for MBSD + HMBD;
Do an experimental evaluation with students.
ICBL- November 5th, 2008 Slide: 20/22
Introduction Background The solution Conclusions
References
Benjamins, R. (1993).Problem Solving Methods for Diagnosis.PhD thesis, University of Amsterdam.
Chittaro, L. and Ranon, R. (2004).Hierarchical model-based diagnosis based on structural abstraction.Artif. Intell., 155(1-2):147–182.
de Kleer, J. and Williams, B. C. (1987).Diagnosing multiple faults.Artif. Intell., 32(1):97–130.
Delgado, K. V. (2005).Diagnostico baseado em modelos num sistema tutor inteligente para programacaoocom padroes pedagogicos.Dissertacao de mestrado, Instituto de Matematica e Estatıstica.
Johnson, W. L. and Soloway, E. (1984).Proust: Knowledge-based program understanding.In ICSE ’84: Proc. of the 7th international conference on Software engineering,pages 369–380, Piscataway, NJ, USA. IEEE Press.
ICBL- November 5th, 2008 Slide: 21/22
Introduction Background The solution Conclusions
References
Mayer, W., Stumptner, M., Wieland, D., and Wotawa, F. (2002).Observations and results gained from the jade project.In Proc. of the 13th International Workshop on Principles of Diagnosis,Semmering, Austria.
Mozetic, I. (1991).Hierarchical model-based diagnosis.Int. J. Man-Mach. Stud., 35(3):329–362.
Reiter, R. (1987).A theory of diagnosis from first principles.Artif. Intell., 32(1):57–95.
Wenger, E. (1987).Artificial intelligence and tutoring systems: Computational and cognitive
approaches to the communication of knowledge.Morgan Kaufmann Press.
ICBL- November 5th, 2008 Slide: 22/22