11EC65R09_Intelligent Tutorial System

8/2/2019 11EC65R09_Intelligent Tutorial System

1/26

INTELLIGENT TUTORIAL SYSTEM &ITS ENHANCEMENT USING

EMOTIONAL FEEDBACK

Soumya Bose

11EC65R09, VIPES

1


2/26

SCOPE

Emergence of Intelligent Tutoring System (ITS)

Central framework of ITS.

Different Modules

New Generation ITS (NGITS)

Need of emotional feedback

Facial Emotion Analysis

Conclusions

Future Work

2


3/26

EXISTING TUTORINGSYSTEMS (CAI)

E-learningWeb Based Tutorials using Audio/Video.

Advantages:

Low Cost.

Learn when you need.Not constrained by geographical location.

Disadvantages:

Based on Simple Computer Aided Instructions

No student teacher feedback.Lack of understanding.

Appears to be boring.

3


4/26

INTELLIGENT TUTORIAL SYSTEM

Outgrowth of Computer Aided Instructions

with added intelligence.

System where teacher-student communicationstrengthens learning process.

Tracks students performance.

Adaptive system where next state of instructionis dependant upon students performance.

4


5/26

CENTRAL FRAMEWORKOF ITS

Traditionally ITS has the following components:

DOMAIN

MODULE

PEDAGOGICAL

MODULE

STUDENT

MODULE

TASK

ENVIRONMENT

Fig.1: ITS Framework 5


6/26

DOMAINMODULE

Knowledge database.

Depends upon the domain in which the ITS is

intended to instruct.

Domain module is prepared with a view ofcognitive psychology of human skill

acquisition.

Knowledge DECLARATIVE or PROCEDURAL

6


7/26

Example

Q. How to evaluate an expression with different arithmetic operands?

Declarative:

Priority1. ( operation )

Priority2. /

Priority3. x

Priority4. +

Priority5. -

Procedural:

Evaluation: (3-4) x 18 / 9 + 5 - 8

Step1. A= (3-4) = -1

Step2. B= 18 / 9 = 2

Step3. C= A x B = -1 x 2 = -2

Step4. D= C+5 = -2+5 = 3

Step5. Ans.= 3-8 = -5

Combination of both provides effective learning process

An ITS to tutor school mathematics.

7


8/26

TASK ENVIRONMENT

Interface for student - teacher communication

GUI may serve the purpose

Tutor displays instructions.

Student gives input through atext editor.

A simulation response to the

student forms a feedback to the

student.

Tutor Feedback helps student

in reasoning.

Problem Statement Tutor Instructions

Output Window Students Worksheet

Fig.2: Typical Task Environment

8


9/26

PEDAGOGICAL / TUTORING MODEL

Structuring the instructions.At curriculum level it is sequence of information.

At problem solving level it can intervene to advisestudents.

Next instructions should be on the basis of presentstate (can be modeled as a tree data structure).

Present state includes (Logical decision):

Current stage of domain module.

Knowledge state of student. 9


10/26

C

1C

2C

3C

4C

n

C

1C

2C

3C

7C

n

C

3

C

6

C

4

C

8

C

6C

7C

8C10

C

n

C

2C

5

C

9

C

1

C

3C

4

C11

C12

C13

C14

C

n

C

n

1. Primitive linear structure

2. Branched structure

3. Multilevel structure

Cn is quantum of domain

knowledge / information in

the nth stage

INSTRUCTIONSTRUCTURES

10


11/26

STUDENT MODULERecord of students knowledge state.

Student module is dynamic: Knowledge state is changing

(Modeling is complex).

e.g: Identifying Operands

Identifying Operators

Doing operations

Evaluating Expressions

Solving Equations

Statistical methods used for estimating students knowledge state (bypsychophysicist Green and Swets, 1973)

Intelligence of Domain Expert module assess students

performance11


12/26

ITS DESIGN FLOW

Modeler

Expert Simulator Knowledge Base Student Model

Tutor

Problem Student

TUTORING SYSTEM

Problem

Information

Data Request

Problem Data

Explanation DataProblem Solving

Situation

Predicted and

Preferred behaviorRelations and

Student Prototypes

Advice &Explanation

Update Model

Students Current state

12


13/26

A TYPICAL ITS

Problem Statement Tutor Instructions

Output Window Students Worksheet

-1 x 2 = -2

Hint: You have done an independentOperation. Calculate 1st priorityoperation

Hint: You have done 2nd priorityoperation which is also independentof the first operation. Calculate 1stpriority operation

Hint: 1st priority operation done.Write down the modified expressionHint: Calculate the 1st priorityOperation now on the modifiedexpression

Hint: Wrong! Now operations aredependant. Calculate 1st operationHint: Calculate 2nd operation.Problem Solved! 1.Evaluate the expression:

Z= (3-4) x 18 / 9 + 58

1. A = -32. B = 2

4. Expression

5. Wrong5. -26. -5

3. C= -1

Hint: Evaluate 1st priority operation

5 - 8 = -318/9 = 23 - 4 = -1-1 x 2 3

2 3 =1-2 3 = -5

13


14/26

PRACTICALLY IMPLEMENTED SYSTEMS

14

1. The PUMP Algebra Tutor (PAT)

(Anderson, Corbett, Koedinger andPelletier, 1995):

Used for tutoring introductory

algebra in Pittsburg Schools.

Fig.3.: The PAT GUI (Courtesy: Ref.4)

2. The SHERLOCK Project (Lesgold,

Laioie, Bunzo and Eggan, 1992;Katz and Lesgold, 1993):

It is a practice environment for

electronics troubleshooting

commissioned by Air-Force.Fig.4.: The SHERLOCK interface(Courtesy: Lawrence Elbaum Associates)


15/26

SHORTCOMINGOF CURRENT SYSTEMS

Evaluation by Current systems (Based on knowledge state):In either of the above cases student will be

assumed to have a knowledge of FORCE.

Q. What is the value of force acting on a body of mass

3Kg and moving with a retardation of 4m/s2?

Ans. : -12N

Ans. : -12N

Confident

Not-Sure

15


16/26

Measuring mental state of learner by bio signals analysis:

Facial expressions

Signals from Brain

Electro-dermal signals

ECG signals, etc.

NEW GENERATION ITS

Aimed at developing more adaptive tutors

Expert Module evaluates both:

Knowledge state

Emotional / Mental state

16


17/26

FACIAL EMOTION ANALYSIS

Involves lot of real time image processing.

Time is the constraint.

Should be processed in parallel with knowledgeevaluation.

Hardware / Software co-design approach is adopted forfast processing.

Challenges:

17


18/26

BASIC STEPS

IMAGE GRABBER PRE PROCESSING

FEATURE

EXTRACTION

MATCHINGINTERPRETATION

COARSEREGION

SEGMENTATION

EDGE

DETECTION

WAVELETTRANSFORM

SOBEL /

LAPLACIANEDGE

GENETICALGORITHM

18


19/26

Labeling : Smoothing small variations in intensity.

Segmentation : Finding edges or sharp transitions.

Smoothening with linear

resistive network blurs edgesof objects.

Resistive Fuse networks are

used to label and

segmenting the image.

Object Segmentation and Labeling

19


20/26

Resistive Fuse Network

The resistive fuse acts as a linear resistor

for |Vdiff| < Voff

Acts as an open circuit for |Vdiff| > Voff

The change in voltage at each node

can be calculated from Kirchoffs current law:

Vout2,2(t+1) Vout2,2(t) =

v[ G(Vout i,j Vout2,2) + (Vin2,2- Vout2,2) ]i,jN

2,2

Fig.5.: Fuse Resistor

Fig.6.: Segmentation Circuit 20


21/26

v[ G(Vout i,j Vout2,2) + (Vin2,2- Vout2,2) ]i,jN

2,2

The equation

can be realized in FPGA:

e = sample pixel

ai = neighbor pixel

Smem = pixel data Vin is stored

Dmem = Output data Vout is stored

LUT1 performs evaluation

LUT2 performs G (fuse value)

per channel (RGB)21

Fig.8.: FPGA implementation (Courtesy: Ref.7)

Fig.7.: Raster Scan of the image(Courtesy: Ref.7)


22/26

Gabor Wavelet Transform (GWT)

GWT is performed over

the segmented object for

feature extraction

Wavelet function is multiplication of a

harmonic function and gaussian function

determines frequency

and the direction

22

Fig.9.: GWTs of the face (Courtesy: Ref.7)


23/26

Matching & Interpretation

Edges of features (eye / lips / nose): Sobel / Gaussian operator

GA is an iterative process .

In each step shapes are

matched with known curves

Termination occurs when

error is minimized.

Final matched curve parameters (like major axis/ minor axis) are matched

with known values to predict emotions:

Happy

Sad

Frustration, etc.

23

Fig.10: Matching eye with an ellipse (Courtesy: Ref.8)

Fig.11.: Matching lips with an irregular ellipse(Courtesy: Ref.8)

Matching: Genetic Algorithm (GA)


24/26

CONCLUSIONS

ITS has been proved efficient and stronger than simple CAI

Involves student in sustained reasoning activity.Problem solving tutor helps conceptual understanding as well as

solving real life problems related to a domain.

High level GUI attracts students for learning

The ability to read the mental state of the learner through facialemotional analysis: Increases Adaptability

Repeated instructions can be delivered on the basis of mental

satisfaction

Helps student in sound understanding

However the accuracy of the emotional analysis can be improved

significantly adding voice information

The main drawback is it is an one-to-one process 24


25/26

Different bio-signals processing: Assess mental state of the

student more correctly.

Electro dermal

signal(GSC): Human

skin is a weak conductor of

electricity.

Brain Signals (EEG): Higher frequency beta waves (15-25Hz)

and low theta waves implies seriousness.

ECG can be analyzed to detect stress, low confidence of the student.

Real time processing several bio-signals will make the design

complex.

But even if half the ability of real human tutor is realized the payoff

to the society will be substantial.

Fig.12: Galvanic Skin Response (Courtesy: Springer Images)

FUTURE DIRECTIONS

25


26/26

1. Abdolhossein Sarrafzadeh, Hamid Gholam Hosseini, Chao Fan, Scott P. Overmyer;Facial

Expression Analysis for Estimating Learners Emotional State in Intelligent Tutoring

Systems; Proceedings of the The 3rd IEEE International Conference on Advanced Learning

Technologies (ICALT03); 2003

2. Morteza BIGLARI-ABHARI, Abbas BIGDELI;FPGA Implementation of Facial Expression

Analysis For Intelligent Tutoring Systems; Proceedings of the II International Conference on

Multimedia and Information & Communication Technologies in Education; ICTE 2003, Spain

3. Sunandan Chakraborty, Devshri Roy, Anupam Basu;Development of Knowledge Based

Intelligent Tutoring System; Indian Institute of Technology, Kharagpur, India; 2001

4. M.Helander, T. K. Landauer, P. Prabhu (Eds), Elsevier Science B. V.; Intelligent Tutoring

Systems, Handbook of Human-Computer Interaction, Second Edition; 1997

5. Arjen Hoekstra and Joris Janssen;Linking Bio-signals to Transfer of Knowledge Towards

Mind-reading ECAs; Faculty of Electrical Engineering, Mathematics and Computer Science

University of Twente, The Netherlands

6. Teppei NAKANO, Hiroshi ANDO, Hideaki ISHIZU, Takashi MORIE, Atsushi IWATA;

Coarse Image Region Segmentation Using Resistive-fuse Networks Implemented in FPGA;

Graduate School of Life Science and Systems Engineering,Kyushu Institute of Technology;

Graduate School of Advanced Sciences of Matter, Hiroshima University; Hiroshima Prefecture

Industrial Research Institute

7. T. Nakano, T. Morie and A. Iwata;A Face/Object Recognition System Using FPGA

Implementation of Coarse Region Segmentation; SICE Annual Conference in Fukui, Fukui

University, Japan; 2003

8. M. Karthigayan, M. Rizon, R. Nagarajan and Sazali Yaacob; Genetic Algorithm and Neural

Network for Face Emotion Recognition; School of Mechatronics Engineering, Universiti

Malaysia Perlis (UNIMAP); 2006

REFERENCES

26

Documents

11EC65R09_Intelligent Tutorial System