ReComment: Towards Critiquing-based Recommendation with Speech Interaction

S C I E N C E P A S S I O N T E C H N O L O G Y

www.tugraz.at

Critiquing-basedRecommendationwith Speech InteractionPeter Grasch (peter.grasch@student.tugraz.at)Alexander Felfernig (afelfern@ist.tugraz.at)Florian Reinfrank (freinfra@ist.tugraz.at),Institute for Software Technology

October 15, 2013

www.tugraz.at

Speech Interaction in Recommender Systems

(Written) natural language input has shown promisein (Shimazu 2001; arnestal, 2004)

Constraint satisfaction using spoken languagepresented by Thompson et al in 2004

Peter Grasch (peter.grasch@student.tugraz.at), Institute for Software TechnologyOctober 15, 20132

www.tugraz.at

Critiquing-based Recommender Systems

Pioneering work as early as 1984: M. Williams’RABBIT [Williams, 1984]

Seminal work by Burke et al: FindMe[Burke et al., 1997]

Continued, active research, especially in the areas ofadvanced critiques[McCarthy et al., 2004, Zhang and Pu, 2006] anduser modeling[Reilly et al., 2005a, McCarthy et al., 2010]

Peter Grasch (peter.grasch@student.tugraz.at), Institute for Software TechnologyOctober 15, 20133

www.tugraz.at

Traditional Critiquing-based Recommender

Figure : QwikShop [Reilly et al., 2005b]

Peter Grasch (peter.grasch@student.tugraz.at), Institute for Software TechnologyOctober 15, 20134

www.tugraz.at

ReComment: Concept

Preference Model

Recommendation

RecommendationStrategy

Preference Elicitation

Peter Grasch (peter.grasch@student.tugraz.at), Institute for Software TechnologyOctober 15, 20135

www.tugraz.at

ReComment: Concept

Preference Model

Recommendation

RecommendationStrategy

Preference Elicitation

Peter Grasch (peter.grasch@student.tugraz.at), Institute for Software TechnologyOctober 15, 20136

www.tugraz.at

ReComment: Rationale

A speech-based natural language interface can allowmore expressive feedback, thus reducing sessionlength.

Peter Grasch (peter.grasch@student.tugraz.at), Institute for Software TechnologyOctober 15, 20137

www.tugraz.at

ReComment: Recommendation Strategy

Incremental unit critiquing-based system[Burke, 2000, Reilly et al., 2005a]

Prior probability based on sales rank

No initial search, relaxed similarity constraint

Custom utility function

Peter Grasch (peter.grasch@student.tugraz.at), Institute for Software TechnologyOctober 15, 20138

www.tugraz.at

ReComment: Recommendation Strategy

P ′ ← {p ∈ P|p satisfies last given critique};maxUtility ← −∞; bestOffer ← rold ;for p ∈ P ′ do

thisUtility ←∞ ;for c ∈ C do

thisUtility ← thisUtility + (1− c.ageMaxAge ) ∗ c.utility(p) ;

endif thisUtility > maxUtility then

maxUtility ← thisUtility ; bestOffer ← p ;end

endreturn bestOfferPeter Grasch (peter.grasch@student.tugraz.at), Institute for Software TechnologyOctober 15, 20139

www.tugraz.at

ReComment: Utility Function

Control rate of change: Implicit goalsdistance = distance(a.value, p[a.id ].value) ∗ r .direction;perfectDist = metaModifier ∗ 0.5;if critiqueViolated then

return −abs(distance − perfectDist);else

if distance < perfectDist thenreturn

√distance

perfectDist ;

elsereturn max(perfectDist − distance + 1, 0.0001);

endend

Algorithm 1: Schematic utility calculation.Peter Grasch (peter.grasch@student.tugraz.at), Institute for Software TechnologyOctober 15, 201310

www.tugraz.at

ReComment: Utility Function

Control rate of change: Implicit goals

Figure : Utility function of the critique x > 50.Peter Grasch (peter.grasch@student.tugraz.at), Institute for Software TechnologyOctober 15, 201311

www.tugraz.at

ReComment: Utility Function

Control rate of change: Implicit goals

Figure : Utility function of subsequent critiques.Peter Grasch (peter.grasch@student.tugraz.at), Institute for Software TechnologyOctober 15, 201312

www.tugraz.at

ReComment: Utility Function

Control rate of change: Implicit goals

Figure : Utility function of subsequent critiques.Peter Grasch (peter.grasch@student.tugraz.at), Institute for Software TechnologyOctober 15, 201313

www.tugraz.at

ReComment: Speech Processing

Speech recognition solution based on CMU SPHINXand Simon [sph, 2013, sim, 2013]

Adapted to recommender situation

Keyword parser

Peter Grasch (peter.grasch@student.tugraz.at), Institute for Software TechnologyOctober 15, 201314

www.tugraz.at

Experiment

Comparison with traditional interface

80 participants

Measuring:

Interaction cyclesPerceived recommendation qualityUsability (adapted SUS)

Peter Grasch (peter.grasch@student.tugraz.at), Institute for Software TechnologyOctober 15, 201315

www.tugraz.at

Experiment: Traditional User Interface

Figure : ReComment: Mouse-based user interface.Peter Grasch (peter.grasch@student.tugraz.at), Institute for Software TechnologyOctober 15, 201316

www.tugraz.at

Experiment: Speech-based User Interface

Figure : ReComment: Speech-based user interface.Peter Grasch (peter.grasch@student.tugraz.at), Institute for Software TechnologyOctober 15, 201317

www.tugraz.at

Experiment: Speech-based User Interface

SentenceI am looking for a camera with 12 megapixel anda weight of around 200 gram.This camera with the same properties justsmaller.An even smaller camera.Optical zoom of 14 times would be better.More optical zoom.[...]

Table : Sample user interaction session.

Peter Grasch (peter.grasch@student.tugraz.at), Institute for Software TechnologyOctober 15, 201318

www.tugraz.at

Results: Feedback Strategies

Category CountDiscarded 49 (12.8%)Unit critique 329 (85.7%)Compound critique (2 attributes) 3 (0.8%)Compound critique (3 attributes) 2 (0.5%)Compound critique (5 attributes) 1 (0.3%)

Table : Types of used commands.

74 sentences (20 %) referred to explicit values.12 sentences (3 %) used modifiers.Peter Grasch (peter.grasch@student.tugraz.at), Institute for Software TechnologyOctober 15, 201319

www.tugraz.at

Results: Speech Processing

03

10

24

0

5

10

15

20

25

1 2 3 4

"Rec

omm

ent u

nder

stan

dsm

y vo

ice

inpu

t"

Figure : Participants’ perception of the speech-recognitionaccuracy ([1, 4], higher is better).

Peter Grasch (peter.grasch@student.tugraz.at), Institute for Software TechnologyOctober 15, 201320

www.tugraz.at

Results: Usability

Figure : Usability evaluation (adapted SUS scores).Peter Grasch (peter.grasch@student.tugraz.at), Institute for Software TechnologyOctober 15, 201321

www.tugraz.at

Results: Recommendation Quality

0

5

10

15

20

25

Mouse−based interface Speech−based interface

Score

1

2

3

4

Figure : User score of last recommended item ([1, 4],higher is better).

Peter Grasch (peter.grasch@student.tugraz.at), Institute for Software TechnologyOctober 15, 201322

www.tugraz.at

Results: Recommender Efficiency

34.5

47.7

79.0250

10

20

30

40

50

Mouse interface(mean)

Mouse interface(median)

Speech interface(mean)

Speech interface(median)

Ses

sion

leng

th (

cycl

es)

Figure : Session length (lower is better).

Peter Grasch (peter.grasch@student.tugraz.at), Institute for Software TechnologyOctober 15, 201323

www.tugraz.at

Conclusion

Spoken language recommender systemsare worth exploring!

Peter Grasch (peter.grasch@student.tugraz.at), Institute for Software TechnologyOctober 15, 201324

www.tugraz.at

Future Work

Explore more natural user interfaces

Advanced sentiment analysis

Use of prosodic features, timing information, etc. toinfer certainty, frustration, etc.

Compare different recommender systems (e.g.,constraint based approaches)

Peter Grasch (peter.grasch@student.tugraz.at), Institute for Software TechnologyOctober 15, 201325

www.tugraz.at

Thank you for your attention.

Q & A

Peter Grasch (peter.grasch@student.tugraz.at), Institute for Software TechnologyOctober 15, 201326

www.tugraz.at

Sources I

[sim, 2013] (2013).

About Simon — Simon.

http://simon.kde.org.

[sph, 2013] (2013).

CMU Sphinx - Speech Recognition Toolkit.

http://cmusphinx.sf.net.

Peter Grasch (peter.grasch@student.tugraz.at), Institute for Software TechnologyOctober 15, 201327

www.tugraz.at

Sources II

[Burke, 2000] Burke, R. (2000).

Knowledge-based recommender systems.

In Encyclopedia of Library and InformationSystems. Marcel Dekker.

[Burke et al., 1997] Burke, R. D., Hammond, K. J.,and Yound, B. (1997).

The findme approach to assisted browsing.

IEEE Expert, 12(4):32–40.

Peter Grasch (peter.grasch@student.tugraz.at), Institute for Software TechnologyOctober 15, 201328

www.tugraz.at

Sources III

[McCarthy et al., 2004] McCarthy, K., Reilly, J.,McGinty, L., and Smyth, B. (2004).

On the dynamic generation of compound critiquesin conversational recommender systems.

In Adaptive Hypermedia and Adaptive Web-BasedSystems, pages 176–184. Springer.

Peter Grasch (peter.grasch@student.tugraz.at), Institute for Software TechnologyOctober 15, 201329

www.tugraz.at

Sources IV

[McCarthy et al., 2010] McCarthy, K., Salem, Y., andSmyth, B. (2010).

Experience-based critiquing: reusing critiquingexperiences to improve conversationalrecommendation.

In Case-Based Reasoning. Research andDevelopment, pages 480–494. Springer.

Peter Grasch (peter.grasch@student.tugraz.at), Institute for Software TechnologyOctober 15, 201330

www.tugraz.at

Sources V

[Reilly et al., 2005a] Reilly, J., McCarthy, K., McGinty,L., and Smyth, B. (2005a).

Incremental critiquing.

Knowledge-Based Systems, 18(4):143–151.

Peter Grasch (peter.grasch@student.tugraz.at), Institute for Software TechnologyOctober 15, 201331

www.tugraz.at

Sources VI

[Reilly et al., 2005b] Reilly, J., Smyth, B., McGinty, L.,and McCarthy, K. (2005b).

Critiquing with confidence.

In Case-Based Reasoning Research andDevelopment, pages 436–450. Springer.

Peter Grasch (peter.grasch@student.tugraz.at), Institute for Software TechnologyOctober 15, 201332

www.tugraz.at

Sources VII

[Williams, 1984] Williams, M. D. (1984).

What makes rabbit run?

International Journal of Man-Machine Studies,21(4):333–352.

Peter Grasch (peter.grasch@student.tugraz.at), Institute for Software TechnologyOctober 15, 201333

www.tugraz.at

Sources VIII

[Zhang and Pu, 2006] Zhang, J. and Pu, P. (2006).

A comparative study of compound critiquegeneration in conversational recommendersystems.

In Adaptive Hypermedia and Adaptive Web-BasedSystems, pages 234–243. Springer.

Peter Grasch (peter.grasch@student.tugraz.at), Institute for Software TechnologyOctober 15, 201334