S C I E N C E P A S S I O N T E C H N O L O G Y
www.tugraz.at
Critiquing-basedRecommendationwith Speech InteractionPeter Grasch ([email protected])Alexander Felfernig ([email protected])Florian Reinfrank ([email protected]),Institute for Software Technology
October 15, 2013
www.tugraz.at
Speech Interaction in Recommender Systems
(Written) natural language input has shown promisein (Shimazu 2001; arnestal, 2004)
Constraint satisfaction using spoken languagepresented by Thompson et al in 2004
Peter Grasch ([email protected]), Institute for Software TechnologyOctober 15, 20132
www.tugraz.at
Critiquing-based Recommender Systems
Pioneering work as early as 1984: M. Williams’RABBIT [Williams, 1984]
Seminal work by Burke et al: FindMe[Burke et al., 1997]
Continued, active research, especially in the areas ofadvanced critiques[McCarthy et al., 2004, Zhang and Pu, 2006] anduser modeling[Reilly et al., 2005a, McCarthy et al., 2010]
Peter Grasch ([email protected]), Institute for Software TechnologyOctober 15, 20133
www.tugraz.at
Traditional Critiquing-based Recommender
Figure : QwikShop [Reilly et al., 2005b]
Peter Grasch ([email protected]), Institute for Software TechnologyOctober 15, 20134
www.tugraz.at
ReComment: Concept
Preference Model
Recommendation
RecommendationStrategy
Preference Elicitation
Peter Grasch ([email protected]), Institute for Software TechnologyOctober 15, 20135
www.tugraz.at
ReComment: Concept
Preference Model
Recommendation
RecommendationStrategy
Preference Elicitation
Peter Grasch ([email protected]), Institute for Software TechnologyOctober 15, 20136
www.tugraz.at
ReComment: Rationale
A speech-based natural language interface can allowmore expressive feedback, thus reducing sessionlength.
Peter Grasch ([email protected]), Institute for Software TechnologyOctober 15, 20137
www.tugraz.at
ReComment: Recommendation Strategy
Incremental unit critiquing-based system[Burke, 2000, Reilly et al., 2005a]
Prior probability based on sales rank
No initial search, relaxed similarity constraint
Custom utility function
Peter Grasch ([email protected]), Institute for Software TechnologyOctober 15, 20138
www.tugraz.at
ReComment: Recommendation Strategy
P ′ ← {p ∈ P|p satisfies last given critique};maxUtility ← −∞; bestOffer ← rold ;for p ∈ P ′ do
thisUtility ←∞ ;for c ∈ C do
thisUtility ← thisUtility + (1− c.ageMaxAge ) ∗ c.utility(p) ;
endif thisUtility > maxUtility then
maxUtility ← thisUtility ; bestOffer ← p ;end
endreturn bestOfferPeter Grasch ([email protected]), Institute for Software TechnologyOctober 15, 20139
www.tugraz.at
ReComment: Utility Function
Control rate of change: Implicit goalsdistance = distance(a.value, p[a.id ].value) ∗ r .direction;perfectDist = metaModifier ∗ 0.5;if critiqueViolated then
return −abs(distance − perfectDist);else
if distance < perfectDist thenreturn
√distance
perfectDist ;
elsereturn max(perfectDist − distance + 1, 0.0001);
endend
Algorithm 1: Schematic utility calculation.Peter Grasch ([email protected]), Institute for Software TechnologyOctober 15, 201310
www.tugraz.at
ReComment: Utility Function
Control rate of change: Implicit goals
Figure : Utility function of the critique x > 50.Peter Grasch ([email protected]), Institute for Software TechnologyOctober 15, 201311
www.tugraz.at
ReComment: Utility Function
Control rate of change: Implicit goals
Figure : Utility function of subsequent critiques.Peter Grasch ([email protected]), Institute for Software TechnologyOctober 15, 201312
www.tugraz.at
ReComment: Utility Function
Control rate of change: Implicit goals
Figure : Utility function of subsequent critiques.Peter Grasch ([email protected]), Institute for Software TechnologyOctober 15, 201313
www.tugraz.at
ReComment: Speech Processing
Speech recognition solution based on CMU SPHINXand Simon [sph, 2013, sim, 2013]
Adapted to recommender situation
Keyword parser
Peter Grasch ([email protected]), Institute for Software TechnologyOctober 15, 201314
www.tugraz.at
Experiment
Comparison with traditional interface
80 participants
Measuring:
Interaction cyclesPerceived recommendation qualityUsability (adapted SUS)
Peter Grasch ([email protected]), Institute for Software TechnologyOctober 15, 201315
www.tugraz.at
Experiment: Traditional User Interface
Figure : ReComment: Mouse-based user interface.Peter Grasch ([email protected]), Institute for Software TechnologyOctober 15, 201316
www.tugraz.at
Experiment: Speech-based User Interface
Figure : ReComment: Speech-based user interface.Peter Grasch ([email protected]), Institute for Software TechnologyOctober 15, 201317
www.tugraz.at
Experiment: Speech-based User Interface
SentenceI am looking for a camera with 12 megapixel anda weight of around 200 gram.This camera with the same properties justsmaller.An even smaller camera.Optical zoom of 14 times would be better.More optical zoom.[...]
Table : Sample user interaction session.
Peter Grasch ([email protected]), Institute for Software TechnologyOctober 15, 201318
www.tugraz.at
Results: Feedback Strategies
Category CountDiscarded 49 (12.8%)Unit critique 329 (85.7%)Compound critique (2 attributes) 3 (0.8%)Compound critique (3 attributes) 2 (0.5%)Compound critique (5 attributes) 1 (0.3%)
Table : Types of used commands.
74 sentences (20 %) referred to explicit values.12 sentences (3 %) used modifiers.Peter Grasch ([email protected]), Institute for Software TechnologyOctober 15, 201319
www.tugraz.at
Results: Speech Processing
03
10
24
0
5
10
15
20
25
1 2 3 4
"Rec
omm
ent u
nder
stan
dsm
y vo
ice
inpu
t"
Figure : Participants’ perception of the speech-recognitionaccuracy ([1, 4], higher is better).
Peter Grasch ([email protected]), Institute for Software TechnologyOctober 15, 201320
www.tugraz.at
Results: Usability
Figure : Usability evaluation (adapted SUS scores).Peter Grasch ([email protected]), Institute for Software TechnologyOctober 15, 201321
www.tugraz.at
Results: Recommendation Quality
0
5
10
15
20
25
Mouse−based interface Speech−based interface
Score
1
2
3
4
Figure : User score of last recommended item ([1, 4],higher is better).
Peter Grasch ([email protected]), Institute for Software TechnologyOctober 15, 201322
www.tugraz.at
Results: Recommender Efficiency
34.5
47.7
79.0250
10
20
30
40
50
Mouse interface(mean)
Mouse interface(median)
Speech interface(mean)
Speech interface(median)
Ses
sion
leng
th (
cycl
es)
Figure : Session length (lower is better).
Peter Grasch ([email protected]), Institute for Software TechnologyOctober 15, 201323
www.tugraz.at
Conclusion
Spoken language recommender systemsare worth exploring!
Peter Grasch ([email protected]), Institute for Software TechnologyOctober 15, 201324
www.tugraz.at
Future Work
Explore more natural user interfaces
Advanced sentiment analysis
Use of prosodic features, timing information, etc. toinfer certainty, frustration, etc.
Compare different recommender systems (e.g.,constraint based approaches)
Peter Grasch ([email protected]), Institute for Software TechnologyOctober 15, 201325
www.tugraz.at
Thank you for your attention.
Q & A
Peter Grasch ([email protected]), Institute for Software TechnologyOctober 15, 201326
www.tugraz.at
Sources I
[sim, 2013] (2013).
About Simon — Simon.
http://simon.kde.org.
[sph, 2013] (2013).
CMU Sphinx - Speech Recognition Toolkit.
http://cmusphinx.sf.net.
Peter Grasch ([email protected]), Institute for Software TechnologyOctober 15, 201327
www.tugraz.at
Sources II
[Burke, 2000] Burke, R. (2000).
Knowledge-based recommender systems.
In Encyclopedia of Library and InformationSystems. Marcel Dekker.
[Burke et al., 1997] Burke, R. D., Hammond, K. J.,and Yound, B. (1997).
The findme approach to assisted browsing.
IEEE Expert, 12(4):32–40.
Peter Grasch ([email protected]), Institute for Software TechnologyOctober 15, 201328
www.tugraz.at
Sources III
[McCarthy et al., 2004] McCarthy, K., Reilly, J.,McGinty, L., and Smyth, B. (2004).
On the dynamic generation of compound critiquesin conversational recommender systems.
In Adaptive Hypermedia and Adaptive Web-BasedSystems, pages 176–184. Springer.
Peter Grasch ([email protected]), Institute for Software TechnologyOctober 15, 201329
www.tugraz.at
Sources IV
[McCarthy et al., 2010] McCarthy, K., Salem, Y., andSmyth, B. (2010).
Experience-based critiquing: reusing critiquingexperiences to improve conversationalrecommendation.
In Case-Based Reasoning. Research andDevelopment, pages 480–494. Springer.
Peter Grasch ([email protected]), Institute for Software TechnologyOctober 15, 201330
www.tugraz.at
Sources V
[Reilly et al., 2005a] Reilly, J., McCarthy, K., McGinty,L., and Smyth, B. (2005a).
Incremental critiquing.
Knowledge-Based Systems, 18(4):143–151.
Peter Grasch ([email protected]), Institute for Software TechnologyOctober 15, 201331
www.tugraz.at
Sources VI
[Reilly et al., 2005b] Reilly, J., Smyth, B., McGinty, L.,and McCarthy, K. (2005b).
Critiquing with confidence.
In Case-Based Reasoning Research andDevelopment, pages 436–450. Springer.
Peter Grasch ([email protected]), Institute for Software TechnologyOctober 15, 201332
www.tugraz.at
Sources VII
[Williams, 1984] Williams, M. D. (1984).
What makes rabbit run?
International Journal of Man-Machine Studies,21(4):333–352.
Peter Grasch ([email protected]), Institute for Software TechnologyOctober 15, 201333
www.tugraz.at
Sources VIII
[Zhang and Pu, 2006] Zhang, J. and Pu, P. (2006).
A comparative study of compound critiquegeneration in conversational recommendersystems.
In Adaptive Hypermedia and Adaptive Web-BasedSystems, pages 234–243. Springer.
Peter Grasch ([email protected]), Institute for Software TechnologyOctober 15, 201334