International Conference on Case-Based Reasoning 2003 ... · –Generalized Shepard nearest neighbor algorithm ... •Planning: –Boosting algorithm to estimate pair-wise ... –Arguments

Copyright (c) 2005. Kevin D. Ashley

International Conference on Case-BasedReasoning 2003: Highlights

Part 1: Overview

Kevin Ashley (Presenter) & Derek BridgeCo-Chairpersons, Program Committee

www.iccbr.org/iccbr03/


CBR System SchematicInput Problem

2. Match indices /Retrieve/compare candidates

3. Select bestcase[s] not tried

4. Apply / adapt best case[s]

7. Explain result;Generalize &update casebase& index

1. Process Description

No

Yes

YesNo

Output

Indexed CaseDatabase

SimilarityMetric

Analytic DomainModel

Try alternative method (Non-CBR)

6. Success?

ANY?

5. Success?


ICCBR-03 Algorithms• Similarity assessment and retrieval:

– Algorithms to assess structural similarity of cases as labeled directed graphs (pp. 92,422)

– Max. likelihood Hebbian learning algorithm to cluster structurally similar cases (p. 107)– Comparison-based recommendation algorithms w/ diversity (pp. 276, 319)– Compromise-driven retrieval algorithm (p. 291)– Entropy-based local weighting algorithm (p. 389)– Rough Sets algorithm for weighting and case reduction (p. 494)– Genetic algorithm to learn local similarity measure w/ adaptability (p. 547)– Genetic algorithm to select among contradictory solutions (p. 567)– Generalized Shepard nearest neighbor algorithm (p. 653)

• Case Base Management:– Multiagent query by committee algorithm for case retention (p. 392)– Discretized highest similarity with pattern solution reuse indexing algorithm (p. 407)

• Planning:– Boosting algorithm to estimate pair-wise preferences among plan steps (p. 35)– Algorithm to reduce case inconsistencies in incomplete hierarchical domain (pp. 665,

679)• Explained Prediction:

– Algorithm for testing hypotheses predicting outcome of legal issues in cases (p. 65)


ICCBR-03 Application Domains• Adhesive product formulation (p. 465)

• Analog electronic circuit diagnosis (p.437)

• Auto assembly variation reduction (p. 306)

• Automated insurance underwriting (p. 5)

• Computer games prediction (p. 161)

• Cost reduction for interactive computersimulations (p. 581)

• E-commerce alternative productrecommendations (pp. 276, 479)

• Expressive musical performance(pp. 20, 552)

• Gas turbine diagnosis (p. 5)

• Geo-spatial imagery retrieval (p. 622)

• Grading plastic resins (p. 96)

• Intelligence analysis (p. 332)

• IT security for e-Governmentservices (p. 362)

• Legal prediction (p. 65)

• Manufacturing process change (p. 522)

• Oceanographic temperatureforecasting (p. 107)

• Predicting repeat criminalvictimization (p. 453)

• Shipping billing codes (p. 567)

• Software reuse & development(pp. 50, 171, 186, 216, 231, 595)

• Systems on chip design (p. 509)


Research Trends in ICCBR-03• Progress on adaptation

– Inducing attribute dependencies to guide substitution (p. 131)

– Selecting best overall rule refinements for faulty adaptation rules (p. 201)

– Generic adaptation based on substitutability/interchangeability in softconstraint satisfaction problems (p. 347)

• Recommender systems comparing and contrasting cases– Uses of diversity in recommending cases (pp. 276, 318)

– Retrieving compromise cases (p. 291)

• Integrating prediction and explanation– Case-based prediction in law and social sciences (pp. 65, 452)

– Arguments explain relation of case evidence to predictive hypotheses(pp. 65, 122, 332)

• Genetic algorithms to…– Learn local similarity measures, refine selection criteria, and optimize

weights (pp. 5, 50, 537, 567)


Highlighted Research Papers

3. Combining CBR and MBR for Predicting Outcome of Legal CasesS. Brüninghaus and K. Ashley

4. Using Evolution Programs to

Learn Local Similarity Measures A. Stahl and T. Gabel

5. Extracting Performers’ Behaviors to Annotate Casesin a CBR System for Musical Tempo TransformationsJ. Arcos, M. Grachten, & R. de Mantaras

2. Global Grade Selector: Recommender

System Supporting Sale of Plastic Resin W. Cheetham

1. A Framework for HistoricalCase-Based ReasoningJ. Ma and B. Knight Input Problem

2. Match indices /Retrieve candidates



7. Explain result;Generalize &update casebase &index


No

Yes

Yes

No

Output

Indexed CaseDatabase

Similarity Metric



6. Success?

ANY?

5. Success?


1. Framework for Historical CBR

• Problem: In medical and other domains, case histories, rather thancase snapshots, are important in reasoning.

• Solution: Framework for relative and absolute temporal knowledge.– Fluents: propositions whose truth values depend on time.– Elemental Cases: time-independent episodes defined as collections of

fluents.– Case histories: list of elemental cases w/ temporal relationships specified via:

• time elements• “meets” relation over time elements, and• “holds” relation of fluents over time elements.

• Non-temporal similarity: elemental case matching.• Temporal similarity:

– Graphs represent temporal information as meets relations and durations.– Similarity as conventional graph similarity measurement.

J. Ma and B. Knight, ICCBR-03 p. 246


2. Global Grade Selector: A RecommenderSystem for Supporting the Sale of Plastic Resin

• Problem: 800 salespeople of GE Plastics sell 3000 grades ofplastic resins worldwide with different properties. Organize!

• Solution: GGS, a CBR recommender system– Input: critical properties of intended application– Output: Candidate grades ranked by similarity score– Casebase of existing grades w/ 50 properties/global tests.– Similarity: weighted sum of similarities for each input property.– If no relevant grade, start new product introduction (NPI) to create one.

• Experience: When GGS introduced, 150 NPI’s in progress.– GGS used to find existing grades to satisfy 11 NPI’s.– 11 grades resulted in $95 million in sales.

Cheetham, W. ICCBR-03 p. 96


3. Combining Case-Based and Model-BasedReasoning for Predicting Outcomes of Legal Cases

• Problem: In predicting legal case outcomes, explanations are asimportant as accuracy.– Statistical methods do not provide explanations.– ML induced rules do not correspond to legal reasoning.

• Solution: Issue-Based Prediction (IBP) generates accuratepredictions and legally-intuitive explanations.– Weak logical domain model identifies issues raised.– CBR poses hypotheses about which side favored on each issue and evaluates

them against cases.– Domain model combines predictions on each issue.

• Experiments:– IBP more accurate than Naïve Bayes, C4.5, Ripper, Nearest-neighbor.– Combining weak model and cases more accurate than either alone.

S. Brüninghaus & K. Ashley, ICCBR-03 p. 65


4. Using Evolution Programs to Learn LocalSimilarity Measures

• Similarity: heuristic estimate of cases’ utility forparticular problem-solving situations.– Metric should include adaptability of cases.

• Problem: hard to acquire knowledge-intensive localsimilarity metrics from expert.– In particular, knowledge re utility and adaptability.

• Solution: Apply machine learning (i.e., geneticalgorithm) to extract info to compute utility of casesfor new problem situations.

Stahl, A. and Gabel, T. ICCBR-03 p. 537, Distinguished Paper Award


5. Extracting Performers’ Behaviors toAnnotate Cases in a CBR System for Musical

Tempo Transformations

• Transforming tempos:– When human performer plays a melody at different tempos, she does not apply

uniform transformation to all of the notes.– Performer makes interpretive decisions, depending on their relative importance.

• plays some additional notes• leaves out some notes, or• changes the duration of notes.

• Problem for automatic generation of expressive musical performance:– Human performers use musical knowledge not explicit in musical scores.

• Solution: Use the knowledge implicit in case examples of musicalperformance recordings.– Extract performers’ decisions and annotate the performance to make it explicit.

• System to annotate cases of musical performances.– Finds optimal alignment between score and transcription of human

performance, using edit distance algorithm

Arcos, J., Grachten, M. & de Mantaras, R. ICCBR-03 p. 20, Runner Up


Conclusions• ICCBR-03 reflects vibrant, mature research community.• CBR not just recommender and e-help systems, but progress in

– Learning knowledge-base similarity metrics.– Case adaptation.– Comparing and contrasting cases.– Integrating prediction and explanation.

• ICCBR-05: www.iccbr.org


Highlights: International Conference onCase-Based Reasoning 2003Part 2: Distinguished Paper Awards

Kevin Ashley (Presenter) & Derek BridgeCo-Chairpersons, Program Committee

www.iccbr.org/iccbr03/


Highlights: International Conference onCase-Based Reasoning 2003

1. Using Evolution Programs to Learn Local Similarity Measures(Stahl, A. and Gabel, T.) ICCBR-03 Proceedings p. 537,Distinguished Paper Award. 2003.

2. Extracting Performers’ Behaviors to Annotate Cases in a CBRSystem for Musical Tempo Transformations(Arcos, J., Grachten, M. & de Mantaras, R.) ICCBR-03Proceedings p. 20. Runner Up. 2003

Part 2: Distinguished Paper Awards


Highlighted Research Papers

1. Using Evolution Programs to

Learn Local Similarity Measures A. Stahl and T. Gabel

2. Extracting Performers’ Behaviors to Annotate Casesin a CBR System for Musical Tempo TransformationsJ. Arcos, M. Grachten, & R. de Mantaras

Input Problem

2. Match indices /Retrieve candidates



7. Explain result;Generalize &update casebase & index


No

Yes

Yes

No

Output

Indexed Case Database

Similarity Metric



6. Success?

ANY?

5. Success?


1. Using Evolution Programs to Learn LocalSimilarity Measures

• Similarity: heuristic estimate of cases’ utility forparticular problem-solving situations.– Metric should include adaptability of cases.

• Problem: hard to acquire knowledge-intensive localsimilarity metrics from expert.– In particular, knowledge re utility and adaptability.

• Solution: Apply machine learning (i.e., geneticalgorithm) to extract info to compute utility of casesfor new problem situations.

Stahl, A. and Gabel, T. ICCBR-03 p. 537, Distinguished Paper Award


Learning Through Utility Feedback

Case 8

Case 6

Case 2Case 5

Case 4

Case 1

Case 7Case 3

Case Order Feedback

Case 2

Case 1

Case 7Case 8

Case 4

Case 6

Case 5Case 3

Retrieval Result

Error Function E

SimilarityMeasure

CBR-System Similarity

Utility

Computes

Query

Feedback

Analyses

SimilarityTeacher

Stahl, A. and Gabel, T. ICCBR-05 p. 542


Why a Genetic Algorithm?• Evolutionary strategies good for searching a complex

search space for an optimum.• Hard to apply optimization techniques that minimize

error function’s value using its derivation.• Local similarity measures can be represented as

individuals within an evolutionary process.– Numeric similarity function is coded as a vector of

sampling points, between which the similarity function islinearly interpolated.

– Symbolic similarity table coded as a matrix.• Define mutation and crossover operators to create

new individuals.• Error function ≡ Index Error.


Experimental Evaluation

• PC domain: 11 attributes (5 numeric, 6 symbolic)– RAM-type, CPU-Clock, etc.– Define local similarity metric for each, but w/o considering

adaptability.• Training data S: 600 random queries yield 20 retrieved cases.

– Adapt these and reorder based on utility case order feedback• Divide S 1:2 into Strain and Stest; apply genetic algorithm to

minimize index error.• Apply newly optimized local similarity metrics to each query

in Stest.• Results: After 300 generations, genetic algorithm improves

two measures of retrieval quality by 26% and 14%.


Conclusion: Learning Local SimilarityMeasures (Stahl & Gabel 2003)

• Where case adaptation is applied, learningalgorithm improves quality of similarity measure.– It takes possibilities of case adaptation into account.

• First step toward learning knowledge-intensivelocal similarity measures automatically:– Avoids problems of manually defining similarity

measures.– Assumes that enough training data are available.


2. Extracting Performers’ Behaviors toAnnotate Cases in a CBR System for Musical

Tempo Transformations

• Transforming tempos:– When human performer plays a melody at different tempos, she does not apply

uniform transformation to all of the notes.– Performer makes interpretive decisions, depending on their relative importance.

• plays some additional notes• leaves out some notes, or• changes the duration of notes.

• Problem for automatic generation of expressive musical performance:– Human performers use musical knowledge not explicit in musical scores.

• Solution: Use knowledge implicit in case examples of musical performancerecordings.– Extract performers’ decisions and annotate performances to make it explicit.

• System to annotate cases of musical performances.– Finds optimal alignment between score and transcription of human

performance, using edit distance algorithm

Arcos, J., Grachten, M. & de Mantaras, R. ICCBR-03 p. 20, Runner Up


Tempo-Express Transforms Tempos

• CBR system to transform tempo of musical performances– Preserves expressivity in context of standard jazz themes.

• Cases comprise 20 scores of musical phrases x 11 annotatedperformances represented as:– Musical score (notes and chords)– Musical model automatically inferred from score– Annotations

• Annotations ≡ notes added, deleted, changed in durationrelative to the score.– Performer’s interpretive decisions


Tempo-Express CBR System

PerformanceAnalysis

SynthesisModule

MusicalAnalysis

AnnotationProcess

Retrieve Reuse

Retain

Score/m.model

Annot.perf.Tn

Annot.perf.T1…

Case Base

wav

PerformanceRecording

.mid

Score

DesiredTempo

wav

To

.xml

Score/m.model

Annot.perf.Ti

.xml

gen.perf.T0


Conclusion: Comparing Performances toScores (Arcos, et al. 2003)

• Crucial to identify optimal alignment:– which performance element corresponds to each note of the melody’s

score.– Liberal jazz interpretations omit or add notes, change expressions.

• Use edit distance for comparison:– Minimum total cost of transforming source sequence into target

sequence,– given set of allowed edit operations and cost of each operation.– Edit operations for performance annotation

• Deletion, insertion, transformation, consolidation, fragmentation

• Main factors in determining which of possible alignmentsbetween score and performance is optimal:– Features of note elements involved in calculating cost of applying each

operation– Relative costs of operations


Conclusions• ICCBR-03 reflects vibrant, mature research community.• CBR not just recommender and e-help systems, but progress in

– Learning knowledge-base similarity metrics.– Case adaptation.– Comparing and contrasting cases.– Integrating prediction and explanation.

• ICCBR-05: www.iccbr.org

Documents

International Conference on Case-Based Reasoning 2003 ... · –Generalized Shepard nearest neighbor algorithm ... •Planning: –Boosting algorithm to estimate pair-wise ... –Arguments