B. Ross Cosc 4f79 1 Knowledge acquisition (Ch. 17 Durkin) • knowledge engineering: building expert systems • knowledge acquisition: process of extracting knowledge from an expert, organizing it, and encoding it into a knowledge base • knowledge elicitation: extracting knowledge from an expert • knowledge acquisition is the principle bottleneck in expert system development • many techniques and theories about how to best do this • more tools are appearing to help in this – early example: inductive inference tables • active research area – psychologists are especially interested in elicitation issues, as it is a fundamental problem of human psychology
Ethical and Legal issuesknowledge acquisition: process of
extracting knowledge from an expert, organizing it, and encoding it
into a knowledge base
knowledge elicitation: extracting knowledge from an expert
knowledge acquisition is the principle bottleneck in expert system
development
many techniques and theories about how to best do this
more tools are appearing to help in this
early example: inductive inference tables
active research area
psychologists are especially interested in elicitation issues, as
it is a fundamental problem of human psychology
B. Ross Cosc 4f79
Some problematic phenomena
1. Paradox of expertise: The more competent a domain expert is, the
less
able she is to describe the knowledge they use to solve
problems.
- studies & experience shows that experts are experts because
they
compile their vast knowledge into compact, efficiently retrievable
form
- as a result, they ignore lots of details about how they derive
conclusions
--> intuition is prevalent; structured principles are
ignored
- for example, experts use lots of generalization and pattern
matching to
solve standard and new problems
2. Experts make bad knowledge engineers
- domain experts are the worst people for formalizing their own
knowledge
- non-objective, unfamiliar with AI technology, ...
- need an objective view of knowledge, which isn’t possible from
expert
- eg. try to formalize how you go about creating a computer
program to solve some problem
B. Ross Cosc 4f79
• unaware of the deep reasoning; use shallow reasoning
ie. often short-term memory isn’t used;rather, long-term memory as
obtained
via past experiences is relied upon
---> huge gaps in knowledge
• because experts don't know the formal structure of their
knowledge,
their descriptions will likely be wrong
- they aren’t used to verbalizing their expertise!
• therefore, knowledge engineer must watch for knowledge that
is...
- irrelevant, incomplete, incorrect, inconsistent
B. Ross Cosc 4f79
- most difficult step
- lots of strategies
3. Analyze:
4. Design:
B. Ross Cosc 4f79
Interviews and questions
Interacting with the expert is the primary means of eliciting
knowledge
17.9, 17.10
Interview strategies
there are different interview techniques; some are suited to
different phases of the elicitation process
Funnel sequencing technique: interview progresses from general,
exploratory questions, to detailed questions
Prompts Indirect Beginning of topic ; General
Probes
SUMMARIZE INTERVIEW
1. Unstructured interview
a spontaneous, natural means to let expert talk freely on anything
in domain
expert verbalizes responses to general questions asked by KE
stream of consciousness sometimes used
KE keeps a minimal level of focus on topics discussed
goal: not to let KE unduly influence early explorations of
knowledge
17.14, 17.15
much more focussed and disciplined than unstructured
interview
KE’s task is to discover concrete information about specific
questions
topic to be explored has been established at earlier sessions
not as exploratory as unstructured --> better for advanced
phases
17.18, 17.19
Interviewing is primary means of knowledge elicitation.
However, there are weaknesses:
easier to “do” than to describe
plus some knowledge (physical, artistic) not easily
verablized
ineffective long-term memory
compiled knowledge is difficult to reconstruct
Case studies: another strategy useful in concert with
interviews
B. Ross Cosc 4f79
ask expert to review and explain a solved case
expert goes over all the steps, explaining as she or he goes
KE will record the protocol: the sequence of problem-solving steps
or strategies used by expert
types of case studies:
general info is obtained
b) unusual case: a new problem hereforeto unseen by expert
good way to get deeper, detailed, more introspective expert
feedback
best for intermediate, later stages
17.22, 17.23
4. Observational case study
rather than giving expert the whole case, just supply the problem
description
then watch & record the expert as he or she solves the
problem
stream of consciousness useful
familiar: more general knowledge obtained
unfamiliar: detailed, deeper insight into problem solving
obtained
17.26, 17.27, 17.30, 17.31
B. Ross Cosc 4f79
Analyzing the knowledge
1. data from expert interview & observation is then transcribed
into text form
important to document all data: date, who, what,...
2. the text is interpreted
identifying “chunks”: labelling key parts of the knowledge
what portions of knowledge? what are they?
3. Analyzing (“sorting”) the knowledge:
interrelating the knowledge with previous sessions
determining it’s representation in domain-friendly notation
converting it to KB language
this is done iteratively and incrementally
must pass it by expert for confirmation and corrections
knowledge dictionary: akin to “data dictionary” in DB systems
a system document that indexes all terms, rules, etc
B. Ross Cosc 4f79
Knowledge analysis
Graphical representation of knowledge is an effective means of
organizing it
both KE and expert can understand
idea is that graphical notations close the “semantic gap” between
expert knowledge and formalized form
Some techniques
inference networks/trees: AND-OR tree
decision tree
B. Ross Cosc 4f79
Conclusion
• research in AI, psychology is forming models of how people &
experts
organize knowledge, learn, and do problem solving
- these models will give means for determining the best way
to
extract knowledge from experts, and encode it into a KB
• in the meantime, knowledge engineers (experts themselves) rely
on
experience for acquiring knowledge and constructing expert
systems
- what about: an expert system for creating expert systems?
• KE is quite an interesting and challenging
- lucrative profession