Advanced Knowledge Technologies

University of Aberdeen University of Edinburgh University of Sheffield Open University University of Southampton http://www.aktors.org

University of Aberdeen University of Edinburgh University of Sheffield Open University University of Southampton http://www.aktors.org

MIAKT Oxford University King’s College, London University of Sheffield Open University University of Southampton

MIAKT Oxford University King’s College, London University of Sheffield Open University University of Southampton

CoAKTinG

University of Edinburgh Open University University of Southampton

CoAKTinG

University of Edinburgh Open University University of Southampton

MIAKT: Medical Informatics and Knowledge Technologies

Supporting triple-assessment (TA) (collaborative decision-making) for the diagnosis and treatmentof breast cancer

Oxford UniversityKings CollegeOpen UniversityUniversity of SheffieldUniversity of Southampton

To support collaboration for the e-Scientist

Intelligent meeting spaces: Decision rationale, group memory capturePlanning, coordination support Instant messaging/presence

Open UniversityUniversity of EdinburghUniversity of Southampton

Advanced Knowledge Technologies

Representation and ReasoningOntologies: domain and process models Interoperability Integration with databases for scalability

(Semantic Web)Reasoning services – local and distributedNatural language processingApplication domain – academic CS

Representation and ReasoningOntologies: domain and process models Interoperability Integration with databases for scalability

(Semantic Web)Reasoning services – local and distributedNatural language processingApplication domain – academic CS

People involved

MIAKTOxford UniversityMike Brady, Jon Whitely King’s College LondonDavid Hawkes, Christine Tanner, Yalin ZhengThe Open UniversityEnrico Motta, John Domingue, Liliana CabralUniversity of SheffieldYorick Wilks, Fabio Ciravegna, Kalina BontchevaUniversity of SouthamptonNigel Shadbolt, Srinandan Dasmahapatra,Paul Lewis, Bo Hu, Hugh Lewis

MIAKTOxford UniversityMike Brady, Jon Whitely King’s College LondonDavid Hawkes, Christine Tanner, Yalin ZhengThe Open UniversityEnrico Motta, John Domingue, Liliana CabralUniversity of SheffieldYorick Wilks, Fabio Ciravegna, Kalina BontchevaUniversity of SouthamptonNigel Shadbolt, Srinandan Dasmahapatra,Paul Lewis, Bo Hu, Hugh Lewis

CoAKTinG

University of EdinburghAustin Tate Stephen PotterJessica Chen-burgerJeff DaltonOpen UniversityMarc EisenstadSimon Buckingham ShumJiri KomzakMichelle BachlerUniversity of SouthamptonDavid De RoureNigel ShadboltDanius MichaelidesRichard BealesKevin PageBen Juby

CoAKTinG

University of EdinburghAustin Tate Stephen PotterJessica Chen-burgerJeff DaltonOpen UniversityMarc EisenstadSimon Buckingham ShumJiri KomzakMichelle BachlerUniversity of SouthamptonDavid De RoureNigel ShadboltDanius MichaelidesRichard BealesKevin PageBen Juby

Breast Cancer – Statistics & Screening

EU: 24% of cancer cases 19% of cancer deaths

1 in 8 of women will develop breast cancer during the course of their lives

1 in 28 will die of the disease. 5 year survival rate for localized breast cancer

is 97% for early detection is 77% if the cancer has spread at diagnosis is 22% if distant metastases are found

Screening for ages 50+ M Brady: MIAS (features), e-Diamond (priors) Knowledge technology support

MIAKT: Patient Management --Triple Assessment

Imaging:Mammography/Ultrasound/MR

Ultrasound usually used for women under 35 (breasts too dense or solid to give a clear picture with mammography) It is also used to see if a breast lump is solid contains fluid (a cyst)

Mammography (X-ray)

Position breast on small flat plate, with X-ray plate under it. Flat plate above your breast. When machine is switched on, breast pressed down between plates by machine to get clearest picture. Two pictures are taken: from above and from the side.

Histopathology

Fine needle aspiration cytology Core biopsy With imaging guidance

Fine needle aspiration cytology Core biopsy With imaging guidance

MIAKT: Triple assessment

Triple Assessment

Clinical and radiological opinions are used independently to decide upon further intervention

The most suspicious opinion prevails (normal/definitely benign, probably benign, indeterminate, probably malignant)

Needle biopsy is mandatory for all abnormalities classified as indeterminate or more suspicious

Needle biopsy results are discussed in the context of imaging and clinical findings at multidisciplinary meetings

Clinical and radiological opinions are used independently to decide upon further intervention

The most suspicious opinion prevails (normal/definitely benign, probably benign, indeterminate, probably malignant)

Needle biopsy is mandatory for all abnormalities classified as indeterminate or more suspicious

Needle biopsy results are discussed in the context of imaging and clinical findings at multidisciplinary meetings

Radiology and Histopathology images: Different scales

SamePatient:multipledescriptors

SamePatient:multipledescriptors

Ontology, Annotation, Language Generation

Annotated images for retrieval in context Memory aids for specialists Report generation from annotations Ontologies for descriptive grounding Correlative reasoning across specialisation

(across scales orders of magnitude apart) Distributed reasoning (grid?)

Annotated images for retrieval in context Memory aids for specialists Report generation from annotations Ontologies for descriptive grounding Correlative reasoning across specialisation

(across scales orders of magnitude apart) Distributed reasoning (grid?)

Knowledge Engineering

Lexical and ontological issues Decision support Records – image and text

Diagnostic Mammography:different views

To pinpoint exact size & location of breast abnormalityand to image surrounding tissue and lymph nodes.

Cranio-caudal (CC) & Mediolateral oblique (MLO) views

med

iola

tera

lla

tero

med

ial

Diagnostic Mammography:BI-RADS Ontology - Masses

2. MARGINS2.a. Circumscribed2.b. Microlobulated2.c. Obscured2.d. Indistinct2.e. Spiculated

2. MARGINS2.a. Circumscribed2.b. Microlobulated2.c. Obscured2.d. Indistinct2.e. Spiculated

3. DENSITY:3.a. High density3.b. Equal density3.c. Low density3.d. Fat containing – radiolucent –

oil cyst, lipoma, or galactocele as well as mixed lesions such as hamartoma or fibroadenolipoma. [and/or histologic terms]

3. DENSITY:3.a. High density3.b. Equal density3.c. Low density3.d. Fat containing – radiolucent –

oil cyst, lipoma, or galactocele as well as mixed lesions such as hamartoma or fibroadenolipoma. [and/or histologic terms]

1. SHAPE 1.a. Round1.b. Oval1.c. Lobular1.d. Irregular

1. SHAPE 1.a. Round1.b. Oval1.c. Lobular1.d. Irregular

MASS: space occupying lesion seen in two different projections.

If potential mass seen in single projection, called DENSITY until 3-D confirmation.

MASS: space occupying lesion seen in two different projections.

If potential mass seen in single projection, called DENSITY until 3-D confirmation.

BI-RADS Ontology – Masses(details)

MASS: space occupying lesion seen in two different projections. If potential mass seen in single projection, called DENSITY until 3-D confirmation. 1. SHAPE a. Round: spherical, ball-shaped, circular or globularb. Oval: elliptical or egg-shaped. c. Lobular: has contours with undulations. d. Irregular: none of the above.

  2. MARGINS [modify the shape of the mass]a. Circumscribed Margins: abrupt transition between the lesion and the surrounding tissue. Without

additional modifiers there is nothing to suggest infiltration. b. Microlobulated Margins: undulate with short cycles producing small undulations. c. Obscured Margins: hidden by superimposed or adjacent normal tissue; cannot be assessed any further. d. Indistinct Margins: poor definition of margins raises concern of infiltration by the lesion; not likely due to

superimposed normal breast tissue. e. Spiculated Margins: lines radiating from margins of mass

3. DENSITY: x-ray attenuation of lesion relative to the expected attenuation of an equal volume of fibroglandular breast tissue; most cancers are of equal or higher density; never fat containing but may trap fat.

a. High densityb. Equal densityc. Low densityd. Fat containing – radiolucent - oil cyst, lipoma, or galactocele as well as mixed lesions such as hamartoma

or fibroadenolipoma. [When appropriate, histologic terms may be included]

Microcalcification

Microcalcifications are the most common mammographic sign of ductal carcinoma in situ

P(micro-Ca | DCIS)=0.9

Microcalcifications are the most common mammographic sign of ductal carcinoma in situ

P(micro-Ca | DCIS)=0.9

Microcalcifications: (<.5mm) specks of calcium in milk ducts.

About half of the cancers detected by mammography appear as a cluster of microcalcifications.

Microcalcifications: (<.5mm) specks of calcium in milk ducts.

About half of the cancers detected by mammography appear as a cluster of microcalcifications.

BI-RADS: Calcifications

Benign calcifications usually larger than malignant ones -- coarser, often round with smooth margins, more visible.

Malignant calcifications very small, require magnifying glass.

When specific aetiology not possible, description of calcifications should include their distribution and morphology.

Benign calcifications only reported if judged to be susceptible to misinterpretation.

Multiple views on domain

Structure, anatomy Function, physiology Pathology Patient history

BI-RADS Ontology: Calcification

Typically benign Intermediate concern Higher probability Distribution modifiers

BI-RADS Ontology: Calcification

1. 1. TYPICALLY BENIGNTYPICALLY BENIGN

1.a. Skin Calcifications 1.a. Skin Calcifications

1.b. Vascular Calcifications1.b. Vascular Calcifications

1.c. Coarse Calcifications1.c. Coarse Calcifications

1.d. Large Rod-Like 1.d. Large Rod-Like CalcificationsCalcifications

1.e. Round Calcifications1.e. Round Calcifications

1.f. Lucent-Centered 1.f. Lucent-Centered CalcificationsCalcifications

1.g. Eggshell or Rim 1.g. Eggshell or Rim CalcificationsCalcifications

1.h. Milk of Calcium 1.h. Milk of Calcium CalcificationsCalcifications

1.i. Suture Calcifications1.i. Suture Calcifications

1.j. Dystrophic 1.j. Dystrophic CalcificationsCalcifications

1.k. Punctate Calcifications1.k. Punctate Calcifications

2. INTERMEDIATE CONCERN

2.a. Amorphous or Indistinct Calcifications

2. INTERMEDIATE CONCERN

2.a. Amorphous or Indistinct Calcifications

3. HIGHER PROBABILITY OF MALIGNANCY

3.a. Pleomorphic or Heterogeneous Calcifications

3.b. Fine, Linear or Branching Calcifications

3. HIGHER PROBABILITY OF MALIGNANCY

3.a. Pleomorphic or Heterogeneous Calcifications

3.b. Fine, Linear or Branching Calcifications

4. DISTRIBUTION MODIFIERS

4.a. Grouped or Clustered

4.b. Linear4.c. Segmental4.d. Regional: 4.e.Diffuse/Scattered

4. DISTRIBUTION MODIFIERS

4.a. Grouped or Clustered

4.b. Linear4.c. Segmental4.d. Regional: 4.e.Diffuse/Scattered

BI-RADS Ontology: Calcification (details)

TYPES AND DISTRIBUTION OF CALCIFICATION:1. TYPICALLY BENIGN -

a. Skin Calcifications: typical lucent centered deposits that are pathognomonic. Atypical forms confirmed by tangential views to be in the skin.

b. Vascular Calcifications: Parallel tracks, or linear tubular calcifications clearly associated with blood vessels.c. Coarse Calcifications: Classic calcifications produced by an involuting fibroadenoma.d. Large Rod-Like Calcifications: Continuous rods, occasionally branching, diameter > 1mm usually, may

have lucent centers, if calcium surrounds rather than fills an ectactic duct. Found in secretory disease, "plasma cell mastitis", and duct ectasia.

e. Round Calcifications: When multiple, of variable size. Considered benign and when small [under 1 mm], frequently formed in acini of lobules. Under 0.5 mm are termed punctate.

f. Lucent-Centered Calcifications: Less that 1 mm to greater than 10 mm, smooth surfaces, round or oval, have lucent center. “Wall" thicker than the "rim or eggshell" type. Included are areas of fat necrosis, calcified debris in ducts, and occasional fibroadenomas.

g. Eggshell or Rim Calcifications: Very thin, under 1mm thickness, appear as calcium deposited on the surface of a sphere. Although fat necrosis can produce these thin deposits, calcifications in the wall of cysts are the most common "rim" calcifications.

h. Milk of Calcium Calcifications: Consistent with sedimented calcifications in cysts. Often less evident in craniocaudal image -- appear as fuzzy, round, amorphous deposits; sharply defined on 90° lateral -- semilunar, crescent shaped, curvilinear (concave up), or linear defining dependent portion of cysts.

i. Suture Calcifications: Ca deposited on suture material, relatively common in post-irradiated breast, typically linear or tubular in appearance and knots are frequently visible.

j. Dystrophic Calcifications: Usually form in irradiated breast or following trauma. Irregular in shape, usually > 0.5 mm, often have lucent centers.

k. Punctate Calcifications: Round/oval, < 0.5 mm with well-defined margins.

Example Mammogram

2 cm mass (tumour) 2 cm mass (tumour) microcalcifications microcalcifications

Significantabnormality

Microcalcifications: Clinical Procedures (guideline)

Further Ultrasound andMammography

Microcalcifications

Discharge!

Clinical exam+

Needle core biopsyWith

Specimen radiography

Triple assessment

Equivocalresult

Treatment

benign

suspicious

Normal or Definitely benign

Clusteredheterogeneous

Histopathology

FFNAC

Descriptors when drawing sample:

Sampling pattern for stereotactic FNAC

mm

•Fine needle aspiration cytology•Core biopsy•With imaging guidance

Histopathology slides

A histological slide has an immense amount of data, the closer you look, the more there is

Histological images are complex with challenges at both the segmentation and feature classification level

Think in terms of two scales – low-power, high-power

HistopathologyHistopathology

Histopathology slides:low power/high power

Histopathology slides: diagnostic criteria

Reporting guidelines

MIAKT: Technology Palette

MIAS – Medical image registration (X-ray, MR)Segmentation and feature extraction Image Classification

AKT – Ontology development (Distributed) reasoning services Image annotation against ontologiesNatural language generationDecision support – belief nets?

Abstract away from the details of TA meeting Collaborative problem solving/decision making Possibly distributed, virtual presence Well-defined goals, well-defined contributory

skill sets Structured protocol Require recall of contents of events of past

meeting Report generation (audit trail)

MIAKTMIAKT

Enhance Technologies

Ontologically annotated audio/video streams Issue handling, tasking, planning and coordination Collective sense-making and group memory capture Enhanced presence management and visualisation Adaptive information systems

Technology Integration

Aim: To support e-Science collaboration by integrating and demonstrating the utility of:

intelligent task-orientated messaging, collaborative planning, issue, activity and constraint management (I-X Process Panels/<I-N-C-A>: Edinburgh)

peripheral awareness of the online presence, availability, attributes and location of colleagues, documents, and devices (BuddySpace: OU)

real time conversational mapping of meetings, providing shared visual focus and group memory capture (Compendium: OU)

multimedia meeting mark-up, replay and navigation (HyStream: Southampton)

Jabber

Jabber is a set of XML-based protocols for real-time messaging and presence notification

Communicates with other instant messaging services through gateways

Many clients available – see

http://www.jabbercentral.org/

JabberJabber

Compendium

Provides a methodological framework, plus an evolving suite of tools, for collective sense-making and group memory.

Intersection of collaborative modelling, organisational memory, computer-supported argumentation and meeting facilitation. 

Centres on face-to-face meetings, potentially the most pervasive knowledge-based activity in working life, but also one of the hardest to do well.

Compendium

BuddySpace

‘Enhanced Presence Management for Collaborative Working, Messaging, Gaming and Beyond’

The concept of presence is a rich combination of attributes that characterise an individual's…– physical and/or spatial location– work trajectory– time frame of reference– mental mood– goals and intentions

http://kmi.open.ac.uk/projects/buddyspace/

BuddySpace

CompendiumCompendium

Process Panels

Based on notion of the representation of a product as a set of nodes making up the components of the product model, along with constraints on the relationship between those nodes and a set of outstanding issues

Investigates the use of shared models for task directed communication between human and computer agents who are jointly exploring a range of alternative options for activity.

Process Panels

HyStream

Smart spaces

Devices in the room enables us to capture continuous (real time, multi-way, multi-cast, ontologically informed, …) metadata

Other devices provide ‘presence’ information Consider an experimental laboratory instead of

a meeting room: Instruments Electronic log books Visualisation