Upload
morris-ryan
View
225
Download
6
Embed Size (px)
Citation preview
An Agent Based Architecture For Designing Flexible Image Analysis Systems
Jagath Samarabandu
Dept. of Electrical and Computer EngineeringUniversity of Western Ontario
Multi-dimensional Imaging System
Capture Processing Visualization
Storage
Confocal Microscopy
Conventional Microscopy with Deconvolution
Tomography with Reconstruction
MR, CT, PET, SPECT, US
ROI Extraction
Digital Filtering
Segmentation
Analysis
Morphometry
Sectional Views
MIP/Sum Projections
Surface Shading
Volume Rendering
Stereo views
Animation Sequences
Interactive displays
Raw data
Region dataProcessed dataHigh level data
Single frameStereo frameMovie sequences
Acknowledgements Kamal Ranaweera – MESc student who
is doing most of the work Dr. Aaron Fenster for his mentorship CITO Round 3.2 research grant
Outline Why image analysis Image processing basics Image analysis paradigm Analysis system example A flexible framework for image analysis Knowledge organization Current status Outlook
Why Image Analysis Image dimensions are increasing Image sizes and resolutions are
increasing Image quality is getting better Cannot visually assess all the voxels
Image Processing Point Operations
Histogram Equalization Contrast Stretching
Spatial Operations Linear Filtering
Low/High pass filtering Edge Detection
Non-Linear Filtering Median filtering Morphological Filtering
Point OperationsHistogram EqualizationIntensity histogram of the output image has a uniform distribution
Spatial Operations Linear Filtering
g(x,y) = ijh(i,j) u(i+x, j+y) where u(x,y) is the input image intensity at
(x,y) and h(i,j) is the value of convolution kernel at (i,j)
1 1 11 1 11 1 1
-1 0 1-1 0 1-1 0 1
-1 -1 -1 0 0 0 1 1 1
MovingAverage
Pair of kernels for edge detection
Example kernels
Edge Detection Operators Gradient Operators
Finite difference approximations of orthogonal gradients (e.g.. hx and hy for 2D images)
gm (x,y) = gx(x,y)2 + gy(x,y)2
g(x,y) = tan -1 (gy(x,y) / gx(x,y))
-1 0 1 -1 -1 -1-1 0 1 0 0 0-1 0 1 1 1 1
-1 0 1 -1 -1 -1-1 0 1 0 0 0-1 0 1 1 1 1
0 1 1 0-1 0 0 -1
Roberts Prewitt Sobel
Edge Detection Operators Compass Operators
Directional gradients at a given number of directions (h1, h2, h3 .. hn)
gm (x,y) = MAXk {| gk (x,y)|} g(x,y) is k at which the maximum occurred
-1 0 1-1 0 1-1 0 1
0 1 1-1 0 1-1 -1 0
1 1 1 0 0 0-1 -1 -1
1 1 0 1 0 -1 0 -1 -1
1 0 -1 1 0 -1 1 0 -1
0 -1 -1 1 0 -1 1 1 0
-1 -1 -1 0 0 0 1 1 1
-1 -1 0-1 0 1 0 1 1
Kirsh templates for 8 directions
Edge Operators
Roberts Prewitt Sobel
Compass Kirsch Laplace
Spatial Operators Linear Filters
Gaussian smoothing Edge detection operators
Non Linear Filters Median filter Rank order filters morphological filters
Spatial Filters for Smoothing
Gaussian Noise Gaussian (7x7) Median (5x5)
Speckle noise Median (5x5)Gaussian (7x7)
High Level Processing Feature Extraction
Edges and Boundaries Shape and Moments Texture
Segmentation Matching Thresholding Boundary Detection
Classification Statistical Classification Clustering
Canny’s Edge OperatorCombines edge operators and adaptive thresholding with
hysterisys to locate edges.
Image Analysis
PictureInterpretation
Image
Picture DomainCues
Scene DomainCues
Model
InstantiatedModel
Scene Domain
Picture Domain
Signal
Physical
Semantic
Kanade’s paradigm
Bottom Up Cycle Picture domain cues are primitive image
features Scene domain cues are the
interpretation of image features This distinction is somewhat blurred for
3D images
Top Down Cycle Model instantiation or hypothesis
generation Verification and refinement
Data Driven Approach Example
Median Filtering
Difference of Gaussian
Low Pass filter
Boundary Forming
Thresholding
Feature Aggregation
Feature Aggregation
This technique was able to suppress concavities
It also tended to diffuse boundaries of small structures
lyxyxH
L
yxyxl
lyx
lyx
lyx
yxeE
i
iii
iii
i
Ekwii
,| and
Otherwise
if
if
Where
,,max
min
,,
1,
1,,
00
0000
Processing Confocal Images
Single section of the original image
2D median filtering
Processing Confocal Images
Difference of Gaussian filter 3D contours of final segmentation
Detecting DNA Replication Sites Bottom up
Intensity maxima are detected on 2D image slices
Sites cross sections are modeled as circles centered on these maxima
Top Down Site boundaries are then refined according to the
local image features These boundaries are then combined to form 3D
structures using 3D connected component labeling
DNA Replication Sites (MIP)
Segmented Replication Sites
Further Analysis
Further Analysis
Blackboard Model Provides us an elegant architecture
within which both data driven and model driven approaches can be incorporated
A problem is solved in an incremental fashion
Contains a set of knowledge sources or tools
Blackboard Model These tools are designed as
independent modules They interact with each other via a
global database called the blackboard Modules that generate solutions are
separated from those that determine their utility
Blackboard Models can handle Large solution space Noisy and unreliable data Variety of input data and a need to
integrate diverse information The need to use many independent or
semi-independent sources of knowledge to co-operate in forming a solution
Blackboard Models can handle The need to use multiple reasoning
methods (eg. Forward and backward) The need for multiple lines of reasoning The need for an evolutionary solution
Requirements for the Framework
Represent and use knowledge about task domain and processing tools
Evaluate the quality of the image processing algorithms
Transfer information between different tools and knowledge sources
Control the flow of processing Allow easy addition of new tools
Requirements for Processing Tools
Implements the underlying processing algorithm
Contains knowledge about its algorithm Can evaluate its suitability (matching
score) to accomplish a given task Communicates with the framework
Knowledge Organization Domain Expertise
Knowledge about the image Intensity characteristics of the structure of
interest Image Processing Expertise
Knowledge about image processing algorithms (e.g. filtering, detection) and their parameters
Domain Knowledge Initial Model (e.g. circle, line, sphere) Parameter value distribution Expected orientation Boundary definition method (e.g.
intensity +/-, texture, color etc.) Boundary definition strength (strong,
medium, weak) Boundary dispersion (short, medium,
long)
Image Processing Knowledge: Filter Class
Filter type (Gaussian, median, morphological etc.)
Frequency response (low pass/high pass) Linearity (linear/non-linear) Filter parameters (e.g. kernel size,
sigma, structuring-element etc.) Performance against different noise
types (Gaussian, shot noise)
Image Processing Knowledge: Detector Class
Detector name Primitive type
Intensity gradient edge Intensity peak Region
Effective parameter range Confidence against
Strong/medium/weak boundary definition short/medium/long boundary dispersion
Knowledge Representation Information is
encoded using XML Attribute names and
value types are defined for each piece of information
Sub-classing to avoid duplicating information
A 0-1 rating is used to classify behavior when applicable
<filter type=“median” linear=“no” sub-type-of “rank”>
<transfer-function type=“lowpass”/>
<parameter name=“window” min=3 max=100/>
<noise-performance type=“gaussian” rating=0.5/>
<noise-performance type=“shot” rating=0.9/>
</filter>
Design of Tools Each tool is an independent object (agent) Able to emit the XML description of self for
use by the framework (or others) All tools are sub-classed from a base type
that has three primary “methods” Judicious use of OOP features such as
“inheritance” and “polymorphism” helps the design of tools and the framework
Implementing the Tools All tools have the following functionality
Self – Describe itself using XML notation Match – Gives a compatibility score against
an input description SetInput – Defines the input data Process – Performs the processing operation GetOutput – Extract processed data
Example current.SetInput(prev);current.Process();
next.SetInput(current); …
Processing
SequenceGeneratio
n
Processing
Sequence Execution
Tool Library
Task Descriptio
n
Image Descriptio
n
Output
User Control
System Architecture
Framework First implementation was script based Simply arranged the utilities in the
prescribed order Performed the sequence of operations Had limited user interface for arranging
operations Contained two detectors (peak and edge) Mostly used as a test bed to develop the
tool interface
Current Implementation Resembles an expert system Inputs are
Detector type (peak, line) Edge strength and diffusion Structure size Noise level
Creates a sequence of operations Performs the sequence at user’s request
Limitations of the Current System
Linear progression of tools Tools with multiple inputs are not yet
supported Iterative processing methods are not
supported XML tools are not fully mature yet. Not enough tools
Future Work Addressing current limitations Using this system as a knowledge
acquisition tool Research on machine learning Possibilities are endless!
Summary Image analysis helps managing large
amounts of data and extract new information
Agent based framework attempts to use multiple processing tools to improve performance
Incorporating knowledge about analysis tools helps create a user centered system
Flexibility comes from being able to add new agents without modifying the framework
Thank You!
Aruni admiring a Water Lilly