Thesis, Image Registration Methods

Image Registration Methods for Reconstructing a Gene Expression

Atlas of Early Zebrafish Embryogenesis

Evangelia BalanouMASTER THESIS

EUROPEAN POSTGRADUATE PROGRAM ON BIOMEDICAL ENGINEERINGUNIVERSITY OF PATRAS – NATIONAL TECHNICAL UNIVERSITY OF ATHENS

DEPARTMENT OF ELECTRONIC ENGINEERING TECHNICAL SCHOOL OF TELECOMMUNICATIONS ENGINEERING

TECHNICAL UNIVERSITY OF MADRID

Outline

Introduction

Motivation

Problem

ImageRegistration

Transformation

Interpolation

Similarity Measure

Optimization

Resampling

Design & Implementation

Concept

Overview

Registration Pipeline

Atlas Construction Pipeline

Implementation

Results & Evaluation

Comparison of Registration Methods

Conclusions & Future Work

• Introduction– Motivation– Problem– Goal

• Image Registration– Components

• Design and Implementation– Concept– Overview– Registration Pipeline– Atlas Construction Pipeline– Tools– Implementation

• Results and Evaluation– Comparison of Registration Methods– Atlas Construction

• Conclusions and Future Work

Outline

IntroductionMotivation

Problem to be solved

Motivation

Early development of a zebrafish embryo

• Study the genes that regulate embryonic development (developmental biology)

• Study embryonic development of vertebrates:– Vertebrate developmental disorders– Human hereditary disease

• Vertebrate model: zebrafish– Rapidly developing transparent embryos– Small size (4-5 cm length)– Short generation time

Outline

Introduction

Motivation

Problem

ImageRegistration

Transformation

Interpolation

Similarity Measure

Optimization

Resampling

Concept

Overview

Implementation

• Quantitative spatio-temporal data at cellular level about gene expression required

Provided by Fluorescence In Situ Hybridization techniques and Laser Scanning Microscopy

Problem

Second gene expression pattern

One gene expression pattern

Outline

Introduction

Motivation

Problem

ImageRegistration

Transformation

Interpolation

Similarity Measure

Optimization

Resampling

Concept

Overview

Implementation

• Qualitative spatio-temporal data at cellular level about gene expression required

Provided by Fluorescence In Situ Hybridization techniques and Laser Scanning Microscopy

Problem

• However, not more than five gene expression patterns simultaneously revealed on the same embryo!

Image processing methods to integrate different expression patterns (from different embryos) into a 3-D gene expression atlas

Outline

Introduction

Motivation

Problem

ImageRegistration

Transformation

Interpolation

Similarity Measure

Optimization

Resampling

Concept

Overview

Implementation

GoalDesign and implement an image processing framework able to register different datasets with different gene expression patterns to a common template at a given developmental stage

Template One datasetTemplate + registered image

“Registration is the process of determining a geometrical transformation that aligns points in one view of an object with corresponding points in another view of that object or another object.”

Outline

Introduction

Motivation

Problem

ImageRegistration

Transformation

Interpolation

Similarity Measure

Optimization

Resampling

Concept

Overview

Implementation

Image RegistrationFundamental task in image processingVarious techniques (data, application)

Image Registration• Intensity-based :

Calculates the transformation using voxel values alone

• Input: 2 images – fixed, moving Output: geometrical transformation

• Optimization problem

• Decomposed into a set of basic elements (defining different methods)

Optimization

Transformation

Resampling

Fixedimage

Movingimage

Registeredimage

TransformationParameters

Registration

Similarity measure

Interpolation

Initial Parameters

Outline

Introduction

Motivation

Problem

ImageRegistration

Transformation

Interpolation

Similarity Measure

Optimization

Resampling

Concept

Overview

Implementation

Transformation

Optimization

Transformation

Resampling

Fixedimage

Movingimage

Registeredimage

Registration

Similarity measure

Interpolation

Initial Parameters

• Defines the type of parameters whose values align the two images (search space)

• Spatial mapping of points from the fixed image space to points in the moving image space (inverse mapping)

Outline

Introduction

Motivation

Problem

ImageRegistration

Transformation

Interpolation

Similarity Measure

Optimization

Resampling

Concept

Overview

Implementation

Interpolation

Optimization

Transformation

Resampling

Fixedimage

Movingimage

Registeredimage

Similarity measure

Interpolation

Initial Parameters

• Evaluate moving image intensities at the mapped, non-grid positions

Outline

Introduction

Motivation

Problem

ImageRegistration

Transformation

Interpolation

Similarity Measure

Optimization

Resampling

Concept

Overview

Implementation

Similarity Measure

Optimization

Transformation

Resampling

Fixedimage

Movingimage

Registeredimage

Similarity measure

Interpolation

Initial Parameters

• A measure of “how well” fixed and transformed moving match each other• Provides a quantitative criterion to be optimized over the search space

(similarity measure function, S(T) )• The desired optimum may be one of the local ones

Outline

Introduction

Motivation

Problem

ImageRegistration

Transformation

Interpolation

Similarity Measure

Optimization

Resampling

Concept

Overview

Implementation

• Correlation Coefficient

Similarity Measures

• Mutual Information

Intensities in two images linearly related As written, function to be maximized

Intensities in two images statistically related As written, function to be maximized

Moving Image intensity (I1)

Outline

Introduction

Motivation

Problem

ImageRegistration

Transformation

Interpolation

Similarity Measure

Optimization

Resampling

Concept

Overview

Implementation

Moving Image intensity (I1)

Optimization

Transformation

Resampling

Fixedimage

Movingimage

Registeredimage

Similarity measure

Interpolation

Initial Parameters

• Most complex component• Starting from an initial set of parameters, iteratively searches the optimal

solution of the similarity measure function over the parameter space defined by the transformation

• Stops when stopping criterion is met

Transformation Parameters

Outline

Introduction

Motivation

Problem

ImageRegistration

Transformation

Interpolation

Similarity Measure

Optimization

Resampling

Concept

Overview

Implementation

Optimization Algorithms• Gradient Descent

• Differential Evolution

Derivative of similarity measure function (S) wrt to each transformation parameterAttracted by local extrema

Stochastic, population-basedGlobal optimization technique – slow in computation

nn pSpp

Outline

Introduction

Motivation

Problem

ImageRegistration

Transformation

Interpolation

Similarity Measure

Optimization

Resampling

Concept

Overview

Implementation

Initialization Mutation Recombination Selection

Cost FunctionOutline

Introduction

Motivation

Problem

ImageRegistration

Transformation

Interpolation

Similarity Measure

Optimization

Resampling

Concept

Overview

Implementation

Local optimization

Outline

Introduction

Motivation

Problem

ImageRegistration

Transformation

Interpolation

Similarity Measure

Optimization

Resampling

Concept

Overview

Implementation

Local optimization

Outline

Introduction

Motivation

Problem

ImageRegistration

Transformation

Interpolation

Similarity Measure

Optimization

Resampling

Concept

Overview

Implementation

Global optimization

Outline

Introduction

Motivation

Problem

ImageRegistration

Transformation

Interpolation

Similarity Measure

Optimization

Resampling

Concept

Overview

Implementation

Global optimization

Outline

Introduction

Motivation

Problem

ImageRegistration

Transformation

Interpolation

Similarity Measure

Optimization

Resampling

Concept

Overview

Implementation

Conclusions & Future Work Capture range of correct optimum (initial parameter range or initialization)

Resampling

Optimization

Transformation

Resampling

Fixedimage

Movingimage

Registeredimage

Similarity measure

Interpolation

Initial Parameters

• Once a stopping criterion is met or iteration number has reached, the last transformation parameters are used to produce the registered image

Outline

Introduction

Motivation

Problem

ImageRegistration

Transformation

Interpolation

Similarity Measure

Optimization

Resampling

Concept

Overview

Implementation

Design & ImplementationImplemented framework’s concept

Different steps it is composed of

Concept

Template embryoPartial views

Nuclei channel

Reference gene channel(goosecoid)

Another gene channel

*All images are 3D and grayscale *Colourmap just for visualization

Outline

Introduction

Motivation

Problem

ImageRegistration

Transformation

Interpolation

Similarity Measure

Optimization

Resampling

Concept

Overview

Implementation

Goal: Design and implement an image processing framework able to register different datasets with different gene expression patterns to a common template at a given developmental stage

ConceptTemplate embryoPartial views

Registration

Gene expression atlas

Nuclei channel

Reference gene channel

Nuclei channel

Reference gene channel

Another gene channel

Partial view of another embryo

*All images are 3D and grayscale *Reference gene (position): goosecoid (gsc)*Colourmap just for visualization

Outline

Introduction

Motivation

Problem

ImageRegistration

Transformation

Interpolation

Similarity Measure

Optimization

Resampling

Concept

Overview

Implementation

Overview

Moving image

Whole embryo view, nuclei channel

Whole embryo view, gsc channel

Partial embryo view, nuclei channel

registration

Registered image

Partial embryo view, gsc channel

Fixed image

addition

initialization

Fixed image

Initialized Moving image

preprocessing

Rotation centre

Partial embryo view, third channel

preprocessing transformation Third channel mapped

Registration pipeline

Atlas construction pipeline

initialization

Gravity centres

Outline

Introduction

Motivation

Problem

ImageRegistration

Transformation

Interpolation

Similarity Measure

Optimization

Resampling

Concept

Overview

Implementation

Moving image

registration

Registered image

Fixed image

addition

initialization

Fixed image

preprocessing

Rotation centre

initialization

Gravity centres

Purpose: Determine the transformation parameters that bring into spatial alignment the template and one partial view

Outline

Introduction

Motivation

Problem

ImageRegistration

Transformation

Interpolation

Similarity Measure

Optimization

Resampling

Concept

Overview

Implementation

Preprocessing & Addition Step

Moving image

registration

Registered image

Fixed image

addition

initialization

Fixed image

preprocessing

Rotation centre

Purpose: Remove noise, blur, downsample, thresholdCombine information from nuclei and gsc channels into a single image

Outline

Introduction

Motivation

Problem

ImageRegistration

Transformation

Interpolation

Similarity Measure

Optimization

Resampling

Concept

Overview

Implementation

Preprocessing & Addition Step

• Preprocessing depends on images (noise, size)• Weighted Addition

additionpreprocessing

preprocessingOriginal gsc channel

Original nuclei channel Combined

Original channels Preprocessed channels Combined image

addition

preprocessing

Resolution: 512 x 512 x 465Voxel size: 1.517 x 1.517 x 1,509μm

Resolution: 128 x 128 x 116Voxel size: 6.068 x 6.068 x 6.036μm

Outline

Introduction

Motivation

Problem

ImageRegistration

Transformation

Interpolation

Similarity Measure

Optimization

Resampling

Concept

Overview

Implementation

Initialization Step

Moving image

registration

Registered image

Fixed image

addition

initialization

Fixed image

preprocessing

Rotation centre

Purpose: Initial positioning of moving to fixed image’s space (no initial parameters in registration)

If NOT sufficient overlapping, registration fails

Outline

Introduction

Motivation

Problem

ImageRegistration

Transformation

Interpolation

Similarity Measure

Optimization

Resampling

Concept

Overview

Implementation

• Based on nature of data (nuclei and gsc channel)

• For both views one gravity centre from each channel

• The resulting four points define a spatial transformation that is applied on the moving image

Initialization StepPreprocessed partial embryo view, nuclei channel (binary)

Preprocessed partial embryo view, gsc channel

Rotation centre

Moving image

Fixed image

initialization

Preprocessed whole embryo view, nuclei channel (binary)

Preprocessed whole embryo view, gsc channel

Outline

Introduction

Motivation

Problem

ImageRegistration

Transformation

Interpolation

Similarity Measure

Optimization

Resampling

Concept

Overview

Implementation

Initialization Step

gscfixed

nfixed

nmoving

gscmoving

Translated nmoving

Rotation axis

Rotation angle

translation

nfixed

gscfixed

nmoving

gscmoving

Fixed (template view)

Moving (partial view)

*Blue/Orange-nuclei Green/Yellow-gsc expression pattern

Translated nmoving

Outline

Introduction

Motivation

Problem

ImageRegistration

Transformation

Interpolation

Similarity Measure

Optimization

Resampling

Concept

Overview

Implementation

Initialization Step

Before initialization

After Initialization

Fixed (template) + Initialized Moving (partial) Partial view before and after initialization

Initialized Moving

Fixed Image

Outline

Introduction

Motivation

Problem

ImageRegistration

Transformation

Interpolation

Similarity Measure

Optimization

Resampling

Concept

Overview

Implementation

Registration Step

Moving image

registration

Registered image

Fixed image

addition

initialization

Fixed image

preprocessing

Rotation centre

Purpose: Find the transformation parameters that register the initialized moving image to the fixed

Outline

Introduction

Motivation

Problem

ImageRegistration

Transformation

Interpolation

Similarity Measure

Optimization

Resampling

Concept

Overview

Implementation

Registration StepOutline

Introduction

Motivation

Problem

ImageRegistration

Transformation

Interpolation

Similarity Measure

Optimization

Resampling

Concept

Overview

Implementation

Global, Rigid 3D Transformation

Resampling

Fixed image

Initialized Movingimage

Registeredimage

Implemented Registration step

Trilinear Interpolation

Initial Parameters (rotation centre)

Mutual Information

Correlation Coefficient

Gradient Descent

Differential Evolution

• Global, rigid transformation-> Assumption: embryos similar in size and shape

-> 3 rotations + 3 translation = 6 transformation parameters

• 2 similarity measures and 2 optimization algorithms

Registration Step

Initialized Moving

Fixed Image

Fixed (template) + Initialized Moving (partial)

Fixed Image

Registered Image

Fixed (template) + Registered Image

Outline

Introduction

Motivation

Problem

ImageRegistration

Transformation

Interpolation

Similarity Measure

Optimization

Resampling

Concept

Overview

Implementation

Atlas Construction PipelineMoving image

registration

Registered image

Fixed image

addition

initialization

Fixed image

preprocessing

Rotation centre

initialization

Gravity centres

Purpose: Transformation of the third channel of the partial viewOnly transformation step is implemented as a new program

Outline

Introduction

Motivation

Problem

ImageRegistration

Transformation

Interpolation

Similarity Measure

Optimization

Resampling

Concept

Overview

Implementation

Registration Pipeline -> Transformation Parameters

Partial view White-nucleiRed-gsc expression patternGreen-snail expression pattern

TemplateOrange-nucleiYellow-gsc expression pattern

Atlas Construction Pipeline -> Apply Transformation Parameters

Outline

Introduction

Motivation

Problem

ImageRegistration

Transformation

Interpolation

Similarity Measure

Optimization

Resampling

Concept

Overview

Implementation

Development• Insight Segmentation and Registration Toolkit

– Available at www.itk.org• CMake

– Available at www.cmake.org • Microsoft Visual Studio 2008

Visualization• Amide– Available at http://amide.sourceforge.net/

• Amira– Commercial product

Outline

Introduction

Motivation

Problem

ImageRegistration

Transformation

Interpolation

Similarity Measure

Optimization

Resampling

Concept

Overview

Implementation

• User’s manual provided• Run from command line configuring parameters

Outline

Introduction

Motivation

Problem

ImageRegistration

Transformation

Interpolation

Similarity Measure

Optimization

Resampling

Concept

Overview

Implementation

• Developmental stage: Shield (6 hpf)

• Framework tested with six datasets (six embryos)

– One template, one whole embryo view

– Partial views of five different embryos

Dorsal

Animal

Ventral

Vegetal

* Images provided by: DEPSN , France

nuclei channel

gsc channel

co-stained gene expression pattern e.g. snail

Template embryoPartial view

Outline

Introduction

Motivation

Problem

ImageRegistration

Transformation

Interpolation

Similarity Measure

Optimization

Resampling

Concept

Overview

Implementation

Questions

1. Does the implemented framework succeed in registering our data?

2. What is the combination of similarity measure and optimization algorithm that results in a successful registration?

In other words…

What is the most appropriate registration method for our application?

Outline

Introduction

Motivation

Problem

ImageRegistration

Transformation

Interpolation

Similarity Measure

Optimization

Resampling

Concept

Overview

Implementation

Preprocessing & Addition

• Framework works with 2 datasets each time• Preprocessing: smoothed, downsampled, nuclei channel turned to binary• Addition: nuclei and gsc channels combined into a single image• 5 partial -> 5 iterations (6 images in total – 1 fixed, 5 moving)

addition

preprocessing

addition

Original channels Preprocessed channels Combined imageTe

tial V

Fixed image

One moving image

Slice Volume rendering

Outline

Introduction

Motivation

Problem

ImageRegistration

Transformation

Interpolation

Similarity Measure

Optimization

Resampling

Concept

Overview

Implementation

Initialization

Initialization looks promising…

Template + Initialized partial 1 Template + Initialized partial 2 Template + Initialized partial 3

Template + Initialized partial 4 Template + Initialized partial 5

Initialized Partial 1

Template ImageInitialized Partial 2

Template Image Initialized Partial 3

Template Image

Initialized Partial4

Template Image

Initialized Partial 5

Template Image

Outline

Introduction

Motivation

Problem

ImageRegistration

Transformation

Interpolation

Similarity Measure

Optimization

Resampling

Concept

Overview

Implementation

Method Evaluation• Four different methods implemented

• Evaluation only by visual inspection of the results

– Optimization algorithms not comparable unless running with optimized parameters

– Lack of golden standard

– Point-to-point correspondence does not exist (different embryos)

Similarity measures Optimization algorithmsCorrelation Coefficient Gradient Descent

Mutual Information Differential Evolution

Outline

IntroductionMotivation

Problem

ImageRegistrationTransformation

Interpolation

Similarity Measure

Optimization

Resampling

Concept

OverviewRegistration Pipeline

Implementation

Comparison of Registration MethodsAtlas

Method EvaluationOptimization algorithm

Similarity measure Gradient descent Differential evolution

Correlation Coefficient

Mutual Information

*After 100 iterations

• Monomodal case, intensities are linearly related (C.C. ideal)• Global optimization algorithm is still computing (D.E. not suitable)• Initialization sufficient (Gradient Descent is “myopic”)

Outline

Introduction

Motivation

Problem

ImageRegistration

Transformation

Interpolation

Similarity Measure

Optimization

Resampling

Concept

Overview

Implementation

• Transformation parameters taken from the registration method with the most coherent performance and successful results

– Correlation Coefficient optimized by the Gradient Descent algorithm

• Initialization parameters (initialization) and transformation parameters (registration) applied on the third channel of three datasets

Atlas Construction

Original view

After mappingOriginal view

After mapping

Original view

After mapping

gsc - spt gsc - snail gsc - chd

chd’s expressionsnail's expression

spt’s expression gsc’s expression

Partial view 3Volume rendering of gsc , third channel, registered gsc and transformed third channel

Outline

Introduction

Motivation

Problem

ImageRegistration

Transformation

Interpolation

Similarity Measure

Optimization

Resampling

Concept

Overview

Implementation

Conclusions & Future work

• Goal achievedDesigned and implemented an image processing framework able to map different gene expression patterns on a common template (for a given developmental stage)

• Key points– Addition: Combine information from two channels– Initialization: Solves the problem of capture range for

optimization– Registration Method: Correlation Coefficient + Gradient

Descent

Summary - ConclusionsOutline

Introduction

Motivation

Problem

ImageRegistration

Transformation

Interpolation

Similarity Measure

Optimization

Resampling

Concept

Overview

Implementation

• Advantages

– Modularity

– Configurability

– Semi-automated

• Future work

– More datasets -> more gene expression patterns

– Other developmental stages

– Validated with known gene regulatory networks

Conclusions-Future workOutline

Introduction

Motivation

Problem

ImageRegistration

Transformation

Interpolation

Similarity Measure

Optimization

Resampling

Concept

Overview

Implementation

Thanks to…

Biomedical Image Technologies Laboratory (BIT)Technical School Of Telecommunications Engineering (ETSIT)

Technical University of Madrid (UPM)

Thank you for your attention

Thesis, Image Registration Methods

Technology

Medical image registration - Sharif University of Technologysbisc.sharif.edu/~miap/Files/Medical Image Registration.pdf · Medical image registration ... whatever the form, together

Elastic Image Registration using Parametric …bigAbstract The main topic of this thesis is elastic image registration for biomedical applications. We start with an overview and classi

Image Registration Using Mutual Information

Registration and Quantitative Image Analysis of SPM Data · 2010-08-25 · Registration and Quantitative Image Analysis of SPM Data ... Registration and Quantitative Image Analysis

Fast Image Registration · 2008-07-16 · Fast Image Registration Ugo Jardonnet Technical Report no0808, JUNE 2008 revision 1868 Abstract: Image registration is a process widely used

CHAPTER 8 Image Registration - Semantic Scholar...450 Image Registration system whose guidance is based on registration between an image and the physical anatomy. Drills, for example,

Image Registration Based on Fast Fourier Transform Using ... · transform, Image registration, Phase correlation. I. ... Image Registration Based on Fast Fourier Transform Using Gabor

Thesis on Image Restoration

Chapter 9 Image Registration

Image registration server and an image mediation distributing system

CHAPTER 8 Image Registration

Image Fusion Image Registration and Radiometric ... · PDF fileIMAGE FUSION, IMAGE REGISTRATION, AND RADIOMETRIC NORMALIZATION FOR HIGH RESOLUTION IMAGE PROCESSING Gang Hong Department

Efficient feature-based image registration by mapping ... · Image registration have been widely studied by diﬀerent research groups. Surveys on the existing image registration

Image Registration with Simulated Annealing and … · Abstract Image Registration with Simulated Annealing and Genetic Algorithms Image registration is the process of ﬁnding correspondence

IMAGE REGISTRATION TECHNIQUES FOR MEDICAL IMAGES · Image registration is a primary step in many real time image processing applications. Registration of images is the bringing of

Fast Parametric Elastic Image Registration

Image Registration Methodsa Survey

Image registration methods: a surveyadessowiki.fee.unicamp.br/media/Attachments/courseIA369O1S2011/MainPage/registration...Image registration methods: a survey Barbara Zitova´*, Jan

Inter-subject Image Registration

Thesis Image