QUT Digital Repository: http://eprints.qut.edu.au/
Fookes, Clinton B. and Bennamoun, Mohammed (2000) Registration of three dimensional medical images.
© Copyright 2000 Queensland University of Technology
Registration of Three Dimensional
Medical Images
Clinton Fo okes and Mohammed Bennamoun
Technical Rep ort
Space Centre for Satellite Navigation
Scho ol of Electrical and Electronic Systems Engineering
Queensland University of Technology
G P O Box Qld Australia
July
c
Space Centre for Satellite Navigation Further repro duction prohibited
ISBN
Copyright Disclaimer
Every attempt has b een made to contact the copyright owners of material presented
in this rep ort Unfortunately in some cases this has not b een p ossible and the authors
would welcome contact from these owners i
Abstract
Image registration is a fundamental problem that can b e found in a diverse range of
elds within the research community It is used in areas such as engineering science
medicine rob otics computer vision and image pro cessing which often require the pro
cess of developing a spatial mapping b etween sets of data In the eld of medical
imaging image registration is required to match images acquired from various imag
ing mo dalities Recent advances in these imaging mo dalities including MRI CTI and
PET now allow the generation of D images that explicitly outline detailed in vivo
information of not only human anatomy but also metab olic function
The amount of time and e ort dedicated to the research of medical image registration is
a testimony to the imp ortance and signi cance that this area holds in the medical eld
This has consequently lead to the development of new and fascinating opp ortunities
for areas involving diagnosis and therapy This includes applications such as surgical
planning image guided surgery and surgery simulation However the creation of such
opp ortunities would not have b een p ossible without the enormous advances made in
computing technology which is required in order to facilitate e cient D image regis
tration
A common task within medical image registration is the fusing of the complimentary
and synergistic information provided by the various imaging mo dalities This pro cess is
known as multimo dal registration Another common task is in the registration of images
of the same patient taken at di erent times and or in di erent p ositions This pro cess
is referred to as mono mo dal registration and can b e used to track any pathological
evolution Other applications include inter patient registration and registration of a
patient s scan with an anatomical atlas The latter application is extremely useful
for further applications such as the statistical analysis of p opulations and automatic
segmentation
In a quest to further understand some of the inherent advantages and disadvantages of
image registration algorithms a literature review was undertaken A classi cation of ii
registration algorithms was also presented along with the literature review This clas
si cation scheme is based on certain characteristics that a registration algorithm may
exhibit The categories include the algorithm s dimensionality nature of the registra
tion algorithm nature and domain of the transformation user interaction optimisation
pro cedure mo dalities involved and the typ e of sub ject and ob jects involved in the reg
istration pro cess
Traditional registration metho ds were based on either manual metho ds or the use of
ducial markers These metho ds either pro duced a p o or accuracy or a greater accuracy
obtained at the exp ense of patient comfort There has since b een a global trend towards
the development of retrosp ective registration metho ds that are non invasive The bulk
of these develop ed techniques are based on intrinsic metho ds that only utilise the in
herent information contained in a patient s image Surface based and intensity based
techniques are currently the most p opular form of intrinsic metho ds where the latter
is slowly setting the standard for registration accuracy
From the literature review it was found that surface based registration metho ds are
currently used the most in clinical applications This is due to the slight sp eed ad
vantage that they have over intensity based metho ds However one of the drawbacks
of surface based metho ds is that they cannot handle cases when the surfaces b eing
matched signi cantly di er from each other To overcome such problems requires the
use of non rigid registration techniques However more research is required into these
approaches as the complexity involved is still to o high to e ectively utilise them in
real time applications This research aims to further develop non invasive retrosp ective
registration techniques that are more accurate robust and fully automatic
This rep ort presents a thorough intro duction into the eld of medical image registration
It includes background on the various imaging mo dalities a lo ok at some relevant
applications of registration a classi cation of registration algorithms and a literature
review on sp eci c techniques The rep ort is then nished with a conclusion and a
discussion on some future directions of registration iii
Contents
Abstract ii
Intro duction
What is Registration
Motivation
Scop e of the rep ort
Medical Imaging Mo dalities
CTI Computed Tomography Imaging
MRI Magnetic Resonance Imaging
PET Positron Emission Tomography
Other Imaging Mo dalities
Related Issues
Applications
Computer Integrated Surgery and Therapy
Ob jectives of Computer Integrated Surgery and Therapy
Other applications iv
CONTENTS CONTENTS
Classi cation of Registration Techniques
Dimensionality D D D
Registration involving Spatial Dimensions
Registration of Time Series
Nature of the Registration Algorithm
Extrinsic Metho ds
Intrinsic Metho ds
Non Image Based Metho ds
Nature Domain of the Transformation
Nature of the Transformation
Domain of the Transformation
Related Issues Regarding the Transformation
Interaction
Optimisation Pro cedure
Mo dalities Involved
Sub ject
Ob ject
Validation and Related Issues
Literature Survey
The Mathematics of Registration
Point Techniques
Registration with p oint corresp ondences
Registration without p oint corresp ondences
Extremal Points
Curve Techniques v
CONTENTS CONTENTS
Marching Lines Algorithm
Surface Techniques
The Head Hat Algorithm
Hierarchical Chamfer Matching
The Iterative Closet Point Algorithm
Intensity Based Techniques
Moments and Principal Axis Techniques
Correlation
Grey Level Co o ccurence Matrices
Non Rigid Registration Techniques
Deformable Mo dels
Physical Continuum Mo dels
Discussions and Conclussion
Bibliography vi
List of Figures
Light b ox showing many cross sections of a Patient s MRI scan
Example CT scanner layout
Example picture of a CT image
Two MRI images Left MR T weighted Right MR proton density
weighted Each submo dality represents the same image but in slightly
di erent intensity values
Example PET image Di erent intensity values represent di erent levels
of brain funciton
Relationship b etween the medical imaging systems Repro duced with
p ermission of authors
Three MRI slices in the transverse coronal and sagittal planes resp ectively
CIST metho dology p erception decision action Lo op Repro duced with
p ermission of authors
Hepatic surgery simulation Repro duced with p ermission of owners c o
Epidaure Research Pro ject INRIA http www sop inria fr epidaure
Anatomical mo del of a skull Repro duced with p ermission of owners c o
Anatomics The Biomo delling Sp ecialists http www anatomics com au
A taxonomy for the classi cation of registration algorithms
Examples of rigid a ne pro jective and curved D transformations in
b oth the global and lo cal domain vii
LIST OF FIGURES LIST OF FIGURES
Examples of registering an ob ject according to rigid elastic and free
deformation motion
Matching two di erent brains using b oth rigid and non rigid matching
Multi resolution registration technique
Registered MRI PET images This allows functional information to b e
viewed in the context of anatomical information
Pre op erative image registered to a video image of a patient Repro
duced with p ermission of owners c o Arti cial Intelligence Lab oratory
Massachusetts Institute of Technology Surgical Planning Lab oratory
Brigham and Women s Hospital
Surface with crest line
Extracted crest lines sup erimp osed on a smo othed iso surface of a brain
Corresp onding p oint selection pro cess for the ICP algorithm
Relationship b etween physical factors viii
Chapter
Intro duction
The eld of medical imaging has exp erienced a p erio d of rapid development over the last
two decades and has consequently revolutionised the way in which mo dern medicine is
practised The emanation of imaging mo dalities such as Magnetic Resonance Imaging
Computed Tomography Imaging and Positron Emission Tomography have b estowed
up on the surgeon and other medical physicians the ability to p eer non invasively into
the human b o dy This provides the surgeon with not only detailed in vivo information
of human anatomy but also an insight into actual human function
The role of medical imaging has progressed far b eyond the simple goal of pro ducing
aesthetic pictures of anatomy as seen in visualisation pro cedures It has
since develop ed into sophisticated to ols for use in clinical applications such as surgical
planning image guided surgery surgery simulation radiotherapy disease monitoring
and many other varied and complex applications The main aims b ehind the use of
medical imaging however may b e summed up by two ob jectives namely diagnosis and
therapy Diagnosis relies on the ability to extract quantify and most imp ortantly to
interpret all the information obtained from the various imaging mo dalities This step
is necessary in order to discriminate disease and also to facilitate further therap eutic
solutions such as radiotherapy and image guided surgery
The capabilities of current medical image pro cessing techniques needed for such clinical
applications however are far b ehind the p ower of the hardware imaging devices It
is not unusual to nd hospitals that contain p owerful and exp ensive D scanners yet
they only p osses D image diagnostic pro cedures In fact the traditional metho ds
of viewing acquired images from any mo dality was based on viewing the lms of two
dimensional cross sections of a patients medical scan on a light b ox An example of
what this light b ox metho d would have lo oked like for the attending physician can
Intro duction Intro duction
b e seen in gure It is therefore necessary to advance current image pro cessing
techniques in order to realise the full p ower that these systems may o er This has
b een one of the underlying motivations b ehind extensive research into image pro cessing
areas
Figure Light b ox showing many cross sections of a Patient s MRI scan
Image analysis and computer vision constitutes a wide and rapidly evolving eld
Recent e orts have b een directed at extending imaging pro cessing techniques from the
computer vision eld into the medical imaging eld Research into advancing certain
image analysis to ols such as registration segmentation and also visualisation has
b een an active area in the research community This has led to the development of
numerous research institutions throughout the glob e that deal solely with the advance
ment of such image pro cessing techniques Examples include the Biomedical Imaging
Resource at the Mayo Clinic the Lab oratory of Neuro Imaging at UCLA and the Ep
idaure Research Pro ject at INRIA just to name a few of the many All of which deal
with state of the art medical image pro cessing techniques and related applications Due
to the e orts of such research institutions and with the advent of mo dern technology
three dimensional visualisation capabilities of image data is now p ossible and as such
is op ening up impressive new areas of p otential for diagnosis and therapy
The fo cus of this research is to extend current techniques employed in the area of medical
image registration for the purp ose of such clinical applications Image registration is
a fundamental and crucial pro cedure in many image pro cessing systems used in the
medical eld If it is not directly involved it may often b e indirectly involved with
other image analysis to ols such as segmentation Although these various disciplines
within image pro cessing are b ecoming increasingly connected it is the aim of this
Intro duction What is Registration
research to concentrate solely on registration techniques incorp orating b oth rigid and
non rigid applications
What is Registration
Registration is a general term that is used to describ e the pro cess of developing a spa
tial mapping b etween sets of data Such a pro cedure can nd applications in many
diverse elds within the research community including engineering science medicine
computer vision rob otics and image pro cessing Within these ma jor elds registra
tion has sp eci c applications in areas such as stereo vision remote sensing image
stabilisation reverse engineering and automated manufacturing satellite navigation
photogrammetry video image compression and co ding pattern recognition tracking
video microscopy and of course medical imaging Some of these applications are only
two dimensional applications however three dimensional techniques are rapidly evolv
ing and proving to b e highly sophisticated and extremely useful for many applications
More sp eci cally the ob jective of registration is to match two or more images that
are acquired for example at a di erent time from di erent sensors or from di erent
viewp oints However due to the immense complexity of the human anatomy
medical image registration turned out to b e a much more di cult pro cess than origi
nally exp ected A numb er of other factors also contribute to this complexity including
the distinct physical realities represented by the imaging mo dalities the di erence in
patient p ositioning and the varying image acquisition parameters Although many
previous registration metho ds exist most are very application sp eci c or su er from
immense computation time Also the simple extension of D techniques into D
registration techniques often results in the computational cost to grow dramatically
A common task within medical image analysis is the automatic registration of D
images of the same patient taken at di erent times and or in di erent p ositions ie
mono mo dality case This task is very useful to detect any pathological evolution
and to compute quantitative measurements of this evolution Another extremely
imp ortant application is in the matching of images taken from di erent mo dalities This
is known as multi mo dal registration This pro cess allows the physician to combine
information from virtually any combination of imaging mo dalities and will prove to
b e extremely b ene cial for the surgeon during any decision making pro cesses An
imp ortant example when this may o ccur is during radiation therapy planning whereby
CT and MRI are b oth used CT is needed to calculate the radiation dose while MRI
is used b ecause of its excellent ability for outlining the contours of the target lesion
Intro duction Motivation
These ab ove two applications constitute the ma jority of rigid registration applications
The term rigid is given when only a rotation and translation is needed in order to bring
any two images into alignment This pro cess is based on six degrees of freedom and is
used to overcome image variations or misalignments due to patient p ositions or di erent
imaging mo dalities During a rigid registration a scale factor is often incorp orated to
account for the di erent spatial resolutions of di ering imaging mo dalities
When problems such as inter patient registration registration of images b etween two
di erent individuals and registration of a patient with an anatomical atlas were en
countered it was found that rigid registration techniques were not adequate for the
job This is due to the inherent anatomical variations b etween di erent individuals
ie di erent individuals may have a di erent brain structure varying in b oth size
and shap e In order to overcome these problems algorithms that allowed an image to
b e deformably matched to another image had to b e invented This class of techniques
are generally referred to non rigid registration techniques Within this class there are
various sub categories which de ne how much deformation is allowed in the matching
pro cess Examples include elastic viscous uid or free deformation registration On
the whole non rigid registration techniques are a much more complex problem
Motivation
Approximately of the information a human receives comes from visual inputs
This fact shows the signi cance and imp ortance that visual information plays in
everyday life and also in our ability to make decisions based on this information Such a
concept has b een the fo cus of the medical imaging community for decades in an endless
endeavour to visualise the interior of the human b o dy
The motivations b ehind any research in the medical imaging eld are to develop to ols
for the medical community that will ultimately lead to b etter medical imaging systems
for diagnosis therapy treatment planning surgery training and surgery assistance
Mo dern trends are towards the development of highly sophisticated computer integrated
surgery and therapy systems that are guided by medical images Such systems lead
to fewer complications and faster rehabilitation for the patient Another underlying
motivation for developing such systems is to provide economical savings by reducing
the nancial costs involved with many medical interventions that o ccur to day This
can b e accomplished by pro ducing automated systems that reduce intervention length
p ost op erative consequences and other complication risks
Intro duction Scop e of the rep ort
With resp ect to registration techniques traditional metho ds were based on either man
ual metho ds or the use of ducial markers Manual metho ds involved many hours sp ent
at a computer by a trained physician who attempted to visually align images together
by matching corresp onding anatomical landmarks These metho ds however not only
pro duced a p o or accuracy and inconsistent results due to human involvement but it
b ecomes infeasible when hundreds of images must b e matched with an equal numb er
of landmarks Metho ds based on the use of fudicial markers do pro duce go o d accuracy
however these are obtained at the exp ense of patient discomfort as fudicial markers
must b e screwed into the patients skull or planted under the skin Thus there is a
global trend towards the development of non invasive retrosp ective registration tech
niques which are more accurate and fully automated This is the fundamental goal
driving this research
Scop e of the rep ort
This technical rep ort aims to give the reader an intro duction into the eld of medical
image registration and provide a comprehensive picture of image registration techniques
by conduction of a literature review The layout of the entire technical rep ort is organ
ised as follows
Chapter two provides an intro duction into the imaging mo dalities that are used in
the medical imaging eld This includes MRI CTI PET and many others This is
b ene cial as it provides the reader with a basic understanding of the physics b ehind each
imaging mo dality and it also presents the raw material that is used by all registration
techniques in the eld of medical imaging For each mo dality a brief description on the
physical parameter to b e imaged and a general overview will b e given A discussion
on some of the common characteristics b etween all mo dalities will then b e presented
Chapter three describ es the current applications of image registration techniques in
terms of computer integrated surgery and therapy The layout of such a system will
b e brie y describ ed along with a discussion on the typ es of registration that are re
quired for the system Other applications such as virtual reality surgery simulation
and anatomical mo delling will also b e brie y describ ed
Chapter four presents a classi cation of current registration algorithms This classi ca
tion is based on a numb er of criteria including dimensionality nature of the registration
algorithm nature and domain of the transformation user interaction optimisation pro
cedure mo dalities involved and the typ e of sub ject and ob jects used in the algorithm
This section is concluded with a discussion on the validation of registration algorithms
Intro duction Scop e of the rep ort
and other related issues This chapter will also help in providing some background
knowledge which will b e useful for putting the next section of the rep ort into prop er
context
A thorough literature review on medical image registration techniques is presented in
chapter ve of the technical rep ort In this section sp eci c registration techniques
will b e describ ed in more detail and shall b e group ed according to the classi cation
outlaid in chapter three This will include a description of several p oint curve surface
and intensity based techniques This review will also entail b oth rigid and non rigid
techniques
The rep ort is then nished in chapter six with a conclusion and a discussion on some
future directions of registration Finally a comprehensive set of references is provided
from which the material contained within this rep ort was gathered
Chapter
Medical Imaging Mo dalities
Advances in the medical imaging eld have brought ma jor improvements to imag
ing mo dalities such Magnetic Resonance Imaging Computed Tomography Images and
Positron Emission Tomography Technology now allows generation of D images that
can b e resolved spatially into D submillimeter levels and temp oral signal changes can
b e resolved at the subsecond level This can provide p owerful to ols for b oth diagnosis
and therapy in the medical eld However b efore commencing a literature review on
medical image registration techniques it will b e useful to rst understand the basics of
the imaging mo dalities that are used in the medical imaging eld This includes MRI
CTI PET and several other mo dalities that are of less concern with resp ect to this
rep ort yet they are still very signi cant For each technique the physical parameter
to b e imaged a short description and a brief comment are given
CTI Computed Tomography Imaging
Computed Tomography otherwise known as Computed Axial Tomography CAT is
resp onsible more than any other single advancement for carrying medicine into the
digital age and has b een called the most imp ortant advancement in radiography since
the discovery of X rays CT is a radiographic metho d that was intro duced in
initially for neurological applications However its diagnostic advantages were so dra
matic that the technology was quickly extended to p ermit examinations of the rest of
the b o dy as well
Computed Tomography images do not represent direct measurements of data rather
they are reconstructed images from a set of X ray attenuation measurements from
various angles around the ob ject The data set obtained from the original measurements
Medical Imaging Mo dalities CTI Computed Tomography Imaging
called a sinogram has little visual relationship to the ob ject under investigation It is
only after reconstruction that the form of the ob ject app ears
Conventional X ray CT scanners utilise a single X ray tub e that rotates through a full
degrees rotation while recording pro jections at small angular increments during ro
tation Using mathematical reconstruction techniques such as ltered back pro jection
or Fourier reconstruction these pro jection images can b e pro cessed to form a tomo
graphic image The X ray b eam is emitted from a p oint source and is collected at a
detector array forming a at fan b eam geometry See gure for an illustration of
this pro cess
Source Detector
Patient
Figure Example CT scanner layout
The measured parameter in CT scanning is the variation in transmission of the X rays
based on attenuation due to the electron density of certain structures within the ob ject
b eing imaged Each slice of a CT scan is a single image ie D However the slice do es
have a de ned thickness and may b e thought of as a volume image comp osed of voxels
Thus combining the successive adjacent CT slices which are each separately acquired
by changing the p osition of the ob ject b eing scanned creates an entire D image
The spatial resolution of CT images can range from to mm in the acquisition
plane and the slice thickness may range from to mm A typical CT scan of a
human head is shown in gure This gure shows a CT slice in the transverse
plane
CT has also found uses outside the medical community It has b een used very success
fully in non destructive testing such as for the insp ection of ro cket motors and turbine
blades Because of the sup eriority of CT it is destined to play a central role in emerg
ing agile manufacturing activities like rapid prototyping rapid to oling and rst article
insp ection
Medical Imaging Mo dalities MRI Magnetic Resonance Imaging
Figure Example picture of a CT image
MRI Magnetic Resonance Imaging
Magnetic Resonance Imaging is p erhaps the most sophisticated current imaging system
and is deemed as state of the art when it comes to medical imaging It is a totally
non invasive yet very exp ensive pro cedure that is exceptional at delineating soft tissues
within the b o dy MRI relies on the resp onse of magnetic elds to short bursts of
radio frequency waves to pro duce computer images that provide structural and also
bio chemical information ab out tissue This imaging technique is based on radio waves
and thus is a much safer technique than other imaging mo dalities such as CT which
may employ the use of X rays or gamma rays
In MRI the patient is placed in within a magnetic coil and radio frequency energy is
applied to the head Note that MRI may also b e used for any other part of the b o dy
as well These harmless radio waves excite protons that form the nuclei of hydrogen
atoms in the brain The protons then give o measurable electrical energy which
can b e used to construct a map of the underlying tissue This technique is based on
the fundamental prop erty that protons p osses a magnetic moment and spin When
placed in a magnetic eld a proton will precess ab out this eld Thus when a radio
frequency having an appropriate magnitude and frequency is applied to the ob ject
b eing scanned the protons will absorb the energy This will disturb the protons from
their current eld of precession ie to an antiparallel state When the applied radio
frequency is removed the absorb ed energy is re emitted and consequently detected by
a radio frequency receiver
The human b o dy is comprised primarily of fat and water which b oth contain many
hydrogen atoms This makes approximately of the human b o dy comprised solely
Medical Imaging Mo dalities MRI Magnetic Resonance Imaging
by hydrogen atoms This fact is useful in understanding why MRI is such an e ective
means of generating tomographic images of the human b o dy More sp eci cally it
is excellent at pro ducing images of soft tissues and the intracranial and intra spinal
contents of the human b o dy However MRI hardly visualises b one due to its lack of
hydrogen atoms
MRI is an imaging technique that pro duces images based on more than one single
prop erty of tissues Images are related to relaxation times or combinations of relax
ation times of certain atoms previously excited by a magnetisation eld at a sp eci c
frequency MRI also contains three common sub mo dalities These are known as T
weighted T weighted and proton density Two of these sub mo dalities can b e seen
in gure The tissue density is re ected in the MR image intensity However the
same tissue can app ear as a di erent intensity in a di erent sub mo dality or due to
gain factors within the scanner
Figure Two MRI images Left MR T weighted Right MR proton density
weighted Each submo dality represents the same image but in slightly di erent intensity
values
The spatial resolution in the acquisition plane ranges from to mm with the slice
thickness ranging from to mm MRI contrast is greatest in soft tissues and it is
capable of detecting a di erence in the signal level which can b e further enhanced
with the use of sp eci c contrasting agents
One disadvantage of MRI is that it has a slightly longer acquisition time compared to
CT which makes it more sensitive to motion artifacts Also as patients b eing examined
must lie quietly inside a narrow tub e the magnetic coil MRI may raise the patient s
anxiety levels esp ecially those with claustrophobia Another drawback of MRI is that
it cannot b e used on patients that contain a pacemaker or any other metal structure
which may b e present in critical areas such as the eye or the brain This is due to
Medical Imaging Mo dalities PET Positron Emission Tomography
the immense magnetic eld that is created during the imaging pro cess and thus plays
havo c on any metal structures within the b o dy
PET Positron Emission Tomography
PET is a form of non invasive nuclear medical imaging that yields transverse tomo
graphic images Despite many discouraging p erceptions in the early development stages
PET has develop ed into a highly recognised imaging mo dality that is worthy not only as
a research to ol but also for clinical applications PET is a unique imaging mo dality
and it di ers signi cantly from the two previously discussed mo dalities b ecause it pro
vides limited anatomical detail of the ob ject b eing scanned It do es however provide
a means of generating functional images which re ects the metab olic activity of the
patient s brain PET is particularly useful for studying heart as well as brain functions
and certain bio chemical pro cesses involving these organs Examples include the study
of glucose metab olism oxygen uptake and cerebral blo o d ow This is of particular
value in the diagnosis of certain degenerative and metab olic disorders
In PET a chemical comp ound lab elled with a short lived p ositron emitting radionuclide
of carb on nitrogen oxygen or uorine is injected into the b o dy A photomultiplier
scintillator is then used to detect and quantitatively measure the activity of such a
radiopharmaceutical throughout the organs which are b eing imaged As the radionu
clide decays p ositrons are annihilated by electrons and give rise to gamma rays that
are detected by the photomultiplier scintillator combinations which are situated on op
p osite sides of the sub ject b eing scanned The detectors collect the data which is then
analysed integrated and nally reconstructed with a computer to pro duce a tomo
graphic image which re ects the tissue bio chemistry and physiology of the organ b eing
scanned A sample PET image in the transverse plane is shown in gure
Two identifying parameters of the PET cameras are its resolution and its sensitivity
The resolution is the system s ability to accurately lo cate the p ositron electron anni
hilation and the sensitivity is the numb er of events that are registered p er unit dose
of isotop e The signal to noise ratio of a PET image is strongly related to the scanner
sensitivity This is due to the fact that the image quality explicitly dep ends on the
numb er of strikes detected Improvement to the system s spatial resolution may b e
made by the utilisation of slowly decaying p ositron emitting isotop es This however
may induce longer e ects on the patient b eing scanned
Medical Imaging Mo dalities Other Imaging Mo dalities
Figure Example PET image Di erent intensity values represent di erent levels of
brain funciton
Other Imaging Mo dalities
The three mo dalities describ ed ab ove are p erhaps the most widely used D imaging
techniques in the medical eld With these three mo dalities the medical physician can
gain an insight into b oth anatomical and functional information of the sub ject b eing
scanned Thus providing the physician with an array of distinct yet complimentary
images that can b e used for diagnosis treatment planning and other clinical applica
tions These describ ed imaging mo dalities however are not the only mo dalities used
in the medical eld Many other forms exist and some of these will now b e discussed
Radiography X ray is one of the oldest and p erhaps the most basic form of trans
mission medical imaging yet it is still widely used in the medical community Radio
graphy uses X rays as the transmission energy source and they fall at the extremely
high end of the electromagnetic sp ectrum Because of this fact patients are generally
only allowed a limited numb er of X rays in a sp eci c amount of time due to the e ects
of its high radiation
An X ray image is not like any of the previously discussed imaging mo dalities as it is
only a D pro jection image Or more sp eci cally a pro jection of a D structure onto
a D image This metho d often causes several structures within the b o dy to overlap
on the pro jection image However even though radiography is not inherently a D
imaging mo dality many image pro cessing techniques created for D images can b e
applied to these D pro jection images
The recorded parameter in radiography is the absorption of X rays This is based on
the X ray attenuation due to variations in electron density of structures in the b eam
path X rays are absorb ed di erentially within the b o dy with structures such as b one
Medical Imaging Mo dalities Other Imaging Mo dalities
or calci cations absorbing more X rays than softer tissues such as fat and muscle
This results in structures such as b one app earing as a lighter intensity on the lm
Also conventional X ray images are now often b eing stored digitally rather than on
traditional radiographic lm This is known as digital radiography
Functional Magnetic Resonance Imaging fMRI is a relatively new technique
which has emerged and has since signi cantly expanded the clinical role of MRI This
technique pro duces images of a patient based on the detection of indirect e ects of
neural activity on lo cal blo o d volume ow and oxygen saturation This creates a
functional image which shows the activated brain regions of the patient b eing scanned
Thus fMRI along with other functional imaging mo dalities such as PET are helping
physicians to bridge the gap in understanding the relationship b etween brain structure
and function
Single Photon Emmision Computed Tomography or SPECT is another form of
nuclear medical imaging such as PET that is used to pro duce an image based on the
radioactive distributions within the b o dy SPECT consists of a rotating gamma ray
detector which is used to pro duce a set of pro jections recording the photons emitted by
radioactive tracers trapp ed in the b o dy However these pro jections are not simple line
integrals due to several factors This includes the imp erfect collimation of the detector
and also the attenuation of the photon s energy through the duration of its ight from
within the b o dy to the collimator SPECT also su ers from other physical phenomena
such as Compton scattering and gamma ray attenuation resulting in SPECT images
that are often very noisy and have a p o or spatial resolution
Ultrasound is the only current imaging mo dality which is not electromagnetic It
employs the use of high frequency acoustic energy or sound waves to pro duce images
of tissue discontinuities This includes blo o d ow information in the form of Doppler
shifts The basic concept b ehind ultrasound imaging is to determine information ab out
intrinsic tissue prop erties based on the observations of the way in which the sound
waves are p erturb ed re ected or scattered by the tissues This technique involves
measurement of the time of ight of the sound waves and the frequency selection is
based on a trade o b etween image resolution and sound attenuation Echography is a
real time imaging system that pro duces images that are corrupted by a strong texture
noise This is due to a sp eckle phenomenon and by distortions that are created by
non constant velo cities in inhomogeneous tissues
Magnetic resonance angiography a unique form of MRI technology can b e used
to nonivasively pro duce an image of owing blo o d This p ermits the visualisation of
arteries and veins without the need for needles catheters or contrast agents This
technique is based on two physical principles in ow and phase e ects The in ow
Medical Imaging Mo dalities Other Imaging Mo dalities
e ects are due to the motion of spins known as entry slice phenomenon Phase e ects
are the result of motions of spins along the direction of the eld gradients employed for
imaging
Digital subtraction angiography or DSA is a technique based on X ray imaging
however it is used to pro duce images of arteries and veins This can b e accomplished by
subtracting a pre injection mask image taken at one time without contrast agents from
another image of a later contrast lled vessel taken at a di erent time and employing
the use of contrast agents DSA systems consist of one movable source detector or a
pair of source detectors DSA systems can also b e used for diagnosing small changes of
internal structures
Biomagnetic imaging or magneto encephalography MEG is a relatively new mo dal
ity that is used for imaging of the brain s electrical activity This technique uses sup er
conducting quantum interference device SQUID detectors to detect magnetic elds
that arise due to active regions within the brain This can give the physician an insight
to brain function
Among the ab ove discussed imaging mo dalities are also quite a numb er of other less
used mo dalities Some of these include electrical imp edance tomography transmission
laser images passive microwave imaging and pressure imaging There are also sev
eral other signals that are pro duced by b o dy activities such as electro encephalograms
EEG which can provide the physician with useful information Another form of
imaging is known as microscopy Several di erent forms exist such as light microscopy
or electron microscopy These techniques however provide information and images of
much smaller structures within the b o dy such as tissue cells This imaging mo dality is
p erhaps one of the most imp ortant to ols available to cell biologists
There are also other means of obtaining information on the internal structure of the
human b o dy without the use of the imaging systems describ ed thus far These are more
invasive metho ds and include p ost mortem tissue cryosectioning and optical intrinsic
signal OIS imaging Cryosectioning involves physical axial slicing and photography
of the entire p ost mortem brain or entire b o dy of a patient This technique yields
p erhaps the highest spatial resolution ab ove all other systems however it has obvious
drawbacks due to the terminal nature of the pro cess Another disadvantage of this
pro cess is that adequate registration algorithms must b e available in order to reconstruct
the entire D structure from the D pictures
OIS imaging is a pro cess whereby part of the brain is physically exp osed to white light
This is frequently done during any neuro surgical pro cedures when the patient s brain
is exp osed By photographic means it is p ossible to view the brain s cortical surface
Medical Imaging Mo dalities Related Issues
and track the distribution of cerebral blo o d ow over the D region of interest It is
also p ossible to obtain certain functional information using this technique However
another obvious disadvantage is that the patient s brain must b e exp osed by surgical
means
Most of the imaging mo dalities discussed in this chapter of the rep ort can b e seen in
gure This gure outlines the relationship b etween the di erent mo dalities and
summarises what transmission energies are used to pro duce their resp ective images
Ionizing Radiations Non Ionizing Radiations
Electromagnetic Waves Acoustic Waves Electromagnetic Waves Electrical Field
Gamma Rays X-rays RF Waves Ultrasound Visible Light Infra Red Microwaves
Impedance Scintigraphy Radiography Echography Endoscopy Thermography MRI Tomography
Microwave Emission CTI Tomography
Tomography
Figure Relationship b etween the medical imaging systems Repro duced with p er
mission of authors
Related Issues
The ab ove imaging mo dalities all present unique ways of obtaining anatomical and or
functional information of the human b o dy Some characteristics that are common to
all the ab ove imaging systems that may b e used as a basis for comparison b etween
them include spatial resolution contrast resolution and temp oral resolution The
spatial resolution refers to the space resolving p ower of the image acquisition system
and image formation mechanism This term determines the dimensions of the pixel or
voxel in the measurement space of an ob ject The resolution or the dimensions of the
pixel voxel may di er for each orthogonal direction represented in the image This is
known as anisotropic If the dimensions are equal in all directions then the imaging
device is isotropic
Medical Imaging Mo dalities Related Issues
The contrast resolution of the imaging system represents the systems ability to detect
di erences in signal intensity b etween two structures The signal intensity is a charac
teristic that is dep endent on the imaging device and it may represent physical prop erties
such as electron density or proton density The contrast resolution is dep endent on not
only the imaging device but also the typ e of energy that is b eing measured It is usu
ally sp eci ed as a p ercentage of the largest signal di erence that can b e detected and
quanti ed by the imaging device
The nal common characteristic of imaging systems is the temp oral resolution This
characteristic is de ned by two terms the ap erture time and the frame rate The
ap erture time refers to the amount of time it takes to capture a single image The frame
rate which is also known as the rep etition rate is de ned by the smallest interval of
time required to pro duce successive images
When discussing images acquired from certain mo dalities terms such as transverse
coronal and sagittal may b e involved These three terms represent the three or
thogonal directions that are present in any three dimensional image These principal
directions are represented in gure which presents three slices of an MRI scan in
the trasnverse coronal and sagittal planes resp ectively Thus it is p ossible to view a
D image as a series of parallel cross sections along any of these three principal axes
Figure Three MRI slices in the transverse coronal and sagittal planes resp ectively
The three main imaging mo dalities that will b e used for the ma jority of this research will
b e MRI CTI and PET As previously mentioned these three mo dalities o er p erhaps
the highest quality form of D imagery It is intended to further extend the research
carried out on D medical image registration techniques for these three mo dalities
All three imaging mo dalities provide complementary and synergistic information which
can b e combined to give the surgeon or physician an overall picture on what is happ en
ing inside the patient MRI is extremely useful for showing the anatomical structure
Medical Imaging Mo dalities Related Issues
and soft tissues within the brain CTI is useful for showing the b one structure and calci
cations and PET gives a functional image of the brain s metab olic activities Thus all
three mo dalities provide the surgeon or physician with valuable information that must
b e combined in order to utilise them to maximum b ene t The pro cess of aligning
images from di erent mo dalities is known as multimo dal image registration
One of the main problems that existing registration techniques su er from is that they
often address the ideal case where the images are of similar intensity and are up to an
unknown geometric transformation This however is not always the case The four
main sources of dissimilarity in medical images are
The representation of the information Eg b ones are low intensity in MRI while
high in CTI
The non redundancy of information Images of di erent mo dalities provide com
plementary information as previously discussed
The measurement noise Distortions in the images which may not necessarily b e
additive Gaussian or stationary
Occlusion Which can o ccur due to the growth of a tumour
Thus it is obvious from this chapter that image registration techniques must b e capable
of handling a broad sp ectrum of information acquired from a diverse range of imaging
mo dalities
Chapter
Applications
Medical image registration is a crucial step in a lot of medical imaging applications It
is the aim of this chapter to provide a brief intro duction to some of these applications
that entail many di erent technological and medical sectors Not only do they involve
the use of image registration algorithms but they may also employ the use of rob otics
or other complex systems in an attempt to provide a much more ambitious service to
b oth the patient and the surgeon in a way that hasn t b een realised until recently
Computer Integrated Surgery and Therapy
This section of the rep ort will provide an intro duction into the fascinating eld of com
puter integrated surgery and therapy CIST Computer integrated surgery and therapy
also represented by the expressions image guided surgery and computer assisted med
ical interventions is an area of medical imaging that has develop ed signi cantly over
the past few years This is solely due to the enormous advances in mo dern computer
technology as such systems require extreme amounts of computational p ower
CIST is a general term that refers to any system that can b e used to provide some
sort of service or assistance during any medical intervention or other medical pro ce
dure According to CIST is de ned as Methods and systems to help the surgeon
or physician use multi modality data in a rational and quantitative way in order to
plan but also to perform medical interventions through the use of passive semi active
or active guidance systems In particular computer integrated surgery and therapy
incorp orates pre op erative intra op erative and p ost op erative data that are obtained
from the various imaging mo dalities Such a system not only provides obvious b ene ts
to the patient but will also provide an economical saving due to the improvement of
Applications Computer Integrated Surgery and Therapy
the success rates of interventions and also a reduction in complications and intervention
lengths
Computer integrated surgery and therapy is a phrase that is often used synonymously
with many other terms in the medical eld It encompasses all disciplines related to
surgery and therapy including many of the medical sp ecialities that b ene t from the
use of such systems Some of the other expressions used to describ e CIST along with
a few examples of medical sp ecialities are listed b elow
Image guided surgery
Medical rob otics
Surgery simulation
Minimally invasive surgery such as laparoscopy
Cranio facial surgery and also simulation
Computer assisted Orthopaedic surgery
Frameless stereotaxy
Radiation therapy
Gamma knife
From the ab ove list it is easy to see the broad expanse of areas that a CIST system must
encompass The creation of a CIST environment in these applications will provide the
surgeon with an enhanced ability to plan navigate and lo calise throughout a surgical
pro cedure A CIST system incorp orates b oth information mo dules which include
images acquired from the various imaging mo dalities and action mo dules This last
mo dule involves the use of guiding systems that are used to aid the surgeon or even the
use of active rob ots that can p erform a certain surgical pro cedure without the assistance
of a surgeon Whatever the case the global aim of CIST technology is not to replace
the surgeon but to provide him or her with advanced to ols that will ultimately help
the surgeons p erform b etter than what they could have on their own
A metho dology pro duced for any CIST system which was employed by will b e
now b e discussed This metho dology is based on a p erception decision action
lo op This is illustrated in gure which succinctly shows the relationship b etween
the three stages involved in any CIST system and also shows the ow of information
in CIST The three stages involved will now b e brie y describ ed
Applications Computer Integrated Surgery and Therapy
PERCEPTION - data acquisition, calibration, segmentation & modelling
REGISTRATION & statistical analysis
PRE-OPERATIVE INTRA-OPERATIVE POST-OPERATIVE INFORMATION INFORMATION INFORMATION
Medical Images Anatomical Models, Medical Images and Information for Medical Images, (MRI,CTI,PET,...) a priori knowledge, etc related information - system control - Clinical results, etc. Open MRI, ultrasound, 3D localisers, shape physiological signals sensors, etc.
REGISTRATION REGISTRATION CONTROL
Partial simulation Definition of the Definition of an of the strategy strategy just before up to date strategy FEEDBACK before intervention intervention on-line
DECISION - surgical planning
REGISTRATION
Passive Systems SYSTEM CONTROL Semi-Active Systems
ACTION - use of guiding systems Active Systems
Figure CIST metho dology p erception decision action Lo op Repro duced with
p ermission of authors
Perception level At the p erception level multimo dal information is acquired using
any of the many imaging mo dalities describ ed in chapter of the rep ort Such in
formation can also b e obtained from more general computer vision sensors such as
D lo calisers and shap e sensors This information can b e acquired pre op eratively
intra op eratively and also p ost op eratively
Decision level At the decision level also referred to as the surgical planning stage
the surgeon de nes an optimal strategy to attempt certain pro cedures based on
the available multi mo dal data It is imp ortant to note however that the typ e
of pro cedures involved are determined by the sp eci c application of the CIST
system
Action level At the action level the attending surgeon can b e assisted with the use
Applications Computer Integrated Surgery and Therapy
of guiding systems during the intervention These guiding systems can b e classed
into three categories based on the amount of help or assistance that they provide
the surgeon with These categories are passive semi active or active guiding
systems
A very imp ortant element used right throughout these three stages is the pro cess of
registration Registration can b e seen as the glue that links all the stages together
This is also illustrated in gure Registration applications in any CIST system
must b e more accurate than in the diagnosis settings Registration is not only used to
align images into a common co ordinate system but it is also used to register op erating
systems and surgical to ols ie to register physical spaces as well as images The main
typ es of data that need to b e registered in CIST can b e categorised into three main
categories
Pre op erative data This includes medical images acquired from the various
sources of imaging mo dalities eg CT MRI PET It also includes any mo dels or
atlases and other pre op erative p ositioning information that provide information
for subsequent registration b etween medical images and intra op erative systems
Intra op erative data This includes medical images once again These are used
in this case for three reasons Firstly to aid in the completion of any last minute
surgical planning needed Secondly to provide real time intra op erative control
of the surgery And thirdly they give p ositioning information that can b e used
for any registration pro cedures It also incorp orates intra op erative p ositioning
information provided by the various D lo calisers and other sensors that are
used for accurate registration Finally it includes intra op erative guiding systems
that have to b e registered with the images on which the surgical planning was
determined
Post op erative data Again this includes medical images and other similar
information that was acquired during the pre op erative stage Registration must
then b e p erformed b etween these sets of data in order to facilitate op erations such
as the evaluation of the e ciency of an intervention This generally requires the
use of statistical techniques
The general ob jectives of any form of computer integrated and therapy system will now
b e presented in the following section
Applications Computer Integrated Surgery and Therapy
Ob jectives of Computer Integrated Surgery and Therapy
An imp ortant ob jective of computer integrated surgery and therapy is more than the
pro duction of technically impressive results The system must bring a clinical added
value ie improvement of a standard therapy or surgical pro cedure Although the
ob jectives of a computer integrated surgery and therapy system are largely dep endent
on the typ e of surgery eg computer assisted orthopaedic surgery craniofacial surgery
neurosurgery a set of common ob jectives can b e pro duced that show the clinical added
value of such a system These ob jectives include
To pro duce less invasive surgery in order to reduce p ost op erative consequences
For instance executing small incisions instead of large op en surgery decreases the
risk of infection and also the duration of the p ost op erative stay of the patient
Increased accuracy of the op eration which in turn results in an e cacious e ect
on the op eration s success rate
Reduction of complication risks by the decreased access size of the op eration
and improvements in the accuracy This also leads to less severe p ost op erative
consequences for the patient
Reduced intervention length is often a side e ect pro duced from aiming to improve
the quality of the intervention This reduced intervention length will reduce
the time under anaesthesia reduce risk of contamination and can allow more
interventions p er day
Reduced stress on the surgeon is often a result of having p owerful reliable to ols
To make some currently imp ossible interventions p ossible due to the added infor
mation provided by a computer integrated surgery and therapy system
Improvement on future surgical planning and surgical proto cols can b e achieved
by the evaluation of data obtained from previous interventions and p ost op erative
results
The use of surgical planning and guiding systems provides a basis for surgical
simulators Developments in this eld will allow therapists to train on virtual
patients instead of the real thing
Applications Other applications
Other applications
Virtual Reality and Surgery Simulation Surgery simulation is a fascinating new
eld incorp orating computer vision computer graphics rob otics and virtual re
ality This typ e of application involves the exploration of D images to planify
simulate and even control some complex therapy Such a system can allow
training of surgeons and other medical practitioners in a virtual environment
thus removing the need for practice on human patients under sup ervision or
other alternatives often involving the use of animals An example picture showing
what a virtual reality environment may lo ok like can b e seen in gure This
gure shows a system develop ed for a hepatic surgery simulator simulation of
a surgical op eration on the liver It involves an interaction with a deformable
mo del of the liver using a force feedback system
Figure Hepatic surgery simulation Repro duced with p ermission of owners c o
Epidaure Research Pro ject INRIA http www sop inria fr epidaure
Anatomical Mo delling Anatomical mo delling involves highly sophisticated sys
tems such as stereo lithography whereby mo dels can b e generated from imaging
mo dalities These computer mo dels can then b e created into an actual physical
mo del that are constructed out of wax or several other comp ounds Techniques
such as this give the surgeon that extra sense of touch that are very b ene cial
in aiding the surgeon to decide on the b est metho ds for surgery therapy or any
Applications Other applications
other typ e of medical pro cedure needed An example of an anatomical mo del
of the skull created using stereolithographic means is shown in gure This
picture was obtained from Anatomics the biomo delling company as a part of the
Queesland Manufacturing Institute
Figure Anatomical mo del of a skull Repro duced with p ermission of owners c o
Anatomics The Biomo delling Sp ecialists http www anatomics com au
Computer vision applications Registration can also nd many applications in the
eld of computer vision Examples include ob ject recognition and visual naviga
tion The aim of ob ject recognition is to match observed data with a set of
previously stored mo dels which represent sp eci c ob jects of interest The role of
visual navigation however is to match observed data in a dynamic environment
and at di erent time intervals in order to infer ob ject motions and to achieve
scene interpretation
There are also many other applications of medical image registration Most of these
other applications will b e brie y discussed during chapter four of the rep ort and as
such will not b e discussed here However these applications are listed b elow for com
pleteness
Disease diagnosis
Neuroscience studies
Therapy evaluation
Treatment monitoring
Multi mo dal image fusion
Chapter
Classi cation of Registration
Techniques
The eld of image registration is an immense and ever expanding eld By the early
stages of there existed over pap ers written on the registration problem as
cited in a comprehensive survey article written by van den Elsen et al One of
the aims of this technical rep ort is to present the reader with a thorough literature
review on all image registration techniques that are used in the medical imaging eld
However b ecause of the sheer volume of pap ers the presented review will b e heavily
condensed and only the principle concepts of registration will b e discussed
This section of the rep ort will present a classi cation of registration techniques This
classi cation is based on a set of criteria that was originally prop osed by van den
elsen et al and also used by Maintz et al at a later date This is not the
only existing classi cation however Brown prop osed a classi cation for general
registration techniques not sp eci c to medical imaging which included the feature
space similarity metric search space and the search strategy However the set of
criteria that is used for this rep ort includes the algorithms dimensionality nature of
the registration algorithm nature and domain of the transformation user interaction
optimisation pro cedure mo dalities involved and the typ e of sub ject and ob jects used
in the algorithm These criteria can b e seen in gure showing the taxonomy of
registration algorithms This section of the rep ort will only deal with the classi cation
of registration algorithms Sp eci c algorithms will b e discussed in more depth in the
next section of the rep ort
Classi cation of Registration Techniques Classi cation of Registration Techniques
Spatial Dimensions Dimesionality Time Series
Extrinsic Nature of the Instrinsic Registration Non-Image Based
Rigid
Nature of the Affine Transformation Projective Curved
Global Domain of the Transformation Local
Interative Medical Image Interaction Semi-Automatic Registration Fully Automatic
Direct Optimisation Procedure Search-Oriented
Mono-modal Multimodal Modalities Involved Modality to Model Modality to Patient
Intra-subject Subject Intersubject Subject to Model
Head Thorax Object Abdomen Pelvis Limbs
Spine
Figure A taxonomy for the classi cation of registration algorithms
Classi cation of Registration Techniques Dimensionality D D D
Dimensionality D D D
The most obvious classi cation that can b e deduced from the ever growing set of image
registration techniques is the numb er of dimensions that are used in the registration
pro cess This can range from a simple D registration pro cess right up to a complex
time series registration of D data ie D pro cess The issues regarding the dimension
ality of registration algorithms shall b e group ed into those that do not deal with time
series registration and those that do Thus the algorithms that do not deal with time
series registration only deal with spatial dimensions
Registration involving Spatial Dimensions
From the algorithms that only deal with spatial dimensions a further grouping may b e
made ie D D Most current research directions are in the area of D registration
which is a much more complex domain than D registration D registration is a
pro cess whereby the set of three dimensional transformation parameters must b e found
in order to nd the correct spatial mapping b etween two sets of D images This typ e
of registration is usually applied to two tomographic data sets
D registration is used for several circumstances within medical imaging The obvious
use for D algorithms is in the registration of two or more D images of an imaging
mo dality This pro cess is a much simpler task compared to D registration b ecause of
the reduced data size and also b ecause of the reduced parameter set which is used in
the registration Another imp ortant application of D algorithms is in the pro cess of
reconstruction This is the pro cess of generating a D image from a series of D slices
Before each separate slice can b e stacked to previous slices it is imp ortant that each
slice b e registered with its neighb ouring slices if the resulting D image is to resemble
the anatomical region that it was originally scanned from
This typ e of registration pro cess is esp ecially crucial for the reconstruction of cryosection
slices As each slice is cut o the b o dy it is digitally photographed The resulting D
images are often so far out of registration that simple stacking would result in a D
image that do es not resemble anything like the sp ecimen it was originally taken from
Reconstruction was also imp ortant b efore the advent of mo dern D imaging systems
Traditionally only one D slice could b e taken at any one time Thus to generate a
whole volume image several D tomographic scans had to b e made in succession If
the patient b eing scanned moved at all b etween these scans then the resulting images
would b e out of registration Thus D registration algorithms would b e needed in
Classi cation of Registration Techniques Dimensionality D D D
order to successfully register each slice to its neighb ouring slices Thus enabling the
entire volume image to b e reconstructed
Another de nable area of the dimensionality criteria is in the registration of D to
D images An example of this is the registration of D tomographic data to a D
pro jection image such as an X ray This pro cess is often employed during surgical
intervention ie intra op erative pro cedures Another case when this may b e relevant
is when a registration must b e made to a D tomographic image and one tomographic
slice acquired at another time This may also b e useful during a surgical intervention
when a surgeon needs to take only one slice in order to validate that certain surgical
pro cedures were carried out as planned It may also b e used to register an entire volume
image of one mo dality to one tomographic slice of another mo dality This is often the
case with an entire volume image such as MRI or CTI that is registered to a single
SPECT image
It is imp ortant to note that any registration algorithm that is involved in intra op erative
pro cedures must b e extremely e cient and fast in order to b e carried out in real time
during the p erio d of intervention However most of the other applications that involve
registration can b e done outside the surgical theatre allowing for less stringent time
constraints on the computation time required to complete the registration pro cess
Thus it is the clinical relevance of the required registration algorithms that should set
constraint on sp eed issues
Registration of Time Series
Another area of registration is in application to a set of images acquired over time ie
time series registration Such a pro cedure allows medical practitioners the ability to
monitor several di erent situations which may arise during a patient s life Perhaps
the most obvious application is in the study of the evolution of a tumour during
which images are acquired over a p erio d of time days weeks months years and are
insp ected to determine factors such as the rate of growth This ability is also extremely
imp ortant for the study of certain other pathologies such as multiple sclerosis and
Parkinson s disease
Multiple sclerosis is a demyelinating pro cess in which the myelin sheaths in the central
nervous system are attacked and destroyed and Parkinson s disease is a degenerative
pro cess of the nervous system Both of these diseases leave physical evidence b ehind to
show that they exist For instance multiple sclerosis leaves numerous neural plaques
throughout the central nervous system predominately the brain which are groups
of damaged axons These neural plaques will also grow with time As a result of
Classi cation of Registration Techniques Nature of the Registration Algorithm
Parkinson s disease certain cells within the brain are damaged and may contain pink
staining spheres called Lewy b o dies These Lewy b o dies are marker s for Parkinson s
Thus by tracking the scans of a patient over time it is p ossible to characterise the
existence of certain diseases and study their b ehaviour
Other applications of time series registration is in the monitoring of b one growth usu
ally in children However this pro cess takes place over a substantial amount of time
Also the monitoring of a patient s healing after a surgical intervention or some other
form of medical pro cedure such as radiotherapy is another extremely imp ortant ap
plication of time series registration This usually o ccurs over a much shorter interval
Other applications include the monitoring and evaluation of the e ect of certain drugs
Nature of the Registration Algorithm
The nature of the registration algorithm is an expression that is used to describ e the
metho d of the registration algorithm in terms of what characteristics or features are
matched b etween images This is an area in which registration algorithms may di er
signi cantly The main distinction that can b e made is b etween those that use extrinsic
or intrinsic metho ds
Extrinsic Metho ds
Extrinsic metho ds were used by the rst breed of registration algorithms that were
develop ed These metho ds are based on the intro duction of foreign ob jects which
are attached in some way to the patient b efore imaging These foreign ob jects often
referred to as ducial markers are designed in such a way so that they app ear visually
in the resulting images acquired after scanning It is also imp ortant that these ducial
markers are easily distinguishable from any other region in the image This is the main
ob jective b ehind extrinsic metho ds as these easily distinguishable features may then b e
used for the registration pro cess
Perphaps the most well known form of ducial marker is the stereotatic frame This
is a device which is screwed into the patients skull b efore imaging and until recently
registration metho ds employing the use of a stereotactic frame were deemed as the gold
standard A stereotactic frame is also heavily used in nuerosurgery for guidance
purp oses and also in other sterotactic pro cedures a blind surgical pro cedure whereby
the target is approached from a small twist drill hole in the patient s skull There
are also other invasive markers that can b e used such as screw mounted markers
Classi cation of Registration Techniques Nature of the Registration Algorithm
As extrinsic metho ds have b een designed with the registration pro cess in mind they are
quite accurate and also remarkably fast This is due to the fact that it is known when
the images are registered as the corresp onding ducial markers will b e appropriately
aligned This removes the need for any elab orate optimisation algorithms that generally
slow the registration pro cess down dramatically Also extrinsic metho ds are often
limited to only rigid transformations since by de nition extrinsic metho ds cannot
include any patient related information However the obvious disadvantages of extrinsic
metho ds includes their prosp ective nature ie steps must b e made prior to the imaging
pro cess and their invasive nature with resp ect to patient comfort
The metho ds discussed thus far in this section have all b een invasive techniques for
obvious reasons There are however a numb er of non invasive techniques that are
still based on extrinsic metho ds These include skin markers which can b e glued to
the patient s skin individualised foam moulds and other head holder frames and also
dental adapters have b een used These non invasive metho ds however do not p osses the
great accuracy of their invasive companions One imp ortant factor when considering
what typ e of material to use for the extrinsic marker is whether it is compatible with
the intended imaging device For example no metal ob jects can b e present during
an MRI due to the extremely high magnetic elds generated during imaging So it is
imp ortant to pick a compatible material that is also easily detectable by the desired
imaging mo dality
Intrinsic Metho ds
Intrinsic metho ds o er an enormous area of exploration for the solution of the registra
tion problem These metho ds are based solely on the information which is contained
within the patient s scan and do es not rely on the intro duction of any arti cial ob jects
into the imaging pro cess Intrinsic metho ds range from simple p oints that corresp ond
to an anatomical landmark to complex D structures that are used in the matching
pro cess More sp eci cally intrinsic metho ds are sub categorised into the following
Anatomical landmarks
Segmentation based
Intensity based
Classi cation of Registration Techniques Nature of the Registration Algorithm
Anatomical Landmarks
The rst form of intrinsic based registration used anatomical landmarks as the match
ing feature b etween images Anatomical landmarks are p oints within the image that
can b e identi ed by a user usually interactively and identi ed by a trained medical
physician and which corresp ond to some distinguishable p oint within the morphol
ogy of the anatomical image Technically the identi cation of anatomical landmarks is a
manual segmentation pro cess However the segmentation based metho ds are reserved
to those that use much more complicated means to segment higher order structures
and or pro cesses that are solely implemented by computational means
Registration metho ds using anatomical landmarks are usually only rigid transforma
tions This is due to the small numb er of p oints which can b e used as stable anatomical
landmarks However if the numb er of p oints obtained were increased then it would b e
p ossible to implement some higher order transformations Since the numb er of p oints
usually available in this pro cess are limited the resulting optimisation pro cedures can b e
quite e cient General measures which are used include the average distance b etween
corresp onding landmarks or other minimal landmark distances which are implemented
iteratively Sp eci c algorithms used for the optimisation of p oint sets will b e presented
in the following chapter of the rep ort
Anatomical landmarks are also often used in conjunction with other metho ds One
p ertinent example is in the use of other registration algorithms that are often prone
to lo cal optima or minima due to the optimisation pro cess By constraining the
optimisation problem with the p ositioning of anatomical landmarks then these lo cal
discontinuities may b e avoided
Segmentation Based
Segmentation based metho ds o er an enormous and comprehensive range of p ossibilities
for solving the registration problem They are called segmentation based b ecause of
the pro cess which must b e undertaken in order to extract the features required for
matching Although not all the metho ds describ ed in this section can b e considered
a direct segmentation as such not in the sense of classifying di erent substructures
within the human anatomy it is given this name due to the fact that a user guided or
a computational pro cedure must b e executed in order to extract features for matching
prior to the actual registration implementation
Geometrical p oints like anatomical landmarks are only p oint features obtained from
an image These are the lowest order segmentation metho ds available and are usually
Classi cation of Registration Techniques Nature of the Registration Algorithm
extracted in an automatic fashion These typ es of features are generally referred to as
geometrical landmarks as they can b e considered stable p oints within the image ie
landmarks which can b e reliably used in the matching pro cess Although these typ e
of p oints can b e considered analogous to anatomical landmarks for the purp ose of this
rep ort they are classi ed into the segmentation based metho ds b ecause of the compu
tational pro cess which must b e undertaken in order to extract the p oints These p oints
are usually generated using some means of di erential geometry and like anatomical
landmarks are generally only used for rigid registration pro cesses
The remaining segmentation based metho ds can b e categorised into curve or surface
metho ds More sp eci cally a segmentation based metho d can b e rigid mo del based or
deformable mo del based A rigid mo del based metho d do es not however imply that
the resulting registration is a rigid pro cess It simply means that the same anatomical
structure is extracted from b oth images that are to b e matched These matched features
are typically surfaces however curves are also quite common
Rigid based segmentation metho ds are currently the most p opular form of registration
algorithms used in clinical applications More sp eci cally it it the surface based ap
proaches that are most p opular The most well known and well used metho d is the
Head Hat algorithm This was designed by Pelizzari et al and it is a surface based
approach that requires a simple segmentation step in order to extract the skin surface
from the imaging mo dality such as MRI or CT This algorithm will b e describ ed in the
following chapter along with another two p opular surface based techniques known as
the Hierarchical Chamfer Matching HCM algorithm and the Iterated Closest Point
ICP algorithm
One disadvantage of surface based techniques is that the registration accuracy is often
restricted by the accuracy of the segmentation step However they are still capable
of p erforming reliable and accurate registration if the pre segmentation step is p er
formed precisely These registration techniques are also commonly automated The
pre segmentation step however is usually executed semi automatically Although sur
face based approaches have dominated the rigid based segmentation metho ds many
other approaches exist Techniques that involve the registration of curves for example
will also b e describ ed in the following chapter
One application that rigid based segmentation metho ds are well suited for is in intra
sub ject registration ie registration of images acquired from the same patient This is
b ecause the surfaces which are b eing matched are generally the same as it is obtained
from the same individual However this typ e of approach is not generally su cient for
inter sub ject registration ie registration of images acquired from di erent patients
This is where deformable mo del based registration metho ds are much more applicable
Classi cation of Registration Techniques Nature of the Registration Algorithm
Deformable based approaches allow one image to b e deformed in order to match it
with another image This allows the inherent anatomical di erences that exist b etween
di erent individuals to b e overcome This typ e of approach is not only well suited for
inter sub ject registration but also registration of a patient s data with an anatomical
atlas These typ e of concepts shall b e discussed in the later sections of this chapter
Derformable mo del segmentation metho ds have two ma jor application areas The rst
is in the matching of b one contours derived from CT images This application is
well suited to derformable mo dels as the b one contours are easily obtained from CT
images and there are usually no other contours in the nearby vicinity to interfere
with the deformation convergence The other application is in the registration b etween
two brians More sp eci cally in the implementation of cortical registration If such a
cortical registration can b e found ie a registration in which the entire suri and guri can
b e matched then automatic segmentation pro cedures can b e implemented by mapping
a segmentation in one image directly onto the other
Deformable mo del based segmentation metho ds rely on the extraction of a structure
from one image also usually surfaces and curves which is tted by a deformable mo del
This structure is then elastically deformed to t the second image The deformable
mo del de ned within the rst image usually referred to as the template mo del is
then either deformed to match a segmented structure within the second image
or it is directly matched to the second image without any pre segmentation steps
Deformable curves which have b een used throughout the literature are referred to as
snakes or active contours More complex D deformable mo dels are known as nets and
ballo ons
The main di erence b etween deformable based and rigid based segmentation metho ds
is that deformable mo dels are not represented by p oint sets as in the rigid mo del based
metho ds They are usually expressed in the form of lo calised functions such as splines
The optimisation criteria employed in deformable approaches is also quite di erent It
is always lo cally de ned and calculated The actual deformation pro cess is restricted
to limits determined by certain elastic mo delling constraints The deformation pro cess
is also done in an iterative fashion ie small p erturbations are carried out for each
iteration of the deformation pro cess
One of the main disadvantages of derfomable mo del approaches is that they generally
need a go o d initial estimate of the required transformation in order to register e ciently
Such a disadvantage is often overcome by the use of hierarchical registration metho ds
in which a rigid registration is rst applied to overcome any global di erences A non
rigid registration may then b e applied to account for lo cal anatomical di erences This
concept of hierarchical registration will b e discussed in section of this chapter
Classi cation of Registration Techniques Nature of the Registration Algorithm
Another predominant error of deformable mo dels arises when any ma jor di erences
exist b etween the anatomy of two images b eing matched This can arise in situations
such as the presence of a tumour or any other pathology When this o ccurs deformable
mo del metho ds often pro duce erroneous results in the registration as it attempts to map
one image onto another image that contains a pathology On the whole deformable
mo dels are b est suited in nding lo cal curved transformations b etween images rather
than nding global rigid or a ne transformations There application promises
to b e very e ective not only in registration applications but also for a wide variety of
other image pro cessing areas including segmentation and motion tracking of anatomical
shap es
Intensity Based
Intensity based registration metho ds also referred to as voxel prop erty based meth
o ds are signi cantly di erent from segmentation based registration metho ds These
metho ds op erate directly on the intensity or grey level values within the image and
thus do not need to utilise complex segmentation pro cedures or other feature extrac
tion metho ds in order to obtain features required for matching These approaches are
generally the most robust of all registration metho ds and o er numerous ways in which
the entire image data may b e utilised in the registration pro cess
The rst form of intensity based metho ds are known as principal axes and moment based
metho ds These metho ds op erate by rst reducing the entire image data into a set of
vectors This is accomplished by the extraction of the zeroth and rst order moments
from within the image The e ect of this is to extract the image s centre of gravity and
its corresp onding principal axes an ellipse for example may b e characterised by its
centre of gravity and two principal directions ie the ma jor and minor axis Thus the
registration pro cess lies in nding a transformation that will match the image s centres
of gravities and overlay their resp ective principal directions
Principal axes and moment based metho ds are not very accurate techniques as they
are often susceptible to any physical di erences in the images b eing matched Thus
these techniques are generally only used for applications that do not require a high
degree of registration precision They are however very commonly used throughout
the literature This can b e accounted for b ecause of its e cient implementation and
the resulting fast computation time Also they are often used in a pre registration step
in order to obtain an initial coarse alignment Sometimes higher order moments are
intro duced into the registration pro cess However this usually makes little impact on
the accuracy of these metho ds
Classi cation of Registration Techniques Nature of the Registration Algorithm
Intensity based registration algorithms that op erate on the entire information contained
within an image are p erhaps one of the most fascinating registration approaches cur
rently available This breed of registration techniques are the most exible and robust
as they make no assumptions on the underlying information contained within an image
Unlike other segmentation based techniques they do not require an entire image to b e
represented by a characteristic structure such as a surface When such approaches are
used it is obvious that the characteristic structure must b e present in the image Thus
metho ds that incorp orate the full content of the image exceed these typ es of approaches
as no prior understanding of the images is needed and the registration algorithms can
b e directly applied
Algorithms that incorp orate the full content of the image have existed for quite a while
however they have only recently started to b e used successfully due to the enormous
advancement of computing technology This is due to the enormous amount of data
which must b e handled during these registration metho ds A typical D image which is
acquired after scanning may contain voxels each containing bits p er
voxel So the resulting size of the image is approximately MB This gives some idea
in how much information must b e handled during a registration pro cedure involving at
least two images that op erates on the entire information contained within the images
These intensity based approaches are currently used in numerous clinical applications
however their use in real time applications is still limited due to the computational
time constraints
Although intensity based approaches are often revered b ecause of their robustness and
versatility their real value can only b e judged individually Certain techniques are well
suited to one application however they may not b e well suited to other applications
Such a concept is veri ed with the example of Mutual information approaches
This technique is very sup erior in application to multimo dal registration however
other intensity based techniques are not as adept in this application Generally all
intensity based metho ds are implemented in an automatic fashion and the ma jority of
the applications are found in intra sub ject mono mo dal and multimo dal registration
Also these applications usually employ only rigid or a ne global transformations
One typ e of intensity based techniques which op erates on the full image content and
was one of the rst to b e develop ed was correlation metho ds These metho ds were in
tro duced to help overcome the problem of the di ering levels of intensity values b etween
images This was accomplished by assuming that there existed some linear correlation
b etween the intensity values of two images However these techniques make a strong
assumption regarding the relationship that exists b etween the intensity values of dif
ferent images As a consequence correlation metho ds have proved to b e of little
Classi cation of Registration Techniques Nature of the Registration Algorithm
service in multimo dal applications and generally only pro duce reasonable results in
mono mo dal applications
Perhaps one of the most promising directions for improvement of registration techniques
is in the utilisation of the mutual information b etween multi mo dality images This
can b e achieved by adjusting the relative p osition and orientation of successive images
until the mutual information b etween the two is maximised This technique provides a
more exible and robust approach compared to most other intensity based techniques
and is rapidly nding itself b eing used in more and more clinical applications
There are also numerous other intensity based algorithms that employ di erent tech
niques Some of these include minimisation of variance of intensity ratios histogram
clustering minimisation of the histogram entropy of di erence images maximisation of
zero crossings in di erence images minimisation of the variance of grey values within
segments Fourier domain based metho ds sum squared di erence and maximum like
liho o d approaches
Non Image Based Metho ds
All registration metho ds discussed thus far ultimately deal with some form of image
pro cessing algorithms However it is p ossible to obtain registration without the use
of such image based registration metho ds Registration of multimo dal images for ex
ample can b e accomplished if the two imaging mo dalities are calibrated to each other
thus ensuring that the resulting images are acquired in the same co ordinate system
For such a pro cess to b e implemented it is required that the two imaging devices are
b oth present in the same situation It is also required that the patient must not move
b etween successive acquisitions of the two imaging mo dalities The use of calibrated
co ordinate systems such as this are also often employed in computer aided surgery
pro cedures where they are used to register images with the p osition of certain surgical
to ols that are mounted on rob otic arms
The use of non image based registration metho ds and also any other patient alignment
systems that do not require any form of image based registration do not utilise any of
the techniques presented in this rep ort As a consequence these metho ds are considered
outside the scop e of this rep ort and will no longer b e discussed
Classi cation of Registration Techniques Nature Domain of the Transformation
Nature Domain of the Transformation
This criteria of the classi cation deals with the nature and domain of the transforma
tion that is employed by certain registration algorithms These concepts will now b e
describ ed in the following subsections
Nature of the Transformation
Registration techniques are often group ed into two very general categories namely rigid
and non rigid registration This is known as the nature of the transformation This
classi cation however is not adequate to su ciently describ e the di erences which may
exist b etween various registration algorithms A more sp eci c approach is to classify
registration algorithms into the following categories
Rigid
A ne
Pro jective
Curved
A rigid transformation is the most fundamental of all registration techniques Such a
transformation involves only a rotation and translation in order to bring the images into
alignment An a ne transformation incorp orates a shearing into the rigid registration
pro cess This e ectively maps parallel lines onto parallel lines A scaling factor is also
often incorp orated into an a ne transformation A pro jective transformation is one in
which any straight line is mapp ed onto another straight line These lines however may
not necessarily b e parallel The nal transformation is a curved transformation This
is one in which any straight line is mapp ed onto a curve A curved transformation is
also often referred to as an elastic transformation
As you descend the transformations listed ab ove the mathematical and thus the compu
tational complexity of the transformation increases dramatically This concept can b e
seen in the implementation of certain registration algorithms in clinical applications
Often rigid registration algorithms are employed in real time surgical interventions
for instance to register intra op erative images to pre op erative images acquired earlier
However it is very rare if not obsolete to nd an elastic based registration that is
involved in a real time application A comparison of issues relating to the computation
complexity of many registration algorithms is presented in
Classi cation of Registration Techniques Nature Domain of the Transformation
Each of these ab ove transformations may b e mo delled as a sp ecial case of its successor
except for the curved transformation which has no successor For example an a ne
transformation can b e represented as a sp ecial case of a pro jective transformation ie
without the scaling factor or a scaling factor that is unity Another fact to consider is
the comp osition of more than one transformation If this is employed in any techniques
then the resulting transformation can b e classed to b e the same as its most complex
transformation used For example if a rigid and a curved transformation is comp osed
into one technique then the resulting transformation is a curved transformation The
mathematics involved in all these transformations discussed ab ove will b e presented in
the following chapter of this rep ort section
Domain of the Transformation
The domain of the transformation deals with the concepts of global or lo cal transfor
mations Before explaining this any further the notion of a transformation must rst
b e understo o d A transformation can b e de ned as a mapping of p oints in one image
to a new set of p oints in another image However this mapping can b e applied to the
image globally or lo cally A global transformation is the case when the entire image
is mapp ed in the same way ie a single equation can b e given which maps the entire
image A lo cal transformation is the case when the image is mapp ed in a di erent way
dep ending on the spatial lo cation Thus lo cal transformations are much more complex
pro cedures and are much harder to express concisely
Figure presents a series of D transformation examples This gure is helpful in
understanding the relationship b etween the nature and the domain of transformation
Related Issues Regarding the Transformation
Global rigid transformations are by far the most utilised registration techniques seen
throughout the literature and in clinical applications This is due to the concept of
the rigid b o dy constraint that can b e successfully applied to many situations in the
registration eld The rigid b o dy constraint is a simple assumption made regarding
the ob ject b eing registered which assumes that the ob ject is a rigid b o dy ie nothing
internal may move This assumption is valid for a large numb er of registrations involving
the head
Most multimo dal registrations for example assume the rigid b o dy constraint Intra
sub ject registration also generally makes this assumption Although the brain may
move internally b etween successive acquisitions of the same patient this assumption
Classi cation of Registration Techniques Nature Domain of the Transformation
Original
Rigid Affine Projective Curved
Global
Local
Figure Examples of rigid a ne pro jective and curved D transformations in b oth
the global and lo cal domain
is still usually valid as a go o d approximation However for inter sub ject registration
registration with an atlas and registration of other parts of the b o dy for example the
ab domen the rigid b o dy constraint is not adequate
Global rigid transformations are quite simple to implement Only six parameters need
to b e calculated in order to characterise them ie three rotation parameters and
translation parameters for each principal direction in any D image This consequently
implies a fast computation time as compared to a lo cal curved transformation It is
also p ossible to compute a global transformation based on the information contained
in only a small p ortion of the image ie a small region of interest The transformation
parameters are then calculated from this small region of interest and can then b e applied
to the entire image
Lo cal transformations are rarely used in the literature esp ecially lower order transfor
mations such as the rigid a ne and pro jective transformations as compared to curved
transformations This is b ecause lo cal transformations usually disrupt the continuity in
an image ie the transformed image contains gaps or tears that were not present in the
images prior to the registration These typ e of lo cal transformations may also b e
unrecoverable This means it is unable to repro duce the original image by application
of the inverse transformation
Pro jective transformations generally are rarely used in the literature They employ a
metho d that has little physical relationship to most applications required in medical
image registration Their main application is in the registration of D D images such
Classi cation of Registration Techniques Nature Domain of the Transformation
as in the matching of a pro jective image such as an X ray with a D tomographic data
set These transformations are well suited for this application as they can account for
the pro jection e ects intro duced into the D X ray images Pro jective transformations
are also sometimes used as a sp ecial case of curved transformations when their successor
p erforms insu ciently or the amount of parameters that need to b e found b ecomes to o
large to b e e ciently or e ectively computed
A ne transformations are used in instances when certain shearing e ects must b e
corrected or when the image scaling factors or gantry tilt information are unknown
or incorrect This sometimes needs to b e employed in the registration of MRI images
as they inherently su er from geometric distortion e ects As a general rule seen
throughout the literature rigid and a ne transformations are global However curved
transformations are generally lo cal This is a logical approach by registration researches
as by de nition the rigid b o dy constraint is global and hence there is no need to lo cally
register certain lo cations in an image any di erently to other lo cations
Lo cal curved transformations are limited almost completely to intrinsic metho ds only
This to o is another logical concept as by de nition any metho d employing lo cal trans
formations must only make use of the available information in that lo cation This
concept also seems logical in another sense that lo cal curved transformations are used
to overcome any lo cal anatomical di erences which may b e present in di ering images
These typ e of metho ds are generally deformable mo del based or intensity based meth
o ds eg Deformable mo dels are usually implemented with certain constraints
applied to them These constraints are usually determined by certain mo dels such as
an elasticity mo dels The mo del used determines the amount of deformation allowed
in the registration pro cess
A classi cation which is helpful in understanding the concepts of deformation with
resp ect to rigid and non rigid registration is one prop osed by Bro Neilsen This
classi cation is used to explain the varying degree of deformation which can b e involved
in the registration pro cess The three main categories are termed rigid transformation
elastic deformation and free deformation Thus any registration will have a deformation
which is contained by one of these or b etween any of these categories as the rst and
third category represent the outer limits of any registration pro cess The concept of
these three categories is illustrated in gure
To further illustrate the use of rigid and non rigid registration gure is presented
This gure shows a brain that is b eing matched to another that di ers in size and shap e
quite signi cantly The rst step shows a rigid registration only rotation and transla
tion then the second step shows the non rigid registration that is used to overcome
the lo cal anatomical di erences
Classi cation of Registration Techniques Interaction
Rigid Transformation
Elastic Deformation
Free Deformation
Figure Examples of registering an ob ject according to rigid elastic and free defor
mation motion
Image 1 Image 2
Rigid Non-rigid Match
Registration Registration
Figure Matching two di erent brains using b oth rigid and non rigid matching
Interaction
This criteria of the classi cation deals with the amount of interaction that is required
by the user of the registration algorithm Within this criteria three sub categories can
b e de ned These are listed b elow
Interactive
Semi Automatic
Automatic
Traditional manual metho ds whereby a trained physician visually found a spatial map
ping b etween two images can b e classed into the interactive category With any in
teractive metho ds the user generally do es all the work manually with some help from
a software based platform that displays the current state of registration to the user
Classi cation of Registration Techniques Interaction
Interactive metho ds may also often supply an initial guess for the registration required
It is then left up to the discretion of the user whether or not this guess is used Fully
interactive systems are seldom used in the literature This is due to the inherent sub jec
tivity which is intro duced into the registration pro cess by the interaction with humans
Also these metho ds rely heavily on adequate and often very exp ensive visualisation
software to display and provide controls to manually manipulate images
Semi automatic metho ds are quite p opular esp ecially in clinical applications as they
still allow some small control inputs from the user This is often a go o d idea in ap
plications where the result of the registration pro cess can have profound e ects on the
end result Thus the attending physician generally desires to oversee the registration
pro cess in order to make sure it has aligned and converged prop erly
Usually two di erent approaches exist for the semi automatic metho ds The rst is
where a user generally has to supply an initialisation to the registration pro cedure
This can involve either a guess to the required transformation or the generation of
features required for the matching eg a user segmentation to extract surfaces The
other semi automatic approach is one in which the user generally oversees the registra
tion pro cedure and provides some input to steer the pro cess in order to make sure it
converges prop erly This may involve the user to accept correct alignments and or to
eliminate false alignments
Fully automatic metho ds have only recently b egun to b e explored They require no
control inputs or initialisations to b e entered by the user and thus are the most ob
jective metho ds currently available In a fully automatic system the user generally
only supplies the images that need to b e registered and mayb e some other parameters
which de ne certain characteristics such as what imaging mo dalities the input images
where acquired from It is these typ e of registration approaches that are currently
exp eriencing a rapid serge in research
Extrinsic metho ds which involve the intro duction of foreign ob jects into the imaging
space are usually automated due to the ducial marker s easy detectabiliy and the re
sulting easy registration implementation Intrinsic metho ds are also usually automated
However numerous techniques employ a semi automatic approach Segmentation based
metho ds for example require the user to provide the segmented structures b efore the
registration pro cess can b egin Other metho ds including geometric landmark based
metho ds ie based on di erential geometry characteristics of surfaces and also inten
sity based metho ds are generally implemented in an automatic fashion
As brie y mentioned ab ove most of the current research is directed towards the devel
opment of fully automated registration systems However not all clinical applications
Classi cation of Registration Techniques Optimisation Pro cedure
necessarily b ene t from this approach There generally exists some form of trade o
relationship b etween the user interaction the resulting sp eed of the registration pro
cess and other design criteria such as the robustness Sometimes user interaction may
improve the sp eed of the registration quite considerably as the user may reduce the
search space and eliminate false registrations However one of the main goals b ehind
the pursuit of fully automatic systems is to eliminate the sub jectivity intro duced from
user interaction ie what one user deems as an adequate registration may b e di erent
from another user s standards Generally if the registration algorithms are robust they
can p erform adequately in the form of a fully automated system
Optimisation Pro cedure
This section of the rep ort discusses the optimisation pro cedures that are employed by
registration algorithms in order to compute the required transformation parameters
The parameters that are obtained are then used to drive the registration pro cess A
global rigid transformation only requires parameters to describ e the transformation
However an elastic or viscous uid registration will typically have hundreds thousands
or even millions of parameters The two main distinctions that can b e made b etween
registration optimisation pro cedures are listed b elow
Direct metho ds
Approximation or search oriented metho ds
A direct metho d obtains the transformation parameters directly from the available in
formation Whereas approximation metho ds search for the transformation parameters
based on some form of optimisation of a function de ned in the parameter space
The computation metho d for the direct approach relies solely on the nature of the
registration algorithm as describ ed in section of the rep ort Generally global trans
formations that compute the required parameters directly only involve small amounts
of information This means that only a small set of features are used in the pro cess
Generally this pro cess is based on lists of corresp onding p oints An example of this
can b e found in As the numb er of p oints increases so to o do es the accuracy
of the registration Lo cal transformations can often b e computed directly from the
lo cal information available in the image and do es not require a set of features to b e
pre de ned
Optimisation pro cedures that search for the transformation parameters generally ac
complish this by formulating the registration mo del in a mathematical equation that
Classi cation of Registration Techniques Optimisation Pro cedure
incorp orates the transformation parameters The transformation parameters are then
computed in order to optimise the de ned function in some sense An example might b e
to nd the transformation parameters that minimise a certain distance metric which is
computed b etween the two images Or it may involve the maximisation of a similarity
criteria such as the correlation or mutual information b etween the two images Mono
mo dal registration pro cedures are usually less complicated as the similarity b etween
images from the same mo dality are easier to de ne and hence compute
If the optimisation pro cedure employed in the registration is well b ehaved ie the
optimisation function forms a quasi convex shap e then any of the numerous standard
and well researched optimisation techniques can b e employed Some examples of these
optimisation metho ds are listed b elow
Powell s metho d
The renowned Newton Raphson iteration metho d
The downhill simplex metho d
Gradient descent metho ds
Geometric hashing
Levenb erg Marquardt optimisation
Brent s metho d and series of D searches
Sto chastic search metho ds
Genetic metho ds
Simulated annealing
Quasi exhaustive search metho ds
If the optimisation function used by the registration pro cess is not well b ehaved then
the only technique that allows the transformation to b e computed is an exhaustive
search of the entire parameter space Such a metho d involves extreme amounts of com
putational time due to the lengthy pro cess of applying each value for each registration
parameter in essentially a trial and error basis Thus exhaustive searches are not very
practical metho ds
Quite often a multi resolution approach is adopted in a registration pro cedure Such
a technique is based on the concept of initially computing a registration on a rough
Classi cation of Registration Techniques Optimisation Pro cedure
scale and then successively re ning the resolution until the nal registration at the
highest level is completed This is accomplished by rst down sampling the images
and then computing the registration based on these down sampled images Figure
illustrates the concept of multi resolution registration Quasi exhaustive searches
b ene t particularly from the use of multi resolution approaches as it reduces the numb er
of transformations to b e examined
Image 1 Registration Image 2 (Level 4) (Quarter Level) (Level 4)
Downsample Interpolate Downsample
Image 1 Registration Image 2 (Level 2) (Half Level) (Level 2)
Downsample Interpolate Downsample
Registration Image 1 (Highest Level) Image 2
Input Image 1 Output Input Image 2
Registered Images
Figure Multi resolution registration technique
Hierarchical registration is another extremely imp ortant technique that is increasingly
b eing used in the literature These techniques pro duce an increase in p erformance
and accuracy and avoid the problems of lo cal minima that are often encountered in
optimisation pro cedures such as gradient descent metho ds The concept of hierarchical
registration is based on the use of low complexity registration techniques to account for
the bulk of the variation in di ering images Higher order transformation metho ds are
then used to re ne the registration until it is complete
Multi resolution techniques are essentially a hierarchical based approach except that
the same algorithm is used but at di ering resolution levels This mimics the use
of registration algorithms that di er in complexity rather than resolution level An
example of a hierarchical approach is one in where a global rigid registration is rst
used to overcome the external di erences b etween images of two di erent patients An
example of an external di erence is the di erence in patient orientations b etween two
images A non rigid registration is then used to overcome the inherent anatomical
di erences b etween the individuals
Classi cation of Registration Techniques Mo dalities Involved
More than one optimisation technique may sometimes b e employed in a registration
pro cess Generally a fast optimisation metho d is used to generate an initial coarse
registration and then a more complex optimisation is adopted to re ne the registration
This is again another form of a hierarchical based approach
There are a few registration techniques that do not employ traditional optimisation
pro cedures An example of this is the ICP algorithm designed by Besl et al This
technique which is discussed in more detail in the next chapter of the rep ort uses an
optimisation technique which is designed sp eci cally for its own purp ose There are
also a numb er of other rigid based registration techniques which are group ed into this
category
Mo dalities Involved
This section of the classi cation deals with the mo dalities that are involved in the
registration pro cess More sp eci cally any registration can b e group ed into one of four
categories dep ending on the mo dalities involved These are listed b elow
Mono mo dality
Multimo dality
Mo dality to mo del or atlas
Mo dality to patient
Mono mo dality registration is the most obvious and most basic form of registration It
involves the registration of images acquired from the same imaging mo dality These
are less complicated registration pro cesses as compared to multimo dal registration as
the variation b etween di erent images is easier to mo del Mono mo dal registration is
often involved with intra sub ject registration ie same mo dality same patient This
typ e of registration can b e used to monitor the evolution of a pathology evaluate the
e ectiveness of a certain treatment and can also b e involved in certain subtraction
imaging pro cesses such as the one required for digital subtraction angiography as
describ ed in other imaging mo dalities section of the rep ort
As mentioned in the intro duction of the rep ort multimo dal registration is the pro
cess of matching images acquired from di erent imaging mo dalities This allows the
fusion of complimentary and synergistic information which is obtained from the vari
ous mo dalities Thus multimo dal registration can allow functional information such
Classi cation of Registration Techniques Mo dalities Involved
as that obtained from PET images to b e represented in the context of soft tissues
images obtained from MRI which can then b e represented in to context of b ones im
ages acquired from CT There are obviously many di erent combinations of imaging
mo dalities that are involved in registration pro cesses Examples include MRI PET
CTI MRI US MRI DSA MRI and many more An example picture showing a com
pleted MRI PET registration can b e seen in gure
Figure Registered MRI PET images This allows functional information to b e
viewed in the context of anatomical information
Another interesting application of multimo dal registration is in the matching of pre
op erative images with intra op erative images One such example is when pre
op erative images are overlaid on video images of the patient that are taken during
the surgical pro cedure This creates an augmented reality and is extremely useful
in helping to direct the surgeon An example of the registration of a pre op erative
tomographic image to a video image is seen in gure This image was acquired
from the medical vision group at the MIT AI Lab Such an approach gives the feeling
that the patient is transparent as one can lo ok inside to see the internal anatomical
structures
A registration that involves an imaging mo dality and a mo del is one in which tomo
graphic images are matched with an anatomical atlas or some other mo del Such a
pro cedure can facilitate automatic segmentation This can b e accomplished once a reg
istration is found b etween the atlas and the image allowing the top ological information
Classi cation of Registration Techniques Mo dalities Involved
Figure Pre op erative image registered to a video image of a patient Repro duced
with p ermission of owners c o Arti cial Intelligence Lab oratory Massachusetts Institute
of Technology Surgical Planning Lab oratory Brigham and Women s Hospital
stored in the atlas to b e transfered directly to the patient s scan Other information
can also b e exchanged during this pro cess This may include functional relational and
other hierarchical information
Mo dality to mo del registration is also b ene cial to many other tasks in the medical
eld Particularly to the statistical analysis of p opulations As many images from many
di erent patients can all b e registered to the same anatomical atlas then the creation
of a statistical database can b e accomplished Thus allowing researches to investigate
anatomic variability in human p opulations and investigation of other characteristics
such as pathologies age gender dissymmetry and any other genetic or demographic
factors
Mo dality to patient registration is slightly di erent in that it do esn t only involve
images or some form of tomographic mo del such as an anatomical atlas Its aim is
to align the actual patient with a mo dality which is often required in some medical
applications Examples include radiotherapy and other intra op erative applications
In the former case the patient is aligned into correct p osition with the aid of X ray
simulation images which are registered to pre op erative images Although this metho d
consists of a registration b etween two di erent typ es of images its purp ose is solely to
correctly align the patient with the mo dality that is to b e used for treatment
Classi cation of Registration Techniques Sub ject
Sub ject
This classi cation refers to the sub ject or sub jects that are involved in the registration
pro cess Such registrations may involve for example the matching of images acquired
from one patient or b etween di erent patients The main distinctions are listed b elow
Intra sub ject
Inter sub ject
Sub ject to mo del or atlas
Intra sub ject registration is the most commonly used throughout the literature This
is the registration of images acquired from the same patient This typ e of registration
can b e used for almost any form of diagnostic or therapy pro cedure including intra
surgical pro cedures It often op erates collectively with multimo dal registration in order
to combine all the complimentary information obtained from the di erent mo dalities
and on the same patient
Inter sub ject registration is slightly more complicated as the transformations must
overcome the inherent anatomical di erences that exist b etween di erent individuals
Hence most of the inter sub ject registration algorithms are based on curved transfor
mations and consequently only op erate on intrinsic characteristics of the images An
inter sub ject registration can also b e implemented by using another approach This
approach is based on the mo dality to a mo del registration that was describ ed in the
previous section If each sub ject s image is registered to the anatomical mo del or at
las then e ectively b oth sub ject s images are registered to each other thus pro ducing
an inter sub ject registration
Sub ject to mo del registration is essentially the same as mo dality to mo del Thus
all the discussions presented in previous sections of the rep ort regarding this typ e of
registration are all applicable in this case as well An interesting approach for the
automatic building of an antomical atlas is presented in
Ob ject
Registration algorithms can nd application in many di erent areas of the human b o dy
Some of the parts of the b o dy which b ene t from the use of registration algorithms as
seen in the literature are listed b elow
Classi cation of Registration Techniques Validation and Related Issues
Head
Thorax
Ab domen
Pelvis
Limbs
Spine
As previously discussed in the rep ort the ob ject involved in the registration pro cess
will control the nature of registration algorithm This concept can b e illustrated with
an example of ab domen registration as compared to an intra sub ject head registration
As the ab domen is certainly not a rigid ob ject the registration can not assume the
rigid b o dy constraint Thus a non rigid or deformable registration will need to b e
employed However the intra sub ject head registration can adequately b e implemented
with a rigid transformation
Although registration is capable of b eing applied to any part of the human anatomy
only registration of head images will b e concentrated on in this research This is due
to the extreme imp ortance that head image registration is given by medical physicians
and researchers alike This however do es not stop the registration algorithms that will
b e develop ed for the head to b e applied to any other application
Validation and Related Issues
The concept of validation is extremely imp ortant in order to assess the relative capa
bilities and weaknesses of any image pro cessing algorithm However it is esp ecially
imp ortant for medical applications as their p erformance may have direct consequences
on the patient s involved The concept of validation is one that has only recently b een
given the imp ortance that it deserves It has consequently seen an increase in the
amount of time and e ort that researchers have exp ended in a quest to solve many of
the problems that are related to it It is the aim of this section to present some of the
problems and ideas involved with the validation of registration algorithms
Validation of registration results is a very hard matter as the algorithms employed are
usually tailored for sp eci c applications Although adhoc techniques can b e very e ec
tive for sp eci c applications they are generally hard to implement for new applications
Perhaps the main di culty in validation is that there exists no gold standard
Classi cation of Registration Techniques Validation and Related Issues
that can b e used as a basis for comparison of di erent registration algorithms It is
also extremely hard to quantitatively measure the accuracy of any particular registra
tion algorithm Although they can b e sub jectively assessed no statistical approaches
adequately quantify the e ectiveness or accuracy of any registration algorithm Some
of the problems involved with the validation pro cess are succinctly describ ed in the
following quote
The diversity in problems and their applications has been the cause of the
development of enumerable independent registration methodologies This
broad spectrum of methodologies makes it di cult to compare techniques
since each technique is often designed for speci c applications and not nec
essarily for speci c types of problems or data
One thing that is a necessity for adequate validation is the development of a quantitative
measure that can b e used for assessing the registration accuracy Such a measure
requires ground truth information However the existence of such information is not
available in clinical practice This means that some other metho ds that can b e used to
describ e the accuracy needs to b e determined Some approaches that have b een used
to characterise registration accuracy involve the use of phantoms Phantom studies
are based on the registration of images that are acquired from either the imaging of a
physical phantom such as a synthetic brain mo del or from a software based phantom
ie simulated images Although certain phantom studies are helpful in providing
some ground truth information that can b e controlled in certain asp ects they are still
limited due to the amount of con dence that can b e instilled in the results obtained
from their use with resp ect to clinical practice
As describ ed in the previous paragraph there exists a need to develop ways to quantify
registration accuracy This task however seems to b e somewhat of a dilemma wrapp ed
in an enigma If such a metho d existed then this measure could b e used in the actual
registration pro cess itself However to date there exists no such metho d
A set of criteria can b e created to characterise validation concerns regarding image
registration techniques This set of criteria was presented by and shall b e discussed
accordingly The list of criteria is as follows
Precision
Accuracy
Robustness stability
Classi cation of Registration Techniques Validation and Related Issues
Reliability
Resource requirements
Algorithm complexity
Clinical use
The rst criteria of precision is a system prop erty that is used to describ e what typ e
of errors are typically encountered in a registration algorithm given a certain input
ie what are the typical systematic errors For example if a registration algorithm
p erforms the required registration with a resolution of voxels then is is exp ected that
the registration will b e p erformed with a precision of within voxels when the input is
ideal Accuracy however is a more direct measure of the registration It refers to the
true error that exists at sp eci c lo cations within an image It is this prop erty that is
of direct interest in clinical applications A go o d example presented by is when a
surgeon may p oint at a screen and say I must make an incision here How accurate
can this lo cation b e determined on the patient It is the accuracy of the registration
that can answer questions such as this
The accuracy of registration algorithms can b e measured b oth qualitatively and quan
titatively A qualitative approach is generally based on visual insp ection by trained
medical physicians to see if corresp onding structures are e ciently overlapp ed onto
each other A quantitative approach however relies on more mathematical or statisti
cal techniques in order to quantitatively measure the accuracy
The robustness of a technique refers to its ability to p erform adequately in noisy en
vironments This criteria is closely related to the stability criteria which implies that
the algorithm should not pro duce erratic results in slightly di erent situations for ex
ample when a tumour may exist in an image The reliability criteria refers to the basic
requirement that the algorithm should p erform the same task rep eatedly in a reliable
fashion
The criteria of resource requirements and the algorithm complexity go hand in hand If
a complex algorithm is required for complex deformations then obviously the resources
required to p erform such a task will b e higher than compared to that required for less
complex rigid registrations These two criteria are also generally heavily related to
the clinical application If an algorithm is required to b e implemented in real time for
use during surgical interventions then it will b e required to b e fast and e cient thus
increasing the resources required
The clinical use criteria refers the the concepts of whether it is a viable prop osition
to implement a certain registration algorithm in clinical applications Or will some
Classi cation of Registration Techniques Validation and Related Issues
other easier and cheap er metho d su ce eg manual metho ds It is seldom that
one particular registration algorithm will meet all of the ab ove criteria However the
criteria that must de nitely b e met will b e determined by the required application and
by common judgement
Few registration pap ers actually follow up on the use of the registration algorithm that
was prop osed in the initial publication This is another reason requiring the conduction
of more validation studies In fact validation studies are only now b eginning to b e
conducted A go o d example of a validation study is presented by West et al
In this study the authors use a prosp ective registration technique based on external
metho ds ducial markers which is used as a gold standard to p erform an ob jective
blinded evaluation of the accuracy of other retrosp ective image registration techniques
One necessary condition for the success of not only validation studies but also general
research into image registration or any other medical imaging techniques is the need
for a thorough collab oration to exist b etween research scientists medical do ctors and
also memb ers of related hardware and software companies
Chapter
Literature Survey
This section of the rep ort will describ e sp eci c registration algorithms in more detail
Due to the immense numb er of existing registration algorithms only signi cant tech
niques and a few classic techniques shall b e discussed These algorithms shall b e
describ ed according to the nature of the registration algorithm as describ ed in chapter
of the rep ort Groupings shall b e made into p oint curve surface intensity based
metho ds and also non rigid based techniques
The Mathematics of Registration
Registration is de ned as the determination of a one to one mapping between the coor
dinates in one space and those in another such that the points in the two spaces that
correspond to the same anatomic point are mapped to each other However b efore
conducting a review of existing image registration techniques it will b e b ene cial to
rst understand the mathematical preliminaries involved in the registration pro cess
This section will outline the principle elements involved in registration for the cases of
rigid a ne pro jective and curved transformations as previously mentioned in section
of the rep ort
The most fundamental form of transformation is the rigid registration This typ e
of transformation is comp osed of only rotations and translations and consequently
preserves the size and shap e of the images b eing registered Thus the problem of
registration can b e de ned as the identi cation of a D rotation R and a D translation
T which when applied to one image Y will bring it into spatial alignment with another
Literature Survey The Mathematics of Registration
image X This concept can b e expressed algebraically as follows
X RY T
This is a simpli ed view at the rigid registration pro cess and succinctly describ es the
way in which it is implemented Note that no particular typ e of arithmetic should
b e implied from equation It simply aims to express the idea that one image is
registered with another by a rotation and a translation
T
Thus any p oint in an image Y Y y y y can b e transformed into the same
T
co ordinate frame as in image X X x x x This can b e expressed as follows
y x
C C B B C C B B
T y R x
A A A A
y x
Where R is the rotation matrix and T is the translation matrix
There are several di erent metho ds that can b e used to represent the rotation matrix
R One of the most p opular metho ds involves the use of Euler angles In this system
three successive rotations through angles of are carried out around the x y
and z axes resp ectively The nal transformation matrix which is a combination of the
three successive rotations just describ ed is shown b elow
cos cos sin cos sin
C B
R
cos sin sin sin cos sin sin sin cos cos cos sin
A
cos sin cos sin sin sin sin cos cos sin cos cos
The disadvantages of Euler representations is that the matrix co e cients are non linear
and secondly the representation is not di erentiable at some singular values where
Another p opular metho d makes use of unit quaternions Quaternions are a generali
sation of complex numb ers and can b e used to succinctly represent b oth D and D rota
tions A unit quaternion may b e represented as a four comp onent vector q q q q q
where q and q q q q A rotation matrix can then b e de ned based
on any unit quaternion Each unit quaternion corresp onds uniquely to a rotation
matrix ie
q q q q q q q q q q q q
C B
R
q q q q q q q q q q q q
A
q q q q q q q q q q q q
Literature Survey The Mathematics of Registration
Unit quaternions have some advantages and are quite p opular for p oint based regis
tration problems However there implementation in more general approaches b ecomes
more complicated due to normalisation factors which must b e taken into account A
metho d used to calculate the registration based on p oint corresp ondences and quater
nion rotations is describ ed in the next section of the rep ort
A ne transformations are comp osed of degrees of freedom as compared to only
degrees of freedom in rigid registration A ne transformations map straight lines onto
straight lines while still preserving parallelism An a ne transformation can also b e
seen as a rigid registration with a scaling and shearing factor added Note that the
scaling can b e either uniform or non uniform The scaling factor can b e intro duced by
T
multiplying the original p oints Y y y y by a scaling matrix
x
C B
A
y
z
The shearing can b e intro duced into the pro cess by multiplying the p oints by a shearing
matrix
xy xz
C B
A
y x y z
z x z y
Where and are the scaling and shearing parameters resp ectively Thus the
degrees of freedom de ned in an a ne transformation are the three rotation three
translation three scaling and three shearing parameters resp ectively
Pro jective transformations again map straight lines onto straight lines however the
parallelism factor is not necessarily kept These typ e of transformations can b e repre
sented by a linear transformation in a higher dimensional space An example of
a D pro jective transformation is given b elow Before the transformation is applied
however a new set of variables are intro duced into the pro cess ie
u
x
w
v
x
w
Where w represents an extra homogeneous co ordinate The transformation is then
expressed as follows
y a a a u
C B C C B B
y a a a v
A A A
a a a w
Literature Survey The Mathematics of Registration
where y y are the original p oints and x x are the p oints obtained after the trans
formation
Another typ e of transformation which is very similar to a pro jective transformation is
bilinear mapping In this typ e horizontal and vertical lines are mapp ed to straight
lines in the transformation however lines falling on any other direction will b e mapp ed
to a curve A D bilinear transformation can b e written as
x a a y a y a y y
x b b y b y b y y
Curved transformations map a straight line into a curve Such a pro cess can b e thought
of as a functional mapping of the co ordinates in the rst image onto the co ordinates in
the second image The can b e expressed in D as
x x F y y
Where F denotes any function that is capable of p erforming the mapping A typical
form of curved transformation is the p olynomial transformation A p olynomial function
in D can b e written as
x a a y a y a y a y y a y
x b b y b y b y b y y b y
For an illustrative description of these transformations refer once again to gure in
chapter of the rep ort
Another example of a p olynomial transformation that is implemented lo cally is in the
use of spline functions The basic format of a D spline transformation can b e given
by the spline co e cients x y z and the spline basis functions B This can b e
expressed mathematically as
P
x x B x y z
B A A A
i j k i j k
i j k
P
y y B x y z
B A A A
i j k i j k
i j k
P
z z B x y z
B A A A
i j k i j k
i j k
The remaining sections of this chapter will now describ e sp eci c registration algorithms
in more detail These techniques are categorised into p oint curve surface and intensity
based techniques A later section will then brie y intro duce the area of non rigid
registration by a review on deformable mo dels and other physical continuum mo dels
Literature Survey Point Techniques
Point Techniques
Point based registration metho ds involve matching sets of p oints b etween di erent
images These p oint sets can b e obtained manually by a trained medical physician ie
anatomical landmarks or they can b e directly computed ie geometrical p oints Once
these p oint sets are obtained an imp ortant problem must then b e addressed This is
the problem of corresp ondence
Registration of images based on p oint techniques can b e categorised into those that are
based on known corresp ondences and those where the corresp ondences are unknown
The concept of corresp ondence is b est describ ed with the use of an example Consider
two sets of p oints x and y each containing N D p oints If it is known a priori that
i i
each p oint x corresp onds to the same physical p oint y then the sets of p oints are
i i
said to b e in corresp ondence This simpli es the registration problem signi cantly and
allows a closed form solution to b e found
Registration with p oint corresp ondences
This section will outline the framework required for a p oint based registration with
known corresp ondences
Assume two corresp onding p oint sets x and y i N which can b e denoted as
i i
X x x and Y y y Thus each p oint describ ed by the three element
N N
vector x corresp onds to y with the same index The registration pro cess then lies in
i i
nding the rotation matrix R and the transformation matrix T such that
x Ry T
i i
The error asso ciated with each transformed p oint can b e describ ed by the following
equation This error is due to imp erfect data or any other noise sources that will
ultimately make it imp ossible to nd a solution to equation for every p oint
e x Ry T
i i i
A numb er of techniques then prop ose to nd the transformation parameters R and T
by minimising the sum of squares of these errors This is formulated in the following
manner
N
X
jjx Ry Tjj
i i
i
Literature Survey Point Techniques
There are several techniques that have b een prop osed to solve this registration with
known corresp ondences Most of these techniques have stemmed from the computer
vision literature Two such metho ds are based on the use of unit quaternions or singular
value decomp osition These techniques are describ ed next Other techniques include
orthogonal matrices by Horn and a dual quaternion metho d
Quaternion Rotations
The metho d describ ed in this section is the rst general analytic solution for determining
the required rotation matrix It was prop osed by Horn and it is based on the
quaternion representation of rotations Refer back to section for a de nition of a
quaternion
The rst step in this registration pro cedure is to translate each p oint set in order to
align their centroids with the co ordinate origin This pro cess is describ ed as follows
P P
N N
Calculate the centroids x and y
X Y
i i
i i
N N
0 0 0 0
y x and y Construct the p oint sets X and Y such that x
Y X
i i
i i
Once these steps have b een completed the quaternion metho d can b e applied as follows
P
T
N
0 0
y Calculate the covariance matrix x
XY
i
i i
T
Form the matrix A
XY
XY
T
Construct the column vector A A A
Form the symmetric matrix Q as follows
T
tr
XY
Q
T
tr I
XY XY
XY
where tr is the trace op erator and I is the identity matrix
Calculate the unit eigenvector q q q q q of Q corresp onding to the largest
R
p ositive eigenvalue
The orthonormal rotation matrix R is calculated from q according to equation
R
given previously in the rep ort
Once the rotation matrix R has b een calculated the transformation matrix T is cal
culated as T R For the full derivation and related techniques refer to
X Y
Literature Survey Point Techniques
Singular Value Decomp osition
Singular value decomp osition was prop osed by Arun et al This technique will now
b e brie y presented The rst step of translating the centroids to the origin is the same
as that used in the unit quaternion approach Thus assuming that the data sets have
already b een transformed to align centroids the remaining approach is as follows
Formulate the covariance matrix as in the quaternion metho d
XY
Then nd the singular value decomp osition SVD of such that
XY XY
T
U V
T
Calculate R UV
If the determinant jRj then set jRj R Else if jX j then the
algorithm has failed and the SVD metho d cannot b e directly used
For this technique it is necessary to check the determinant of the result in order to
validate the result obtained A determinant of implies that the calculated rotation
matrix is actually a re ection This however is not the desired result In order to
comp ensate for this the authors show that certain steps can b e taken to ensure that
only rotations are p ossible See for a complete description of the technique and
for related applications
Registration without p oint corresp ondences
Registration without a priori knowledge of corresp ondences is a much more di cult
task than that discussed in the previous section This is b ecause the algorithm must
simultaneously estimate the corresp ondences and the rigid transformation
Solutions to this problem were prop osed by several researchers One well known ap
proach is the use of the ICP algorithm designed by Besl and McKay This technique
however is capable of solving not only the p oint set registration but also the curve
and surface based registration problems Thus this algorithm is describ ed along with
the other surface based approaches in section of this chapter Another technique
similar to the ICP algorithm is prop osed by Chen et al This technique and also
the ICP approach is based on an iterative least squares solution
Other metho ds also exist to solve the corresp ondence problem Graph matching meth
o ds are one such approach An example of a graph matching technique is discussed in
the following section
Literature Survey Point Techniques
Graph Matching Approach
This section will brie y outline a graph matching approach used to solve the corresp on
dence problem which was prop osed by Cheng et al
The approach is based on a recursive descent tree traversal algorithm that is used to
calculate the p oint corresp ondences b etween two views Given two sets of p oints which
may b e corrupted by noise the p oints in the rst set are constructed as a graph and
then a second graph is constructed from the second set of p oints so that a maximal
matching p oint and minimal matching error is obtained The matching algorithm for
the case when there exists an equal numb er of p oints in b oth sets is given b elow
Assume two sets of D feature p oints P P P and Q Q Q
n n
Construct a graph from the rst data set G P P P whose vertices
n
are the p oints P P P G can b e a p olygon a tree a path or any typ e of
n
connection This graph will b e used to compare the graphs generated from the
second data set
P is the ith vertex in G and d P P is the length of the ith edge P P
i i i i i
Thus nd an edge Q Q in the second data set which is close to the rst edge
i j
P P of graph G within a threshold ie
jd Q Q d P P j
i j
Once a match has b een found generate the next successive vertex and store
all the matched vertices in a working graph G Continue this pro cess until all
i
n p oints have b een visited and a nished candidate graph is generated G
Q Q Q
n
The solution graph G will b e chosen from those candidate graphs G with the
i
minimal k th order error from G ie
jG G j jG G j for all i
i
k k
Where the k th order error measure b etween G and G is de ned b elow Note
that k sp eci es the numb er of p oints in the nal graph and can range anywhere
from to n The case when k n sp eci es the complete graph
min k i
i n
X X
jd Q Q d P P j jG G j
i i j i i j
k
i j
Literature Survey Point Techniques
The larger k is the less p ossible an error corresp ondence will b e found b ecause it is
closer to the real or optimal solution For a complete description of the metho d see
In general though graph matching approaches are extremely computationally
exp ensive Esp ecially for large data sets the computation time will b e exp onentially
prop ortional to the size of the data set They are however a unique approach for the
determination of corresp ondences b etween di erent p oint sets
Extremal Points
The approach describ ed in this section is a metho d used to generate a set of geometrical
p oints based on di erential characteristics obtained from an image This metho d was
prop osed by Thirion These p oints known as extremal p oints are stable
landmarks that can b e used for registration A brief description of the metho d will now
b e presented
This metho d used by Thirion to extract extremal p oints is based on a mo di ed version
of the Marching Lines algorithm The marching lines algorithm is a metho d used
to extract crest lines from surfaces in an image This technique will b e discussed in the
next section of the rep ort A crest line also known as an extremal lines is de ned as
the intersection of two implicit surfaces They are known to b e geometric invariant D
curves Extremal p oints however are de ned as the intersection of extremal lines with
a third implicit surface
The following gure is helpful in understanding the physical relationship b etween
a crest line and a surface Note that the total set of curvatures at any p oint on a
surface can b e describ ed with two principle directions t and t and two asso ciated
principle curvatures k and k except for umbilic p oints These parameters are also
illustrated in gure
Thus an extremal p oint is de ned as the intersection of three implicit surfaces f I
e and e where f represents the intensity value of the image I is an iso
intensity threshold and e and e are extremality functions de ned as the directional
derivative of k and k in the direction of t and t resp ectively ie
e rk t
e rk t
The two extremality functions are geometric invariants of the implicit surface ie in
variant to rigid transformations Therefore the relative p ositions of extremal p oints
are also geometric invariants
Literature Survey Point Techniques
Crest Line Normal n
t2 t1 Principal Directions
k1 Maximal
Curvature
Figure Surface with crest line
The extremal p oints can b e calculated using a metho d whereby the extremality func
tions e and e are calculated for all the voxels in the D image directly from the
di erentials of the image intensity function Then each cubic cell cell formed with
voxels is considered individually to see if an extremal p oint can b e extracted For
each vertice of the cub e values can b e de ned f e and e
Thus the extraction pro cess requires rst checking to see if the iso surface f I crosses
the cell Then interp olating the extremality co e cients to these intersection p oints
so that they can b e used to check if a crest line exists If so then the extremality
co e cients are further interp olated to the end p oints of the crest line to nally check if
an extremal p oint exists Tri linear interp olation is used as a go o d rst order approxi
mation for the interp olation pro cess
Once a set of extremal p oints are extracted from two images they can b e registered
This step is quite simple In fact it is the extraction of the stable and invariant D
p oints that present the most problems due to its complicated nature The registration
metho d used by Thirion is based on a prediction veri cation scheme with an iterative
improvement of the nal transform based on a least square t and relying on a quater
nion representation His technique also uses geometrical invariant attributes which
can b e calculated for each extremal p oint to reduce the numb er of p oints that need to
b e considered in the corresp ondence selection For a complete description of extremal
p oints extraction and registration refer to
Literature Survey Curve Techniques
Curve Techniques
The next step up the ladder of complexity in terms of features that can b e matched
b etween images is curve based techniques Any curve that can b e derived intrinsically
from an image can b e used in the matching pro cess Such a curve could b e repre
sentative of some anatomical structure that can b e outlined by a medical physician or
they can b e derived in an automatic fashion
One interesting technique prop osed by Thirion et al is the extraction of crest
lines based on di erential geometry of surfaces This technique which utilises the
marching lines algorithm can b e implemented in an automatic fashion and is describ ed
next in the rep ort
Marching Lines Algorithm
The marching lines algorithm is used to extract D lines corresp onding to the inter
section of two iso surfaces from two di erent D images with sub pixel accuracy This
technique is an extension of the marching cub es algorithm which is used to extract iso
value surfaces from D images In the pap er presented by Thirion et al they also
describ e a way to compute di erential characteristics of iso surfaces This technique
can then b e applied to the extraction of crest lines in D images
For an illustrative description of a crest line refer back to gure that was presented
in the discussion of extremal p oints The extraction of such crest lines has b een studied
previously by Monga et al They present formulas for computing the curvature
of ob ject surfaces based on the rst and second derivatives of the image and conse
quently provide a metho d to compute the principal curvatures and directions of D
surfaces Their de nition of a crest line is a lo cus of p oints whose maximal curvature
ie maximum absolute value of the two principal curvatures is a lo cal maximum in the
corresp onding principal direction An example picture showing a set of crest lines is
shown in gure These crest lines are shown sup erimp osed on top of an iso intensity
surface of a brain that was extracted from a heavily smo othed MR image
Once a set of curves have b een extracted from corresp onding images using this tech
nique then steps can b e undertaken in order to register them Gueziec et al
employed a technique based on the smo othing of crest lines and used a B spline rep
resentation to rigidly register a set of CT images Another approach using similar
metho ds is presented by Thirion et al It is also p ossible to non rigidly regis
ter images using this approach A nal comment on techniques such as the marching
Literature Survey Surface Techniques
Figure Extracted crest lines sup erimp osed on a smo othed iso surface of a brain
lines algorithm is that they generally must have very high resolution images in order
to e ciently extract stable crest lines
An extension of the marching lines algorithm and the extremal p oints metho d is pro
p osed by Thirion This technique is based on the fusion of b oth of these metho ds
which can b e used to extract a set of characteristic geometric features known as the
extremal mesh
Surface Techniques
Registration based on the existence of corresp onding surfaces b etween images is an
extremely p opular approach in medical image registration Typically the skin or b one
surface is used as these surfaces can b e automatically extracted in a simple fashion
The extraction of more detailed surfaces such as the brain surface is slightly more
complicated Most of the surface based approaches attempt to minimise the distance
b etween the two surfaces More sp eci cally they attempt to minimise the distance
b etween a mo del surface S and a data surface S ie
A B
D distance S T S
A B
There are three p opular surface based registration techniques that have dominated over
most other techniques They are the Head hat algorithm by Pelizarri et al the
Literature Survey Surface Techniques
Hierarchical Chamfer Matching HCM algorithm which has b een formulated and ex
tended by several researchers Borgefors Jiang et al and nally the Iterated
Closest Point ICP algorithm by Besl and McKay These techniques solve the regis
tration problem by nding the optimal transformation based on information contained
in the surfaces of corresp onding images The principal features of these three techniques
will now b e presented along with their inherent advantages and disadvantages
The Head Hat Algorithm
The head hat algorithm is one of the reasons why rigid based segmentation techniques
has b een so p opular in clinical applications This algorithm was originally develop ed for
head image registration of PET images to either MRI or CT images In this technique
a simple pre segmentation step is used to extract the skin surface of the head from all
of the imaging mo dalities used in the registration The surface of one of the images
is then used as the head and a set of p oints is extracted from the surface contour of
the second image to represent the hat The algorithm is presented b elow according to
This rst step in this technique is to represent the mo del surface S by a set of planar
A
contours segmented on the MRI or CT images The second surface SB is then rep
where i M The head hat algorithm then resented as a set of D p oints P
B
i
estimates the rigid b o dy transformation T that minimises the following quantity
X
D d S T P
s A B j
j
This technique minimises the distance metric that is de ned as the distance b etween
the hat p oints from the head p oints in a direction along a line from the p oint of the
centroid G of the head surface Once the centroid is computed the distance d is
S
de ned in terms of the Euclidean distance b etween the p oint P and the intersection
p oint Q b etween the surface S and the line GP The algorithm attempts to nd the
six parameters of the rigid transformation T that minimises D and must b e supplied
with an initialisation quite close to the nal solution The head hat technique uses a
standard gradient descent of Powell s algorithm to compute the minimisation of the
distance metric D
The main disadvantages of the head hat approach is that it is often prone to fall into lo
cal minima during the registration pro cess This is generally caused from either the im
plementation of the distance measure or due to the inaccuracy of the pre segmentation
step Because of this drawback user interaction is generally required The second
Literature Survey Surface Techniques
pitfall is in the de nition of the distance from the centroid as used in equation
This use of the centroid G suggests that the surface must in fact b e a spherical shap e
The nal disadvantage is that the distance measure can only b e used when the surfaces
are su ciently close to each other during the entire registration pro cess The e ects of
some of these disadvantages can b e quite critical and as such metho ds were prop osed
to replace the centroid by a central curved axis This metho d was found e cient for
registering D images based on the use of b ony structures as references However this
has still not lead to the formation of a general metho d
Hierarchical Chamfer Matching
The hierarchical chamfer matching algorithm uses a chamfer distance map that is gen
erated from the surface of one of the images This distance map is then
used as a p otential function for surface p oints that are contained in the second image
The total p otential is then minimised in order to nd the transformation parameters
This metho d along with several others was created in an attempt to overcome some
of the drawbacks mentioned in the previous technique description The aim was to
minimise a more general distance measure A brief description of the steps leading to
the intro duction of the HCM algorithm and other metho ds will now b e presented
where First assume that the data surface S can b e represented by D data p oints P
B B
i
i M as in the previous section Thus any surface can simply b e decomp osed into
these p oint representations The rst assumption is that the surface S is included in S
B A
after the registration pro cess The next step is to then estimate the rigid transformation
T that minimises the quantity D as de ned in equation where d is the distance
S
b etween a surface S and a p oint M and is de ned by the minimum Euclidean distance
ie
d S M min d P M
S P 2S i
eucl
i
Many algorithms have b een prop osed to compute this minimum Euclidean distance
However di erences exist due to the various representations of the surfaces One
example is a parametric representation of a surface Eg
x f u v
x
y f u v
y
z f u v
z
In this case f f and f could represent spline functions Thus for each p oint M
x y z
the distance d P M as seen in equation is de ned as a function of u and v
i
eucl
Literature Survey Surface Techniques
Thus d S M can b e computed with a non linear bi dimensional minimisation in u
S
and v using the derivatives of f f and f This technique however is often prone
x y z
to encounters with lo cal minima during the minimisation Approaches that involve
the intro duction of higher order representations have b een prop osed However for
implementation of complex shap es this approach can b ecome quite cumb ersome
The next approach was to compute the distances and store the results in a distance map
that is constructed as a lattice contained within a volume V Thus for any p oint M
inside V the distance d S M can b e approximated by a combination of the distances
S
computed at the lattice p oints near M These distances stored in the D distance
map can b e computed using the exact distance from the p oint to surface However
this is a computationally exp ensive approach A p opular approach was to extend D
chamfer distance maps into D ie the hierarchical chamfer matching algorithm
describ ed at the b eginning of this section
In this approach the mo del surface is represented as a set of voxels contained within the
image Euclidean distances are then calculated and stored at each of the voxels inside
the surface These Euclidean distances are approximated by propagating a mask of
lo cal distances each lo cal distance b eing asso ciated with an elementary displacement
in the lattice A further extension of this approach was prop osed by in order
to increase the accuracy of the distance map near the surface by use of o ctree spline
distance maps
The Iterative Closet Point Algorithm
As mentioned ab ove the ICP algorithm was develop ed by Besl and McKay in
and owes much of its p opularity to its versatility as it may b e used for p oint sets curves
and also surfaces The ICP algorithm is also quite simple as it only requires a pro cedure
to determine the closet p oint on a geometric ob ject to a certain p oint and a pro cedure
to determine the rigid transformation b etween two p oint sets
The ICP algorithm is designed to nd a match b etween a data shap e Y and a mo del
shap e X This pro cess is formulated under the assumption that the data set is formed
by a p ossibly noisy sampling of the mo del A distance metric d is then de ned b etween
a data p oint y Y and the mo del X as
d y X min kx y k
x2X
A p oint x is de ned as that for which d y x d y X where x X Thus x is the
c c c
c
closest p oint on X to Y If we denote X as the set of closest p oints in X corresp onding
c
Literature Survey Surface Techniques
to each p oint in Y we can de ne an op erator C such that
X C Y X
c
This notion is illustrated in gure where two shap es are b eing matched The
mo del is shown by the smo oth line and the data shap e is given by the dashed line The
corresp onding p oints b etween shap es ie control p oints on Y and closet p oints on X
are connected by another dashed line
Closet Points Xc
Model Shape X
Measured Data Shape Y
Control Points on Y
Figure Corresp onding p oint selection pro cess for the ICP algorithm
To continue on to the next stage an assumption ab out the existence of a pro cedure
to match two sets of corresp onding p oint sets must b e made Such a pro cedure which
will b e denoted by Q may b e implemented by any of the techniques discussed in
Q shall b e de ned such that
R T d Q X Y
C
returns the transformation parameters R rotation matrix and T translation matrix
which can b e used to register the corresp onding p oint sets X and Y optimally in some
C
sense The pro cedure also returns d which is the mean squared distance b etween the
two p oint sets
Now that these preliminaries are covered the ICP algorithm can b e describ ed as follows
Initialise the cumulative transformation parameters R and T to the identity ma
trix Reset k the iteration counter to zero
For each discrete p oint y in the data set compute the closet p oint in terms of
k
Euclidean distance x which lies on the mo del surface ie X C Y X
k C k k
Literature Survey Surface Techniques
Using the corresp ondences from the ab ove step nd the transformation parame
ters R and T which minimise a certain cost function or distance metric via a
k k
sp eci ed p oint registration metho d ie R T d Q Y X A common
k k k C k
example of a cost function or distance metric is the least squared distance metric
X
min kx Ry T k
i i
R T
i
where y represents p oints in the data shap e and x represents p oints in the mo del
i i
shap e If b oth the mo del and data are b oth surface descriptions then solving
equation is approximately equivalent to the volume b etween the surfaces
according to Hauser et al
Apply the incremental transformation parameters from the previous step to all
data p oints y ie Y R Y T Then up date the cumulative transforma
k k k k
tion parameters R and T based up on the incremental transformation parameters
R and T
k k
If a certain stopping criteria is satis ed then end the registration pro cess else
goto step
There are several di erent options which may b e used for the stopping criteria of the
ICP algorithm Originally the algorithm was stopp ed once the change in mean squared
error d d fell b elow a certain threshold Replacing this threshold by the
k k
p
P P
value tr where is the covariance matrix of the mo del data p oints will make
x x
the threshold fairly ob ject indep endent If the registration pro cedure Q p erforms an
optimal transformation in the least squares sense then it can b e validated that the ICP
algorithm is guaranteed to monotonically converge to a lo cal minimum with resp ect to
the mean square distance b etween corresp onding p oints
Some of the other options for stopping criteria which may b e used are listed b elow
Stop if the magnitude of the incremental transformation parameters are b oth less
than thresholds
jT j jR j
k k
and
Rr T r
jRj jTj
Stop if the absolute magnitudes of the incremental transformation parameters are
b oth less than thresholds
jR j and jT j
Ra T a
k k
Literature Survey Intensity Based Techniques
Stop if the change in residual error from equation is less than a threshold
X
kx Ry T k
i i LS E
i
Stop if the total numb er of iterations exceeds a certain threshold k k This
max
last condition is often used in conjunction with another stopping criteria
Besl and McKay also prop osed another version of the ICP algorithm in which the
solution tra jectory in the parameter space is predicted with a linear estimator in order to
fasten the p oint of convergence This approach was compared with traditional gradient
descent like metho ds that require vector gradients to b e calculated It was argued that
the approach of the ICP algorithm will b e approximately one order of magnitude faster
than any vector gradient technique
As mentioned earlier the ICP algorithm can b e incorp orated with varying registration
techniques Most registration techniques use a cost function or other distance metric
similar to that given in equation However there are numerous other techniques
One variation includes the insertion of weighting terms into the cost function The
weights are used to scale the imp ortance of sp eci c corresp ondences based on a pri
ori knowledge often including noise estimates Betting et al also prop osed the
incorp oration of surface normals into the cost function
The ICP approach is an extremely versatile technique for many applications as
it can b e used to match any typ e of surfaces assuming that a closest p oint function
is available Also the ICP algorithm do es not require any calculation of di erential
quantities of the surface data which is often required in other techniques
Intensity Based Techniques
Intensity based techniques are generally the most robust of all registration metho ds
This breed of registration approach do es not involve complex segmentation pro cesses in
order to extract features required for matching ie it do es not reduce the entire image
to a set of characteristic features Instead it op erates directly on the intensity values
within the image and as such do es not make any assumptions on the underlying
information contained within the image Intensity based techniques have also made
signi cant leaps towards the improvement of multimo dal image registration
This section of the rep ort shall outline some of the p opular intensity based metho ds
including moments and principal axis techniques correlation and also mutual infor
mation techniques
Literature Survey Intensity Based Techniques
Moments and Principal Axis Techniques
This family of techniques are based on the principle that an image is a b o dy and thus
has mass moments of inertia principal axes centroids and the like By assuming that
the b o dy is an ellipse or ellipsoid then their corresp onding principal axes can b e found
Thus the registration pro cess lies in trying to nd a transformation that will overlay
the resp ective centroids of each ob ject and also align the principal axes The basic
mathematics involved in a rigid registration by moments will now b e presented
For each p oint in an image it has a corresp onding weight assigned to it This weight
is equal to the intensity value at that p oint Firstly let I x denote the intensity value
T
at the co ordinates x x x x in an N N N image Thus p oints that have
a high intensity level will b e given more attention regarding the transformation The
rst order moment of an image is calculated as follows
P P P
N N N
I x x
x x x
1 2 3
E x
P P P
N N N
I x
x x x
1 2 3
The second order moment known as the disp ersion matrix is given by
P P P
N N N
T
I x x E x x E x
x x x
1 2 3
D x
P P P
N N N
T T
I x E xx E x E x
x x x
1 2 3
Thus the registration pro cess lies in nding the transformation parameters R and T
such that
E x RE y T and
T
D x RD y R
Where y are the co ordinates from an image Y b eing registered to the co ordinates x
in an image X The solution to equation can b e found by simply nding the
transformation parameters that satisfy the following expressions
T RE y
T
D y R R
Many researchers have frequently employed the use of moments and principal axis
techniques However despite their p opularity they are susceptible to erroneous results
if any shap e di erences exist Such a case could arise due to the growth of a tumour or
other kinds of pathology Also registration of pre op erative images with intra op erative
images will result in large errors due to the fact that the brain will move during the
surgical intervention
Literature Survey Intensity Based Techniques
A common use of the moments and principal axis metho ds is in the course acquisition
of registration ie this metho d is employed to get an initial rough estimate of the
transformation which is further re ned by some other metho d
Correlation
One of the most frequently used similarity measures employed in the registration of
medical images is based on the correlation b etween successive images The correlation
b etween an image X and a transformed image Y based on the transformation de ned
by T is de ned as follows
P
X x Y T x
x2X
p p
C t
P P
X x X x Y T x Y T x
x2X x2X
The normalisation that is shown in the ab ove equation results in the nal answer of
the correlation to fall somewhere within the interval Such a result is de ned
as a qualitative measure This correlation measure is also often used to evaluate the
e ectiveness of other transformations
A go o d formulation of the correlation approach is presented in by Ro che et al In
this pap er they b egin with the basics of statistical theory by de ning the correlation
co e cient in terms of two random variables X and Y as
E XY E X E Y cov X Y
X Y
var X var Y E X E X E Y E Y
The authors then lead through the relevant workings and arrive at the de nition of the
correlation ratio which is de ned as
var E Y jX
Y jX
var Y
This measure is then used for rigid multimo dal registration of MRI CT and PET
images Ro che et al suggest that the correlation ratio proves to b e a go o d trade
o b etween accuracy and robustness
Grey Level Co o ccurence Matrices
Correlation techniques use a similarity measure based on the voxel values of coinciding
voxels Another technique similar to this is based on Grey Level Coocurrence Matrices
GLCM In essence GLCM generalises the investigation of the intensities of coinciding
voxels
Literature Survey Intensity Based Techniques
The GLCM is de ned as a D plot or matrix of grey values of the voxels in an image
with a xed displacement b etween them The GLCM is generated by accumulating
the grey value pairs i p i p u and forming a D histogram for all voxels in the
image where i p is the grey value or the intensity value of the voxel at p osition
T
p x y z in the image and u x y z is the displacement vector b etween
u u u
corresp onding voxels Normalising the GLCM is a way of obtaining an estimate of the
joint probability distribution of the voxels i p and i p u
An extension to the de nition of the displacement vector u can b e made in order to
change it to a displacement vector b etween not only voxels in one image but also
b etween voxels in di erent images Thus the GLCM represents a matrix where the
rows represent the intensity scale in one image and the columns represent the inten
sity scale in another image Thus every p osition in the matrix corresp onds to a pair
of coinciding voxels b etween two images The GLCM can b e constructed by simply
counting co o ccurring intensity values ie
X
GLC M
i j
S x i T t x j
x2S
where is the Dirac delta function Thus this extension transforms the normalised
i j
GLCM into an estimate of the joint probability distribution of the voxels in two images
as opp osed to the joint p df of voxels in the same image
Some work has b een carried out by Bro Nielsen to extend the application of GLCMs
from the texture analysis eld into the medical imaging eld or more generally any typ e
of rigid image registration application Bro Nielsen identi ed several voxel similarity
measures from the texture analysis eld evaluated their p erformance for multi mo dality
image registration and also compared them to previously known measures Some of
the GLCM measures that were considered include energy variance entropy relative
entropy mutual information inertia diagonal moment and the correlation co e cient
just to name a few
Mutual Information
Mutual information is a new information theoretic approach and is quickly b ecoming
one of the most p opular techniques available for multimo dal image registration The
basic concept b ehind this approach is to nd transformation which when applied to the
images will maximise the mutual information b etween the two images This technique
along with other intensity based techniques works directly with the image data and
negates the need for any pre pro cessing or segmentation The mutual information
Literature Survey Intensity Based Techniques
technique however is more exible and much more robust than other intensity based
techniques like correlation
Most of the pioneering work on the concept and application of mutual information
in the image registration area has b een p erformed by Paul Viola Results of
image registration using mutual information has proven to b e very successful and has
since seen a dramatic increase of the use of mutual information registration techniques
in other research directions and most imp ortantly clinical applications It has b een
applied successfully to several di erent mo dality combinations which include MR CT
PET and also SPECT Some other applications of mutual information found in the
literature include
The implementation of mutual information registration techniques is quite e cient as it
is based on a sto chastic approximation It is also considered to b e a very general mea
sure as it makes very few assumptions regarding the relationship that exists b etween
the image intensities Assumptions regarding linear correlation or even functional cor
relation are not made It only assumes a statistical dep endence Even though di erent
mo dalities pro duce images which can di er greatly the underlying concept of mutual
information can b e stated as follows given that di erent modalities are imaging the
same underlying anatomy then there wil l be some inherent mutual information between
the images
In terms of statistical theory the representation of mutual information can b e as follows
Given two images X and Y then their joint probability density function P i j may
b e calculated by a simple normalisation of their D histogram other metho ds also
exist Also denote P i and P j as their corresp onding marginal probability density
x y
functions Thus the mutual information b etween the two images is de ned as
X
P i j
P i j log I X Y
P i P j
x y
i j
This ab ove expression presents the basic mathematical formulation of mutual informa
tion However a more complex implementation metho d used by Viola et al will now
b e discussed
The aim of this pro cess is to register two volumes of image data namely the reference
volume and the test volume A voxel within the reference volume is denoted by u x
and a voxel within the test volume is denoted by v x where x are the co ordinates
of the voxel The registration pro cess is aiming to nd a transformation T that maps
co ordinates in the reference volume to co ordinates in the test volume Thus v T x is
a test volume voxel that is asso ciated with the reference volume voxel u x
Literature Survey Intensity Based Techniques
The aim is to seek an estimate of the transformation that registers the two volumes
together by maximising their mutual information ie
T arg max I u x v T x
T
where I is the mutual information and is de ned in terms of the entropy within the im
ages Entropy can b e interpreted as a measure of uncertainty variability or complexity
Thus the mutual information is given by
I u x v T x h u x h v T x h u x v T x
where h x is the entropy of a random variable and h x y is the joint entropy of two
random variables These two terms are given by the following expressions
Z
h x p x ln p x dx
Z
h x y p x y ln p x y dxdy
The mutual information de ned in equation is split into three comp onents The
rst term is the entropy of the reference volume The second term is the entropy of
the test volume into which the reference volume pro jects Notice that the second term
is a function of the transformation T however the rst term is not The nal term is
the joint entropy of b oth the reference and the test volume and it contributes to the
whole expression when the two volumes are functionally related
Finding the entropy of the image volumes involves integration of their probability den
sity functions as describ ed earlier This presents some problems as the image prob
ability density functions are unknown However an estimate can b e calculated from a
sample of the image data The rst step is to approximate the underlying probability
density p z by a sup erp osition of functions centered on the elements of a sample A
drawn from z
X
z p z P R z z
j
N
A
z 2A
j
In this equation P z is the Parzen window density estimate N is the numb er of
A
trials in the sample A and R is a window function which integrates to
To simplify subsequent analysis it will b e assumed that the window function is a
Gaussian density function however this is not necessary The Gaussian density is
given by
n 1
T
2 2
z z G z j j exp
Literature Survey Intensity Based Techniques
where is the co variance of the Gaussian function Thus evaluation of the entropy
integral b ecomes
Z
1
P z ln P z dz h z E ln P z
z
1
This equation however is extremely di cult to solve and b ecause of this it is approx
imated as a sample mean
X
h z ln P z
i
N
B
z 2B
i
where N is the numb er of trials in the second sample B Thus an approximation for
B
the entropy of a random variable z may b e written as
X X
ln G z z h z h z
i j
N N
B A
z 2B z 2A
i j
To brie y re cap the entropy of a density can b e approximated by two samples One
to estimate the density and the second to estimate the entropy
The next step in this pro cess is to estimate the entropy of the term v T x which is
a function of the transformation T In order to nd a maximum of the entropy or
mutual information then the gradient with resp ect to the transformation T must b e
ascended The derivative of the entropy can b e shown to b e
X X
d d
T
W v v v v h v T x v v
v i j i j i j
dT N dT
B
x 2B x 2A
i j
where v v T x v v T x v v T x and
i i j j
k k
v v G
i j
v
P
W v v
v i j
v v G
i
k
v
x 2A
k
where W v v is a weighting factor which takes on values b etween one and zero
v i j
The approximation of the entropy as seen in equation can now b e used to evaluate
the mutual information b etween the two images as seen in equation However to
obtain a maximum of the mutual information an approximation to its derivative will
have to b e calculated This is shown as follows
d d d d
h u x h v T x h u x v T x
dT dT dT dT
Note that the reference volume is not a function of the transformation Thus its
derivative which is the rst term on the right hand side is zero The other two terms
can b e computed using equation Thus the registration of the two images can
b e computed For a more complete derivation and description of the implementation
refer to
Literature Survey Non Rigid Registration Techniques
Non Rigid Registration Techniques
This section will brie y outline some of the theory and techniques involved in non rigid
registration applications The theory of deformable mo dels will rst b e presented along
with a brief discussion on their use in registration A brief intro duction to the use of
physical continuum mo dels such as elasticity and viscous uid mo dels will then b e
discussed
Deformable Mo dels
Deformable mo dels o er a unique an e ective approach to certain image analysis ap
plications Their use can b e found in areas such as segmentation shap e representation
tracking and of course registration Deformable mo dels were originally intro duced in
the computer vision eld for use in computer graphics and have since b een recognised
as p owerful techniques for use in medical imaging asp ects The mathematical basis
of deformable mo dels can b e derived from geometry physics and also approximation
theory The physical interpretation p erceives deformable mo dels as elastic b o dies that
resp ond naturally to applied forces and constraints
The most p opular form of D deformable mo del is known as a snake Snakes oth
erwise known as deformable contour mo dels will b e presented in the following section
This mathematical formulation b ehind the snakes will b e presented according to
This formulation draws from the theory of optimal approximation involving functionals
Energy Minimising Deformable Mo dels
In geometric terms a snake is a parametric contour contained in the image plane
T
x y This contour is represented as s x s y s where x y are the
co ordinate functions and s is the parametric domain Thus the shap e of the
contour sub ject to an image I x y is determined by the following functional
E S P
This functional can b e interpreted as a representation of the energy of the contour The
nal shap e of the contour corresp onds to the minimum of this energy The rst term
in this expression is known as the internal deformation energy and is expressed as
Z
w s S ds w s
s s
Literature Survey Non Rigid Registration Techniques
The two parameters seen in equation characterise the deformation of the contour
w s and w s control the tension and the rigidity of the contour resp ectively
The second term in equation refers to the coupling of the snake to the image This
is expressed as follows
Z
P P s ds
Where P x y denotes a scalar p otential function de ned on the image plane The
implementation of snakes to images is accomplished by designing external p otentials
whose lo cal minima coincide with intensity extrema in the image ie edges or any other
features of interest Thus this will attract the snake to lie on these p oints of interest
A nal note regarding snakes is that the contour s which minimises the energy
E must satisfy the Euler Lagrange equation
w rP s t w
s s s s
This partial di erential equation signi es the balance of b oth internal and external
forces when the contour is at equilibrium The three terms in this equation represent
the internal stretching forces the b ending forces and the external forces that couple
the snake to the image data resp ectively Solving this equation is generally p erformed
with the use of numerical techniques
Dynamic Deformable Mo dels
The approach presented in the previous section can b e thought of as a static problem
However it is p ossible to formulate this approach in a dynamic way such that the
snake can evolve to equilibrium Such an approach is termed as dynamic deformable
mo dels This technique not only allows the quanti cation of static shap e but also shap e
evolution through time These typ e of mo dels are valuable to medical image analysis as
most anatomical structures are deformable and continually undergo non rigid motion
in vivo
An example of a dynamic deformable mo del is a dynamic snake that can b e extended
from the previous static application by inserting a time varying contour s t
T
x s t y s t with a mass density s and a damping density s Thus the
Lagrange equations of motion for a snake with the internal energy as given in equation
and external energy as given in equation is
w rP s t w
t t s s s s
Literature Survey Non Rigid Registration Techniques
In this expression the terms from left to right represent the inertial forces damping
forces internal stretching b ending forces and external forces resp ectively
One ma jor application where deformable mo del approaches have seen go o d use is in
the atlas matching problem In this case the atlas is mo delled as a physical ob ject and
is given elastic prop erties After an initial alignment the atlas deformably matches
itself to a patient s image scan in resp onse to forces derived from the image features
There are however several problems with this approach Firstly the technique is quite
sensitive to initial p ositioning This often requires a go o d initial rigid registration in
order to overcome this problem The approach is also susceptible to any neighb ouring
features that may cause incorrect matching
Finally user interaction is generally required in order to ascertain the matching of the
atlas with complicated ob ject b oundaries One approach to overcome such problems
is to incorp orate the use of pre pro cessing the images ie edge detection and morpho
logical op erations This allows the generation of a more smo othed representation of
a brain surface for example which will consequently allow easier convergence of the
deformable surface mo del Generally deformable mo dels require some user interaction
in order to pro duce accurate results
Physical Continuum Mo dels
A large ma jority of the recent work conducted on non rigid registration has involved the
use of physical continuum mo dels These mo dels have stemmed from computer vision
and computer graphics and are now currently nding many applications in the area of
registration These physical continuum mo dels examples include elasticity and viscous
uid mo dels ensure that the deformation eld is physically smo oth This section of the
rep ort will present a very brief intro duction into this immense and complicated eld
Any physical continuum mo del can b e describ ed in terms of a sequential relationship
b etween four factors These are the displacement strain stress and force This
relationship is illustrated in gure The forces and displacements are external
factors while the stress and strain are internal factors The relationship b etween these
last two is determined by the physical mo del used
In terms of the mo del of elasticity the deformations u x are linked to the external
forces f x by the linear op erator of elasticity L such that
Lu x f x
Literature Survey Non Rigid Registration Techniques
External Factors
Internal Factors
Force Stress Strain Displacement
Physical Model
Figure Relationship b etween physical factors
where x x x x are the image co ordinates and u x is the displacement eld
The linearisartion of the elastic mo del implies that the ab ove equation is only valid for
small deformations The linear op erator of elasticity is de ned by
T
Lu x r u x r r u x
T
where r r r is the Laplacian op erator Equation with L de ned in equation
shall b e referred to as the linear partial di erential equation of elasticity Several
metho ds exist to solve this PDE Examples include the successive over relaxation algo
rithm as used in nite elements and convolution as prop osed in The amount
of deformation that o ccurs is determined by the prop erties of the ob ject ie the elastic
constants and as seen in equation The deformation is then determined by
solving equation sub ject to a b o dy force f
The forces are derived from factors such as the image intensities and image gradients
Also when the constants in equation are large compared to the forces then the
ob ject will b e rigid One of the main disadvantages of the elasticity mo del is that the
elastic deformations are characterised by restoring forces that increase linearly with the
deformation preventing the images from b eing fully registered To overcome some
of these disadvantages Christensen intro duced the viscous uid mo del
In a simple viscous ow mo del equation and equation applies to the instan
taneous velo city eld v x t instead of the displacement eld A viscous uid mo del
di ers from the elasticity mo del b ecause the restoring internal forces are relaxed over
time thus allowing the images to b e fully registered This mo del also p ermits much
larger deformations while still maintaining the top ological prop erties of the deformed
ob ject The viscous ow partial di erential equation used by Christensen is shown
b elow
T
r v x t r r v x t b x u x t
Literature Survey Non Rigid Registration Techniques
where v x t is the instantaneous velo city of the displacement eld u x t at time t
The term b x u x t represents the applied forces and the parameters and are the
viscous uid co e cients For a more complete description of these physical continuum
mo dels see
Chapter
Discussions and Conclussion
It is apparent from the discussions presented in the rep ort thus far and also from the
large numb er of researchers who are still devoting time and resources in this area that
registration is far from a solved problem The many diverse techniques develop ed for
various applications such as a simple registration of D images to highly sophisticated
image guided surgery applications show the imp ortance that image registration still
holds in the medical imaging eld Although some sections of the image registration
problem have b een adequately solved there are a numb er of areas that still need further
research in order to fully develop the p ower that registration can o er This section will
attempt to answer some of the questions involved with the future research of medical
image registration
The traditional metho ds of medical image registration have b een based on either manual
metho ds or the use of ducial markers It is obvious from the discussion thus far that
there is a de nite trend to move away from manual metho ds towards more automated
means This can clearly b e deduced from the many fully automatic systems that are
now b eing prop osed in the literature It is also quite obvious that there is a de nite
trend towards the use of more non invasive registration metho ds Non invasive metho ds
do not involve the use of ducial markers which are devices that do not go hand in
hand with patient comfort
Generally these trends can b e summed up as a movement from extrinsic to intrinsic
based registration metho ds This typ e of approach also implies that the registration
is b ecoming retrosp ective in nature This term has b een previously mentioned in the
rep ort It refers to the idea that registration can b e p erformed after the imaging of
the patient and no steps prior to the imaging need to b e taken into consideration
Prosp ective registration on the other hand refers to metho ds that involve the use of
Discussions and Conclussion Discussions and Conclussion
ducial markers or other metho ds whereby certain provisions must b e made b efore
the imaging of the patient can b e conducted Thus there is a trend from extrinsic
prosp ective registration metho ds to intrinsic retrosp ective metho ds
In terms of registration metho ds that are currently the most p opular surface based
metho ds and also intensity based approaches are de nitely the winners by far Intensity
based techniques are also now b eginning to set the standard for registration accuracy
Such a standard was only created by extrinsic metho ds in the past One advantage
that surface based approaches have over intensity based metho ds is that they still have
a slight sp eed advantage in terms of the computation time they require This can b e
accredited to the fact that intensity based metho ds must handle a lot more information
than surface based approaches ie they op erate on the entire image data rather than
a reduced set of characteristic features
Surface based metho ds are also dominant in intra op erative pro cedures where patient
to mo dality registration is often required The reason for this domination is that sur
face information can b e easily acquired with the use of devices such as laser scanners
Obtaining voxel based information however is a much more di cult Although this
is generally the case to day a new breed of imaging devices are currently b eing slowly
intro duced into clinical practice For example the advent of op en MRI imaging devices
now allow high quality intra op erative images to b e acquired which may b e used for
patient to mo dality registration along with many applications
One of the drawbacks of surface based metho ds is that they cannot handle cases when
the surfaces b eing matched di er signi cantly from each other This may often o ccur
during any intra surgical conditions when a brain my shift once it has b een exp osed
by op ening the patient s skull This is where non rigid techniques are more sup erior
b ecause they can account for any deformations that may b e intro duced during the inter
vention However the complexity involved with non rigid registration is still to o high
to e ectively and e ciently use them in real time applications Thus more research is
still needed on these techniques in order to allow for faster real time implementation
metho ds
One area that intensity based metho ds will have a strong future in is in radiotherapy
treatment related registration For such a pro cedure it is required to know exactly
where an anatomical structure is p ositioned within a patient in order to correctly aim
the radiation dose at it Targets such as an intra cranial mass or other forms of tumours
will need to b e accurately outlined This is almost solely based on imaging techniques
Thus with this form of imaging available in this application the use of intensity based
registration is seen as a prime candidate for any related registration problems
Discussions and Conclussion Discussions and Conclussion
Mono mo dal registration problems on the whole have b een solved adequately
This is b ecause the registration problem can b e formulated much easier as compared
to other registration problems ie the relationships b etween images are easier to mo del
b ecause the images are acquired from the same mo dality Although certain areas still
need some consideration there is no ro om for any new ma jor contributions to b e made
Multimo dal registration however is a much more sub jective area as it is much harder
to quanti ably validate the accuracy of such algorithms
Multimo dal registration is still an evolving eld This is due to not only the continued
improvement of certain techniques but also due to the improvement of many imaging
mo dalities Thus the registration algorithms must evolve with the imaging mo dalities
in order to make full use of the information they o er As previously mentioned
multimo dal registration validation is based on a much more sub jective approach There
are also several layers of sub jectivity within this global approach For example a small
di erence in the completed registration of MRI CT images is easily discernible by the
human eye However validation of matched ultrasound or PET images is a much harder
task due to the nature of the images eg PET images are often blurred and ultrasound
is heavily sp eckled with noise This makes quanti cation of this typ e of registration
virtually imp ossible
The main application of multimo dal registration is in diagnostic areas Although many
pap ers describ e how e ective multimo dal registration can b e for this application it is
used very little In the authors describ e that this can b e accounted for by logistic
factors such as multimo dal images are generally acquired in di erent departments at
di erent times by di erent p eople and often evaluated by di erent sp ecialists This
concept is also comp ounded by the fact that it is still unclear as to how registration
can b e used optimally in the diagnostic setting Thus there is still much research to
b e done in this area
Perhaps one of the main areas that need signi cant development is in the validation
of registration accuracy This topic was heavily discussed in section of the rep ort
In that same section an idea was prop osed that there exists a need for an extensive
collab oration to exist b etween relevant parties This is a crucial factor that must
b e ful lled if any research in the medical imaging eld is to b e conducted successfully
Only together can researchers engineers medical practitioners software and hardware
manufacturers and the like solve the many problems involved with image registration
and general medical image analysis problems
Bibliography
M N Ahmed Novel Image Segmentation and Registration Algorithms for the Study
of Brain Structure and Function PhD thesis University of Louisville Decemb er
K S Arun T S Huang and S D Blostein Least squares tting of two d p oint
sets Trans PAMI Septemb er
N Ayache Volume image pro cessing results and research challenges Technical
rep ort INRIA Septemb er
N Ayache Medical computer vision virtual reality and rob otics Image and
Vision Computing May
P J Besl and N D McKay A metho d for registration of d shap es IEEE Trans
actions on Pattern Analysis and Machine Intel ligence February
F Betting and J Feldmar d d pro jective registration of anatomical structures
with their pro jections In Y Bizais and C Barillot editors Information Processing
in Medical Imaging pages Dordrecht Kluwer Academic Publishers
F Betting J Feldmar N Ayache and F Devernay A new framework for fusing
stereo images with volumetric medical images Computer Vision Virtual Reality
and Robotics in Medicine pages
G Borgefors Hierarchical chamfer matching A parametric edge matching algo
rithm IEEE Transactions on Pattern Analysis and Machine Intel ligence
M Bro Nielsen Medical Image Registration and Surgery Simulation PhD thesis
Department of Mathematical Mo delling Technical University of Denmark July
BIBLIOGRAPHY BIBLIOGRAPHY
L G Brown A survey of image registration techniques ACM Computing Surveys
Decemb er
P Cachier X Pennec and N Ayache Fast non rigid matching by gradient de
scent Study and improvements of the demons algorithm Technical Rep ort
INRIA June
M Chen D Deformable Registration Using a Statistical Atlas with Applications
in Medicine PhD thesis The Rob otics Institute Carnegie Mellon University
Octob er
Y Chen and G Medioni Ob ject mo delling by registration of multiple range
images Image and Vision Computing April
J C Cheng and H S Don A graph matching approach to d p oint corresp on
dences International Journal of Pattern Recognition and Arti cial Intel ligence
G E Christensen Deformable Shape Models for Anatomy PhD thesis Washington
University August
G E Christensen Mimd vs simd parallel pro cessing A case study in d medical
image registration Paral lel Computing
D L Collins D Model based segmentation of individual brain structures from mag
netic resonance imaging data PhD thesis Department of Biomedical Engineering
McGill University Octob er
D L Collins A P Zijdenb os V Kollokian J G Sled N J Kabani C J Holmes
and A C Evans Design and construction of a realistic digital brain phantom
IEEE Transactions on Medical Imaging June
J Declerck J Feldmar M L Goris and F Betting Automatic registration and
alignment on a template of cardiac stress and rest sp ect images Technical Rep ort
INRIA Sophia Antip olis
H Delingette G Subsol S Cotin and J Pignon A craniofacial surgery simulation
testb ed Technical Rep ort INRIA Sophia Antip olis
G J Ettinger Hierarchical Three Dimensional Medical Image Registration PhD
thesis Massachusetts Institute of Technology June
J Feldmar and N Ayache Rigid a ne and lo cally a ne registration of free form
surfaces Technical Rep ort INRIA Sophia Antip olis
BIBLIOGRAPHY BIBLIOGRAPHY
W R Fright and A D Linney Registration of d head surfaces using multiple
landmarks IEEE Transactions on Medical Imaging Septemb er
C Gramkow Registration of d and d medical images Master s thesis Depart
ment of Mathematical Mo delling University of Denmark
A Gueziec Large deformable splines crest lines and matching In International
Conference on Computer Vision pages Los Alamitos California
IEEE Computer So ciety Press
A Gueziec and N Ayache Smo othing and matching of d space curves In
Proceedings SPIE on Visualization in Biomedical Imaging volume pages
A Gueziec X Pennec and N Ayache Medical image registration using geo
metric hashing IEEE Computational Science and Engineering Octo
b er Decemb er
A Guimond A Ro che N Ayache and J Meunier Multimo dal brain warping
using the demons algorihtm and adaptative intensity corrections Technical Rep ort
INRIA Sophia Antip olis
S Gurke A new approach for geometrically deformable mo dels by integration of
knowledge In B Giro d H Niemann and H P Seidel editors Vision Modeling
and Visualization pages Erlangen Germany Novemb er in x
I H Habb oush K D Mitchell R V Mulkern P D Barnes and S T Treves Regis
tration and alignment of three dimensional images An interative visual approach
Radiology
N Hata T Dohi S War eld W Wells R Kikinis and F A Jolesz Multimo dality
deformable registration of pre and intraop erative images for mri guided brain
surgery In Proceedings of the First International Conference on Medical Image
Computing and Computer Assisted Intervention pages
D L Hauser and D L Taylor Optimal alignment of geometric mo dels for com
parison In First Conference on Visualization in Biomedical Computing pages
Atlanta Georgia May IEEE Computer So ciety Press
M V Herk and H M Ko oy Automatic three dimensional correlation of ct ct ct
mri and ct sp ect using chamfer matching Med Phys
BIBLIOGRAPHY BIBLIOGRAPHY
D L G Hill D J Hawkes J E Crossman M J Gleeson T C S Cox E C M L
Bracey A J Strong and P Graves Registration of mr and ct images for skull
base surgery using p ointlike anatomical features British Journal of Radiology
B K P Horn Closed form solution of absolute orientation using unit quaternions
Journal Opt Soc Am A April
B K P Horn H M Hilden and S Negahdarip our Closed form solution of absolute
orientation using orthonormal matrices Journal Opt Soc Am A
July
H Jiang K S Holton and R A Robb A new approach to d registration of mul
timo dal medical images by chamfer matching In Proceedings of SPIE Biomedical
Image Processing and Three dimensional Microscopy volume pages
K Kanatani Analysis of d rotation tting IEEE Transactions on Pattern
Analysis and Machine Intel ligence May
T Kapur Model Based Three Dimensional Medical Image Segmentation PhD
thesis Arti cal Intelligence Lab oratory Massachusetts Institute of Technology
May
Z Z Karu Fast Subpixel Registration of D Images PhD thesis Massachusetts
Institute of Technology Septemb er
S Lavallee R Szeliski and L Brunie Matching d smo oth surfaces with their
d pro jections using d distance maps In SPIE Geometric Methods in Computer
Vision volume pages San Diego California July
T Lehmann A Sovakar W Schmitt and R Repges A comparison of similarity
measures for digital subtraction radiography Comput Biol Med
M E Leventon A registration tracking and visualization system for image guided
surgery Master s thesis Massachusetts Institute of Technolog May
F Maes A Collignon D Vandermeulen G Marchal and P Suetens Multimo dal
image registration by maximization of mutual informaiton In Mathematical Meth
ods in Biomedical Image Analysis pages Los Alamitos California
IEEE Computer So ciety Press
J B A Maintz and M A Viergever A survey of medical image registration Medical
Image Analysis
BIBLIOGRAPHY BIBLIOGRAPHY
C R Maurer Registration of Multimodal Three Dimensional Medical Images Using
Points and Surfaces PhD thesis Vanderbilt University Decemb er
C R Maurer and M J Fitzpatrick a review of medical image registration In
R J Maciunas editor Interative Image Guided Neurosurgery pages Park
Ridge
T McInerney and D Terzop oulos Deformable mo dels in medical image analysis
a survey Medical Image Analysis
C R Meyer J L Bo es B Kim P H Bland K R Zasadny P Kison K Koral
K A Frey and R L Wahl Demonstration of accuracy and clinical versatility of
mutual information for automatic multimo dality image fusion using a ne and thin
plate spline warp ed geometric deformations Medical Image Analysis
C R Meyer G S Leichtman J A Brunb erg R L Wahl and L E Quint Simul
taneous usage of homologous p oints lines and planes for optimal d linear reg
istration of multimo dality imaging data IEEE Transactions on Medical Imaging
March
O Monga and S Benayoun Using partial derivatives of d images to extract
typical surface features Technical rep ort INRIA Sophia Antip olis
S B Narayan Registration and Analysis of Magnetic Resonance Images of the
Brain PhD thesis University of Cincinnati
C Nikou F Heitz J P Armspach and I J Namer Single and multimo dal sub
voxel registration of dissimilar medical images using robust similarity measures In
SPIE Conference on Image Processing volume pages San Diego
California February
S Ourselin A Ro che G Subsol X Pennec and C Sattonnet Automatic align
ment of histological sections for d reconstruction and analysis Technical Rep ort
INRIA Sophia Antip olis
J K Ousterhout Tcl and the Tk Toolkit Addison Wesley Publishing Company
Sydney
C A Pelizarri G T Y Chen D R Sp elbring R R Weichselbaum and C T Chen
Accurate three dimensional registration of ct p et and or mr images of the brain
Journal of Computer Assisted Tomography
X Pennec and J P Thirion Validation of d registration metho ds based on p oints
and frames Technical Rep ort INRIA Sophia Antip olis
BIBLIOGRAPHY BIBLIOGRAPHY
X Pennec and J P Thirion A framework for uncertainty and validation of d reg
istration metho ds based on p oints and frames International Journal of Computer
Vision
T Pun G Gerig and O Ratib Image analysis and computer vision in medicine
Computerized Medical Imaging and Graphics
P Viola and W M Wells Alignment by maximization of mutual information In
ternational Journal of Computer Vision
D Rey G Subsol H Delingette and N Ayache Automatic detection and seg
mentation of evolving pro cesses in d medical images Application to multiple
sclerosis Technical Rep ort INRIA Sophia Antip olis
R A Robb Three Dimensional Biomedical Imaging Principles and Practice
John Wiley and Sons Brisbane
A Ro che G Malandain and N Ayache Unifying maximum likeliho o d approaches
in medical image registration Technical Rep ort INRIA Sophia Antip olis
A Ro che G Malandain X Pennec and N Ayache Multimo dal image regis
tration by maximization of the correlation ratio Technical Rep ort INRIA
Sophia Antip olis
J G Rosenman E P Miller G Tracton and T J Cullip Image registration An
essential part of radiation therapy treatment planning International Journal of
Radiation Oncology Biol Phys
C Roux and J L Coatrieux editors Contemporary Perspectives in Three
Dimensional Biomedical Imaging volume of Studies in Health Technology and
Informatics IOS Press Netherlands
D A Simon Fast and Accurate Shape Based Registration PhD thesis Carnegie
Mellon University Decemb er
D A Simon M Heb ert and T Kanade Techniques for fast and accurate intra
surgical registration The Journal of Image Guided Surgery April
C Studholme D L G Hill and D J Hawkes Automated d registration of mr
and ct images of the head Medical Image Analysis
G Subsol J P Thirion and N Ayache First steps towards automatic building of
antomical atlases Technical Rep ort INRIA Sophia Antip olis
BIBLIOGRAPHY BIBLIOGRAPHY
G Subsol J P Thirion and N Ayache A general scheme for automatically build
ing d morpholometric anatomical atlases application to a skull atlas Technical
Rep ort INRIA Sophia Antip olis
R H Taylor S Lavallee G C Burdea and R Mosges editors Computer
Integrated Surgery Technology and Clinical Applications MIT Press Cambridge
P Thevenaz and M Unser Spline pyramids for inter mo dal image registration
using mutual information In SPIE Conference on Wavelet Applications in Signal
and Image Processing V volume pages
J P Thirion Extremal p oints de nition and application to d image registra
tion In Computer Vision and Pattern Recognition pages Los Alamitos
California IEEE Computer So ciety Press
J P Thirion Fast non rigid matching of d medical images Technical Rep ort
INRIA Sophia Antip olis
J P Thirion The extremal mesh and the understanding of d surfaces Interna
tional Journal of Computer Vision
J P Thirion New feature p oints based on geometric invariants for d image reg
istration International Journal of Computer Vision
J P Thirion and S Benayoun Image surface extremal p oints new feature p oints
for image registration Technical Rep ort INRIA Sophia Antip olis
J P Thirion and G Calmon Deformation analysis to detect and quantify active
lesions in d medical image sequences Technical Rep ort INRIA Sophia
Antip olis
J P Thirion and A Gourdon The d marching lines algorithm and its application
to crest lines extraction Technical Rep ort INRIA Sophia Antip olis
J P Thirion and A Gourdon The marching lines algorithm new results and
pro ofs Technical Rep ort INRIA Sophia Antip olis
J P Thirion A Gourdon O Monga A Gueziec and N Ayache Fully automatic
registration of d cat scan images using crest lines In Proceedings of the Annual
International Conference IEEE Engineering in Medicine Biol Soc volume
pages
J P Thirion S Prima and G Subsol Statistical analysis of dissymmetry in
volumetric medical images Technical Rep ort INRIA Sophia Antip olis
BIBLIOGRAPHY BIBLIOGRAPHY
P Thompson and A W Toga Handbook of Medical Image Processing chapter
Elastic Image Registration and Pathology Detection Academic Press
A W Toga and P M Thompson The role of image registration in brain mapping
Image and Vision Computing Journal
P A van den Elsen J B A Maintz E J D Pol and M A Viergever Automatic
registration of ct and mr brain images using correlation of geometrical features
IEEE Transactions on Medical Images
P A van den Elsen E J D Pol and M A Viergever Medical image matching a
review with classi cation IEEE Engineering in Medicine and Biology
March
P A Viola Alignment by Maximization of Mutual Information PhD thesis Ar
ti cal Intelligence Lab oratory Massachusetts Institute of Technolog June
M W Walker L Shao and R A Volz Estimating d lo cation parameters using
dual numb er quaternions CVGIP Image Understanding
W M Wells P Viola H Atsumi S Naka jima and R Kikinis Multi mo dal vol
ume registration by maximization of mutual information Medical Image Analysis
J West J M Fitzpatrick M Y Wang B M Dawant C R Maurer and et al
Comparison and evaluation of retrosp ective intermo dality image registration tech
niques In Medical Imaging Newp ort Beach California February
J A Williams and M Bennamoun Multiple view d registration using statistical
error mo dels In B Giro d H Niemann and H P Seidel editors Vision Modeling
and Visualization Erlangen Germany Novemb er in x
S T C Wong R C Knowlton R A Hawkins and K D Laxer Multimo dal image
fusion for noninvasive epilepsy surgery planning IEEE Computer Graphics and
Applications
R P Wo o ds J C Mazziotta and S R Cherry Mri p et registration with automated
algorithm Journal of Computer Assisted Tomography July August