Upload
temima
View
41
Download
0
Embed Size (px)
DESCRIPTION
Optimizer based on particle swarm optimization and LBG (PSO-LBG) — application in vector quantization. Liao Huilian SZU TI-DSPs LAB Aug 27, 2007. School of Software Engineering, Shenzhen University. Outline. Vector quantization (VQ) LBG Particle swarm optimization (PSO) - PowerPoint PPT Presentation
Citation preview
Faculty of Information Engineering, Shenzhen UniversityFaculty of Information Engineering, Shenzhen University
Liao HuilianLiao HuilianSZU TI-DSPs LABSZU TI-DSPs LAB
Aug 27, 2007Aug 27, 2007
Optimizer based on particle swarm Optimizer based on particle swarm optimization and LBG (PSO-LBG)optimization and LBG (PSO-LBG)
—— application in vector quantizationapplication in vector quantization
School of Software Engineering, Shenzhen University
Faculty of Information Engineering, Shenzhen UniversityFaculty of Information Engineering, Shenzhen University
OutlineOutlineVector quantization (VQ)Vector quantization (VQ) LBGLBG
Particle swarm optimization (PSO)Particle swarm optimization (PSO)Optimizer based on PSO and LBG (PSO-Optimizer based on PSO and LBG (PSO-LBG)LBG) PSO-LBGPSO-LBG 2-dimensional simulation2-dimensional simulation Performance comparisonPerformance comparison
ConclusionConclusionAcknowledgementAcknowledgement
School of Software Engineering, Shenzhen University
Faculty of Information Engineering, Shenzhen UniversityFaculty of Information Engineering, Shenzhen University
OutlineOutlineVector quantization (VQ)Vector quantization (VQ) LBGLBG
Particle swarm optimization (PSO)Particle swarm optimization (PSO)Optimizer based on PSO and LBG (PSO-Optimizer based on PSO and LBG (PSO-LBG)LBG) PSO-LBGPSO-LBG 2-dimensional simulation2-dimensional simulation Performance comparisonPerformance comparison
ConclusionConclusionAcknowledgementAcknowledgement
School of Software Engineering, Shenzhen University
Faculty of Information Engineering, Shenzhen UniversityFaculty of Information Engineering, Shenzhen University
Vector quantization (VQ)Vector quantization (VQ)
School of Software Engineering, Shenzhen University
Faculty of Information Engineering, Shenzhen UniversityFaculty of Information Engineering, Shenzhen University
LBGLBG
LBG, a well-known method of VQ, was proposed LBG, a well-known method of VQ, was proposed by Linde, Buzo and Gray in 1980 by Linde, Buzo and Gray in 1980
Apply two optimality criteria iteratively:Apply two optimality criteria iteratively: Nearest neighbour criterion during assigning training vNearest neighbour criterion during assigning training v
ectorsectors Centroid criterion during updating codewords (code vCentroid criterion during updating codewords (code v
ectors)ectors)
Drawbacks: Drawbacks: Local optimization Local optimization Sensitive to the selection of initial codebookSensitive to the selection of initial codebook
School of Software Engineering, Shenzhen University
Faculty of Information Engineering, Shenzhen UniversityFaculty of Information Engineering, Shenzhen University
LBGLBG
School of Software Engineering, Shenzhen University
Faculty of Information Engineering, Shenzhen UniversityFaculty of Information Engineering, Shenzhen University
LBGLBG
School of Software Engineering, Shenzhen University
Faculty of Information Engineering, Shenzhen UniversityFaculty of Information Engineering, Shenzhen University
OutlineOutlineVector quantization (VQ)Vector quantization (VQ) LBGLBG
Particle swarm optimization (PSO)Particle swarm optimization (PSO)Optimizer based on PSO and LBG (PSO-Optimizer based on PSO and LBG (PSO-LBG)LBG) PSO-LBGPSO-LBG 2-dimensional simulation2-dimensional simulation Performance comparisonPerformance comparison
ConclusionConclusionAcknowledgementAcknowledgement
School of Software Engineering, Shenzhen University
Faculty of Information Engineering, Shenzhen UniversityFaculty of Information Engineering, Shenzhen University
Particle swarm optimizationParticle swarm optimization
PSO was proposed by Eberhart and Kennedy in PSO was proposed by Eberhart and Kennedy in 19951995
Advantages:Advantages: Simplicity of implementationSimplicity of implementation Few parameters Few parameters High convergence rate High convergence rate
Population based optimizationPopulation based optimization Remember the best location of itself (Pbest)Remember the best location of itself (Pbest) Remember the best experience in the swarm (Gbest)Remember the best experience in the swarm (Gbest)
School of Software Engineering, Shenzhen University
Faculty of Information Engineering, Shenzhen UniversityFaculty of Information Engineering, Shenzhen University
Particle swarm optimizationParticle swarm optimization
School of Software Engineering, Shenzhen University
Faculty of Information Engineering, Shenzhen UniversityFaculty of Information Engineering, Shenzhen University
OutlineOutlineVector quantization (VQ)Vector quantization (VQ) LBGLBG
Particle swarm optimization (PSO)Particle swarm optimization (PSO)Optimizer based on PSO and LBG (PSO-Optimizer based on PSO and LBG (PSO-LBG)LBG) PSO-LBGPSO-LBG 2-dimensional simulation2-dimensional simulation Performance comparisonPerformance comparison
ConclusionConclusionAcknowledgementAcknowledgement
School of Software Engineering, Shenzhen University
Faculty of Information Engineering, Shenzhen UniversityFaculty of Information Engineering, Shenzhen University
PSO-LBGPSO-LBG
Based on conventional PSO and LBG Based on conventional PSO and LBG algorithmsalgorithms
PSO-LBGPSO-LBG Structure of particleStructure of particle Particle-pair model Particle-pair model Updating processUpdating process
Apply in Vector Quantization (VQ)Apply in Vector Quantization (VQ)
School of Software Engineering, Shenzhen University
Faculty of Information Engineering, Shenzhen UniversityFaculty of Information Engineering, Shenzhen University
Structure of particleStructure of particle
School of Software Engineering, Shenzhen University
Faculty of Information Engineering, Shenzhen UniversityFaculty of Information Engineering, Shenzhen University
Updating model Updating model
School of Software Engineering, Shenzhen University
Faculty of Information Engineering, Shenzhen UniversityFaculty of Information Engineering, Shenzhen University
Updating processUpdating process
PSO-LBG performs three steps at each itePSO-LBG performs three steps at each iteration:ration: Step1: Basic PSO operationsStep1: Basic PSO operations Step2: Classical vector quantizer, i.e. LBG algStep2: Classical vector quantizer, i.e. LBG alg
orithmorithm Step3: Deal with codewords “flying” over the bStep3: Deal with codewords “flying” over the b
oundary of training vector spaceoundary of training vector space
School of Software Engineering, Shenzhen University
Faculty of Information Engineering, Shenzhen UniversityFaculty of Information Engineering, Shenzhen University
Step1Step1 -- Basic PSO operationsBasic PSO operations
Difference between PSO-LBG and PSODifference between PSO-LBG and PSO Velocity updating: (additive inertia weight )Velocity updating: (additive inertia weight )
The parametersThe parameters , and are much smaller than general PSO-, and are much smaller than general PSO-based algorithmsbased algorithms
Apply a particle-pair instead of a large number of particlesApply a particle-pair instead of a large number of particles
1c 2c
School of Software Engineering, Shenzhen University
Faculty of Information Engineering, Shenzhen UniversityFaculty of Information Engineering, Shenzhen University
Why small parameters?Why small parameters?One point larger parametersOne point larger parameters
The solution of PSO-LBG represents The solution of PSO-LBG represents NN points in the training vector spacepoints in the training vector space
School of Software Engineering, Shenzhen University
Faculty of Information Engineering, Shenzhen UniversityFaculty of Information Engineering, Shenzhen University
Why just two particles?Why just two particles?
Three particles consisting of two codewordThree particles consisting of two codewords: s: PP11={={yy11, , yy22}; }; PP22={={yy22, , yy11} and } and PP33={={yy33, , yy44}. }. PP33
has a poorer performance has a poorer performance
During the following iterations, particle During the following iterations, particle PP11 aa
ndnd PP22 are comparative are comparative
The fly direction of particle The fly direction of particle PP33 is uncertain is uncertain
School of Software Engineering, Shenzhen University
Faculty of Information Engineering, Shenzhen UniversityFaculty of Information Engineering, Shenzhen University
Stable convergence Unstable convergenceStable convergence Unstable convergence
Why just two particles?Why just two particles?
School of Software Engineering, Shenzhen University
Faculty of Information Engineering, Shenzhen UniversityFaculty of Information Engineering, Shenzhen University
Updating steps 2 & 3Updating steps 2 & 3
Apply LBG with only 3 iterations to avoid cApply LBG with only 3 iterations to avoid converging earlyonverging early
Deal with the codewords “flying” over the bDeal with the codewords “flying” over the boundary of search space: Replace this kinoundary of search space: Replace this kind of codeword with the training vector that d of codeword with the training vector that has higher distortionhas higher distortion
School of Software Engineering, Shenzhen University
Faculty of Information Engineering, Shenzhen UniversityFaculty of Information Engineering, Shenzhen University
Demonstration in Demonstration in 2-dimensional space2-dimensional space
Three objectives Three objectives PSO-LBG intends to achieve: intends to achieve: Disperse codewords Disperse codewords Move towards global optimum codebook Move towards global optimum codebook Codewords are settled reasonably both in high densitCodewords are settled reasonably both in high densit
y regions and low density areas of training vectors spy regions and low density areas of training vectors space ace
0 50 100 150 200 2500
50
100
150
200
250
0 50 100 150 200 2500
50
100
150
200
250
0 50 100 150 200 2500
50
100
150
200
250
School of Software Engineering, Shenzhen University
Faculty of Information Engineering, Shenzhen UniversityFaculty of Information Engineering, Shenzhen University
Demonstration in Demonstration in 2-dimensional space2-dimensional space
Initial codebook =561.15
LBG =61.26
PSO-LBG =46.85
0 50 100 150 200 2500
50
100
150
200
250
School of Software Engineering, Shenzhen University
0 50 100 150 200 2500
50
100
150
200
250
0 50 100 150 200 2500
50
100
150
200
250
Faculty of Information Engineering, Shenzhen UniversityFaculty of Information Engineering, Shenzhen University
Performance comparisonPerformance comparison
Performance is evaluated by Performance is evaluated by and PSN and PSNRR : : Mean square error between the training vMean square error between the training v
ectors and corresponding nearest codeworectors and corresponding nearest codewordsds
PSNR: Peak signal to noise ratioPSNR: Peak signal to noise ratio
2255PSNR 10log (dB)
/D L
School of Software Engineering, Shenzhen University
Faculty of Information Engineering, Shenzhen UniversityFaculty of Information Engineering, Shenzhen University
Performance comparisonPerformance comparison
Comparison is conducted among:Comparison is conducted among: LBGLBG Fuzzy k-means (FKM)Fuzzy k-means (FKM) Fuzzy reinforced learning vector quantization Fuzzy reinforced learning vector quantization
(FRLVQ)(FRLVQ) FRLVQ-FVQ: Apply FRLVQ as the pre-FRLVQ-FVQ: Apply FRLVQ as the pre-
process of FVQ process of FVQ PSO-LBGPSO-LBG
School of Software Engineering, Shenzhen University
Faculty of Information Engineering, Shenzhen UniversityFaculty of Information Engineering, Shenzhen University
Experimental imagesExperimental images
LenaPepper
Cameraman
Kgirl
School of Software Engineering, Shenzhen University
Faculty of Information Engineering, Shenzhen UniversityFaculty of Information Engineering, Shenzhen University
PSNR comparison on LenaPSNR comparison on Lena
School of Software Engineering, Shenzhen University
1 2 3 4 5 6 7 8 9 10
28.8
29
29.2
29.4
29.6
29.8
30
30.2
30.4
30.6
Number of runs
PS
NR
(dB
)PSO-LBG
FRLVQ-FVQ
FRLVQFKM
LBG
Faculty of Information Engineering, Shenzhen UniversityFaculty of Information Engineering, Shenzhen University
Convergence comparison on LenaConvergence comparison on Lena
School of Software Engineering, Shenzhen University
5 10 15 20 25 3026
27
28
29
30
31
32
Number of iterations
PS
NR
(dB
)PSO-LBG
FRLVQ-FVQ
FRLVQFKM
LBG
Faculty of Information Engineering, Shenzhen UniversityFaculty of Information Engineering, Shenzhen University
Computation time on LenaComputation time on Lena
0102030405060708090
100
FKM FRLVQ FRLVQ-FVQ
PSO-LBG
ti me (mi n)
School of Software Engineering, Shenzhen University
Faculty of Information Engineering, Shenzhen UniversityFaculty of Information Engineering, Shenzhen University
Codebook characteristic on LenaCodebook characteristic on Lena
D
0 100 2000
50
100
150
200
250
0 2 4
x 104
0
50
100
150
200
250
0 2000 40000
50
100
150
200
250Codebook Size AV. D D
School of Software Engineering, Shenzhen University
Faculty of Information Engineering, Shenzhen UniversityFaculty of Information Engineering, Shenzhen University
PSNR comparison on pepperPSNR comparison on pepper
School of Software Engineering, Shenzhen University
1 2 3 4 5 6 7 8 9 1029.5
30
30.5
31
31.5
Number of runs
PS
NR
(dB
)PSO-LBG
FRLVQ-FVQ
FRLVQFKM
LBG
Faculty of Information Engineering, Shenzhen UniversityFaculty of Information Engineering, Shenzhen University
PSNR comparison on cameramanPSNR comparison on cameraman
School of Software Engineering, Shenzhen University
1 2 3 4 5 6 7 8 9 10
27
27.5
28
28.5
29
29.5
30
30.5
31
Number of runs
PS
NR
(dB
)PSO-LBG
FRLVQ-FVQ
FRLVQFKM
LBG
Faculty of Information Engineering, Shenzhen UniversityFaculty of Information Engineering, Shenzhen University
PSNR comparison on KgirlPSNR comparison on Kgirl
School of Software Engineering, Shenzhen University
1 2 3 4 5 6 7 8 9 1031
31.5
32
32.5
33
33.5
34
Number of runs
PS
NR
(dB
)PSO-LBGFRLVQ-FVQFRLVQFKMLBG
Faculty of Information Engineering, Shenzhen UniversityFaculty of Information Engineering, Shenzhen University
ConclusionConclusion
Experimental results demonstrate that PSExperimental results demonstrate that PSO-LBG Outperforms existing algorithms in tO-LBG Outperforms existing algorithms in the field of vector quantizationhe field of vector quantization
Future workFuture work Application in gene clusteringApplication in gene clustering
School of Software Engineering, Shenzhen University
Faculty of Information Engineering, Shenzhen UniversityFaculty of Information Engineering, Shenzhen University
AcknowledgementAcknowledgement
My supervisor:My supervisor:
Prof. JiProf. Ji
All of youAll of you
School of Software Engineering, Shenzhen University
Faculty of Information Engineering, Shenzhen UniversityFaculty of Information Engineering, Shenzhen University
Thank you!Thank you!
School of Software Engineering, Shenzhen University