Dictionary-based phase retrieval for space-time super resolution

using lens-free on-chip holographic videoZihao Wang1

Advisor: Oliver Cossairt1Co-authors: Qiqin Dai1 (EyeVerify), Donghun Ryu2, Kuan He1, Roarke Horstmeyer3,

Aggelos Katsaggelos1

Affiliations: 1 Electrical Engineering & Computer Science, Northwestern University, USA

2 Electrical Engineering, California Institute of Technology, USA3 Charite ́ Medical School, Humboldt University of Berlin, Germany

sperm,protozoa,etc.

T. Su et al. Sci. Rep. 3 (2013)

non-invasive large field-of-view high speed localization

lens-freeon-chipvideo

CMOS sensor

LED

Biological sample

Blepharisma

largeField-of-View,compact,cost-effective2.2x2.2umpixelresolution

fromhologramtoobject– phaseretrieval

back propagation

refocusing

GF

GF:GaussianFiltering

Th Dl Er

Th:Thresholding Dl:Dilation Er:Erosion

10th 20th

fromhologramtoobject– phaseretrieval

update amplitude

update support

Z. Wang et al. Digital Holography & 3-D Imaging Tu2A.4 (2017)

Blepharisma

Paramecium

temporalresolutionenhancement

0 20 40 60Temporal resoluion (FPS)

0

1

2

3

4

5

Spat

ial r

esol

utio

n (p

ixel

s)

#106

Goal

subsampledphaseretrieval

z0

Factor of 9 Factor of 16 Factor of 25

Rectilin

ear

Sem

i-ra

ndom

Inte

rpola

tion

(b) Reconstructed phase(a) Full field-of-view holograms

(c1) Factor of 9 (c2) Factor of 16 (c3) Factor of 25

𝟏𝟎𝟎𝝁𝒎

𝟏𝒎𝒎

𝟐𝟎𝝁𝒎

rad

-30 -15 0 15 30

30

0510152025

𝐗 (micron)

-30 -15 0 15 30

30

0510152025

𝐗 (micron)

-30 -15 0 15 30

30

0510152025

𝐗 (micron)

Theory Rectilinear Semi-random Interpolation

0

-3

D. Ryu et al. Biomed. Opt. Exp. 8(3) 1985-1991 (2017)

z0

Bilevel dictionaryHR-LR training pairs

Captured LR

Sparse representation

Captured HR

Recovered HR

LR Dictionary (𝐃")

HR Dictionary (𝐃#)

Simulated LR

Amplitude Phase

lens-freeon-chipdictionary?

J. Yang et al. Computer Vision and Pattern Recognition pp. 2360-2367 (2012)

Captured holograms

Recovered amplitude

Recovered phase

-𝜋 𝜋0 1Amplitude Phase (rad)44𝜇𝑚

datacollection

Peranema Spirostomum Didinium Euplotes Paramecium Blepharisma

Peranema

Spirostomum

Didinium

Euplotes

Paramecium

Blepharisma 2x2 Bilevel - Bicubic2x2 Bicubic

3x3 Bilevel - Bicubic3x3 Bicubic

4x4 Bilevel - Bicubic4x4 Bicubic

results– numericalsubsampling

23.31 23.04 21.20 20.65 20.01 19.92

23.52 24.00

28.62 25.78

21.62 22.28

25.57 24.04

20.02 21.36

21.60 21.93

SPR

Bicubic

Bilevel

-𝜋 𝜋0 1Amplitude Phase (rad)

2x2 3x3 4x4Full resolution

PSNR

PSNR

PSNR

results– numericalsubsampling

2x2 3x3 4x4

13.6 FPS 24.8 FPS 36.9 FPS

results– experimentalvideosrecovery

Full resolution 2×2Sub-sampling

3×3 4×4

0.00 s

3.25 s

results– 3DtrackingofEuglena

summary

• used holographic lens-free on-chip video for monitoring in vivo biological samples;

• used single frame subsampling for temporal resolution enhancement;

• constructed a bilevel dictionary for recovering subsampled holograms;

• performed phase retrieval on recovered holograms;

Thanks for your attention!

Zihao Wangzwinswang@gmail.com