Imaging through extreme scattering in extendeddynamic mediaA. V. KANAEV,1,* A. T. WATNIK,1 D. F. GARDNER,1 C. METZLER,2 K. P. JUDD,1 P. LEBOW,1

K. M. NOVAK,3 AND J. R. LINDLE1

1U.S. Naval Research Laboratory, 4555 Overlook Ave., Washington, DC 20375, USA2Department of Electrical and Computer Engineering, Rice University, Houston, Texas 77005, USA3Tekla Research, 3700 Fettler Park Drive, Dumfries, Virginia 22025, USA*Corresponding author: andrey.kanaev@nrl.navy.mil

Received 5 January 2018; revised 16 May 2018; accepted 18 May 2018; posted 30 May 2018 (Doc. ID 318130); published 21 June 2018

Critical to navigation, situational awareness, and objectidentification is the ability to image through turbid waterand fog. To date, the longest imaging ranges in such envi-ronments rely on active illumination and selection ofballistic photons by means of time gating. Here we showthat the imaging range can be extended by using time-gatedholography in combination with multi-frame processing.Instead of simply summing the intensity of the frames,we use the complex fields retrieved through digital holo-graphic processing and coherently add the frames. We dem-onstrate imaging through extended bodies of turbid waterand fog at one-way attenuation lengths of 13 and 13.6,respectively. Compared to equivalent traditional time-gatedsystems, gated holography and coherent processing require20× less laser illumination power for the same imagingrange.

OCIS codes: (110.0113) Imaging through turbid media; (090.0090)

Holography; (100.0100) Image processing.

https://doi.org/10.1364/OL.43.003088

Sensing through scattering media is one of the fundamentalproblems of modern optics. Recent research has emphasizedimaging and focusing through biological samples and thinscreens, both with very limited spatial extent [1–6]. However,there is long-standing interest in imaging through dynamicmacroscopic-size natural scattering environments such asturbid water and dense fog [7–9]. This problem assumes a chal-lenging sensing configuration, in which the source and sensorare separated from the object by a bulk scattering media.Photons from the source must traverse the media to and fromthe object before detection on the sensor. The difficulty asso-ciated with imaging through any scattering media is usuallyscaled with its thickness measured in mean free paths or attenu-ation lengths (ALs), i.e., distances over which the intensity ofunscattered or ballistic light is reduced by a factor of e.Attenuation of light in natural environments consists of absorp-tion and scattering. Underwater absorption depends on an

electromagnetic wavelength and establishes an opticallytransparent window in the blue-green spectrum. However,scattering effects are dominant for underwater propagationof blue-green light and for fog propagation of visible/IR light.The size distributions of particulate matter in water and waterdroplets in fog are typically larger than visible light wavelengthsleading to Mie scattering, which is characterized by dominantforward and backward scattering lobes. Conceptually, an imag-ing range is limited by the number of detected photons, but lossof contrast due to forward-scattered light cuts the range evenshorter. This scattering pattern is different from Raleighscattering observed in haze, smoke, or aerosols, wherein par-ticles are smaller than wavelengths of visible light.

Recently developed approaches for imaging through scatter-ing using scattered or diffuse light such as the transmission ma-trix or memory effect are applicable only to microscopic andmesoscopic scales [2,3,5,6]. Wavefront shaping techniques re-quire stationary or relatively slowly changing environmentssuch as biological samples and atmospheric or underwaterturbulence, and are often not suited for the considered configu-ration, in which a detector–source pair is separated from anobject by a scattering environment [1,10,11]. Methods utiliz-ing unscattered photons represent ballistic photon detectionbased on temporal, spatial, coherence, or polarization proper-ties [12]. The modern history of imaging through extendedscattering media started with coherence gating, in which theduration of the interfered pulses played the role of the timegate [13]. Later, this approach for imaging through macro-scopic-size scattering media was largely replaced by time gatingenabled by fast cameras, photomultipliers, and avalanche photodiodes [9,14,15]. To date, time-gated or time-resolved sensinghas demonstrated the best results in imaging through macro-scale natural environments, in particular, a pulsed laser coupledwith multi-aperture photomultiplier tube or a silicon singlephoton avalanche diode has attained a record range ofapproximately 9 AL (one-way attenuation in two-way imagingexperiments) [9,15].

To address the long-standing challenge of imaging throughnatural macroscopic-size scattering environments at theextreme limit of only a few unscattered photons, we present

3088 Vol. 43, No. 13 / 1 July 2018 / Optics Letters Letter

0146-9592/18/133088-04 Journal © 2018 Optical Society of America

a novel approach combining electronic time gating, digitalholography, and coherent multi-pulse processing. In thisLetter, we initially detect ballistic and near-forward scatteredphotons using a nanosecond time-gated sensor, isolate ballisticphotons via coherence gating with off-axis holography and, fi-nally, recover the object image employing a coherent processingalgorithm based on alternating minimization. We demonstraterecord underwater and fog imaging ranges of 13 and 13.6 AL,respectively and, for the first time, to the best of our knowledge,outperform traditional time-gated systems in sensing throughextended scattering media.

In our experiment, holograms are generated with a double-pulsed laser system, which is schematically depicted in Fig. 1.The system begins with a double-pulsed 1064 nm yttrium fiberlaser operating at a 5 ns pulse width (NP Photonics). The sep-aration between pulses is tunable, via an electronic control sig-nal allowing for a range of object distances without the need foroptical delay lines. Traditional approaches relying on fixed de-lay lines require careful alignment of delay stages and becomeimpractical for objects at range. Instead, our setup allows us toimage objects at a few meters to a few hundreds of meterswithout hardware changes owing to 30 km coherence lengthof the seed CW laser. The pulses are frequency doubled to ob-tain 532 nm light, resulting in 0.9 mJ/pulse-pair energy andsplit with a Pockels cell and a polarizing beam splitter (BS).The first pulse is steered to the object through the scatteringmedia illuminating a 1.3 cm diameter spot, while the secondpulse serves as the reference. The signal and reference pulses arecombined at a small angle with a BS to complete an off-axisholographic setup. The angle is tuned such that the interferencefringes are sampled at approximately 3× the Nyquist frequency.Before the BS, the reference pulse passes through a tunable neu-tral density (ND) filter, in which the optical density is adjustedfor a particular attenuation range to prevent detector saturation.The holograms from the interfering pulses are recorded on anintensified charged coupled device (ICCD) that provides a 2 nsgating window (Stanford Computer Optics). This experimentalsetup allows for transformation into a traditional time-gatedsystem by blocking the reference arm and adding an imagingf ∕10 lens (50 mm focal length) in front of the camera. Itenables straightforward performance comparison of theproposed time-gated holographic sensor and an equivalenttime-gated system.

The chamber holding scattering media has dimensions of1.22 × 0.2 × 0.2 m and is constructed out of Plexiglas withanti-reflection-coated high-transmission glass entrance/exitwindows. To produce the aqueous turbid state, the chamber

is filled with tap water and a fine dust (Power Technologies)consisting of a mean particle diameter of 3 μm and a nearlyGaussian particle distribution with a 2 μm variance. The dustparticles tend to settle over minute timescales; therefore, foursubmersible circulation pumps are installed to stir the water.The variability of beam attenuation during the time of imagecollection is observed to be 0.04 AL. Alternatively, thechamber can be filled with ultrasonically generated fog (Cyclon,Nutramist) containing average size particles of approximately5 μm. Fog dissipation is observed to occur on second time-scales; therefore, automatic cycling of fog generator is employedto maintain a fog density. Due to the kinetic nature of the me-dia, few frames at a particular attenuation range are availableand, therefore, images and holograms collected at similar at-tenuations are binned together in groups of 20 for processing.The distance from the object to the detector is 7.5 m in thewater and 7.14 m in the fog experiments.

Beam attenuation through the scattering media is measuredcontinuously during image collection using one-way propaga-tion of a separate CW 15 mW, 532 nm laser and a powermeter(Thorlabs PM16-130). To minimize the collection of scatteredlight into the powermeter, its acceptance angle is constrainedwith a spatial filter (Thorlabs KT310) to 2 mrad, which iswithin the range of the acceptance angles of commercial waterbeam transmissometers [16].

We consider a holographic signal model, in which the com-plex-valued field reflected off the object xϵCn is mapped onto2-D Fourier transformed and windowed measurements yϵCn

in spatial frequency space:

y � Fx � w, (1)

where F describes the linear mapping between them andw ∼ CN �0, σ2I�,wϵCm denotes a noise vector. In pupil planeoff-axis holography, the operator F is the composition of free-space field propagation from the pupil plane to the objectplane, a Fourier transform at the hologram plane, and a win-dowing of the object signal. Signal-to-noise (SNR) improve-ment in low-light holography can be achieved by averagingintensities of multiple object images. However, SNR increasesfaster with the number of images if complex fields are used.Our experiments show that when a set of measurements l �1…L for yl is recorded, the measurements capture identicallydistributed values of x up to nuisance amplitudes αl and phasesβl , which are due to the dynamic nature of the environmentand laser phase drifts. Thus, equations serving as a basisfor coherent processing of L measurements can be writtenin vector form:

y � Fα,βx � w, (2)

where the following definitions are used:

y �

2664

y1

..

.

yL

3775, Fα,β �

2664

α1FB1

..

.

αLFBL

3775, w �

2664

w1

..

.

wL

3775: (3)

Bl � diag�βl � and wl ∼ CN �0, σ2I�, wϵCm are independentnoise realizations.

Without assuming any prior information, maximum likeli-hood (ML) can be used to estimate the object’s reflection co-efficients x. Each measurement yl depends on the unknownreflection coefficient x, the nuisance phase Bl , and the nuisanceFig. 1. Experimental setup of the time-gated holographic system.

Letter Vol. 43, No. 13 / 1 July 2018 / Optics Letters 3089

amplitude αl making the reconstruction of x from y a highlynonconvex optimization problem. Our alternating minimiza-tion approach, based loosely on methods proposed inRef. [17], consists of the following three steps repeated severaliterations:

1. Fix x and α1…αl ; determine the ML estimate ofB1…BL.

2. Fix x and B1…BL; determine the ML estimateof α1…αl .

3. Fix α1…αl and B1…BL; determine the ML estimateof x.

In Step 1, nuisance phases Bl are expanded using the first kZernike basis vectors ΨϵRn×k, βl � exp�iΨθl �. Then the MLestimate of the expansion coefficients θlML is given by

θlML � arg minθl

kyl − αlFX exp�iΨθl �k22, (4)

where X � diag�x�. There are a number of methods to solvethis minimization problem. If constant nuisance phases orpistons, (k � 1), are assumed, then Eq. (4) has a closed formsolution:

θlML � −∠�yHl Fx�, (5)where ∠�·� denotes the angle of the argument and �·�H denotesHermitian transposed matrix. The steepest gradient descent isused for k > 1; however, the results suggest that retainingexpansion terms above the piston phases is unnecessary.

For Step 2, given x and Bl , the ML estimate of nuisanceamplitudes αl is

αlML � Re�yHFBl x�xHBH

l FHFBl x

: (6)

In addition, in a final Step 3, given α1…αl and B1…BL, anML estimate of the object’s reflection coefficient x is computed:

xML � �FHα,β�θ�Fα,β�θ��−1FHα,β�θ�y: (7)

Inversion of the operator FHα,β�θ�Fα,β�θ� is nontrivial; therefore,a least square solution is obtained using Matlab’s lsqr func-tion. The computational time for 50 holograms on NvidiaTitan X GPU is 28 s.

In underwater imaging experiments, the object consists of aletter “X ”made of light absorbing material and positioned on ahighly reflective surface (mirror). Both the time-gated imageand hologram collections use 2 ns time gates. An importantacquisition parameter in holography is the ratio of the referenceand signal pulse energies. Ideally, a higher ratio provides greateramplification of the signal reflected from an object, but im-provement is constrained by the shot noise of the referencebeam and dynamic range of the camera. Thus, imaging throughincreasing attenuation requires corresponding reduction of thereference pulse energy (the lower limit for a ICCD camera is 1photon per pixel of the reference pulse) and, as a result, therange of acceptable ratios shrinks. At the maximum attenuationof 13 AL achieved in underwater experiments, the estimatedreference-signal pulse energy ratio on the focal plane arrayreaches 105.

Figure 2(a) displays the image of the object collected in anexperimental configuration corresponding to the time-gatedsystem through the tank filled with clear tap water (1 ALrange). The maximum range of the traditional time-gated sys-tem, at which the object is still visible, is 11.4 AL [Fig. 2(b)].

Figure 2(c) demonstrates that the system fails to resolve anydiscernable image of the object after the range is increasedto 13 AL, despite averaging over 103 frames. For comparison,Figs. 3 and 4 display the images computed from the data re-corded by the time-gated holographic system at analogousranges. Holograms contain approximately 20 × 20 resolutionelements, and time-gated imagery contains approximately 70 ×70 resolution elements. The incoherent intensity averaging ofimages recovered from time-gated holograms leads to the im-aging limit similar to the one found for the time-gated system ascan be expected from their analogous SNR considerations[Fig. 3(c)]. Applying the alternating minimization algorithmin average achieves 6 dB improvement in peak signal-to-noiseratio (PSNR) at 11.4 AL [Fig. 4(b)] and makes 50 hologramssufficient to obtain a discernable image of the object at 13 AL[Fig. 4(c)]. This imaging range corresponds to a round-triplaser energy attenuation of ∼5 · 10−12, resulting in an orderof 102 arriving photons or an order of 10−4 photons per pixelfrom the signal pulse. Comparison to the state-of-the-artunderwater imaging results can be made in terms of numberof photons reaching the detector that needed to form an image.Both system parameters and target characteristics such as typeof reflection (Lambertian or specular) and reflectivity must be

Fig. 2. Imaging through water using the traditional time-gated sys-tem through (a) clear water 1 AL (average of 50 frames), (b) turbidwater 11.4 AL (average of 50 frames), and (c) turbid water 13 AL(average of 1000 frames).

Fig. 3. Images of the object obtained by incoherent averaging of 50time-gated holograms collected through (a) clear water 1 AL, (b) turbidwater 11.4 AL, and (c) turbid water 13 AL.

Fig. 4. Images of the object obtained by coherent processing of 50time-gated holograms collected through (a) clear water 1 AL, (b) turbidwater 11.4 AL, and (c) turbid water 13 AL.

3090 Vol. 43, No. 13 / 1 July 2018 / Optics Letters Letter

taken into account. The range limit of the state-of-the-artunderwater imaging systems that demonstrated previous re-cords of ∼9 AL is estimated to be above 104 photons [10,17].Thus, our experiment demonstrates 100× improvement interms of sensitivity. To emphasize the difficulties of imagingat 13 AL and the benefits of the developed processing,Figs. 5(a) and 5(b) display an image recovered from a singletime-gated hologram and an image recovered with 103 frameprocessing, respectively. Although increasing the number ofholograms in coherent processing leads to a significant PSNRimprovement reflecting contrast enhancement, it saturatesquickly above 102 frames [Fig. 5(c)]. In contrast, other mea-sures of image quality such as structural similarity index(SSIM) and normalized mutual entropy index (NMI) empha-sizing distortions and texture continue to improve with anincreasing number of holograms.

In imaging through fog experiments, the object is the centralpart of the glass plate with dark letters “THORLABS”(Thorlabs R1L3S5P) positioned in front of highly reflectivesurface (mirror). Figures 6(a)–6(c) display the images of theobject obtained at 12.0–12.1 AL ranges using the three pre-vious approaches. As in the underwater experiments, the reso-lution of the fog holograms (30 × 30 resolution elements) islower than the resolution of the time-gated imagery. Imagingthrough attenuation above 12.1 AL exceeds performance limitsof time-gated system augmented with frame averaging[Fig. 6(d)]. Incoherent averaging of time-gated holograms failsto resolve the object, while the alternating optimizationalgorithm reveals the object at ∼13.6 AL range [Figs. 6(e)and 6(f )]. The differences in absolute values of the attainableranges in two environments can be attributed to slightdifferences in physical properties of water and fog such asvolume scattering function and scattering albedo.

In conclusion, time-gated holographic imaging through ex-tended bodies of two dynamic natural scattering environments,

fog and water, is demonstrated. Generally, imaging rangesdepend on particular characteristics of the sensing system suchas laser pulse energy, illumination area, and light gatheringcapability of the aperture, as well as environmental conditionssuch as scatterer properties and object reflectivity. Consideringthese factors, we demonstrated 102 improvement in sensitivitycompared to the state-of-the-art underwater imaging tech-nique. The experiments in this Letter show that time-gatedholographic data collection in macroscale scattering media,combined with coherent processing, attains ∼1.5 AL longerimaging ranges compared to equivalent time-gated sensing.Alternatively, this advantage translates to 20× improvementof laser pulse power at the same range.

REFERENCES

1. Z. Yaqoob, D. Psaltis, M. S. Feld, and C. Yang, Nat. Photonics 2, 110(2008).

2. S. Popoff, G. Lerosey, M. Fink, A. C. Boccara, and S. Gigan, Nat.Commun. 1, 81 (2010).

3. A. Badon, D. Li, G. Lerosey, A. C. Boccara, M. Fink, and A. Aubry, Sci.Adv. 2, e1600370 (2016).

4. T. Chaigne, O. Katz, A. C. Boccara, M. Fink, E. Bossy, and S. Gigan,Nat. Photonics 8, 58 (2014).

5. A. P. Mosk, A. Lagendijk, G. Lerosey, and M. Fink, Nat. Photonics 6,283 (2012).

6. S. Kang, S. Jeong, W. Choi, H. Ko, T. D. Yang, J. H. Joo, J.-S. Lee,Y.-S. Lim, Q.-H. Park, and W. Choi, Nat. Photonics 9, 253 (2015).

7. K. A. Stetson, J. Opt. Soc. Am. 57, 1060 (1967).8. J. S. Jaffe, IEEE J. Ocean. Eng. 40, 683 (2015).9. F. R. Dalgleish, A. K. Vuorenkoski, and O. Bing, Marine Technol. Soc.

J. 47, 128 (2013).10. I. M. Vellekoop and A. P. Mosk, Opt. Lett. 32, 2309 (2007).11. E. G. van Putten, D. Akbulut, J. Bertolotti, W. L. Vos, A. Lagendijk, and

A. P. Mosk, Phys. Rev. Lett. 106, 193905 (2011).12. C. Dunsby and P. M. W. French, J. Phys. D 36, R207 (2003).13. N. Abramson, Opt. Lett. 3, 121 (1978).14. G. Gariepy, N. Krstajic, R. Henderson, C. Li, R. R. Thomson, G. S.

Buller, B. Heshmat, R. Raskar, J. Leach, and D. Faccio, Nat.Commun. 6, 6021 (2015).

15. A. Maccarone, A. Halimi, A. McCarthy, R. Tobin, S. McLaughlin, Y.Petillot, and G. S. Buller, Conference on Lasers and Electro‐Optics(2017), paper SF2M.2.

16. E. Boss, W. H. Slade, M. Behrenfeld, and G. Dall’Olmo, Opt. Express17, 1535 (2009).

17. S. Samadi, M. Cetin, and M. A. Masnadi-Shirazi, IET Radar SonarNavig. 5, 182 (2011).

Fig. 5. Imaging through 13 AL turbid water using (a) single time-gated hologram, (b) coherent processing of 1000 time-gatedholograms, and (c) image quality metric dependences on numberof holograms in coherent processing.

Fig. 6. Imaging through fog and processing 20 frames at ranges of(a)–(c) 12–12.1 AL, (d) 12.3–12.5 AL, and (e), (f ) 13.5–13.8 AL.

Letter Vol. 43, No. 13 / 1 July 2018 / Optics Letters 3091