65
May, 28, 2004 Topics in Preservation Science Lecture Series 1 Carl Haber LBNL. The Reconstruction of Mechanically Recorded Sound by Image Processing Update on Collaboration with the Library of Congress Carl Haber Lawrence Berkeley National Lab rotational stage + encoder cylinder optical probe fiber controller host PC linear stage + encoder motion control and driver

May, 28, 2004 Topics in Preservation Science Lecture Series 1 Carl Haber LBNL. The Reconstruction of Mechanically Recorded Sound by Image Processing Update

Embed Size (px)

Citation preview

Page 1: May, 28, 2004 Topics in Preservation Science Lecture Series 1 Carl Haber LBNL. The Reconstruction of Mechanically Recorded Sound by Image Processing Update

May, 28, 2004 Topics in Preservation ScienceLecture Series

1 Carl HaberLBNL.

The Reconstruction of Mechanically Recorded Sound by Image Processing

Update on Collaboration with the Library of Congress

Carl Haber

Lawrence Berkeley National Lab

rotational stage + encoder

cylinder

optical probe

fiber

controller

host PC

linear stage + encoder

motion control and driver

Page 2: May, 28, 2004 Topics in Preservation Science Lecture Series 1 Carl Haber LBNL. The Reconstruction of Mechanically Recorded Sound by Image Processing Update

May, 28, 2004 Topics in Preservation ScienceLecture Series

2 Carl HaberLBNL.

CollaboratorsVitaliy Fadeyev, Carl Haber,Zach Radding, and Jim TriplettLawrence Berkeley National Lab

Christian MaulTaicaan Technology, U.K.

John W. McBrideUniversity of Southampton, U.K.

Mitch Golden

Peter AlyeaLarry ApplebaumMark RoosaSam BrylawskiThe Library of Congress

Bill KlingerARSC

George HornFantasy Records, Berkeley

Page 3: May, 28, 2004 Topics in Preservation Science Lecture Series 1 Carl Haber LBNL. The Reconstruction of Mechanically Recorded Sound by Image Processing Update

May, 28, 2004 Topics in Preservation ScienceLecture Series

3 Carl HaberLBNL.

Lawrence Berkeley National Labwww.lbl.gov

• Founded in 1931 by E.O.Lawrence• Oldest of US National Labs• Operated by the University of California

for the US DoE• 4000 Staff, 800 Students, 2000 Guests• 14 Research Divisions including

– Physics, Nuclear Science– Materials, Chemical Science– Life Sciences, Physical Bioscience– Energy and Environment, Earth– Computing

• Major user facilities- – Advanced Light Source– Nat. Center for Electron Microscopy– Nat. Energy Research Super Computer Center

Page 4: May, 28, 2004 Topics in Preservation Science Lecture Series 1 Carl Haber LBNL. The Reconstruction of Mechanically Recorded Sound by Image Processing Update

May, 28, 2004 Topics in Preservation ScienceLecture Series

4 Carl HaberLBNL.

Outline• Introduction

• Summary of method (mostly a repeat)*

• Towards a real 2D machine (I.R.E.N.E.)**

• 3D reconstruction of an Edison cylinder***

• Plans for the 3D research program

• Conclusions* V.Fadeyev & C. Haber, J. Audio Eng. Soc., vol. 51, no.12, pp.1172-1185 (2003 Dec.).

** IRENE Proposal 12-Feb-2004

*** V. Fadeyev et al, LBNL Report-54927

Page 5: May, 28, 2004 Topics in Preservation Science Lecture Series 1 Carl Haber LBNL. The Reconstruction of Mechanically Recorded Sound by Image Processing Update

May, 28, 2004 Topics in Preservation ScienceLecture Series

5 Carl HaberLBNL.

Introduction• We have investigated the problem of optically recovering

mechanical sound recordings without contact to the medium• Address concerns of the preservation, archival, and research

communities:– The reconstruction of delicate or damaged media– Mass digitization of diverse media

• The approach evolved naturally out of methods of optical metrology, pattern recognition, and image processing.

• First shown at the LC in July 2003.• Research is now supported by an LC/DOE agreement.• Message to take away from today’s presentation:

– The techniques yield good reproductions and some improvements.– Measurement, data storage, and computing technologies may be

approaching performance levels required for this application.– Strong development program for the near future in both the mass

digitization and analytic aspects.

Page 6: May, 28, 2004 Topics in Preservation Science Lecture Series 1 Carl Haber LBNL. The Reconstruction of Mechanically Recorded Sound by Image Processing Update

May, 28, 2004 Topics in Preservation ScienceLecture Series

6 Carl HaberLBNL.

Traditional Contact Playback

Bulky stylus riding in a narrow groove => Issues with• tracking• condition of the groove

•debris and contamination• wear

Presence of trained manpower or supervision is de-facto required.

Modulation is lateral for most discmedia, and vertical for Edisoncylinders

Transduction may be electrical oracoustic

Groove width = 160 mLateral modulation

radius at bottom

width

depthcontact depth

groove angle

parameter 78 rpm coarse 33 1/3 ultrafine width 0.006 - 0.008 0.001 depth ~0.0029 ~0.0006 contact depth 0.0008 0.0004 radius 0.0015-0.0023 0.00015 angle 82 - 98 87 - 92 units are inches or degrees

stylus

Page 7: May, 28, 2004 Topics in Preservation Science Lecture Series 1 Carl Haber LBNL. The Reconstruction of Mechanically Recorded Sound by Image Processing Update

May, 28, 2004 Topics in Preservation ScienceLecture Series

7 Carl HaberLBNL.

A Non-contact Method• Using digital optical techniques, the pattern of

undulations in a surface can be imaged.• Cover surface with sequential views or grid of points.• Views can be stitched together: surface map• The images can be processed to remove defects and

analyzed to model the stylus motion.• The stylus motion model can be sampled at a

standard frequency and converted to digital sound format.

• Real time playback is not required de-facto, method is aimed at reconstruction and digitization.

Page 8: May, 28, 2004 Topics in Preservation Science Lecture Series 1 Carl Haber LBNL. The Reconstruction of Mechanically Recorded Sound by Image Processing Update

May, 28, 2004 Topics in Preservation ScienceLecture Series

8 Carl HaberLBNL.

Parameter 78 rpm, 10 inch Cylinder

Cut Lateral Vertical

Revolutions per minute 78.26 80-160

Max/Min radii inches (mm) 4.75/1.875 (120.65/47.63)

2– 5 fixed (50.8-127)

Area containing audio data (mm2 ) 38600 16200 (2”)

Total length of groove 152 meters 64-128 meters

Groove width at top inches (m) 0.006 (160) variable

Lines/inch (mm) Gd 96-136 (5.35-3.78) 100-200 (3.94-7.87)

Line spacing microns 175 – 250 125 – 250

Ref level peak velocity@1KHz 7 cm/sec (11 m) NA

Max groove amplitude (microns) 100 - 125 ~10

Groove depth 80 m “~20 m”

Noise level below reference, S/N 17-37 dB ?

Dynamic range 30-50 dB ?

Groove max ampl@noise level 1.6 - 0.16 m ?

Page 9: May, 28, 2004 Topics in Preservation Science Lecture Series 1 Carl Haber LBNL. The Reconstruction of Mechanically Recorded Sound by Image Processing Update

May, 28, 2004 Topics in Preservation ScienceLecture Series

9 Carl HaberLBNL.

c

de

a

b

Page 10: May, 28, 2004 Topics in Preservation Science Lecture Series 1 Carl Haber LBNL. The Reconstruction of Mechanically Recorded Sound by Image Processing Update

May, 28, 2004 Topics in Preservation ScienceLecture Series

10 Carl HaberLBNL.

Imaging Methods: Electronic Camera

• 2D method, CCD or CMOS image sensor: frame or line format• Cameras contains 1 x 4000, or 768 x 494 pixels, or up to few Mega-Pixels• 1 pixel = ~ 1 micron on the disc surface• Magnification and pixel size yield sufficient resolution for audio data

measurement due to pixel interpolation• Frames acquired at 30-100 fps, lines at 10-20K lps

surface

scratch

dust

groovebottom

160 m

Coaxially illuminated grooveon a shellac 78 rpm disc

Page 11: May, 28, 2004 Topics in Preservation Science Lecture Series 1 Carl Haber LBNL. The Reconstruction of Mechanically Recorded Sound by Image Processing Update

May, 28, 2004 Topics in Preservation ScienceLecture Series

11 Carl HaberLBNL.

Imaging Methods: Confocal Scanning Probe3D methodAcquire measurementsover a grid of points atup to 4 KHzGrid spacing may be optimized in design of the scan.

Surface of an Edison cylinder

Page 12: May, 28, 2004 Topics in Preservation Science Lecture Series 1 Carl Haber LBNL. The Reconstruction of Mechanically Recorded Sound by Image Processing Update

May, 28, 2004 Topics in Preservation ScienceLecture Series

12 Carl HaberLBNL.

Imaging Methods: White Light Interferometry

•3D method, requires depth scan.•Acquire frames at 1-20 seconds/frame, depending upon depth and slope•Frame size ~0.5 – 2.5 mm2

•Stitch together adjacent frames

Scratch in a 78 rpm shellac discWax cylinder surface

Page 13: May, 28, 2004 Topics in Preservation Science Lecture Series 1 Carl Haber LBNL. The Reconstruction of Mechanically Recorded Sound by Image Processing Update

May, 28, 2004 Topics in Preservation ScienceLecture Series

13 Carl HaberLBNL.

Issue of Aliasing• Sampling theorem1. Sample at 2*f where f is highest frequency of interest

2. Apply low pass filter above f to prevent aliased components appearing in data unless noise above f can be neglected.

• In optical approach sampling is done by pixelization of image.1. High sampling frequency

2. Use of pixel size to achieve effective low pass filtering

Page 14: May, 28, 2004 Topics in Preservation Science Lecture Series 1 Carl Haber LBNL. The Reconstruction of Mechanically Recorded Sound by Image Processing Update

May, 28, 2004 Topics in Preservation ScienceLecture Series

14 Carl HaberLBNL.

Comparison of SpecificationsParameter 2D Digital Camera W.L.I. Confocal

Acquisition n (x m) pixels n x m pixels point

Transverse resolution 0.5 - 1 m Pixel projection 1-10 m

1.5 - 10 m

Vertical resolution NA 10 nanometers 10 nanometers

Points/measurement 4000 480x540 = 259200 1

Max Time/measurement 50 s 1-10 seconds 250 s

Effective time/point 12.5 ns 4-40 s 250 s

Low pass filtering? Image field twice w. offset

Image field twice w. offset

2 passes w. offset

Depth of field ~ 0 – 75 m Depth is scanned 20 m – mm

Cost of probe only ~$7K >$100K ~$30K

Page 15: May, 28, 2004 Topics in Preservation Science Lecture Series 1 Carl Haber LBNL. The Reconstruction of Mechanically Recorded Sound by Image Processing Update

May, 28, 2004 Topics in Preservation ScienceLecture Series

15 Carl HaberLBNL.

Speed and Data• 2D scans for lateral discs

– Line scan camera: ~5-15 min/scan for 10” 78 rpm disc

– 190 Mbytes / 1 s of raw audio images

• 3D scans for vertical media– Confocal methods depends upon grid.

~12 - 24 hours, 450 M points.– Interferometric frame methods 1 - 5 hours

• 3D for deep groove lateral discs– Confocal with optimized grid ~15 - 60 hours for 10” 78 rpm disc– Interferometric frame method ~10 - 100 hours (groove depth)

Lateral groove

Vertical groove

Key 3D issues are slope and depth

Page 16: May, 28, 2004 Topics in Preservation Science Lecture Series 1 Carl Haber LBNL. The Reconstruction of Mechanically Recorded Sound by Image Processing Update

May, 28, 2004 Topics in Preservation ScienceLecture Series

16 Carl HaberLBNL.

Image Processing• Intensity profile and edge

finding: measure features• Shape recognition• Dilation operation can

remove dust particles• Example is 2D but

generalizes to 3D• Knowledge of groove

geometry provides a powerful constraint for rejecting debris and damage

dilation

Edge finding

Page 17: May, 28, 2004 Topics in Preservation Science Lecture Series 1 Carl Haber LBNL. The Reconstruction of Mechanically Recorded Sound by Image Processing Update

May, 28, 2004 Topics in Preservation ScienceLecture Series

17 Carl HaberLBNL.

Signal Analysis• For recording and playback, signal is proportional to stylus velocity

“electrical”: magnetic induction

“acoustic”: plane wave approximation, air pressure and velocity are proportional and in-phase

• Electrical recordings are (deliberately) mediated by equalization scheme to attenuate low frequencies and boost high frequencies

• Acoustic recordings are (naturally) mediated by the frequency response of horns and diaphragms.

• Potential to improve fidelity with modeling of acoustic component response

• Groove data is in digital form, numerical analysis

• Determine velocity by numerical differentiation Max. Slope

Wavelength

Amplitude

f

vA pp 2

Page 18: May, 28, 2004 Topics in Preservation Science Lecture Series 1 Carl Haber LBNL. The Reconstruction of Mechanically Recorded Sound by Image Processing Update

May, 28, 2004 Topics in Preservation ScienceLecture Series

18 Carl HaberLBNL.

RM

+x direction

hstylus

mechanical linkage

diaphragm of radius R

listener

source

D

media surface

time

xo

exponential horn

conical horn

0

0.2

0.4

0.6

0.8

1

1.2

1 10 100 1000 10000

Frequency

Tra

nsm

issi

on

Co

effi

cien

t

Exponential

Conical

Response of horn and diaphragm atlow frequency can modify response anddeviations from “constant velocity” characteristic.

Page 19: May, 28, 2004 Topics in Preservation Science Lecture Series 1 Carl Haber LBNL. The Reconstruction of Mechanically Recorded Sound by Image Processing Update

May, 28, 2004 Topics in Preservation ScienceLecture Series

19 Carl HaberLBNL.

Response of one real horn

From Maxfield, J.P. and Harrison, H.C., The Bell System Technical Journal, Volume V, No.3, July 1926, pp. 493-524

Page 20: May, 28, 2004 Topics in Preservation Science Lecture Series 1 Carl Haber LBNL. The Reconstruction of Mechanically Recorded Sound by Image Processing Update

May, 28, 2004 Topics in Preservation ScienceLecture Series

20 Carl HaberLBNL.

(Dis)Advantages of Imaging Method

♪ Delicate samples can be played without further damage.♪ Independent of record material and format – wax, metal, shellac, acetates…♪ Effects of damage and debris (noise sources) reduced by image processing.

Scratched regions can be interpolated, re-assemble broken media♪ Discrete noise sources are resolved in the “spatial domain” where they

originate rather than as a random effect in the audio playback.♪ Dynamic effects of damage (skips, ringing) are reduced.♪ Classic distortions (wow, flutter, tracing and tracking errors, pinch effects

etc) are absent or resolved as geometrical corrections♪ Operator intervention during transcription is reduced, mass digitization.

Data intensive, storage and manipulation of large data-setsScanning speed, 3D methods may be quite slow, use for special cases only?Is ultimate resolution sufficient to provide required fidelity?

Page 21: May, 28, 2004 Topics in Preservation Science Lecture Series 1 Carl Haber LBNL. The Reconstruction of Mechanically Recorded Sound by Image Processing Update

May, 28, 2004 Topics in Preservation ScienceLecture Series

21 Carl HaberLBNL.

Relationship to Other Work

• Laser turntables (www.ELPJ.com): reflected laser spot, susceptible to damage, debris, and surface reflectivity.

• Stanke and Paul, (“3D Measurement and modelling in cultural applications”, Inform. Serv. & Use 15 (1995) 289-301): depth sensed from greyscale in 2D image of “galvano”, states the basic approach in general

• S.Cavaglieri et al, Proc of AES 20th International Conference, Budapest, Oct 5-7, 2001: photographic contact prints and scanner to archive groove pattern in 2D – no 3D analog.

• O.Springer (http://www.cs.huji.ac.il/~springer/): use of desk top scanner on vinyl record – lacks resolution

• W.Penn: (First Monday, volume 8, number 5 (May 2003)) real time interferometric cylinder playback system in development.

Page 22: May, 28, 2004 Topics in Preservation Science Lecture Series 1 Carl Haber LBNL. The Reconstruction of Mechanically Recorded Sound by Image Processing Update

May, 28, 2004 Topics in Preservation ScienceLecture Series

22 Carl HaberLBNL.

Test of Concept using 2D Imaging • Precision optical metrology

system “SmartScope” manufactured by Optical Gauging Products.

• Video zoom microscope with electronic camera and precision stage motion in x-y-z.

• Image acquisition with pattern recognition and analysis & reporting software

• Wrote program to scan groove, report, and process data (offline).

• Study of 78 rpm shellac disc ~1950

Page 23: May, 28, 2004 Topics in Preservation Science Lecture Series 1 Carl Haber LBNL. The Reconstruction of Mechanically Recorded Sound by Image Processing Update

May, 28, 2004 Topics in Preservation ScienceLecture Series

23 Carl HaberLBNL.

Offline Data Processing

Reformatting data in one global coordinate system

Removal of big outliers

Filtering by selecting on the distance between interval pair;merging two sides into one.

Matching the adjacent frames

Fit the groove shape R’=R0+C*’+A*sin2(’ + 0)

Numerical Differentiation and resampling

Multiple runs addition; conversion to WAV format

1

2

3

4

5

6

7

Page 24: May, 28, 2004 Topics in Preservation Science Lecture Series 1 Carl Haber LBNL. The Reconstruction of Mechanically Recorded Sound by Image Processing Update

May, 28, 2004 Topics in Preservation ScienceLecture Series

24 Carl HaberLBNL.

Raw measurementof groove bottom edges

Averaged and filteredfor known width R<cut

Frames aligned

Stylus velocity by numerical differentiation4th order polynomial fit of15 points about each sample

Width across groove bottom

Measurement spacing alongtime axis ~ 66 KHz

R

R distribution

Page 25: May, 28, 2004 Topics in Preservation Science Lecture Series 1 Carl Haber LBNL. The Reconstruction of Mechanically Recorded Sound by Image Processing Update

May, 28, 2004 Topics in Preservation ScienceLecture Series

25 Carl HaberLBNL.

Waveform comparison

• Clear reduction in “clicks and pops”• Similarity of fine waveform structure

optical

stylus

CD

19.1 seconds 40 ms

Page 26: May, 28, 2004 Topics in Preservation Science Lecture Series 1 Carl Haber LBNL. The Reconstruction of Mechanically Recorded Sound by Image Processing Update

May, 28, 2004 Topics in Preservation ScienceLecture Series

26 Carl HaberLBNL.

Sound Comparison

Sound from the optical readout.

Sound from the mechanical (stylus) readout.

Sound from the CD of re-mastered tape.

“Goodnight Irene” by H. Ledbetter (Leadbelly) and J.Lomax, performedby The Weavers with Gordon Jenkins and His Orchestra ~1950

optical + commercial noise reduction

Page 27: May, 28, 2004 Topics in Preservation Science Lecture Series 1 Carl Haber LBNL. The Reconstruction of Mechanically Recorded Sound by Image Processing Update

May, 28, 2004 Topics in Preservation ScienceLecture Series

27 Carl HaberLBNL.

Frequency Spectra

•FFT spectra of optical (top), stylus (middle), and CD/tape (bottom)•Audio content in range 100 - 4000 Hz very similar•More high frequency content in stylus and CD versions.•Effects of equalization and differentiation?•Low frequency structure in optical sample (audible).

2 4 14 34 134 1234 5234 20K

Page 28: May, 28, 2004 Topics in Preservation Science Lecture Series 1 Carl Haber LBNL. The Reconstruction of Mechanically Recorded Sound by Image Processing Update

May, 28, 2004 Topics in Preservation ScienceLecture Series

28 Carl HaberLBNL.

Waveform Comparison: 2nd Sample

optical

stylus

CD

Page 29: May, 28, 2004 Topics in Preservation Science Lecture Series 1 Carl Haber LBNL. The Reconstruction of Mechanically Recorded Sound by Image Processing Update

May, 28, 2004 Topics in Preservation ScienceLecture Series

29 Carl HaberLBNL.

Sound Comparison

Sound from the optical readout.

Sound from the mechanical (stylus) readout.

Sound from the CD of re-mastered tape.

“Nobody Knows the Trouble I See ” , traditional, performedby Marion Anderson, Matrix D7-RB-0814-2A, 1947

optical + commercial noise reduction

Page 30: May, 28, 2004 Topics in Preservation Science Lecture Series 1 Carl Haber LBNL. The Reconstruction of Mechanically Recorded Sound by Image Processing Update

May, 28, 2004 Topics in Preservation ScienceLecture Series

30 Carl HaberLBNL.

Directions

In July 2003, at the LC, some possible future directions were discussed:

1. The 2D test was promising, what would it take to make a machine to run near real-time on discs? Could this address mass digitization needs?

2. 3D may be the ultimate goal, could you do a small feasibility study similar to the 2D test?

3. Can you propose a research program to further the 3D technology?

Page 31: May, 28, 2004 Topics in Preservation Science Lecture Series 1 Carl Haber LBNL. The Reconstruction of Mechanically Recorded Sound by Image Processing Update

May, 28, 2004 Topics in Preservation ScienceLecture Series

31 Carl HaberLBNL.

Design of a 2D Machine

• In Fall of 2003 LBNL supported design study for a 2D machine based upon methods shown in the test.

• Specifications vetted with LC staff.• Media sample collection provided by LC.• Basic design developed by LBNL mechanical and

software engineering staff.• Design, cost, and schedule passed internal reviews at

LBNL.• Documents submitted to LC in 2/2004

Page 32: May, 28, 2004 Topics in Preservation Science Lecture Series 1 Carl Haber LBNL. The Reconstruction of Mechanically Recorded Sound by Image Processing Update

May, 28, 2004 Topics in Preservation ScienceLecture Series

32 Carl HaberLBNL.

Basic Features and Goals

• I.R.E.N.E. (Image, Reconstruct, Erase Noise, Etc.)• Follow 2D approach used in test, image groove bottom and/or

top since that is known to work at some level. Quality consistent with test or better(?).

• Emphasize throughput.• Encompass as much variation in media as possible.• Handle broken or partial discs.• Facility to (temporarily) flatten flexible media (Memovox)• User friendly interface.• Commercial off-the-shelf components• Provide a test bed for the mass digitization application.

Page 33: May, 28, 2004 Topics in Preservation Science Lecture Series 1 Carl Haber LBNL. The Reconstruction of Mechanically Recorded Sound by Image Processing Update

May, 28, 2004 Topics in Preservation ScienceLecture Series

33 Carl HaberLBNL.

Media Condition Survey

• Approx 40 discs provided by LC

• 65% GOOD: should reconstruct well

• 25% FAIR: may require additional software developments to reconstruct well

• 10% POOR: reconstruct with excess noise or distortions, or not at all.

• Similar breakdown on a random set of 78 rpm shellac discs.

Page 34: May, 28, 2004 Topics in Preservation Science Lecture Series 1 Carl Haber LBNL. The Reconstruction of Mechanically Recorded Sound by Image Processing Update

May, 28, 2004 Topics in Preservation ScienceLecture Series

34 Carl HaberLBNL.

Media Condition: Shellacgood

fair

fair

poor

Rough groovebottomedge

Multipleedges

Page 35: May, 28, 2004 Topics in Preservation Science Lecture Series 1 Carl Haber LBNL. The Reconstruction of Mechanically Recorded Sound by Image Processing Update

May, 28, 2004 Topics in Preservation ScienceLecture Series

35 Carl HaberLBNL.

Media Condition: Acetate

good

Exudateddeep cleaned

Exudatedsurface cleaned

Page 36: May, 28, 2004 Topics in Preservation Science Lecture Series 1 Carl Haber LBNL. The Reconstruction of Mechanically Recorded Sound by Image Processing Update

May, 28, 2004 Topics in Preservation ScienceLecture Series

36 Carl HaberLBNL.

System Layout

Page 37: May, 28, 2004 Topics in Preservation Science Lecture Series 1 Carl Haber LBNL. The Reconstruction of Mechanically Recorded Sound by Image Processing Update

May, 28, 2004 Topics in Preservation ScienceLecture Series

37 Carl HaberLBNL.

Components

Page 38: May, 28, 2004 Topics in Preservation Science Lecture Series 1 Carl Haber LBNL. The Reconstruction of Mechanically Recorded Sound by Image Processing Update

May, 28, 2004 Topics in Preservation ScienceLecture Series

38 Carl HaberLBNL.

• 4000 pixel, 18 KHz line scan sensor

• Magnify to 1 pixel = 1 m

• 7.6 x 105 lines/outer ring– 390 KHz sampling

• Time/ring = 40 seconds

• 73 mm / 4 mm = 19 rings

• 19 x 40 sec = 13 minutes

• Reduce with variable speed on inner rings: 9 minutes

• Scan time increased if warping is large.

direction of radial scan

direction of azimuthal scan

indicates radial width of 1 sweep

73 mm

sensor field of view projected onto surface 1 x 4000 microns

4.000

bluebird

Based upon 10 inch, 78 rpm geometry

Page 39: May, 28, 2004 Topics in Preservation Science Lecture Series 1 Carl Haber LBNL. The Reconstruction of Mechanically Recorded Sound by Image Processing Update

May, 28, 2004 Topics in Preservation ScienceLecture Series

39 Carl HaberLBNL.

PerformanceFeature Targets Best Effort Upgrade

Coarse grooves, track and measure groove bottom or top X Measure additional features on groove wall X X Use of coaxial lighting X Use of alternative lighting approach X X Reconstruct GOOD based on LOC sample set (Figure 7) X Reconstruct FAIR based on LOC sample set (Figure 8) X X Reconstruct POOR based on LOC sample set (Figure 9) ? Scan time of 6-15 minutes on flat samples X X Processing time of 3-5 minutes X X Scan time on warped samples – as fast as practical X Reconstruct Memovox or other highly warped items X Reconstruct broken records, good quality debris X Reconstruct broken records, lower quality debris X X Exudated lacquers – well cleaned (Figure 11a left) X Exudated lacquers–moderate cleaning (Figure 11a right,b) X Available sampling rates < 200 KHz X Available sampling rates > 200 KHz X Reconstruct stampers X Data and statistical results on quality of scan and sample X

Page 40: May, 28, 2004 Topics in Preservation Science Lecture Series 1 Carl Haber LBNL. The Reconstruction of Mechanically Recorded Sound by Image Processing Update

May, 28, 2004 Topics in Preservation ScienceLecture Series

40 Carl HaberLBNL.

Software Interface

Page 41: May, 28, 2004 Topics in Preservation Science Lecture Series 1 Carl Haber LBNL. The Reconstruction of Mechanically Recorded Sound by Image Processing Update

May, 28, 2004 Topics in Preservation ScienceLecture Series

41 Carl HaberLBNL.

Software Features

• Control framework• Data acquisition functions• Image processing, data reduction• Analysis package (filter, noise reduction), data

quality monitor• Calibration tools• Directory structure• Configuration management• Display tools and diagnostics• Commented source code

Page 42: May, 28, 2004 Topics in Preservation Science Lecture Series 1 Carl Haber LBNL. The Reconstruction of Mechanically Recorded Sound by Image Processing Update

May, 28, 2004 Topics in Preservation ScienceLecture Series

42 Carl HaberLBNL.

Issue of Alternative Lighting• Addition of diffuse ring light adds

new features to image.• Diameter of light source is important• Depends strongly on location of

features which reflect back into optics• Inclusion of these features requires

additional algorithms• Usefulness still needs to be tested

1 ring

2 rings

Page 43: May, 28, 2004 Topics in Preservation Science Lecture Series 1 Carl Haber LBNL. The Reconstruction of Mechanically Recorded Sound by Image Processing Update

May, 28, 2004 Topics in Preservation ScienceLecture Series

43 Carl HaberLBNL.

IRENE Summary

• Design study completed and documented.

• Projected scan time reasonable for a production-like machine.

• Includes a suite of attractive features (GUI, broken, warped discs, data quality plots etc.)

• Would also provide a powerful test-bed for further development.

• Provides a new statistical view of disc media

Page 44: May, 28, 2004 Topics in Preservation Science Lecture Series 1 Carl Haber LBNL. The Reconstruction of Mechanically Recorded Sound by Image Processing Update

May, 28, 2004 Topics in Preservation ScienceLecture Series

44 Carl HaberLBNL.

Test of 3D Scanning

• 2 methods were identified: confocal scanning and white light interferometry

• Collaboration with Taicaan Technology and Dept of Engineering Sciences, U of Southampton, UK: confocal tools in hand

• LBNL, MG designed the experiment, performed analysis and interpretation.

• UK Group configured the hardware and performed the scan.

• Scan speed was not emphasized, wanted to perform a proof-of-concept.

Page 45: May, 28, 2004 Topics in Preservation Science Lecture Series 1 Carl Haber LBNL. The Reconstruction of Mechanically Recorded Sound by Image Processing Update

May, 28, 2004 Topics in Preservation ScienceLecture Series

45 Carl HaberLBNL.

3D Study of an Edison Cylinder

rotational stage + encoder

cylinder

optical probe

fiber

controller

host PC

linear stage + encoder

motion control and driver

Utilize confocal scanning probe at 300 Hz,Along axis at 3 mm/sec (10 m points)Angular increment = 0.01o = 96 KHz

~59 hours for 30 seconds

Page 46: May, 28, 2004 Topics in Preservation Science Lecture Series 1 Carl Haber LBNL. The Reconstruction of Mechanically Recorded Sound by Image Processing Update

May, 28, 2004 Topics in Preservation ScienceLecture Series

46 Carl HaberLBNL.

Frequency – Distance Map

diameter (inches) 2.1875

RPM 80 90 120 144 160

surface velocity (mm/s) 232.740 261.832 349.109 418.931 465.479

frequency wavelength m)

10 23274.0 26183.2 34910.9 41893.1 46547.9

100 2327.4 2618.3 3491.1 4189.3 4654.8

500 465.5 523.7 698.2 837.9 931.0

1000 232.7 261.8 349.1 418.9 465.5

5000 46.5 52.4 69.8 83.8 93.1

10000 23.3 26.2 34.9 41.9 46.5

15000 15.5 17.5 23.3 27.9 31.0

20000 11.6 13.1 17.5 20.9 23.3

44100 5.3 5.9 7.9 9.5 10.6

88200 2.6 3.0 4.0 4.7 5.3

100000 2.3 2.6 3.5 4.2 4.7

Page 47: May, 28, 2004 Topics in Preservation Science Lecture Series 1 Carl Haber LBNL. The Reconstruction of Mechanically Recorded Sound by Image Processing Update

May, 28, 2004 Topics in Preservation ScienceLecture Series

47 Carl HaberLBNL.

Confocal Probe Specifications

Parameter Value

Probe Model STIL CHR150

Depth of field 350 microns

Spot size 7.5 microns

Sampling Frequency 300 Hz

Vertical Resolution 10 nanometers

Vertical Accuracy 100 nanometers

Step size across grooves 10 microns

Step size along grooves (circumferential) 0.01o (= 5 microns on circumference)

Linear scan speed (parallel to cylinder axis) 3 mm/second

Page 48: May, 28, 2004 Topics in Preservation Science Lecture Series 1 Carl Haber LBNL. The Reconstruction of Mechanically Recorded Sound by Image Processing Update

May, 28, 2004 Topics in Preservation ScienceLecture Series

48 Carl HaberLBNL.

Methodology

• Parabolic fits to each groove bottom with fixed curvature to determine depth. Damage and debris are filtered with shape constraint.

• No explicit low pass filter applied but high sampling avoids most of the noise.

• Overall shape is distorted but can constrain with an averaged measurement of ridge heights

• Signal is (approximately) stylus velocity, perform numerical differentiation using Discrete Fourier Transform

Page 49: May, 28, 2004 Topics in Preservation Science Lecture Series 1 Carl Haber LBNL. The Reconstruction of Mechanically Recorded Sound by Image Processing Update

May, 28, 2004 Topics in Preservation ScienceLecture Series

49 Carl HaberLBNL.

Overall Shape of Cylinder

• Perfect cylinder would be flat in this view

• Off center rotation• Out of round – elliptical• Local deformity• Heard as “rumble” and

other low frequency effects

Page 50: May, 28, 2004 Topics in Preservation Science Lecture Series 1 Carl Haber LBNL. The Reconstruction of Mechanically Recorded Sound by Image Processing Update

May, 28, 2004 Topics in Preservation ScienceLecture Series

50 Carl HaberLBNL.

Page 51: May, 28, 2004 Topics in Preservation Science Lecture Series 1 Carl Haber LBNL. The Reconstruction of Mechanically Recorded Sound by Image Processing Update

May, 28, 2004 Topics in Preservation ScienceLecture Series

51 Carl HaberLBNL.

1

0

1

0

1

)()()(1

)()()(

1)](

)()(

)(N

k

nTik

N

k

nTikDF

ekCkMikN

ekCkMnTd

d

NkCF

nTd

dnTA

nTd

d

The filtering factor M is defined as follows.

)23(

2.50

2.5,8.44.0

8.40.1

8.4,201

200

KHzffor

KHzKHzfforf

KHzHzffor

Hzffor

M

1.The cut below 20 Hz removes the low frequency structure in the bottom-only data due to the cylinder shape irregularity. 2.The 400 Hz wide transition to zero at 5.0 KHz was used to avoid the interference-like pattern triggered by jumps in the data. 3.The cut above 5.2 KHz satisfies the Nyquist criteria before re-sampling to a lower digital audio standard.

Numerical Differentiation and Filtering

Page 52: May, 28, 2004 Topics in Preservation Science Lecture Series 1 Carl Haber LBNL. The Reconstruction of Mechanically Recorded Sound by Image Processing Update

May, 28, 2004 Topics in Preservation ScienceLecture Series

52 Carl HaberLBNL.

Waveforms

bottoms

tops

top - bottom

Page 53: May, 28, 2004 Topics in Preservation Science Lecture Series 1 Carl Haber LBNL. The Reconstruction of Mechanically Recorded Sound by Image Processing Update

May, 28, 2004 Topics in Preservation ScienceLecture Series

53 Carl HaberLBNL.

Comparison of Waveforms

optical(t – b)

stylus

Page 54: May, 28, 2004 Topics in Preservation Science Lecture Series 1 Carl Haber LBNL. The Reconstruction of Mechanically Recorded Sound by Image Processing Update

May, 28, 2004 Topics in Preservation ScienceLecture Series

54 Carl HaberLBNL.

Sound Comparison• “Just Before the Battle, Mother”, composed by George F.

Root, performed by Will Oakland and Chorus 1909, 1516 (..76; 4M-297-2) originally as Amberol #297 1909

• with stylus, flat equalization

• Optical version, flat equalization

• + commercial noise reduction + low frequency boost *

*thanks to George Horn, Fantasy Records, Berkeley

Page 55: May, 28, 2004 Topics in Preservation Science Lecture Series 1 Carl Haber LBNL. The Reconstruction of Mechanically Recorded Sound by Image Processing Update

May, 28, 2004 Topics in Preservation ScienceLecture Series

55 Carl HaberLBNL.

Frequency Spectra

3D optical versionwith top-bottom shape correction

Version playedon modern cylindersystem with electricalstylus

Page 56: May, 28, 2004 Topics in Preservation Science Lecture Series 1 Carl Haber LBNL. The Reconstruction of Mechanically Recorded Sound by Image Processing Update

May, 28, 2004 Topics in Preservation ScienceLecture Series

56 Carl HaberLBNL.

Groove bottoms only

Ridge tops only (T)

(B)

T – B (version played already)

Frequency Spectra

Page 57: May, 28, 2004 Topics in Preservation Science Lecture Series 1 Carl Haber LBNL. The Reconstruction of Mechanically Recorded Sound by Image Processing Update

May, 28, 2004 Topics in Preservation ScienceLecture Series

57 Carl HaberLBNL.

3D Research Plan

• Setting up similar scanning system at LBNL.• Acquired 4000 Hz confocal probe with high intensity

xenon light source.• Configure stages for raster and helical scans.• Study data quality versus probe speed and grid

spacing to optimize overall scan time.• Study media with mould growth and other damage.• Development of scratch correction code.• Some 3D studies of disc media may be possible.• Possibility of WLI study (?)

Page 58: May, 28, 2004 Topics in Preservation Science Lecture Series 1 Carl Haber LBNL. The Reconstruction of Mechanically Recorded Sound by Image Processing Update

May, 28, 2004 Topics in Preservation ScienceLecture Series

58 Carl HaberLBNL.

Page 59: May, 28, 2004 Topics in Preservation Science Lecture Series 1 Carl Haber LBNL. The Reconstruction of Mechanically Recorded Sound by Image Processing Update

May, 28, 2004 Topics in Preservation ScienceLecture Series

59 Carl HaberLBNL.

Page 60: May, 28, 2004 Topics in Preservation Science Lecture Series 1 Carl Haber LBNL. The Reconstruction of Mechanically Recorded Sound by Image Processing Update

May, 28, 2004 Topics in Preservation ScienceLecture Series

60 Carl HaberLBNL.

Conclusions• Image based methods have sufficient resolution to reconstruct

audio data from mechanical media and reduce impulse noise.• Basic process is data intensive compared to simple stylus

playback.– 2D approach may be suitable for mass digitization. How general is the

2D image quality? IRENE design can address these and other key issues.

– At present 3D methods may be suitable for reconstruction of particular samples since they require ~hours per scan.

• Upcoming 3D research program should clarify some issues of ultimate scan time, address damaged media

• Reports available – 2D: LBNL-51983, JAES Dec 2003 – 3D: LBNL-54927, to be submitted to JAES

• Info at URL www-cdf.lbl.gov/~av

Page 61: May, 28, 2004 Topics in Preservation Science Lecture Series 1 Carl Haber LBNL. The Reconstruction of Mechanically Recorded Sound by Image Processing Update

May, 28, 2004 Topics in Preservation ScienceLecture Series

61 Carl HaberLBNL.

Extra Slides

Page 62: May, 28, 2004 Topics in Preservation Science Lecture Series 1 Carl Haber LBNL. The Reconstruction of Mechanically Recorded Sound by Image Processing Update

May, 28, 2004 Topics in Preservation ScienceLecture Series

62 Carl HaberLBNL.

Measurement of Noise at Rmin & Rmax

Optical readingsUpper sample is at outer radiusLower sample is at inner radiusFrom “Goodnight Irene” discIf noise is dominated by surface structuresof constant size distribution, the outer radiusamplitude and frequency should be greater due to greater linear speed there

Page 63: May, 28, 2004 Topics in Preservation Science Lecture Series 1 Carl Haber LBNL. The Reconstruction of Mechanically Recorded Sound by Image Processing Update

May, 28, 2004 Topics in Preservation ScienceLecture Series

63 Carl HaberLBNL.

Physical Origin of Noise in Optical Reconstruction

• View of raw groove shape data from region of pause, before differentiation into velocities.

• Upper plot is 0.6 second portion.

• Lower plot shows deviations about 10 Hz waveform.

• Each point is an independent edge detection across the groove bottom.

• Clear structures, spanning multiple points are resolved of typical scale:

100 microns (0.2 ms) x 0.2 microns !!!

0.1 seconds

Page 64: May, 28, 2004 Topics in Preservation Science Lecture Series 1 Carl Haber LBNL. The Reconstruction of Mechanically Recorded Sound by Image Processing Update

May, 28, 2004 Topics in Preservation ScienceLecture Series

64 Carl HaberLBNL.

Parameter 78 r.p.m., 10 inch 33 1/3 r.p.m., 12 inch

Groove width at top 150-200 m 25-75 m

Grooves/inch (mm) Gd 96-136 (3.78-5.35) 200-300 (7.87-11.81)

Groove spacing 175-250 m 84-125 m

Reference level peak velocity@1KHz 7 cm/sec 7 cm/sec (0.0011 cm)

Maximum groove amplitude 100-125 m 38-50 m

Noise level below reference, S/N 17-37 dB 50 dB

Dynamic range 30-50 dB 56 dB

Groove max amplitude at noise level 1.6 - 0.16 m 0.035 m

Maximum/Minimum radii 120.65/47.63 mm 146.05/60.33 mm

Area containing audio data 38600 mm2 55650 mm2

Total length of groove 152 meters 437 meters

Page 65: May, 28, 2004 Topics in Preservation Science Lecture Series 1 Carl Haber LBNL. The Reconstruction of Mechanically Recorded Sound by Image Processing Update

May, 28, 2004 Topics in Preservation ScienceLecture Series

65 Carl HaberLBNL.

Cylinder Sample

Parameter Value

Cylinder issue Edison Blue Amberol

Diameter 2 inch (2.1875 inches)

Artist Will Oakland and Chorus

Title “Just Before the Battle, Mother”

Serial number 1516 (..76; 4M-297-2) originally as Amberol #297 1909

Date of original recording 1909

Date of manufacture ~1920’s

Tracks per inch (t.p.i.) 200

Groove spacing 127 m