Upload
others
View
1
Download
0
Embed Size (px)
Citation preview
VIDEO BASED STRUCTURAL HEALTH MONITORING
USING VIRTUAL VISUAL SENSORS
by
Ali Shariati
A dissertation submitted to the Faculty of the University of Delaware in partial
fulfillment of the requirements for the degree of Doctor of Philosophy in Civil
Engineering
Summer 2016
© 2016 Ali Shariati
All Rights Reserved
All rights reserved
INFORMATION TO ALL USERSThe quality of this reproduction is dependent upon the quality of the copy submitted.
In the unlikely event that the author did not send a complete manuscriptand there are missing pages, these will be noted. Also, if material had to be removed,
a note will indicate the deletion.
All rights reserved.This work is protected against unauthorized copying under Title 17, United States Code
Microform Edition © ProQuest LLC.
ProQuest LLC.789 East Eisenhower Parkway
P.O. Box 1346Ann Arbor, MI 48106 - 1346
ProQuest
Published by ProQuest LLC ( ). Copyright of the Dissertation is held by the Author.
ProQuest Number:
10191680
10191680
2016
VIDEO BASED STRUCTURAL HEALTH MONITORING
USING VIRTUAL VISUAL SENSORS
by
Ali Shariati
Approved: __________________________________________________________
Harry W. Shenton III, Ph.D.
Chair of the Department of Civil and Environmental Engineering
Approved: __________________________________________________________
Babatunde A. Ogunnaike, Ph.D.
Dean of the College of Engineering
Approved: __________________________________________________________
Ann L. Ardis, Ph.D.
Senior Vice Provost for Graduate and Professional Education
I certify that I have read this dissertation and that in my opinion it meets
the academic and professional standard required by the University as a
dissertation for the degree of Doctor of Philosophy.
Signed: __________________________________________________________
Thomas Schumacher, Ph.D.
Professor in charge of dissertation
I certify that I have read this dissertation and that in my opinion it meets
the academic and professional standard required by the University as a
dissertation for the degree of Doctor of Philosophy.
Signed: __________________________________________________________
Harry W. Shenton III, Ph.D.
Member of dissertation committee
I certify that I have read this dissertation and that in my opinion it meets
the academic and professional standard required by the University as a
dissertation for the degree of Doctor of Philosophy.
Signed: __________________________________________________________
Michael J. Chajes, Ph.D.
Member of dissertation committee
I certify that I have read this dissertation and that in my opinion it meets
the academic and professional standard required by the University as a
dissertation for the degree of Doctor of Philosophy.
Signed: __________________________________________________________
Branko Glisic, Ph.D.
Member of dissertation committee
iv
“… the highest appreciation is not to utter words, but to live by them” John F.
Kennedy
First, I thank my advisor Prof. Thomas Schumacher, who always believed in
me and was sometimes even more excited than myself in this research. I also thank my
committee members Profs. Shenton and Chajes for their support and input, without
which the completion of this dissertation would not have been possible. Measurement
of natural frequencies of the Streicker Bridge as presented in Chapter 3 was made
possible by Prof. Glisic, to whom I owe a big “thank you”. He also served as the
fourth committee member and I appreciate his input and critical questions.
Furthermore, I thank my family, who lives far away from me, yet their infinite
love has kept my heart warm and hopeful. Especial thanks go to my mom for all of her
sacrifice and love, the two very words that are synonymous to her precious life.
Finally, my wife. She has known me for the past 5 years, which has been the
most hectic, challenging, and busy phase of my life. I thank her for her unstoppable
care and her love during these years.
ACKNOWLEDGMENTS
v
LIST OF TABLES ...................................................................................................... viii
LIST OF FIGURES ....................................................................................................... ix
ABSTRACT ................................................................................................................ xiii
Chapter
1 INTRODUCTION .............................................................................................. 1
2 MONITORING OF STRUCTURES AND MECHANICAL SYSTEMS
USING VIRTUAL VISUAL SENSORS FOR VIDEO ANALYSIS:
FUNDAMENTAL CONCEPT AND PROOF OF FEASIBILITY .................... 5
2.1 Motivation ................................................................................................. 5
2.2 Proposed Sensing Approach ...................................................................... 6
2.2.1 Background .................................................................................... 6
2.2.2 Methodology .................................................................................. 7
2.2.3 Theoretical Basis ........................................................................... 8
2.3 Experimental Verification ....................................................................... 12
2.3.1 Candidate Virtual Visual Sensors ................................................ 14
2.3.2 Accuracy of Virtual Visual Sensors ............................................ 17
2.3.3 Frequency Analysis over Range of Pixels ................................... 18
2.4 In-Service Monitoring Example .............................................................. 19
2.5 Conclusion and Outlook .......................................................................... 22
3 EULERIAN-BASED VIRTUAL VISUAL SENSORS TO DETECT
NATURAL FREQUENCIES OF STRUCTURES .......................................... 23
3.1 Introduction and Background .................................................................. 24
3.2 Motivation and Objectives ...................................................................... 26
3.3 Description of Theory and Proposed Methods ........................................ 27
TABLE OF CONTENTS
vi
3.3.1 Theoretical Considerations .......................................................... 27
3.3.2 Quantization Error and Signal-to-Noise Ratio ............................ 27
3.3.3 Patch Processing .......................................................................... 30
3.3.4 Linear Gradient Pattern Targets .................................................. 31
3.3.5 Noise Reduction Strategies for LGPTs ....................................... 32
3.4 Laboratory Experiments .......................................................................... 33
3.4.1 Experimental Test Setup .............................................................. 33
3.4.2 Three-Story Structure Tests ......................................................... 33
3.4.2.1 Cameras Used ............................................................... 33
3.4.2.2 Reference Data from Accelerometers .......................... 34
3.4.2.3 Results from GoPro Camera ......................................... 35
3.4.2.4 Results from Photron FASTCAM U100 Camera ......... 38
3.4.2.5 Comparison of Results ................................................. 41
3.4.3 Steel Beam Tests ......................................................................... 42
3.5 Field Test ................................................................................................. 43
3.6 Conclusions and Outlook ........................................................................ 45
4 EULERIAN-BASED VIRTUAL VISUAL SENSORS TO MEASURE
DYNAMIC DISPLACEMENTS OF STRUCTURES ..................................... 47
4.1 Introduction ............................................................................................. 48
4.2 Background and Theory .......................................................................... 49
4.3 Experiments ............................................................................................. 55
4.3.1 Laboratory Setup and Instrumentation ........................................ 55
4.3.2 Data Acquisition and Data Preprocessing ................................... 56
4.4 Results ..................................................................................................... 57
4.4.1 Accuracy of Proposed Approach ................................................. 57
4.4.2 Relationship of Noise and Patch Size .......................................... 63
4.4.3 Dynamic In-Service Load Test on the Streicker Bridge ............. 64
4.5 Discussion and Conclusions .................................................................... 67
5 MEASUREMENT OF TENSION IN CABLES USING VIRTUAL
VISUAL SENSORS ......................................................................................... 69
5.1 Introduction ............................................................................................. 70
5.2 Theory ...................................................................................................... 71
vii
5.3 Laboratory Experiments .......................................................................... 73
5.4 Real World Applications ......................................................................... 79
5.5 Conclusion, Future Work ........................................................................ 81
6 SUMMARY AND OVERALL CONCLUSIONS, FUTURE WORK ............ 83
REFERENCES ............................................................................................................. 85
viii
Table 3.1: Comparison of SNR from three-story structure. ..................................... 41
Table 4.1: Summary table of accuracy evaluation. .................................................. 63
Table 5.1: Measured natural frequencies in Hz using the accelerometer and the
camera, and Δ𝑓/𝑓 as a measure of error. ** ........................................... 77
Table 5.2: Range of load cell values and their average vs. the calculated tension
based on natural frequencies measured by accelerometer and the
camera: .................................................................................................... 78
LIST OF TABLES
ix
Figure 2.1: Proposed methodology of a virtual visual sensor (VVS) to measure
structural vibrations. xp and yp represent fixed coordinates of the
monitored pixel, P. (Figure courtesy of Thomas Schumacher) ................. 8
Figure 2.2: Example of experimental data extracted from a VVS: brightness of (a)
red; (b) green; and (c) blue; (d) computed intensity (used for
subsequent analyses); I, and (e) smoothed intensity (for illustrative
purposes). .................................................................................................. 9
Figure 2.3: Illustration of the relationship between vibration of motion in
direction (one direction only for simplification), amplitude of
vibration A, pixel location xP and pixel size, and shape of the grey-
scale intensity curve I(x). (Figure courtesy of Thomas Schumacher) ..... 10
Figure 2.4: Cantilever test:(a) Snapshot of moving cantilever and (b) candidate
virtual visual sensors (VVS). (Figure courtesy of Thomas
Schumacher) ............................................................................................ 14
Figure 2.5: Time history (left column) and frequency (right column) data for (a)
accelerometer; (b) pixel A; (c) pixel B; and (d) pixel C. Note: the
intensity time histories were centered about 0 vertically to avoid a
large DC component in the frequency domain. ....................................... 16
Figure 2.6: Measurements correlation (a) Table and (b) plot showing correlation
between physical accelerometer and virtual visual sensor (VVS). ............. 18
Figure 2.7: Outline of the moving object (a) Snapshot of cantilever; (b) snapshot
with highlighted pixels of same peak frequency; and (c) snapshot with
normalized highlighted pixels of same peak frequency. ......................... 19
Figure 2.8: Histogram of peak frequencies from all pixels in the bridge video.
Insert: (a) Snapshot of video clip; (b) Colored pixels with same
frequencies. .............................................................................................. 21
Figure 3.1: Experimental test setups for: a) Three-story structure and b) steel
beam. (Figure courtesy of Thomas Schumacher) .................................... 24
LIST OF FIGURES
x
Figure 3.2: Linear gradient pattern targets a) Sample linear gradient pattern
targets (LGPT), b) intensity values captured by the camera and linear
curve-fit, c) calculated noise, and d) the histogram of the noise. ............ 26
Figure 3.3: The effect of non-linearity in the spatial domain. Rows one to four
(top to bottom) show intensity in the spatial domain, I(x),
displacement, d(x), observed intensity in the time domain, I(t), and the
Fourier transform of the former, respectively. Column a) shows the
case of 𝐼𝑥 = 𝑥3, b) the linear case of 𝐼𝑥 = 𝑥, and c) the case of an
impulsive change of intensity modeled by 𝑥 = 𝑥99. .............................. 29
Figure 3.4: Two noised reduction strategies for LGPTs: a) Patch on an LGPT to
average out the noise b) linear regression approach: measured
intensities are mapped onto linear curve-fit. (Figure courtesy of
Thomas Schumacher) .............................................................................. 32
Figure 3.5: Cameras used for the laboratory experiments: a) Photron FASTCAM
UX100, b) Photron FASTCAM SA-X2 and c) GoPro Hero 3. ............... 34
Figure 3.6: Sample data from the accelerometers: a) Second story and b) third
story. The left and right column shows data in the time and frequency
domain, respectively. ............................................................................... 35
Figure 3.7: Data extracted from videos taken by the GoPro camera without
LGPTs: a) One pixel in the middle of first floor, and b) a 50 x 50
patch of pixels in the middle of the first floor. ........................................ 36
Figure 3.8: Data extracted from videos taken by the GoPro camera with LGPTs:
a) One pixel on the gradient target, b) A patch of 5 x 5 pixels on the
target c) Linear regression applied to 10 randomly chosen points on
the gradient. ............................................................................................. 37
Figure 3.9: Data extracted from videos taken by the FASTCAM UX100 camera
with and without LGPTs: a) One pixel at the very bottom, b) a patch
of pixels at the mid-level of the first floor, c) one pixel on the LGPT,
d) a 5 x 5 pixel patch on the LGPT, and e) linear regression applied to
10 random pixels on the LGPT. .............................................................. 40
Figure 3.10: The steel beam test results: (a) Results from the FASTCAM SA_Z
camera, and (b) results from one accelerometer. ..................................... 42
Figure 3.11: Real world application a) View of the Streicker Bridge and b) test
setup, camera position and LGPTs (insert). ............................................ 44
xi
Figure 3.12: Frequency response of the Streicker Bridge from (a) the VVS located
on an LGPT and (b) the fiber-optic sensor system. ................................. 45
Figure 4.1: Illustration of the proposed methodology to measure structural
vibrations using Eulerian-based virtual visual sensors (VVS).
Acronyms: BWT = black-and-white target, FFT = Fast Fourier
transform, IDT = intensity-to-displacement transform. (Courtesy of
Thomas Schumacher) .............................................................................. 49
Figure 4.2: Illustration of the VVS measurement process using a black-and-white
target (BWT): the target, which is attached to the vibrating structural
element, moves in the x-direction relative to a fixed patch of pixels,
i.e. having Eulerian-coordinates, as a function of time, t. (Figure
courtesy of Thomas Schumacher) ........................................................... 51
Figure 4.3: Illustration for the case where the camera is not oriented perpendicular
to the displacement component of interest, uact. O denotes the camera
location and T the center location of the target mounted to the
vibrating structure. a and b represent horizontal and vertical distance
between the camera and the center of the target. (Figure courtesy of
Thomas Schumacher) .............................................................................. 52
Figure 4.4: Experimental setup: (a) high-speed data acquisition system, (b)
potentiometer to measure displacements at the first story mass, (c)
digital camera to collect VVS data, and (d) three-degree-of-freedom
laboratory structure. ................................................................................. 56
Figure 4.5: Photo of BWT target with two VVS patch sizes: (a) 40 x 50 pixel
patch and (b) 40 x 100 pixel patch. ......................................................... 58
Figure 4.6: Comparison of results using independent calibration factors, 40 x 50
pixels (left column) and 40 x 100 pixels (right column): (a) Time
history signals of VVS patch data and potentiometer, (b) correlation
between the two measurements, (c) absolute value of the difference
between the two signals (errors), and (d) histogram of the errors. .......... 60
Figure 4.7: Comparison of results using calibration factors based on minimized
difference between measurements, 40 x 50 pixels (left column) and 40
x 100 pixels (right column): (a) Time history signals of camera and
potentiometer, (b) correlation between the two measurements, (c)
absolute value of the difference between the two signals (error), and
(d) histogram of the error. ....................................................................... 62
xii
Figure 4.8: Noise power and the signal-to-noise ratio (SNR): (a) Power of the
noise vs. the number of pixels in the VVS patch (N), (b) histogram of
noise in one pixel, (c) histogram of noise in a 10 x 10 pixel patch, and
(d) the SNR values vs. width over length of the patch. ........................... 64
Figure 4.9: Photograph of the Streicker Bridge showing the measurement setup
and the location of the VVS. The insert shows the location of the 60 x
20 pixel VVS patch (red rectangular). The target was installed by Dr.
Maria Feng’s research team from Columbia University [46]. ................ 66
Figure 4.10: Results from the dynamic load test on the Streicker Bridge: (a)
Computed actual vertical displacement time history and (b) frequency
response of signal (a). .............................................................................. 67
Figure 5.1: Test setup showing: a) load cell: Omegadyne LC101, b)
accelerometer: Entran EGC, c) camera: GoPro Hero 3, and d) cable. .... 74
Figure 5.2: Samples of typical data: a) FFT of acceleration, b) FFT of VVS, c)
FFT of load cell, and d) tension force vs. time. ....................................... 76
Figure 5.3: Lab test results a) The minimum, maximum, and average tension
force in the cable measured by the load cell (Min LC, Max LC and
Mean LC) vs. the average calculated values of tension based on
natural frequencies of accelerometer (Mean Acc) and camera (Mean
Cam), b) The same values divided by the average load cell values. ....... 79
Figure 5.4: Typical pluck test on a cable of the Indian River Inlet Bridge a) The
crane, b) moving the cable to initiate free vibration. ............................... 80
Figure 5.5: Measured natural frequency for the cable 219 E ..................................... 81
xiii
Structural health monitoring (SHM) has become a viable tool to provide
owners of structures and mechanical systems with quantitative and objective data for
maintenance and repair. Traditionally, discrete contact sensors such as strain gages or
accelerometers have been used for SHM. However, distributed remote sensors could
be advantageous since they don’t require cabling and can cover an area rather than just
a few discrete points. Along this line a novel monitoring methodology based on video
analysis is proposed. By employing commercially available digital cameras combined
with efficient signal processing methods, measurement of natural frequencies using a
computationally less demanding algorithm was possible. In this dissertation, the basic
concept of the proposed so-called Eulerian-based virtual visual sensors (VVS) is first
introduced. In order to improve the signal-to-noise ratio (SNR), the application of
oversampling as well as two different targets were considered. The proposed
methodology was evaluated on a set of laboratory experiments to demonstrate the
accuracy of the considered approach. In-service monitoring examples of different
bridges are further provided to show the practical aspects. A discussion of further
work to improve the methodology is also discussed.
ABSTRACT
1
INTRODUCTION
Structural vibrations contain important information about a structural system’s
dynamic characteristics. Changes over time in the vibration response can be caused by
alterations in the loading, boundary conditions, or degradation of the structural system.
Several approaches have been developed to extract information from the measured
vibrational characteristics of structures, which fall in a category well known as
“vibration-based structural health monitoring (SHM)” [1]–[14]. These methods
essentially utilize natural frequencies and mode shapes acquired from different types
of physical sensors such as accelerometers.
A critical step in the process of measuring mode shapes and natural
frequencies, is the gathering of the vibration data using sensors or sensor networks.
The ultimate goal is to have a sensing system that produces objective, quantitative, and
accurate data, inexpensively. Conventional contact-type sensors such as strain gages
or accelerometers that are attached to specific locations of a structure are capable of
measuring the response at that specific point. Accessibility of the member of interest
combined with wiring issues in addition to high local-only sensitivity are some
drawbacks of conventional sensors, which have stimulated innovation to develop non-
contact sensors.
Alternatively, optical devices such as laser Doppler vibrometers, which can be
applied remotely, have been used but are expensive. Video-based methods have
Chapter 1
2
recently been introduced for a variety of applications in structural health monitoring
(SHM). Patsias and Staszewski [15] analyzed digital videos for edge detection and to
approximate the mode shape of a cantilever in a laboratory experiment. By applying a
wavelet transform to the mode shape they were able to detect the location of damage
which was introduced by cutting a groove with decreasing depth into the cross-section.
Lee et al. [16] devised a real-time method to measure in-plane displacements and
rotations using feature tracking techniques based on a Lagrangian approach, and
applied it to a target bridge. Zaurin and Catbas[17] developed a method using digital
video data to locate and measure applied loads on a bridge and devised an index called
unit influence line (UIL) as a measure of the health of bridges. Elgamal et al.[18]
developed a framework to integrate different data types including computer vision data
to create a “decision-support system” for bridges and other lifelines. In a SHM review
on wind turbines by Ciang et al. [19], it is noted that digital image correlation (DIC)
techniques can also be used for these structures, but the 3-D version of these methods
should be investigated in more depth if they are to be applied. Song et al. [20]
modified the Hough Transform to track numerous markers on a beam with a
computationally efficient algorithm and fitted a spline curve to the tracked shape in
order to detect the location of the damage.
The use of digital videos for SHM is only in the beginning stage. With the
availability of inexpensive yet high-quality digital video cameras, there is great
potential that has not been fully explored yet. In this dissertation a methodology to
measure natural frequencies and dynamic displacement is investigated with
commercially available camera technology as well as high speed professional camera.
With an efficient and yet simple methodology to capture and compute structural
3
vibration data from digital videos, this technique has been shown to be effective,
accurate, and have a high signal to noise ratio.
The development of this technique is presented in Chapters 2 to 4. In Chapter 2
the concept of Eulerian virtual visual sensors (VVS) is introduced and based on that
the estimation of the first fundamental natural frequency of vibration of structures is
studied [21]. The methodology is based on the original idea presented in [22] and
further applied in [23]. The essence of the original idea is that single pixels can carry
essential information about minute changes of objects that are not visible to the naked
eye, but can be made visible by a technique called Eulerian motion magnification [22].
It has been shown experimentally, that change of intensity in certain pixels of a digital
video can be related to the natural frequency of a vibrating structure. It should be
noted that this approach is in its very nature is different from what is used in feature
tracking or block matching algorithms such as DIC. It has been shown that in this
technique to achieve the natural frequency the calculation of the displacement is not
necessary.
It is later on shown in Chapter 3 that by employing the basic methodology
introduced in Chapter 2, 2nd and 3rd natural frequencies of a laboratory-scale three-
story structure are not observable. It is discussed that this problem is caused partially
by the low signal-to-quantization noise ratio (SQNR) due to the limited and usually
uncontrollable range of change in intensity. Employment of gradient pattern targets,
which are mounted to the structure at locations of interest, expand the range of
intensity values and thus reduce the quantization noise, is thoroughly investigated to
address this problem. Also another approach to increase the SQNR, which is
oversampling in time is discussed and studied. The idea of using a patch of pixels
4
rather than a single pixel to alleviate challenges related to occlusion due to large
displacements and two efficient ways to reduce the noise are discussed as well.
In Chapter 4, a simple yet effective alternative way to measure dynamic
displacement using Eulerian-based virtual visual sensors (VVS) is proposed, for which
the fundamental basis is developed in earlier Chapters and in the author’s publications
[21], [24]–[27]. It is shown for a black-and-white target (BWT), a linear intensity-
displacement relationship exists for a patch of pixels on the boundary of the target. It
should be noted that this same transform can also be applied without using BWTs, for
example to the edge of a structural member where a distinct boundary between the
member and the background exists.
In Chapter 5, an application of the methodology for the measurement of natural
frequencies of cables in cable stayed bridges is presented. A laboratory study and field
test show that the methodology, as introduced in Chapter 2, can be applied to measure
natural frequencies of cables accurately. Finally, in Chapter 6 overall conclusions are
made and future work is discussed.
5
MONITORING OF STRUCTURES AND MECHANICAL SYSTEMS USING
VIRTUAL VISUAL SENSORS FOR VIDEO ANALYSIS: FUNDAMENTAL
CONCEPT AND PROOF OF FEASIBILITY
In this chapter, the concept of Eulerian virtual visual sensors (VVS), the theory
behind it and application in laboratory and real-world settings are studied. The first
natural frequency of vibration of the structures is studied [21]. As mentioned before
this method genuinely is based on the idea presented in [22]. The original idea is that
minute changes in a single pixel reveals essential information about the dynamic
characteristics of the captured context. It should be emphasized that there is a
fundamental difference between the proposed technique and block matching and
feature tracking algorithms. This chapter is published in Sensors 2013, 13(12), 16551-
16564; doi:10.3390/s131216551 and currently has 7 citations as reported by Google
Scholar.
2.1 Motivation
The objective of this study was to evaluate a novel sensing approach for
structural health monitoring (SHM) purposes which is contactless, inexpensive, and
flexible in its application. Vibration data are important in a number of disciplines such
as mechanical and structural engineering. A comprehensive review on structural health
monitoring (SHM) shows the efforts put forth to estimate damage and damage
Chapter 2
6
location based on observed changes in natural frequencies of vibration [5]. The
literature contains different resources addressing vibration-based SHM as well [8], [9],
[28]–[32]. Finally, natural frequencies from in-service structures are often used to
calibrate finite element (FE) models [33], [34].
2.2 Proposed Sensing Approach
2.2.1 Background
In a recent inspiring paper on Eulerian video magnification, Wu et al. [22]
present an innovative yet beautifully simple approach to magnify subtle motions in
digital videos so that they become visible to the naked eye. This was done using an
Eulerian specification where a pixel with a fixed coordinate is selected and its value
monitored in time. In contrast, in a Lagrangian specification one would attempt to
track a specific feature in a video in time and space. One of the examples presented,
which may have great potential for application in the medical field, measures the pulse
of a person by analyzing a video taken from the person. The inventors found that the
minute change in intensity in the red content, R, of the person’s skin was significant
enough to be analyzed to accurately compute the person’s pulse. Another example was
a video of a person’s wrist where the expansion and contraction of the veins were
amplified to be clearly visible. The advantage is that this approach is contactless and
can be performed continuously without interfering with the person. Motivated by this
article a methodology based on the same fundamental idea for potential use in the field
of structural health monitoring (SHM) for structures and mechanical systems is
proposed.
7
2.2.2 Methodology
It is proposed that every pixel in a digital video taken from a structure
represents a candidate virtual visual sensor (VVS) that may be used for SHM purposes
(first suggested by Patsias and Staszewskiy [15]). The term “VVS” follows the
terminology suggested by Song, Bowen, et al. [35]. Although the approach presented
in the latter paper may appear similar, it is fundamentally different as they were
employing a Lagrangian specification where a target (or feature) is tracked in space
and time.
The proposed methodology uses an Eulerian specification where a specific
pixel is selected and monitored which is illustrated in Figure 2.1; the intensity of the
pixel at location xp and yp is monitored over time and analyzed using the Fast Fourier
Transform (FFT) [36], [37] to reveal the fundamental frequency of vibration. Note that
the pixel value in the time domain represents gray-scale intensity and does not directly
correspond to the amplitude of vibration, i.e., displacement. As a result, at this
point it is not possible to estimate the amplitude but only the frequency of the motion
which represents a limitation.
8
Figure 2.1: Proposed methodology of a virtual visual sensor (VVS) to measure
structural vibrations. xp and yp represent fixed coordinates of the
monitored pixel, P. (Figure courtesy of Thomas Schumacher)
2.2.3 Theoretical Basis
Digital videos are a sequence of digital images captured at a specified frame
rate. Typical frame rates of commercially available cameras are 25, 30, or 60 fps
(=frames per second). In this study a range of cameras are used, including two
inexpensive point-shoot cameras with 25 and 30 fps, respectively, and a new high-
speed camera mainly used in the adventure sports community that can capture videos
up to 240 fps. Video frames are typically stored in RGB (red-green-blue) color mode
as measured by the camera’s image sensor. A single grey-scale value, called intensity,
I, is assigned to each pixel where 0 and 255 represent black and white, respectively.
MATLAB uses a linear combination to calculate I based on RGB values that
eliminates hue and saturation information while retaining the luminance [38]:
0.2989 0.5870 0.1140 I R G B (2.1)
An example of experimental data extracted from a VVS and the resulting
intensity curves are shown in Figure 2.2. If the intensity value is smoothed using a 5-
9
point moving average as shown in Figure 2.2 e, the quantization effects that exist in
the raw brightness values (Figure 2.2c) and the computed grey-scale intensity curve
(Figure 2.2d) can effectively be removed to reveal a relatively harmonic motion. For
this study, only raw intensities (example shown in Figure 2.2d) were used for the
computation of frequencies.
Figure 2.2: Example of experimental data extracted from a VVS: brightness of (a)
red; (b) green; and (c) blue; (d) computed intensity (used for subsequent
analyses); I, and (e) smoothed intensity (for illustrative purposes).
Figure 2.3 illustrates the factors that influence the accuracy and reliability of
the proposed VVS. The dotted line represents the grey-scale intensity curve I(x) along
a path x. For this theoretical example, the background is assumed to be light colored
and the object of interest dark colored. The location and size of the monitored pixel is
depicted by the grey square denoted with P(x,t). L represents the length over which the
10
intensity changes. Figure 2.3 represents a snapshot and as time t progresses the
intensity curve I(x) will vibrate horizontally (in the x-direction) with an amplitude A
causing the pixel to oscillate vertically about xp, following the I(x) curve.
Figure 2.3: Illustration of the relationship between vibration of motion in direction
(one direction only for simplification), amplitude of vibration A, pixel
location xP and pixel size, and shape of the grey-scale intensity curve
I(x). (Figure courtesy of Thomas Schumacher)
If the object, characterized by the intensity I(x), is vibrating at a natural
frequency ώ0 and without losing generality we can write 0sinx A t , hence the
intensity value becomes 0sinI x I A t . If we want to consider the effect of
the function I(x) on our measured peak frequencies we can write:
0sin j tF I x I A t e dt
(2.2)
If I(x) is a linear function and by subtracting the DC term we are able to
compute the exact peak frequency (neglecting any quantization noise)
0 0sin sinI A t C t so that:
11
0 0F I t Cj
(2.3)
where C is a constant. If I x is a nonlinear function, e.g., nx , with the
identity 0sin
2
j t j te et
we can write:
02
0
12 2
n nj t j t nk j n k tj t j t
k
ne e AF I t A e dt e e dt
k
(2.4)
Taking the summation out of the integral we obtain:
0
0
2 1 22
n nk
k
nAF I t n k
k
(2.5)
Equation (2.5) reveals that any nonlinearity of degree n in I(x) produces peak
frequencies at (n – 2k)ώ0 for 0 < k < n. It is important to note that if I(x) can be written
as a power series, the magnitude of the spurious impulses in the frequency domain can
be calculated based on the above equation. From the discussion above, the following
can be concluded: The extreme positions (peak amplitude points) of the VVS with
respect to the intensity curve I(x) should be located on an approximately linear portion
of I(x) and within L. If I(x) is nonlinear, spurious frequency peaks will occur as can be
observed in Figure 2.5b which will be explained in more detail in Section 2.3.1. The
important implications of the above equations are as follows:
The intensity range max minI I I should be maximized, i.e., a small
range will increase quantization noise. This can be achieved by selecting
proper background and lighting conditions.
The number of pixels across A should be maximized which is directly
related to the spatial resolution.
12
The amplitudes of vibration should be small, i.e., the maximum amplitude
of vibration A should lie within L in order to avoid the appearance of
nonlinear system behavior.
The size of the pixel (or VVS) with respect to the length should be small to
avoid averaging of measured intensity values and additional quantization
noise.
Additionally, the following factors influence the accuracy of the VVS:
High sampling rates, i.e., a large number of frames per second, decrease the
quantization noise. Minimum sampling rates as given by the Nyquist-
Shannon sampling theorem [39].
The total signal duration T directly influences the resolution and thus the
uncertainty of the VVS, i.e., the resolution of a signal in the frequency
domain is 1f T .
Finally, moving and shaking of the camera, changing illumination, and
noise in the image sensor influence the signal-to-noise ratio and therefore
the accuracy of the computed peak frequency.
2.3 Experimental Verification
To verify the validity and accuracy of the proposed approach, a cantilever
beam with adjustable length, LC (i.e., variable stiffness) and a concentrated constant
mass on top, equivalent to a single-degree-of-freedom (SDOF) system, was tested as
shown in Figure 2.4 a. The test was initiated by creating an initial displacement (by
hand) and then letting the cantilever vibrate in its natural mode of vibration.
Acceleration was measured using a high-accuracy capacitive accelerometer (Model 2260-
010 by Silicon Design, sampling at 1 kHz) attached to the mass. Additionally, a digital
13
video was taken during the test capturing the motion of the cantilever using two
different cameras: a commercially available digital camcorder (Model ViXIA HFS100
HD by Canon, recording at 30 fps) for frequencies up to 10 Hz and a relatively high-
speed camera (Model Hero 3 by GoPro, recording at 120 fps) for higher frequencies. It
should be noted that, as for any digitally sampled signal, the Nyquist-Shannon [39]
sampling theorem applies, i.e., the sampling rate needs to be set to at least twice the
highest anticipated frequency to be distinguishable in the signal. Anti-aliasing filters
were set to one half of the selected sampling frequency for the accelerometer. For the
cameras, such an option is currently not available, and one of the goals was to
determine whether this represents a problem. The selection of the pixel to be
monitored (candidate VVS) turned out to be critical to obtain meaningful frequency
data from the videos and a thorough discussion is included in the following section.
14
Figure 2.4: Cantilever test:(a) Snapshot of moving cantilever and (b) candidate
virtual visual sensors (VVS). (Figure courtesy of Thomas Schumacher)
2.3.1 Candidate Virtual Visual Sensors
The advantage that every pixel in the video represents a candidate virtual
visual sensor (VVS) is also the challenge. In this section we compare and discuss the
signals computed from a number of different candidate pixels. For this evaluation the
cantilever length, LC was kept constant at 25 in (635 mm). Figure 2.4 b shows three
candidate VVS: Pixels A and B are located near the top and the bottom of the
cantilever where the largest and smallest amplitudes of vibration occur, respectively.
Pixel C is located away from the cantilever but capturing its shadow. Intuitively one
might pick pixel A since it is located where the largest motion takes place which
15
should produce the best data. However, in the case of our proposed approach this does
not work well as it is discussed next.
Figure 2.5a shows example data collected with the accelerometer. Although
pixel B is at the bottom of the cantilever, a place with the smallest motion which can
hardly be observed by the naked eye, the change of intensity (grey scale pixel value) is
represented by a relatively harmonic signal (Figure 2.5c). For pixel A located near the
top of the cantilever, where the amplitude of vibration is largest, the intensity value
experiences periodic impulses due to the sudden occlusion of the mostly grey
background by the beam. As a result, the FFT produces a periodic function as well,
showing pronounced harmonic peaks, as can be observed in Figure 2.5b. This can be
mathematically explained by a highly non-linear function such as I(x) = xn where n is a
large number. Although the peak frequency is present and correct, the upper
harmonics are very strong as well which makes the analysis more difficult.
Alternatively, pixel C is found to produce a relatively harmonic signal as well (Figure
2.5 d). Although it is not located on the structure, it can capture the motion of its
shadow. This represents an opportunity to observe vibrations indirectly, in case the
actual structure is not directly observable. These facts suggest that a reliable candidate
VVS for measuring frequency is a point with small amplitudes of vibration near the
physical boundary (i.e., edge) of the structure where the change in intensity is most
pronounced (a theoretical discussion is presented in Section 2.2.3). For the cantilever
study this can also be a point near the top of the cantilever after damping has reduced
the amplitudes of vibration significantly. For practical purposes, this condition is
usually satisfied considering the camera is relatively far from the structure compared
to the amplitude of vibrations to be captured.
16
Figure 2.5: Time history (left column) and frequency (right column) data for (a)
accelerometer; (b) pixel A; (c) pixel B; and (d) pixel C. Note: the intensity
time histories were centered about 0 vertically to avoid a large DC
component in the frequency domain.
Although the total average amplitude of vibration for the data shown in Figure
2.5a, c was approximately 100 mm and 0.3 mm, respectively, the signal-to-noise ratios
17
for the frequency plots are comparable. This further highlights the potential of this
sensing approach to capture small vibrations.
2.3.2 Accuracy of Virtual Visual Sensors
In order to determine the accuracy of the proposed approach, a pixel close to
the bottom of the cantilever was selected to compute the frequency as described
previously and shown in Figure 2.4a. The length, LC was varied between 50 and 635
mm to produce a range of natural frequencies. Figure 2.6 shows the correlation
between the physical accelerometer and the frequencies computed from the selected
VVS. The computed frequencies listed in Figure 2.6a are given as f ± f/2 to account
for the uncertainty where f = 1/T with T being the duration of the original signal
length in seconds. Although we used zero padding to run the FFT in some cases,
which will provide smoother peaks in the low frequency range, higher accuracy is not
achieved. As can be observed in Figure 2.6b, there is excellent correlation between the
frequencies computed from the two measurements. The squared correlation coefficient
and standard error between accelerometer and the camera’s computed frequency were
found to be 99.993% and 0.0295, respectively. Figure 2.6 a Table and b plot showing
correlation between physical accelerometer and virtual visual sensor (VVS).
18
Figure 2.6: Measurements correlation (a) Table and (b) plot showing correlation
between physical accelerometer and virtual visual sensor (VVS).
2.3.3 Frequency Analysis over Range of Pixels
An extended approach to determine candidate VVS is to analyze a selected
area of pixels in a video around the vibrating structure and then highlighting the pixels
that have the same frequency in the image as illustrated in Figure 2.7. It should be
noted that this only works well for small amplitudes of vibration for reasons
previously discussed. This involves the following steps:
1. Select range of pixels to be analyzed within video (shown as white box in
Figure 2.7b,c).
2. Compute time history of intensity values for each of the selected pixels.
3. Compute the peak frequency for each pixel as described in Section 2.3.2 and
create a histogram.
4. Highlight the pixels with the same peak frequency in the selected range (Figure
2.7b).
5. Normalize the color values with the magnitude of the FFT transform to reduce
noise (optional, Figure 2.7c).
19
6. Repeat steps 1 to 5 if more than one significant frequency is present in the
histogram.
Figure 2.7: Outline of the moving object (a) Snapshot of cantilever; (b) snapshot
with highlighted pixels of same peak frequency; and (c) snapshot with
normalized highlighted pixels of same peak frequency.
As can be observed from Figure 2.7, the result of this analysis is essentially an
image of the outline of the vibrating parts of the structure. Note that this was done for
a period where the cantilever was experiencing small amplitudes of vibration to avoid
problems as discussed in the previous section. This analysis could also be used to
average peak frequencies from several measurements rather than using one
measurement.
2.4 In-Service Monitoring Example
In order to examine the applicability of this method for practical purposes, a
video of an existing major bridge in Oregon was evaluated. The bridge consists of a
20
continuous steel truss and some of the vertical hangers have experienced extensive
torsional vibrations due to the high transverse winds which caused vortex shedding.
Concerns regarding fatigue at the connections have been raised and as a result, the
State Department of Transportation has recently retrofitted some of the susceptible
members. The reason for the vibrations is the low torsional stiffness of the used I-
sections. In a recent research project the problem was investigated in the laboratory to
make predictions on the remaining fatigue life [40].
A video taken by DOT personnel showing torsional oscillations was made
available to us. The movie was taken with an inexpensive point-and-shoot-type
camera recording at 25 fps by hand (no mechanical stabilization) with the intent to
qualitatively document such an occurrence and not necessarily for quantitative
analysis purposes. The video captured severe torsional vibrations of two vertical
hangers labeled (1) and (2) in Figure 2.8a, simultaneously. It should be noted that only
four seconds from the original video were usable which directly influences the
resolution in the frequency domain. For this analysis, all pixels were analyzed and
their peak frequency values computed as described in the Section 2.3.3. Figure 2.8
shows a histogram of all computed peak frequencies. As can be observed, the majority
of computed peak frequencies are close to zero which essentially means that the
majority of the pixel intensity values don’t change. Frequencies between 0 and 1 can
be associated with the fact that the camera was held by hand and therefore probably
not completely fixed. A closer look reveals that there are two distinct frequency peaks
that can be associated with the torsional vibrations of the two hangers.
21
Figure 2.8: Histogram of peak frequencies from all pixels in the bridge video. Insert:
(a) Snapshot of video clip; (b) Colored pixels with same frequencies.
By filtering out pixels that are not within the desired frequency range, the
outline of the oscillating hangers is obtained (Figure 2.8b) as described in Section
2.3.3. The frequencies computed from one selected pixel for members (1) and (2) are
6.1 ± 0.125 and 7.1 ± 0.125 Hz, respectively. A finite element (FE) analysis of a
hanger modeled after one of these two members [40] predicted a torsional vibration
frequency of approximately 6.5 Hz. This result is not the actual measurement but
proves that our computed frequencies obtained from the VVS are plausible. This
demonstrates that, although the movie was never taken with the intent for analysis
with our proposed methodology, it was still possible to deduce useful frequency of
vibration information.
22
2.5 Conclusion and Outlook
The concept of virtual visual sensors (VVS) offers new opportunities for
structural health monitoring of structural and mechanical systems. The following
conclusions can be drawn from this study:
The fundamental frequency of vibration of single-degree-of-freedom
(SDOF) systems can be accurately computed using the proposed
methodology of virtual visual sensors (VVS).
VVS are inexpensive non-contact sensors with great application flexibility.
Multiple independently vibrating elements in one video can be distinguished
and their fundamental frequency of vibration computed.
The accuracy and resolution of the measurements depends on a variety of
factors such as sampling rate, quantization noise (function of pixel size and
location with respect to intensity curve), image sensor quality and size, and
lens type.
By highlighting the pixels with a distinct frequency, the outline of the
vibrating elements in a video can be recovered.
Application of the methodology to multi-degree-of-freedom (MDOF) and
continuous systems will be discussed in Chapter 2. Furthermore, investigations of
approaches that not only give frequency but also amplitude of vibration are presented
in Chapter 3. The sensitivity and reliability of the measurements are discussed and
finally, an application to estimate the tensile force of cables based on this
methodology is proposed in Chapter 5.
23
EULERIAN-BASED VIRTUAL VISUAL SENSORS TO DETECT NATURAL
FREQUENCIES OF STRUCTURES
Benefits of the introduced approach in Chapter 1, are that it allows for
distributed sensing and it is non-contact. However, as intensity does not reflect any
physical quantity, such as displacement, and the range of values is difficult to control,
the signal-to-noise-ratio (SNR) can be relatively low. Furthermore, impulsive changes
of intensity caused by large deformations compared to the pixel size can result in an
impulse train in the frequency domain which leads to ambiguity in determining peak
frequencies as discussed in previous chapter. As a result, it is often only possible to
measure the first fundamental mode of vibration. In this chapter, strategies using
targets mounted to the structure combined with signal processing methods that
significantly improve the signal-to-noise ratio (SNR) and allow for detecting higher
natural frequencies of vibration are presented. The concepts, their mathematical
background, laboratory tests to prove the accuracy and enhancement of SNR, as well
as an example of an in-service pedestrian bridge are presented and discussed. This
chapter is published in the Journal of Civil Structural Health Monitoring 5.4 (2015):
457-468 DOI: 10.1007/s13349-015-0128-5.
Chapter 3
24
3.1 Introduction and Background
By employing the basic methodology introduced in the previous chapter and in
[21] observing all natural frequencies of a laboratory-scale three-story structure was
not possible (see Fig. 3.1 a)). This was partially because of the low signal-to-
quantization noise ratio (SQNR) due to the limited and usually uncontrollable range of
change in intensity. One solution may be to employ gradient pattern targets, which are
mounted to the structure at location of interest, and expand the range of intensity
values and thus reduce the quantization noise. As it is discussed in Section 2.2.3, non-
linear intensity functions produce multiple harmonic frequencies in the frequency
domain. It was decided to evaluate grey-scale linear gradient pattern targets (LGPT)
with a theoretical range of intensities from 0 (= black) to 255 (= white) (see Fig 3.2
a)).
Figure 3.1: Experimental test setups for: a) Three-story structure and b) steel beam.
(Figure courtesy of Thomas Schumacher)
25
The idea is to introduce a linearly varying background avoiding non-linear or
impulse-like behavior. In Section 3.3.3 we discuss that another solution to increase the
SQNR, which is oversampling in time. The idea of using a patch of pixels rather than
a single pixel to alleviate challenges related to occlusion due to large displacements is
discussed in Section 3.3.3. Section 3.3.4 presents the basic idea of LPGTs and in
Section 3.3.5 we discuss two efficient ways to reduce their noise. In Section 3.4 the
laboratory experiments are presented and the results from a number of different digital
cameras used on two different structural systems are discussed. The results from
monitoring of an in-service pedestrian bridge during an impact test are deliberated in
Section 3.5. Finally, in Section 3.6 the conclusions are made and further work are
proposed. It is shown in the previous chapter, experimentally, that change of intensity
in certain pixels of a digital video can be related to the natural frequency of a vibrating
structure. It should be noted that this approach is in its very nature different from what
is used in feature tracking or block matching algorithms such as DIC.
26
Figure 3.2: Linear gradient pattern targets a) Sample linear gradient pattern targets
(LGPT), b) intensity values captured by the camera and linear curve-fit,
c) calculated noise, and d) the histogram of the noise.
3.2 Motivation and Objectives
The objectives of this study were to develop strategies for Eulerian-based
virtual visual sensors (VVS) that will (a) minimize non-linear effects and (b) improve
the signal-to-noise ratio of the recorded data to enable the detection higher natural
frequencies of vibration, which was not previously possible. This is achieved by
introducing linear pattern gradient targets (LGPTs) that are mounted to the structure
and by developing a number of signal processing steps. It is demonstrated that the
strategies work with a set of laboratory experiments as well as a field test on a real
bridge.
27
3.3 Description of Theory and Proposed Methods
3.3.1 Theoretical Considerations
As mentioned previously, non-linearity in the intensity function in the spatial
domain, I(x) will result in higher harmonics in the frequency domain [21]. This is
shown by the following theoretical relationship that is the same as Equation 2.5 which
comes here again for the sake of convenience:
0
0
, 2 1 22
n nk
k
nLI F I x t n k
k (3.1)
where L is the amplitude of vibration, n is the degree of non-linearity, k the
counter from 0 to n, is the radial frequency, 0 is the object’s radial frequency, and
the Delta-Dirac function. For example, if the intensity function in the spatial domain,
I(x) is a third degree polynomial, 3)( xxI as shown in in Figure.3.3 a), and the
displacement follows a sinusoidal function with a frequency of 1 Hz (Figure. 3.3,
second row), the observed intensity response in the time domain, I(t) is not a
sinusoidal function, as illustrated in Figure. 3.3 a), third row. In the absence of noise,
the Fourier transform of the intensity values have two peak frequencies at 1 and 3 Hz,
as shown in the Figure. 3.3 a), fourth row, which verifies Equation. (3.1). On the other
hand, if I(x) is linear, the resulting I(t) is sinusoidal, as shown Figure. 3.3 b). A highly
non-linear case such as I(x) = x99 results in an impulsive response of the intensity, I(t)
in turn leading to multiple peaks in frequency domain as shown in Figure. 3.3 c). This
illustrates the effect of occlusion discussed earlier in this paper and in [21].
3.3.2 Quantization Error and Signal-to-Noise Ratio
Analog-to-digital (A/D) conversion involves two main steps: sampling in time
and quantization. Errors due to quantization and their effect on the signal is a known
28
issue addressed in the literature, e.g. in [41]. Assuming that the quantizer uniformly
covers the limit values and its error is independent of the original signal, it can be
deduced that the error is equivalent to an additive white noise. The white noise model
can also be used with high-resolution quantization, which satisfies the independence
condition. In practical signal processing, in a process called “dithering”, some random
noise within the range of quantization is added to the analog signal prior to digitization
to satisfy the independence of the error from the signal [41].
29
Figure 3.3: The effect of non-linearity in the spatial domain. Rows one to four (top to
bottom) show intensity in the spatial domain, I(x), displacement, d(x),
observed intensity in the time domain, I(t), and the Fourier transform of
the former, respectively. Column a) shows the case of 𝐼(𝑥) = 𝑥3, b) the
linear case of 𝐼(𝑥) = 𝑥, and c) the case of an impulsive change of
intensity modeled by (𝑥) = 𝑥99.
In commercially available cameras, the quantization resolution to reflect the
amount of absorbed energy in CCD sensors is usually 8 bit. However, as discussed
previously in the proposed methodology, this energy (or intensity value) does not
30
correspond to any physical quantity such as displacement or any of its derivatives. In
other words, higher amounts of displacement, velocity, or acceleration do not
necessarily cause higher change of intensity. Assuming that the quantization error can
be modeled as white noise, increasing the sampling frequency will decrease its
amplitude in the frequency domain. It can also be shown that by doubling the
sampling frequency, the power of quantization noise decreases by 3 dB. This means
that by doubling temporal sampling rate, the maximum theoretical increase in the
signal-to-noise ratio (SNR) is 3 dB. The frame rates of commercially available
cameras are in the range of 30, 60 and 120 Hz, which is reasonably sufficient for
measuring frequencies in large structural systems such as bridges but may not be
sufficient to detect all of the natural frequencies due to the high quantization error.
High-speed cameras represent an effective yet expensive solution to this issue, which
is also shown earlier [42]. A discussion of high-speed cameras can be found in Section
3.4.
3.3.3 Patch Processing
As discussed earlier and visualized in Figure. 3.3, choosing one pixel in a
digital video recorded at a comparatively low frame rate and resolution can lead to
ambiguous peaks in the frequency domain, which makes the detection of higher
frequencies difficult or often impossible. To solve the problem of occlusion, which
produces periodic impulses in time and frequency domain, one can choose a patch of
pixels and monitor their average value though time. In other words, by choosing a
patch of pixels, we virtually decrease the ratio of displacement to the pixel size which
makes the change of intensity smoother. Patch processing can be applied to videos
31
where no targets are used or combined with LGPTs as discussed in more detail in
Section 2.5.
3.3.4 Linear Gradient Pattern Targets
In this study the idea of using linear gradient pattern targets (LGPT) mounted
to locations of interest on the structure is proposed. For this case, we do need access to
the structure in order to mount the targets, which may require ladders or lifts,
depending in the situation. Also, the target needs to be oriented in the expected
direction of motion to capture the vibration amplitudes accurately. We employed
LGPTs with different sizes in our experiments as shown in Fig. 3.2 a). The idea of
these targets is to create a well-defined, linearly-varying background to avoid non-
linear behavior as discussed in Section 3.2. The criteria for size is to optimize
maximum amplitude of vibration, A with the length of the target, L. A typical cross
section of an LGPT as it is captured and represented by the camera is shown in Figure.
3.2 b). The intensity value, although designed to be linear, contains noise as is shown
in Figure. 3.2 c). This noise was computed by subtracting the linear curve from the
captured intensity curve. A histogram of the noise is shown in Figure. 3.2 d).
32
Figure 3.4: Two noised reduction strategies for LGPTs: a) Patch on an LGPT to
average out the noise b) linear regression approach: measured intensities
are mapped onto linear curve-fit. (Figure courtesy of Thomas
Schumacher)
3.3.5 Noise Reduction Strategies for LGPTs
A strategy to reduce the noise would be to average the intensities of a patch of
pixels on the LGPT as shown in Figure. 3.4 a). From Figure. 3.2 d) it can be seen that
the average of the noise is close to zero so it can be deduced that by averaging the
pixel values, we essentially reduce the noise. Another strategy for noise reduction is to
choose random pixels on the gradient and fitting a linear function through them
(Figure. 3.4 b)). Tracking the constant part of this linear function through time can
lead to a much less noisy signal, in the case of our laboratory experiments it improved
the SNR by 3 to 6 dB. The requirement for these computationally inexpensive noise
reduction techniques is that during the whole vibration phase, the selected pixels
should never leave the LGPT range, otherwise artificial non-linear behavior is
introduced. A solution to this is to employ LGPTs that consist of several patterns with
different lengths as shown in Figure. 3.2 a). The most appropriate target can then be
33
picked after the digital video has been collected, which is one of the advantages of this
approach.
3.4 Laboratory Experiments
3.4.1 Experimental Test Setup
Two laboratory experiments were performed: Free vibration of a laboratory-
scale three story structure as shown in Figure. 3.1 a) and free vibration of a simply-
supported steel beam (Figure. 3.1 b)). In the first test, an initial displacement was
manually imposed on the structure by hand. Following a sudden release, the system’s
free vibration was recorded until it was damped out. The 3.6 m long simply-support
steel beam with a W15x87 cross-section was struck with an instrumented hammer to
impose structural vibrations. In both experiments, high-precision capacitive
accelerometers were attached to the structures to verify the frequencies estimated from
the VVS data.
3.4.2 Three-Story Structure Tests
3.4.2.1 Cameras Used
Three different cameras were used for the laboratory experiments. For the
three-story structure experiment, a GoPro Hero 3 camera and a Photron UX100
(Figure. 3.5 a) and c)) were used. The resolution of the GoPro camera was 1280 x 720
pixels and the frame rate was 120 fps. The Photron camera was used with 500 fps and
its full resolution of 1280 x 1024 pixels to evaluate the ability of detecting higher-
order frequencies with high-speed cameras. Finally, for the beam experiment, and to
explore the limits of our proposed methodology, a Photron FASTCAM SA-X2 (Figure
34
3.5 b)) with 5000 fps and its full resolution of 1024 x 1024 pixels was employed. It is
important to note that there is a trade-off between resolution and frame rate due to the
bandwidth limit of the camera hardware. Also, based on our own observations, the
spatial noise power in high-speed cameras is relatively high, i.e. higher than in regular
cameras. Three LGPTs with dimensions 8 x 60 mm were attached to the three
different masses of the three-story structure as shown in Figure. 3.1 a). For our
analysis VVS data collected from the first story mass as it was best suited for the size
of our LGPTs is used.
Figure 3.5: Cameras used for the laboratory experiments: a) Photron FASTCAM
UX100, b) Photron FASTCAM SA-X2 and c) GoPro Hero 3.
3.4.2.2 Reference Data from Accelerometers
Two high-fidelity capacitive accelerometers were attached to the side at the
height of masses two and three (see Figure. 3.1 a)) and sampled at 1 kHz using a high-
speed data recorder. The natural frequencies computed from the acceleration data from
the second and third story was essentially the same for all of the experiments, as
shown in Figure. 3.6 a) and b), respectively. The only difference in the frequency
domain was that the magnitude of the peaks slightly varied. This, however, had no
influence on the value of the peak frequency. The fundamental frequencies of
35
vibration were found to be 4.70 Hz, 14.0 Hz and 20.9 Hz for both stories. It can be
observed that the SNR for the second story is higher, which is due to the fact that the
third and second natural frequencies of vibration have a much stronger contribution
than for the third (= top) story.
Figure 3.6: Sample data from the accelerometers: a) Second story and b) third story.
The left and right column shows data in the time and frequency domain,
respectively.
3.4.2.3 Results from GoPro Camera
Fig. 3.7 shows the results from the GoPro Camera without the use of LGPTs.
In Fig. 3.7 a), it can be seen that the first and second peak frequencies are detectable
but at the same time there are several higher harmonics in the frequency domain which
made it difficult to choose the right natural frequency. By using a patch of 50 x 50
pixels, however, it is possible to detect all of the natural frequencies (Figure. 3.7 b))
36
bearing in mind that other peak frequencies are just multiples of the first one.
Although the magnitude of the third mode is not very large, it is still detectable
(Figure. 3.7 b)). The duration of the signals are about 10 s, which produces a
resolution in the frequency domain of 0.10 Hz.
Figure 3.7: Data extracted from videos taken by the GoPro camera without LGPTs:
a) One pixel in the middle of first floor, and b) a 50 x 50 patch of pixels
in the middle of the first floor.
As proposed, linear gradient pattern targets (LGPT) should significantly
improve the results in Figure 3.7. By selecting a pixel on the gradient target of the first
floor, all three natural frequencies could be recovered as is evident in Figure 3.8 a).
Figure 3.8 b) shows the same data processed using a patch of 5 x 5 pixels on the LGPT
which reduced the noise by 2.7 dB. Figure 3.8 c) shows the data when the linear
regression approach is employed. As can be observed, this processing step is capable
37
of reducing the noise even better than the patch if applied for the case when LGPTs
are used.
Figure 3.8: Data extracted from videos taken by the GoPro camera with LGPTs: a)
One pixel on the gradient target, b) A patch of 5 x 5 pixels on the target
c) Linear regression applied to 10 randomly chosen points on the
gradient.
As can be observed from the inserts in Figure 3.8 in the frequency domain,
because of the relatively low SNR in Figure 3.8 a), accurate estimation of the third
peak frequency is difficult. Hence, the small difference in the third peak frequency can
be associated with the low SNR. As can be seen in Figure 3.8 b) and Figure 3.8 c), this
38
problem is resolved using patch processing and linear regression, respectively, which
notably improved the SNR of the signal. It should be noted that the use of LGPTs
improves the contrast and decreases quantization intervals, simultaneously.
Comparing the time history part of Figure 3.8 and Figure 3.7 shows that the range of
change in intensity values is much larger when LGPTs are used. This, as previously
mentioned, helps to reduce the effect of quantization noise, which is partially
responsible for missing peak in the frequency domain (Figure 3.7 a)).
3.4.2.4 Results from Photron FASTCAM U100 Camera
As discussed before, high-speed cameras can help improve the signal-to-
quantization noise ratio. For high-speed cameras, picking a pixel at the bottom of the
three-story structure without any noise reduction strategy can reveal all of the natural
frequencies (Figure 3.9 a)). Selecting a patch of pixels on the other hand will produce
a much less noisy signal and avoid the presence of artificial peaks in the frequency
domain as shown in Figure 3.9 b). As described before, the Photron FASTCAM U100
was used for this experiment. The problems associated with this type of camera are
their limited storage, which leads to shorter recording time, limited bandwidth, which
results in a sacrifice of spatial resolution with higher temporal resolution, and also
higher spatial noise. Also, the higher the frame rate, the better illuminated the medium
should be in order to have high-quality videos. The use of LGPTs for high-speed
cameras can be beneficial as well. As can be seen from Figure 3.9 c), although the
signal is noisy, the peaks are more pronounced. Using a patch of pixels on the LGPT
(Figure 3.9 d)) reduces the noise, increases SNR by 9 dB and shows the peaks even
clearer. Also linear regression can increase SNR by almost 12 dB (Figure 3.9 e). This
40
Figure 3.9: Data extracted from videos taken by the FASTCAM UX100 camera with
and without LGPTs: a) One pixel at the very bottom, b) a patch of pixels
at the mid-level of the first floor, c) one pixel on the LGPT, d) a 5 x 5
pixel patch on the LGPT, and e) linear regression applied to 10 random
pixels on the LGPT.
41
3.4.2.5 Comparison of Results
Table 3.1 shows the SNR for all experiments on the three-story structure. In
the case of the high-speed camera, the level of SNR of one pixel where no LGPTs are
used is artificially higher due to the exponential decay of the signals where LGPTs are
used. Also, a comparison of cases using low- and high-speed cameras with LGPTs
evidently shows an increase in the SNR from 3.4 to almost 6 dB, which is close to the
theoretical bound due to the improved SQNR. It should be noted that the setup of this
experiment with high speed camera was a little bit different from the previous tests
and that the duration of the signal was 7 s, which gives a resolution around 0.14 Hz.
These factors explain the slight difference in third peak frequency in Figure 3.9, and
the accelerometers’.
Table 3.1: Comparison of SNR from three-story structure.
Capacitive Accelerometer (Reference) with 1 kHz
Second story 31.8
GoPro Camera with 120 fps
Without LGPT With LGPT
One Pixel 14.7 18.0
Patch of pixels 17.1 20.7
Linear regression - 26.0
FastCam UX-100 with 500 fps
One Pixel 21.7 17.4
Patch of pixels 23.4 26.6
Linear regression - 29.4
42
3.4.3 Steel Beam Tests
Finally, and in order to explore the limits of our proposed methodology, we
conducted a test on a steel beam as shown in Figure 3.1 b). The stimulus was provided
by a hammer strike imposed at the mid-span location. In this test, as shown in Figure
3.10 a) and b), several peak frequencies were deducible from the accelerometer data.
These reference data were collected using the same high-precision capacitive
accelerometer as used for the three-story structure. Monitoring a patch of pixels at the
boundary of the steel beam where the gradient of the intensity is maximum (the edge),
it was possible to detect several peak frequencies in congruence with the measured
accelerometers’ peaks.
Figure 3.10: The steel beam test results: (a) Results from the FASTCAM SA_Z
camera, and (b) results from one accelerometer.
Figure 3.10 a) shows the peak frequencies detected by the camera, while
Figure 3.10 b) show the data from the accelerometers for comparison. As can be seen,
even without the LGPT, by using a patch of pixels, several peak frequencies could be
43
detected. The interesting point about this experiment is that it involves a continuous
system where the high-frequency displacements are extremely small, completely
unobservable by the naked eye. However, it was possible to identify frequencies as
high as 764 Hz using our proposed VVS methodology. The resolution in the frequency
domain is approximately 1 Hz.
3.5 Field Test
In order to evaluate the real-world performance of our proposed approach, we
conducted a field test on the Streicker Bridge (Figure 3.11 a)): a prestressed concrete
pedestrian bridge located on Princeton University’s campus in Princeton, NJ. As can
be seen from Figure 3.11 a), the bridge has a unique design with a main span and four
horizontally curved legs. The main span consists of a deck-stiffened arch. The bridge
is equipped with a structural health monitoring (SHM) system consisting of embedded
fiber-optic sensors. The data were made available to us by Prof. Branko Glisic and
allowed for a direct comparison with our measurements. The dynamic stimulus was
provided in form of a group of students jumping in unison for a few seconds at the
location of our LGPTs.
44
Figure 3.11: Real world application a) View of the Streicker Bridge and b) test setup,
camera position and LGPTs (insert).
LGPTs were mounted on the inside of a curved leg to measure vertical
vibrations while the cameras were on the other side of the street, approximately 8 m
away from the targets. The camera used was a Canon T4i with 60 frames per second
and 128 x730 resolution. The test was performed on April 23, 2014, with adequate
lighting conditions and some wind. We verified that the wind did not affect the
measurements by comparing several measurements taken at different instances in
time. Figure 3.12 shows the results in the frequency domain for both measurements.
As can be observed, the two main frequencies of vibration of the leg, namely 3.0 Hz
and 3.6 Hz, were detected by both sensing approaches. The low-frequency content in
the VVS data (Figure 3.12 a)) can possibly be explained by slightly periodically-
varying lighting conditions due to trees rocking with the wind. In this case this is not a
real issue since the motion did not result in an additional peak that could be
45
misinterpreted. Although it was not possible in this case to calculate a SNR, it can be
observed that the two frequency plots are of very comparable quality.
Figure 3.12: Frequency response of the Streicker Bridge from (a) the VVS located on
an LGPT and (b) the fiber-optic sensor system.
3.6 Conclusions and Outlook
In this chapter, we evaluate and discuss a number of strategies to detect higher
frequencies of vibration using the earlier proposed Eulerian-based virtual visual
sensors (VVS) [21]. It can clearly be seen that, based on the experiments, the use of
linear gradient pattern targets (LGPT) and high-speed cameras can improve the signal-
to-noise-ratio (SNR) and help detecting multiple frequencies in multi-degree-of-
freedom (MDOF) structural and mechanical systems. From the study, we conclude the
following:
The introduction of LGPTs increases the SNR and enables detecting higher
natural frequencies which is particularly helpful when standard digital
video cameras are use.
Analyzing a patch of pixels rather than a single pixel can be employed
when no LGPTs are used to smooth the change of intensity, i.e. minimize
impulse-type response in the signal.
46
By analyzing a patch of pixels or apply a linear regression approach, the
SNR of LGPTs can further be improved.
While high-speed camera technology is still expensive and mostly used by
researchers, the use of commercially-available digital video cameras in
conjunction with LGPTs allow for accurate and reliable detection of
multiple natural frequencies.
High-speed cameras benefit from lower noise amplitude due to
oversampling and are able to detect higher frequencies even without
LGPTs.
The methodology also works in the field where we found the same peak
frequencies compared to the existing structural health monitoring (SHM)
system.
A limitation of the approach is that it can only accurately capture vibrations
that are perpendicular to the line of view of the camera and in the direction
of the LGPT.
Correlation between intensity and actual displacement is considered in the next
chapter. Evaluation of signal to noise ratio, the effect and the distribution of noise is
investigated.
47
EULERIAN-BASED VIRTUAL VISUAL SENSORS TO MEASURE DYNAMIC
DISPLACEMENTS OF STRUCTURES
The pixels in the vicinity of the boundary of a vibrating structural element
contain useful frequency information, which we have been able to demonstrate in
earlier chapters. The ultimate goal, however, is to be able to compute dynamic
displacements, i.e. actual displacement amplitudes in the time domain. In order to
achieve that the use of simple black-and-white targets (BWT) that are mounted to
locations of interest on the structure are introduced. By using these targets, intensity
can be directly related to displacement, turning a video camera into a simple,
computationally inexpensive, and accurate displacement sensor with notably low
signal-to-noise ratio (SNR). We show that subpixel accuracy with levels comparable
to computationally-expensive block matching algorithms can be achieved using the
proposed targets. The methodology can be used for laboratory experiments, on real
structures. In this chapter the concept and theory of the proposed methodology is
introduced, laboratory experiments to evaluate the accuracy of the proposed BWT
target are presented, and the results from a field test of an in-service bridge are
discussed. This chapter was submitted as a paper to the Journal of Structural Control
and Health Monitoring and is currently in review.
Chapter 4
48
4.1 Introduction
Digital image correlation (DIC) and other block matching algorithms use
digital video data to measure static displacement fields with high accuracy[43], [44].
However, the computational cost of these methods is relatively high. Efficient yet
accurate non-contact methods are needed that are computationally inexpensive and
work with standard digital video cameras. In this chapter, we propose a simple
alternative way to measure structural vibrations using Eulerian-based virtual visual
sensors (VVS), for which the fundamental basis that have been developed earlier
chapters [21], [24]–[27] It is shown that for a black-and-white target (BWT), a linear
intensity-displacement relationship exists for a patch of pixels on the boundary of the
target. Which is referred to as intensity-to-displacement transform (IDT). It should be
noted that this same transform can also be applied without using BWTs, for example
to the edge of a structural member where a distinct boundary between the member and
the background exists. Figure 4.1 illustrates the concept of our proposed methodology:
a BWT target is attached to a location of interest. From the digital video extracted
from the camera, a VVS located on the BWT is selected. The change of intensity is
recorded over time (green curve) and by employing a linear transformation (= IDT)
converted to actual displacement (blue curve). Also shown is a frequency analysis (red
curve), which can be obtained directly from the intensity time history by using a Fast
Fourier transform (FFT). The focus of this chapter is on how to compute displacement
time histories (blue curve) and the nature of the error in such measurement.
49
Figure 4.1: Illustration of the proposed methodology to measure structural vibrations
using Eulerian-based virtual visual sensors (VVS). Acronyms: BWT =
black-and-white target, FFT = Fast Fourier transform, IDT = intensity-to-
displacement transform. (Courtesy of Thomas Schumacher)
4.2 Background and Theory
As it is discussed in previous chapters [18, 20], change of intensity observed
by the Eulerian-based virtual visual sensor (VVS) did not directly correspond to a
physical quantity such as displacement, velocity, or acceleration. In this chapter, the
use of simple printed black-and-white targets (BWT) that allow the measurement of
actual dynamic displacements, as visualized in Figure 4.2, is evaluated. For an ideal
BWT target observed in a digital video with dimensions Lt x Wt (pixel x pixel), the
pattern colors are represented by the minimum and maximum intensity values (Imin.
and Imax., respectively) corresponding to 0 (= black) and 255 (= white), respectively.
The displacement varies linearly with VVS patch intensity, Ip(t):
50
max min
p p
p
I II t x t n t
L
(4.1)
Where Ip(t) is the average pixel intensity across the patch area, Ap = Wp x Lp
(pixel x pixel) for a frame at time instant, t (sec), x(t) (pixel) measured displacement,
and Lp (pixel) is the length of the patch. It is assumed that intensity across the width of
the patch, Wp (pixel) is constant and it therefore does not appear in Equation. (4.1). It
should be noted that the patch length, Lp (pixel) should be large enough to account for
the maximum displacement amplitude, A (pixel), i.e. Lp > A. At the same time, the
length of the target, Lt (pixel) needs to be able to accommodate for the patch length, Lp
(pixel), i.e. the target cannot leave the patch, otherwise the relationship becomes non-
linear. Finally, there is no perfect BWT target (with perfect black (I = 0) or white (I =
255) intensity values) in a real setting and measurement noise is always present. The
total average noise of the patch, np(t) can be defined as:
1
1 N
p i
i
n t n tN
(4.2)
where N is the total number of pixels in the VVS patch and ni is the noise present in
pixel i.
51
Figure 4.2: Illustration of the VVS measurement process using a black-and-white
target (BWT): the target, which is attached to the vibrating structural
element, moves in the x-direction relative to a fixed patch of pixels, i.e.
having Eulerian-coordinates, as a function of time, t. (Figure courtesy of
Thomas Schumacher)
As an alternative to Equation. (4.1), VVS patch intensity, Ip(t) can also be
computed as the average intensity value of all pixels across the patch area, Ap:
1
1 N
p i
i
I t I tN
(4.3)
where N is the total number of pixels in the VVS patch and Ii is the intensity value of
pixel i.
52
Figure 4.3: Illustration for the case where the camera is not oriented perpendicular to
the displacement component of interest, uact. O denotes the camera
location and T the center location of the target mounted to the vibrating
structure. a and b represent horizontal and vertical distance between the
camera and the center of the target. (Figure courtesy of Thomas
Schumacher)
For the case where the camera is not oriented perpendicular to the
displacement component of interest, a geometric correction factor, C applies. This
factor is calculated based on the location of the camera (O) and the center location of
the target (T), as illustrated in Figure 4.3:
1
1
cos tan
Cb
a
(4.4)
Considering that Ip(t) is known and by introducing a calibration constant, B
(pixel/mm), which can be estimated from a frame extracted from the digital video, and
the geometric correction factor, C, the actual dynamic displacement of the target,
uact(t) can be computed using the following relationship:
act pu t B C I t in (mm) (4.5)
53
The presence of noise may require implementation of a noise reduction
technique. Fortunately, for a BWT the averaging process (expressed by Equations.
(4.1) and (4.3)) by itself helps reducing the noise, i.e. the power of the noise reduces
directly with the number of pixels in the patch, N. Assuming that the noise is
independent of the signal and can be represented by a stationary process, we arrive at:
2 2
max min2 2
2
p
p
p
nI II x
L N
(4.6)
Equation (4.6) relates the size of the VVS patch and the power of the noise and
the signal. As can be seen from the second term of the right hand side of Equation
(4.6), as the number of pixels increases the power of the noise decreases. However,
increasing the length of the patch will have the same effect on the power of the signal,
hence keeping the length as short as possible and the width as large as possible will
maximize the SNR. Substituting N for Wp.Lp in Equation (4.6), which is width x
length of the patch, we can get the following expression for the signal-to-noise ratio
(SNR):
2
2max min
2 22
max min2 2
p p
pp p
p p
I Ix
L W xSNR I I
Ln n
W L
(4.7)
Equation (4.7) shows that in order to reach the maximum SNR one has to
maximize the Wp-to-Lp ratio keeping in mind that Lp should be able to cover the
maximum displacement amplitude, A, as discussed earlier. For a specific camera and
lighting conditions, the pixel noise power can be assumed constant and the appropriate
size of the patch can be specified based on the desired SNR. The other factor that
should be discussed in Equation (4.7) is the second factor, max minI I , which has a
54
more significant effect on the SNR. It can be concluded from Equation (4.7) that the
higher the contrast between black and white in the target, the higher the SNR will be.
Using a confidence level and the noise power in a pixel the patch size for the
desired accuracy can be calculated. Assuming that noise has zero mean and in each
pixel it is independent and identically distributed (iid) RV, based on central limit
theorem we can say that as the patch size become bigger the average noise converge to
normal distribution. Using Equation (4.1) we’ll have:
−1
2𝛼
(𝐼𝑤ℎ𝑖𝑡𝑒 − 𝐼𝑏𝑙𝑎𝑐𝑘)
𝑙≤ 𝑛𝑝 ≤
1
2𝛼
(𝐼𝑤ℎ𝑖𝑡𝑒 − 𝐼𝑏𝑙𝑎𝑐𝑘)
𝑙 (4.8)
In which 𝛼 is the desired accuracy level with 𝛼 = 1 giving the pixel level
accuracy and 𝛼 < 1 the subpixel accuracy. Assuming the normal distribution for the
noise in the patch with zero mean and standard deviation of 𝜎2(𝑛𝑖)
𝑁 and defining the
confidence level 𝛽 with 𝐾𝛽 = Φ−1(1 −𝛽
2) we can write:
𝑃(−𝐾𝛽 < 𝑛𝑝
𝜎(𝑛𝑖)
√𝑁
< 𝐾𝛽) = 1 − 𝛽 (4.9)
So then with the confidence level 𝛽 we can say if:
𝐾𝛽 . 𝜎(𝑛𝑖)
√𝑁=
1
2𝛼
(𝐼𝑤ℎ𝑖𝑡𝑒 − 𝐼𝑏𝑙𝑎𝑐𝑘)
𝑙 (4.10)
Which is equivalent to:
√𝑙
𝑤=
1
2𝛼
(𝐼𝑤ℎ𝑖𝑡𝑒 − 𝐼𝑏𝑙𝑎𝑐𝑘)
𝐾𝛽 . 𝜎(𝑛𝑖) (4.11)
Then we will have:
𝑃 (−1
2𝛼
(𝐼𝑤ℎ𝑖𝑡𝑒 − 𝐼𝑏𝑙𝑎𝑐𝑘)
𝑙≤ 𝑛𝑝 ≤
1
2𝛼
(𝐼𝑤ℎ𝑖𝑡𝑒 − 𝐼𝑏𝑙𝑎𝑐𝑘)
𝑙) = 1 − 𝛽 (4.12)
55
Using the Equation 4.8 one can specify the size of the patch based on the desired
accuracy (𝛼) and the level of confidence 𝛽.
4.3 Experiments
4.3.1 Laboratory Setup and Instrumentation
A laboratory-scale three-degree-of-freedom structural system (Total height =
610 mm (2 ft) as shown in Figure. 4.4 (d) was used to evaluate the accuracy of the
methodology proposed in Section 4.1. The structure was excited by introducing
random initial displacements at two locations on the structure by hand followed by a
sudden release to initiate free vibration. A digital camera (GoPro Hero 3, shown in
Figure 4.4 (c))) capturing the free vibration response was located 305 mm (1 ft) away
from the structure. The displacement of the first floor was also measured using a 12.7
mm (0.5 in) amplitude potentiometer (Figure 4.4 (b)) connected to a high-speed data
acquisition system (Figure 4.4 (a)) using a sampling frequency of 1200 Hz. The frame
rate of the digital camera was 60 frames per seconds (fps).
56
Figure 4.4: Experimental setup: (a) high-speed data acquisition system, (b)
potentiometer to measure displacements at the first story mass, (c) digital
camera to collect VVS data, and (d) three-degree-of-freedom laboratory
structure.
4.3.2 Data Acquisition and Data Preprocessing
In order to compare the two measured signals, two steps have to be taken: (1)
synchronization of the signals in the time domain and (2) multiplication of the
measured signals by their appropriate calibration factors to obtain actual displacement
from the measured data. For step (1), both camera and potentiometer data were
interpolated linearly to two equivalent 3000 Hz signals. Based on the maximum
correlation between the two signals, the time lag between the two signals was
calculated and one of the signals shifted so that they had a common time axis. In order
57
to achieve actual displacement for step (2), the potentiometer was calibrated against a
precision height gage. The mean calibration factor was found to be 1.257 mm/V
(0.0495 in/V). For the camera intensity data, the known target length, Lt was measured
in a video frame in terms of pixels, which produced a mean calibration factor of 0.279
mm/pixel (0.011 in/pixel).
In addition to the independent application of calibration factors as described
above, it was also possible to multiply the VVS intensity data by a factor that
minimizes the second norm of difference, or error, between the two measurements.
This case represents the optimal estimate of the displacement for the VVS, assuming
the potentiometer represents an accurate reference measurement. Obviously, in a real
life scenario only the first approach can be used where a calibration factor has to be
estimated from the video data. It should be noted that the potentiometer does not
necessarily produce the more accurate displacement. This was particularly visible at
the peak displacement points and is discussed in more detail in Section 4.4.1.
4.4 Results
4.4.1 Accuracy of Proposed Approach
Two VVS patch sizes, Wp x Lp = 40 x 50 and 40 x 100 pixels, were selected to
study how the accuracy of the measurements change with the size of the VVS patch,
Ap. Figure 4.5 shows a snapshot of a video frame with the target and the two evaluated
VVS patch sizes.
58
Figure 4.5: Photo of BWT target with two VVS patch sizes: (a) 40 x 50 pixel patch
and (b) 40 x 100 pixel patch.
Figure 4.6 shows a comparison of a sample measurement using independent
calibration factors, as described in Section 4.3.2. As can be observed from Figure.4.6
(a), the displacements measured by the VVS and the potentiometer are, qualitatively,
in close agreement for both patch sizes. However, the inserts in Figure 4.6 (a) reveal
that the end of the signal of the 40 x 100 pixel patch resembles the potentiometer’s
measurement more closely. A direct correlation between the two measurements
(Figure 4.6 (b)) shows approximately a straight line with a slope of 1 (black dashed
line). It is also evident that the patch size with the greater Wp-to-Lp ratio correlates
better with the potentiometer’s measurement. Furthermore, in Figure 4.6 (c), which
shows the absolute value of the difference between the two measurements, less than
2% of the signal difference is greater than a pixel size and roughly 90% of the time the
difference is less than half of a pixel size. It can further be observed that the difference
shows distinct evenly-spaced peaks that are highest at the beginning of the signal.
Also, they appear to coincide with the peak amplitudes of the signal. The difference is
likely due to an error in the potentiometer measurement, when the direction of the
displacement changes. Unfortunately, it was not possible to ascertain this claim
59
completely. Despite this uncertainty, the data shows that subpixel-level accuracy is
achievable with high confidence. The actual difference in terms of noise can be
observed in Figure 4.6 (c), between 4 and 6 s of the difference. In addition, it should
be mentioned that the maximum error can further be reduced by increasing the Wp-to-
Lp ratio of the patch. The distribution of the error with mean, and standard deviation,
is shown in Fig. 6 (d). It can be observed from the distribution of the signal
difference that it appears to follow a Normal distribution, as assumed in Section 4.2.
60
Figure 4.6: Comparison of results using independent calibration factors, 40 x 50
pixels (left column) and 40 x 100 pixels (right column): (a) Time history
signals of VVS patch data and potentiometer, (b) correlation between the
two measurements, (c) absolute value of the difference between the two
signals (errors), and (d) histogram of the errors.
61
Figure 4.7 shows the case where the calibration factor for the VVS was
optimized as discussed in Section 4.3.2. Figure 4.7 (a) compares with Figure 4.6 (a)
while correlation plots shown in Figure 4.7 (b) are even better compared to Figure 4.6
(b). The maximum signal difference (Figure 4.7 (c)) is reduced by almost half of a
pixel size as compared to Figure 4.6 (c). Comparing the patch sizes in Figure 4.7 (d), it
can be observed that the standard deviation of the pixel error has been significantly
decreased from 0.23 pixels to 0.12 pixels for the 40 x 50 pixel patch compared to the
40 x 100 pixel patch, respectively. Also, with a confidence of more than 90%, the
error in the smaller patch is less than one third of a pixel size while in the bigger patch
it is less than one fifth of a pixel size. Again, this approach represents the case where
the calibration factor for the VVS sensor was optimized by minimizing the difference
between the two measurements.
In conclusion from Figures 4.6 and 4.7, we have demonstrated that subpixel
accuracy can be achieved with high confidence even without implementing a
computationally-expensive block matching algorithm.
62
Figure 4.7: Comparison of results using calibration factors based on minimized
difference between measurements, 40 x 50 pixels (left column) and 40 x
100 pixels (right column): (a) Time history signals of camera and
potentiometer, (b) correlation between the two measurements, (c)
absolute value of the difference between the two signals (error), and (d)
histogram of the error.
63
Table 4.1 summarizes the main results of the accuracy evaluation presented in
this section. It can be seen that the larger VVS patch (40 x 100 pixels) was closer to
the potentiometer reading compared to the smaller patch (40 x 50 pixels). For the
displacements computed using the independent factors, the larger sized VVS patch
was closer to the potentiometer measurement. However, the standard deviation of the
difference for the larger patch remained the same because of the calibration issues
explained earlier.
Table 4.1: Summary table of accuracy evaluation.
Independent
calibration factors
(see Fig. 5)
Minimization of signal
difference
(see Fig. 6)
Size of the patch 40 x 50 40 x 100 40 x 50 40 x 100
Correlation Coefficient 0.998 1.000 0.998 1.000
Maximum difference in pixel size 1.5 1.2 1.5 0.6
Mean of the difference 0.105 0.007 0.018 -0.014
Standard deviation of the
difference 0.277 0.300 0.236 0.124
4.4.2 Relationship of Noise and Patch Size
Figure 4.8 (d) shows the relationship between patch noise and number of pixels
in a VVS patch as defined theoretically by Equation (4.7) and observed
experimentally. As can be seen in Figure 4.8 (a), the power of the noise is close to the
theoretical values. Figure 4.8 (b) and (c) show the distribution of the noise for one
pixel and a patch of 10 x 10 pixels. As can be observed, the noise in the patch follows
a Normal distribution and its power is one order of magnitude smaller than that for one
pixel. Also, the SNR values approximately change linearly with the width to length
64
ratio of the patch as predicted from Equation (4.7). This validates the theoretical
framework presented in Section 4.2.
Figure 4.8: Noise power and the signal-to-noise ratio (SNR): (a) Power of the noise
vs. the number of pixels in the VVS patch (N), (b) histogram of noise in
one pixel, (c) histogram of noise in a 10 x 10 pixel patch, and (d) the
SNR values vs. width over length of the patch.
4.4.3 Dynamic In-Service Load Test on the Streicker Bridge
In order to evaluate the applicability of the proposed methodology in a real-
world scenario, the Streicker Bridge was tested dynamically with a black-white target
(BWT) to compute displacement. Located on the Princeton University campus, the
bridge has a unique design with a straight main deck section supported by a steel truss
system underneath and four curved ramps leading up to the straight sections, as shown
65
in Figure 4.9. One of the ramps was instrumented with a fiber-optic measurement
system during construction by Br. Branko Glisic from Princeton University [45]. For
the test, we installed an off-the-shelf Canon EOS Rebel T4i camera with a standard
Canon EF 75-30mm zoom lens aimed at one of the ramps, to take a 60 fps video while
a number of volunteers jumped up and down on it. A VVS patch having 60 x 20 pixels
was chosen to compute displacements. As can be seen in Figure 4.9, the VVS is
located at a black-and-white edge on a target mounted to the edge of the bridge slab.
This target was set up by Dr. Maria Feng’s research team from Columbia University,
who collected data for evaluation of their own video-based monitoring methodology
[46]. The relationships in Equations (4.1) and (4.5) were used to compute the actual
vertical dynamic displacement from the collected VVS patch. The calibration
coefficient, was estimated from the target size as 17.3 (pixel/mm); the geometric
correction factor, C estimated to be 1.02.
66
Figure 4.9: Photograph of the Streicker Bridge showing the measurement setup and
the location of the VVS. The insert shows the location of the 60 x 20
pixel VVS patch (red rectangular). The target was installed by Dr. Maria
Feng’s research team from Columbia University [46].
The computed vertical displacement response of the ramp section due to the
described dynamic forcing for a duration of 15 seconds is presented in Figure 4.10. In
the earlier chapter it is already reported that the natural frequencies for this same test
were found to be the same as those measured by the fiber-optic measurement system.
67
Although there were no other physical measurement available to directly compare and
verify our computed displacement, it is comparable in amplitude to what the Columbia
University team reported[46]. Also, the frequency peak is exactly the same as reported
by the same group.
Figure 4.10: Results from the dynamic load test on the Streicker Bridge: (a) Computed
actual vertical displacement time history and (b) frequency response of
signal (a).
4.5 Discussion and Conclusions
The objective of this study was to evaluate the possibility of computing actual
dynamic displacements using Eulerian-based virtual visual sensors (VVS). This is
based on the idea that either an edge of a vibrating structural element or a black-and-
white target (BWT) can be monitored by a patch of pixels. The noise in the VVS
68
sensor was found to be inversely related to the patch size. The following conclusions
can be made from on our study:
The use of BWT allows for accurate computation of dynamic
displacements of a vibrating structural element.
The laboratory tests demonstrated that sub-pixel accuracy can be achieved
similar to block matching algorithms.
The accuracy in the measurement of the displacement implies that change
of intensity is highly sensitive to even tiny amounts of movement, which
results in the fact that natural frequencies can be measured as proposed in
[21], [25], [26] even if the displacement is much less than a pixel size.
The proposed approach also works in the field, as demonstrated by the
measurements of the Streicker Bridge. A direct validation was not possible
since no other physical displacement data was available, which is typically
the case for field measurements. However, the frequency content of the
signal has already been verified and the displacement amplitude as well as
the frequency peak is comparable to what the team from Columbia
University found [46].
The influence of camera movement and changes in lighting conditions
need to be addressed further in future research.
69
MEASUREMENT OF TENSION IN CABLES USING VIRTUAL VISUAL
SENSORS
A total of 48 cable-stayed bridges and 87 suspension bridges [47], [48] are
currently in service in the United States and as Scientific American [49] points out, we
have to expect the number of the cable-stayed bridges to increase in the future. Having
a critical role in transferring the load from the deck to the pylons, cables need to be
monitored over time fir changes in their tensile force. A way to do this non-
destructively is by measuring the natural frequencies of vibration and estimating from
those the tensile force using the well-known taut cable theory [50]. However, the
implementation and maintenance cost of traditional sensors such as accelerometers for
monitoring bridge cables is relatively high. This cost includes, and is not limited to,
the cost of sensor hardware, its installation and wiring, the data acquisition system,
and data storage, management, and interpretation. As shown in the previous chapters,
standard inexpensive digital video cameras can be used to measure natural frequencies
employing a simple algorithm with high accuracy and low computational cost. In this
chapter, the theories that relate the tensile force to the natural frequencies of a cable
are briefly explained. Controlled laboratory experiments were conducted to investigate
the above-mentioned theory and the capability to measure the natural frequencies of
vibration. Finally, the tensile forces of a cable in a cable-stayed bridge was estimated.
Chapter 5
70
It is demonstrated that digital video cameras can be successfully used to measure
natural frequencies and hence estimate tensile forces of in-service stay cables.
5.1 Introduction
Several structure types including long-span bridges such as stay-cable and
suspension bridges, lift bridges, and communication antenna towers employ cables for
load transfer mechanisms. Relaxation of the tensile force in the cables can cause
critical stresses in other parts of the structure and lead to damage. Playing a critical
role in these types of lifeline structures, cables need to be monitored using non-
destructive approaches to estimate their tension force. Vibration-based methods, which
use natural frequencies of vibration to calculate the tensile force of cables have been
developed. The theory of inextensible cables goes back to the 1730s [51]. Irvine and
Clough refined the linear theory of free vibration in cables by taking into account their
extensibility [51]. Shinke et al. proposed simplified formulae to estimate the tension
force using lower modes of vibration considering the cable’s flexural rigidity [52].
Experimental, numerical, and heuristic methods have been widely investigated to
study various parameters such as flexural rigidity, sag, and the boundary conditions on
the natural frequencies of the cables [53]–[63].
Measurement of natural frequencies in cables are usually done through
acceleration response measurements. This requires that a number of accelerometers
are attached to the cables and connected to a data acquisition system that records the
data. The cost of the instruments and their installation and maintenance cost are
significantly higher compared to the cost of a digital video camera. Also, finding the
right location to get the best signal to noise ratio (SNR) because of the high local
sensitivity of these sensors is another issue. Digital videos have been successfully used
71
for the measurement of displacement fields. Block matching algorithms such as digital
image correlation (DIC) replace the cost of traditional sensors but come with high
computational cost [43], [44]. This research has demonstrated that small changes of
color intensity in each pixel of a video carries essential information about the
vibrations of a structural element. In the previous chapters it has been shown that by
implementing Eulerian-based virtual visual sensors (VVS), measurement of
frequencies even with a subpixel amount of displacement is feasible [21], [24]–[27].
In this chapter, the linear theory of taut cables is reviewed and a practical
formula that is applied to correlate natural frequencies to the tensile force is utilized.
Laboratory experiments that are conducted for this correlation and the basic
observations in these experiments are discussed in Section 5.3. Finally, the Indian
River Inlet Bridge was used as a real-world application example. First, the natural
frequencies for three cables are compared with accelerometer measurements from
previous tests. Based on the discussed formulation in Section 5.2, a tensile force was
estimated, which is compared against the earlier measured values made available by
Prof. Tripp Shenton and his team [64].
5.2 Theory
In the previous chapters it has been shown that the selection of the VVS pixels
should be at locations where the gradient of intensity is high, which is typically along
the edges of the structural element being monitored. The greater the gradient, the
better will be the signal-to-noise ratio (SNR).
Based on the linear-elastic theory of cables, several equations have been
developed to give a better approximation of their dynamic behavior taking into
account the axial and flexural rigidities and the effect of the existing boundary
72
conditions [51]–[61], [65]–[71]. Since the focus of this dissertation is on the
development of Eulerian-based VVS to measure vibrations, and not on advancing the
theory of vibrating cables, the effects of flexural rigidity and boundary conditions
were neglected. The behavior of the cable for such conditions can be approximated
using the classical wave equation:
𝑇𝑑2𝑦
𝑑𝑥2= 𝑚
𝑑2𝑦
𝑑𝑡2 (5.1)
in which T is the tension in the cable, y is the deflection of the cable, x is the length
variable along the cable, m is the mass per unit length of the cable, and t is the time
variable. Separating the variables and enforcing pinned boundary conditions, we
obtain:
𝜔𝑘 =𝜋𝑘
𝑙√
𝑇
𝑚 (
𝑟𝑎𝑑
𝑠) 𝑜𝑟 𝑓𝑘 =
𝑘
2𝑙√
𝑇
𝑚 (𝐻𝑧) (5.2)
where 𝜔𝑘 and 𝑓𝑘 are the kth natural frequencies with l being the length of the cable.
From Equation (5.2) we can now solve for the cable tension force, Tfk:
𝑇𝑓𝑘=
4𝑚𝑙2𝑓𝑘2
𝑘2 (5.3)
Equation (5.3) gives the tension force with the assumption that the flexural rigidity
and the boundary conditions have no effect on the behavior of the cable. It is
important to note that an error in the measured frequency results in an error in the
calculation of the tensile force. Using the notation 𝑇𝑓𝑘+Δ𝑓 =4𝑚𝑙2(𝑓+Δ𝑓)2
𝑘2 and by
neglecting the second order term (Δ𝑓
𝑓)
2 𝑇𝑓𝑘
, it can be seen that:
𝑇𝑓𝑘+Δ𝑓
𝑇𝑓𝑘
= (𝑓 + Δ𝑓
𝑓 )
2
= 1 + 2Δ𝑓
𝑓+ (
Δ𝑓
𝑓)
2
≈ 1 + 2Δ𝑓
𝑓 (5.4)
73
Equation (5.4) shows that an error in the measurement by Δ𝑓 will result in an error of
2Δ𝑓
𝑓𝑇𝑓𝑘
, which for low frequency measurements can be significant. Hence, it is
advised that the duration of the recording is appropriately long and to take measures as
explained in Chapter 3 to increase the SNR so that the error in the tensile force
calculation is minimized.
5.3 Laboratory Experiments
In order to examine the applicability of the proposed methodology to measure
the natural frequencies of vibration of cables and estimate their tensile force,
preliminary laboratory tests were carried out. A seven-strand, 187 inch (4.75 m) long
and 1/4 inch (9.53 mm) thick cable (Figure 5.1 d)) with a unit mass of 0.08 lb/ft (118
gram/m) was connected to a load cell to measure its tensile force. Also, the cable was
monitored using a GoPro Hero3 camera and an accelerometer to measure its natural
frequencies. The instruments used for this purpose and the test setup are shown in
Figure 5.1. The S-shaped load cell (Figure 5.1 a) is an Omegadyne LC101, capable of
measuring up to 1200 lbs. (5.34 kN), the accelerometer is an Entran EGC sensor
located at 78 ½ inch (2.00 m) from the left end of the cable (Figure 5.1 b), while the
same GoPro camera that was introduced in previous chapters was used for these tests.
The sampling frequency of the accelerometer and the load cell was 100 Hz whereas
the camera was recording at 120 frames per second.
The tensile force was adjusted by tightening the screw hook at the one end of
the cable before each of the eight performed pluck tests. Each time data sets from
camera, accelerometer, and load cell were collected during the pluck test. The natural
frequencies of each data set were then plugged into Equation (5.3) and the respective
74
tension forces computed. The estimated tension force was then compared to the
average measured tensile force in the load cell.
Figure 5.1: Test setup showing: a) load cell: Omegadyne LC101, b) accelerometer:
Entran EGC, c) camera: GoPro Hero 3, and d) cable.
Figure 5.2 shows samples of typical accelerometer, camera (or VVS), and load
cell data. As evident from Figure 5.2 a) and b), the VVS measured frequencies are
close to those measured by the accelerometer and, in fact, the camera was able to
measure more natural frequencies. This may be because the location of the
accelerometer is close to mid-span coinciding to a node for the non-symmetrical mode
shapes, which results in a small displacement amplitude, or it may be an error in the
VVS because of the nonlinearity issue discussed in Chapters 2 and 3. Also, it can be
seen from Figure 5.2 c) that the load cell data contains the same first natural
frequency, which implies that the invariability of the tensile force with respect to time
is not a true assumption.
75
All eight measurements are summarized in Table 5.1, which shows in each row
the measured natural frequencies using the accelerometer, camera, and the load cell
data. The Δ𝑓
𝑓=
|𝑓𝑎𝑐𝑐−𝑓𝑐𝑎𝑚|
𝑓𝑎𝑐𝑐 also gives a measure of error between the two
measurements. On the other hand, Table 5.2 shows the estimated tensile force using
the natural frequency data from Table 5.1.
76
Figure 5.2: Samples of typical data: a) FFT of acceleration, b) FFT of VVS, c) FFT
of load cell, and d) tension force vs. time.
77
Table 5.1: Measured natural frequencies in Hz using the accelerometer and the
camera, and Δ𝑓
𝑓 as a measure of error. **
Test Mode 1 2 3 4 5
1
Accelerometer 3.10 6.15
Camera 3.10 6.18
Δ𝑓
𝑓 0.00 0.00
2
Accelerometer 4.13 8.06
Camera 4.10 9.63 14.43
Δ𝑓
𝑓 0.01 0.19*
3
Accelerometer 5.45 10.28
Camera 5.39 10.83 21.67
Δ𝑓
𝑓 0.01 0.05
4
Accelerometer 6.08 11.89
Camera 6.06 11.95 18.24 24.36
Δ𝑓
𝑓 0.00 0.01
5
Accelerometer 7.47 14.80
Camera 7.46 14.90 22.24 29.80 37.12
Δ𝑓
𝑓 0.00 0.01
6
Accelerometer 7.62 15.12
Camera 7.61 15.20 22.87 30.36 37.92
Δ𝑓
𝑓 0.00 0.01
7
Accelerometer 8.10
Camera 8.10 16.13 24.18 32.30
Δ𝑓
𝑓 0.00
8
Accelerometer 8.84 17.68
Camera 8.84 17.63 26.70
Δ𝑓
𝑓 0.00 0.00
*Maximum Δ𝑓
𝑓
** Unobserved natural frequencies are left blank
78
Table 5.2: Range of load cell values and their average vs. the calculated tension
based on natural frequencies measured by accelerometer and the camera:
Load Cell (lb) Accelerometer (lb) Camera (lb)
Test Min Max Average Average Average
1 19.32 26.59 22.94 22.78 22.91
2 35.95 44.26 39.54 39.78 42.22
3 65.04 77.51 69.56 67.05 69.87
4 89.99 98.30 92.83 86.39 87.50
5 144.03 149.22 146.72 132.09 132.24
6 151.30 156.50 153.94 137.65 138.07
7 172.09 177.30 174.85 156.77 155.80
8 209.50 214.70 212.31 186.72 187.22
Figure 5.3 a) shows the average tension force computed from the natural
frequency data of the accelerometer and the VVS vs. the range and average values in
the load cell (the minimum and maximum values of the load cell). As can be seen
from Figure 5.3, the calculated values from the natural frequencies obtained using the
VVS and the accelerometer data match well. Also, as shown in Figure 5.3 b) the
calculated values of tension based on natural frequency data have an error of less than
12 percent.
79
Figure 5.3: Lab test results a) The minimum, maximum, and average tension force in
the cable measured by the load cell (Min LC, Max LC and Mean LC) vs.
the average calculated values of tension based on natural frequencies of
accelerometer (Mean Acc) and camera (Mean Cam), b) The same values
divided by the average load cell values.
5.4 Real World Applications
The Indian River Inlet Bridge located on the Delaware coast is one of the most
densely-instrumented bridges in the country. This bridge has a structural health
80
monitoring (SHM) system consisting of more than 100 sensors, which include strain
gauges, accelerometers, and inclinometers to name only a few types.
The load of the deck of the bridge is transferred using four pylons, which have
19 cables on each side of them. The accelerometers are located at three out of four
pylons, quarter points of each span, and on 11 cables [72]. The purpose of the
accelerometers on the cables was to monitor the tension loss in the cables and the
force time-history during extreme events such as hurricanes. On May 11th 2016, a
pluck test was performed on the cables as shown in Figure 5.4.
Figure 5.4: Typical pluck test on a cable of the Indian River Inlet Bridge a) The
crane, b) moving the cable to initiate free vibration.
Using the GoPro Hero camera that was used in all of our studies, the natural
frequency of a cable was measured and is in accordance with the measurement found
in [64]. For this cable , which is numbered 219 E, the acceleration data was recorded
81
during the test. As can be observed from Figure 5.5 b), the natural frequencies were
approximately the same. From the data in [64], it can be concluded that the tension
force in Cable 219 E is 1356 ± 68.94 kips. The target tension force for this cable by
the end of construction was estimated to be 1438 kips, while the estimated force based
on the pluck test on May 5th 2013 was 1347 kips. Given the uncertainty in our
estimation based on VVS, it can be concluded that the tension force has not changed
between 2013 and our recent measurement.
Figure 5.5: Measured natural frequency for the cable 219 E
5.5 Conclusion, Future Work
The methodology introduced in Chapter 2 has been successfully applied to
measure natural frequencies of cables and estimate their tension force. As previously
mentioned, the advantage of a camera over accelerometers is that it covers a large area
across the element of interest and hence might allow to capture a better range of
natural frequencies since one does not need to be worried about the proximity of the
sensor to a node. On the other hand, the sensitivity of the accelerometer is higher and
independent of weather and lighting conditions. The proposed methodology provides
82
the opportunity for everyone to measure natural frequencies of cables using their
point-and-shoot camera or their cell phones given that the used device has a sufficient
frame rate. It also makes it possible to analyze existing digital videos, as was
demonstrated in Section 2.4.
83
SUMMARY AND OVERALL CONCLUSIONS, FUTURE WORK
In this work the general concept of Eulerian Virtual Visual Sensors (VVS) is
proposed and studied. It is shown in Chapter 2 that measurement of the first natural
frequency is possible by observing the minute change in the intensity values through
time and taking it to frequency domain using the Fast Fourier Transform (FFT). It is
shown however that nonlinear change of intensity with motion will cause higher
harmonics in frequency domain as presented in Equation (2.5). In Chapter 3 the SNR
issue, the quantization noise and the intensity-displacement correlation were
discussed. Methods to tackle each problem to increase the SNR in measurement of
natural frequencies were proposed and compared. In Chapter 4 application of new
black and white target is considered to give a measure of the accuracy and sensitivity.
It is revealed through the laboratory experiments that with a high confidence level,
subpixel displacement accuracy is achievable. The theory in Chapter 4 was developed
to give measures for the patch size based on the desired confidence level of accuracy
and SNR. Through the findings of Chapter 4 it is implied that in order to achieve the
subpixel sensitivity in measurement of natural frequencies it is best practice to choose
places where displacement is in the same direction of maximum spatial gradient of
intensity ∆𝐼 = (𝐼𝑚𝑎𝑥 − 𝐼𝑚𝑖𝑛). Finally, in Chapter 5 an application of the proposed
methodology to monitor tension forces in cables is discussed. It is shown through the
laboratory experiments that VVS can be a feasible alternative to the accelerometers in
Chapter 6
84
natural frequency measurements. The effect of error in measurement of natural
frequency on the calculation of the tensile force is briefly discussed. Also for the pluck
tests on the Indian River Inlet Bridge, for one cable it is shown that the measured
natural frequency is close to the one independently measured by the accelerometers in
the health monitoring system.
The proposed methodology can be simply used for measurement of natural
frequencies or measurement of displacement with subpixel accuracy level using the
discussed targets. In the future, a combination of this method and block matching
algorithms could be used in a multiresolution displacement measurement scenario.
Since the proposed method works well in measurement of pixel-level displacement
with almost no computational cost, in conjunction with a block matching algorithm it
can give highly accurate displacement measurements in a significantly lower amount
of time.
85
[1] A. Alvandi and C. Cremona, “Assessment of vibration-based damage
identification techniques,” J. Sound Vib., vol. 292, no. 1–2, pp. 179–202, Apr.
2006.
[2] A. Bagheri, G. Ghodrati Amiri, and S. A. Seyed Razzaghi, “Vibration-based
damage identification of plate structures via curvelet transform,” J. Sound Vib.,
vol. 327, pp. 593–603, 2009.
[3] E. P. Carden and P. Fanning, “Vibration Based Condition Monitoring: A
Review,” Struct. Heal. Monit., vol. 3, no. 4, pp. 355–377, Dec. 2004.
[4] A. Deraemaeker and K. Worden, New trends in vibration based structural
health monitoring. 2012.
[5] S. W. S. Doebling, C. R. C. Farrar, M. B. M. Prime, and D. W. D. Shevitz,
“Damage identification and health monitoring of structural and mechanical
systems from changes in their vibration characteristics: a literature review,”
1996.
[6] C. R. Farrar and K. Worden, “An introduction to structural health monitoring.,”
Philos. Trans. A. Math. Phys. Eng. Sci., vol. 365, no. 1851, pp. 303–315, 2007.
[7] M. I. Friswell, “Damage identification using inverse methods.,” Philos. Trans.
A. Math. Phys. Eng. Sci., vol. 365, no. 1851, pp. 393–410, 2007.
[8] C. Fritzen, “Vibration-based structural health monitoring–concepts and
applications,” Key Eng. Mater., vol. 294, pp. 3–18, 2005.
[9] D. Montalvao, N. Maia, and A. Ribeiro, “A review of vibration-based structural
health monitoring with special emphasis on composite materials,” Shock Vib.
Dig., vol. 38, no. 4, pp. 295–324, 2006.
[10] W. Ostachowicz and A. Güemes, New trends in structural health monitoring.
2013.
REFERENCES
86
[11] A. Pandey, M. Biswas, and M. Samman, “Damage detection from changes in
curvature mode shapes,” J. Sound Vib., vol. 145, pp. 321–332, 1991.
[12] O. S. Salawu, “Detection of structural damage through changes in frequency: a
review,” Eng. Struct., vol. 19, no. 9, pp. 718–723, Sep. 1997.
[13] Z. Sun and C.-C.-C. Chang, “Vibration based structural health monitoring:
Wavelet packet transform based solution,” Struct. Infrastruct. Eng., vol. 3, no.
4, pp. 313–323, Dec. 2007.
[14] Y. Zou, L. Tong, and G. P. Steven, “Vibration-Based Model-Dependent
Damage (Delamination) Identification and Health Monitoring for Composite
Structures — a Review,” J. Sound Vib., vol. 230, no. 2, pp. 357–378, Feb. 2000.
[15] S. Patsias and W. J. Staszewskiy, “Damage Detection Using Optical
Measurements and Wavelets,” Struct. Heal. Monit., vol. 1, no. 1, pp. 5–22, Jul.
2002.
[16] J. J. Lee and M. Shinozuka, “Real-Time Displacement Measurement of a
Flexible Bridge Using Digital Image Processing Techniques,” Experimental
Mechanics, vol. 46, no. 1. pp. 105–114, 2006.
[17] R. Zaurin and F. Necati Catbas, “Structural health monitoring using video
stream, influence lines, and statistical analysis,” Struct. Heal. Monit., vol. 10,
no. 3, pp. 309–332, Jun. 2010.
[18] M. Fraser, A. Elgamal, X. He, and J. Conte, “Sensor network for structural
health monitoring of a highway bridge,” J. Comput. Civ. Eng., no. February, pp.
11–24, 2010.
[19] C. C. Ciang, J.-R. Lee, and H.-J. Bang, “Structural health monitoring for a wind
turbine system: a review of damage detection methods,” Meas. Sci. Technol.,
vol. 19, no. 12, p. 122001, Dec. 2008.
[20] Y.-Z. Song, C. R. Bowen, A. H. Kim, A. Nassehi, J. Padget, and N. Gathercole,
“Virtual visual sensors and their application in structural health monitoring,”
Struct. Heal. Monit., vol. 13, no. 3, pp. 251–264, Feb. 2014.
[21] T. Schumacher and A. Shariati, “Monitoring of structures and mechanical
systems using virtual visual sensors for video analysis: fundamental concept
and proof of feasibility.,” Sensors (Basel)., vol. 13, no. 12, pp. 16551–64, Jan.
2013.
87
[22] H. Wu, M. Rubinstein, and E. Shih, “Eulerian video magnification for revealing
subtle changes in the world,” 2012.
[23] J. Chen, N. Wadhwa, and Y. Cha, “Structural modal identification through high
speed camera video: Motion magnification,” Top. Modal Anal. I, vol. 7, no.
191–197, 2014.
[24] A. SHARIATI and T. SCHUMACHER, “SHM Using Eulerian-based Virtual
Visual Sensors: Introduction of a New Black-and-white Target for Improved
SNR,” in Structural Health Monitoring 2015, 2015.
[25] A. Shariati and T. Schumacher, “Oversampling in virtual visual sensors as a
means to recover higher modes of vibration,” in AIP Conference Proceedings
(Proceedings of QNDE 2014, July 20-25, Boise, ID.), no. Dic, pp. 1–7.
[26] A. Shariati, T. Schumacher, and N. Ramanna, “Eulerian-based virtual visual
sensors to detect natural frequencies of structures,” J. Civ. Struct. Heal. Monit.,
vol. 5, no. 4, pp. 457– 468, 2015.
[27] A. Shariati, T. Schumacher, and N. Ramanna, “Exploration of Video-Based
Structural Health Monitoring Techniques.”
[28] M. Friswell, “Damage identification using inverse methods,” Phylo Trans. R.
Soc. A …, no. December 2006, pp. 393–410, 2007.
[29] T. Duffey and S. Doebling, “Vibration-based damage identification in
structures exhibiting axial and torsional response,” Trans. …, 2001.
[30] K. D’Souza and B. Epureanu, “Damage detection in nonlinear systems using
system augmentation and generalized minimum rank perturbation theory,”
Smart Mater. Struct., vol. 14, pp. 989–1000, 2005.
[31] C. Farrar and K. Worden, “An introduction to structural health monitoring,”
Philos. Trans. …, vol. 365, no. 1851, pp. 303–315, 2007.
[32] C. Farrar and N. Lieven, “Damage prognosis: the future of structural health
monitoring,” … R. …, vol. 365, no. 1851, pp. 623–632, 2007.
[33] J. E. Mottershead and M. I. Friswell, “Model Updating In Structural Dynamics:
A Survey,” J. Sound Vib., vol. 167, no. 2, pp. 347–375, Oct. 1993.
[34] D. Ribeiro, R. Calçada, R. Delgado, M. Brehm, and V. Zabel, “Finite-element
model calibration of a railway vehicle based on experimental modal
parameters,” Veh. Syst. Dyn., vol. 51, no. 6, pp. 821–856, Jun. 2013.
88
[35] Y. Song, C. Bowen, H. Kim, and A. Nassehi, “Virtual Visual Sensors and Their
Application in Structural Health Monitoring,” urmassn11.iids.org.
[36] G. Strang, Linear Algebra and Its Applications. Harcourth Brace Jovanovich,
1988.
[37] G. Strang, Introduction to Linear Algebra, Second Edition. Wellesley-
Cambridge Press, 1993.
[38] “Convert RGB image or colormap to grayscale - MATLAB rgb2gray.”
[Online]. Available:
http://www.mathworks.com/help/images/ref/rgb2gray.html. [Accessed: 31-Jul-
2013].
[39] C. E. Shannon, “Communication in the presence of noise,” Proc. IRE, vol. 86,
no. 2, pp. 447–457, Feb. 1949.
[40] P. Keller, “Wind induced torsional fatigue behavior of truss bridge verticals,”
Oregon State University, 2012.
[41] B. Widrow and I. Kollár, “Quantization noise,” Cambridge Univ. Press, 2008.
[42] A. Shariati and T. Schumacher, “Oversampling in virtual visual sensors as a
means to recover higher modes of vibration,” in AIP Conference Proceedings
(Proceedings of QNDE 2014, July 20-25, Boise, ID.).
[43] C. A. Murray, W. A. Take, and N. A. Hoult, “Measurement of vertical and
longitudinal rail displacements using digital image correlation,” Can. Geotech.
J., vol. 52, no. 2, pp. 141–155, Feb. 2015.
[44] J. Peddle, A. Goudreau, E. Carson, and E. Santini-Bell, “Bridge displacement
measurement through digital image correlation,” Bridg. Struct. Des. Constr.,
vol. 7, no. 4, 2011.
[45] B. Glisic, J. Chen, and D. Hubbell, “Streicker Bridge: a comparison between
Bragg-grating long-gauge strain and temperature sensors and Brillouin
scattering-based distributed strain and temperature sensors,” in SPIE Smart
Structures and Materials + Nondestructive Evaluation and Health Monitoring,
2011, p. 79812C–79812C–10.
89
[46] D. Feng, M. Q. Feng, E. Ozer, and Y. Fukuda, “A Vision-Based Sensor for
Noncontact Structural Displacement Measurement.,” Sensors (Basel)., vol. 15,
no. 7, pp. 16557–75, Jan. 2015.
[47] “Category:Cable-stayed bridges in the United States.” [Online]. Available:
https://en.wikipedia.org/wiki/Category:Cable-
stayed_bridges_in_the_United_States.
[48] “Category:Suspension bridges in the United States.” .
[49] A. Nordrum, “The Rise of the Cable-Stayed Bridge,” Scientific American, 20-
Jan-2015. [Online]. Available:
http://www.nature.com/doifinder/10.1038/scientificamerican0215-15.
[Accessed: 04-Jul-2016].
[50] A. Chopra, Dynamics of structures. 2012.
[51] H. M. Irvine and T. K. Caughey, “The Linear Theory of Free Vibrations of a
Suspended Cable,” Proc. R. Soc. London. A. Math. Phys. Sci., vol. 341, no.
1626, pp. 299–315, 1974.
[52] T. Shinke, K. Hironaka, H. Zui, and H. Nishimura, “Practical Formulas for
Estimation of Cable Tension By Vibration Method,” J. Struct. Eng., vol. 122,
no. 6, pp. 651–656, 1996.
[53] J. C. Russell and T. J. Lardner, “Experimental Determination of Frequencies
and Tension for Elastic Cables,” J. Eng. Mech., vol. 124, no. 10, pp. 1067–
1072, 1998.
[54] A. B. Mehrabi and H. Tabatabai, “Unified Finite Difference Formulation for
Free Vibration,” J. Struct. Eng., vol. 124, no. November, pp. 1313–1322, Nov.
1998.
[55] Y. Q. Ni, J. M. Ko, and G. Zheng, “Dynamic Analysis of Large-Diameter
Sagged Cables Taking Into Account Flexural Rigidity,” J. Sound Vib., vol. 257,
no. 2, pp. 301–319, 2002.
[56] R. Geier, G. De Roeck, and R. Flesch, “Accurate cable force determination
using ambient vibration measurements,” Struct. Infrastruct. Eng., vol. 2, no.
January, pp. 43–52, 2006.
90
[57] B. H. Kim and T. Park, “Estimation of cable tension force using the frequency-
based system identification method,” J. Sound Vib., vol. 304, no. 3–5, pp. 660–
676, Jul. 2007.
[58] M. A. Ceballos and C. A. Prato, “Determination of the axial force on stay
cables accounting for their bending stiffness and rotational end restraints by free
vibration tests,” J. Sound Vib., vol. 317, no. 1–2, pp. 127–141, 2008.
[59] G. Ricciardi and F. Saitta, “A continuous vibration analysis model for cables
with sag and bending stiffness,” Eng. Struct., vol. 30, no. 5, pp. 1459–1472,
2008.
[60] R. A. Sousa, R. M. Souza, F. P. Figueiredo, and I. F. M. Menezes, “The
influence of bending and shear stiffness and rotational inertia in vibrations of
cables: An analytical approach,” Eng. Struct., vol. 33, no. 3, pp. 689–695, 2011.
[61] H. Nam and N. T. Nghia, “Estimation of cable tension using measured natural
frequencies,” Procedia Eng., vol. 14, pp. 1510–1517, 2011.
[62] E. Nazarian, F. Ansari, and H. Azari, “Recursive optimization method for
monitoring of tension loss in cables of cable-stayed bridges,” J. Intell. Mater.
Syst. Struct., pp. 1–11, 2015.
[63] W. Pakos, Z. Wójcicki, J. Grosel, K. Majcher, and W. Sawicki, “Experimental
research of cable tension tuning of a scaled model of cable stayed bridge,”
Arch. Civ. Mech. Eng., vol. 16, no. 1, pp. 41–52, 2016.
[64] K. Davidson, “Estimation of tension in stay cables at the Indian River Inlet
Bridge using frequency based methods,” University of Delaware, 2013.
[65] H. Irvine, “Studies in the statics and dynamics of simple cable systems,” 1974.
[66] A. S. K. Kwan, “Simple technique for calculating natural frequencies of
geometrically nonlinear prestressed cable structures,” Comput. Struct., vol. 74,
no. 1, pp. 41–50, 2000.
[67] S. Cho, J. P. Lynch, J.-J. Lee, and C.-B. Yun, “Development of an Automated
Wireless Tension Force Estimation System for Cable-stayed Bridges,” J. Intell.
Mater. Syst. Struct., vol. 21, no. 3, pp. 361–376, 2009.
91
[68] T. Park, D. Hyung Lee, and B. Hwa Kim, “Estimation of tension force in
double hangers by a system identification approach,” Inverse Probl. Sci. Eng.,
vol. 18, no. 2, pp. 197–216, 2010.
[69] S. Cho, J. Yim, S. W. Shin, H.-J. Jung, C.-B. Yun, and M. L. Wang,
“Comparative Field Study of Cable Tension Measurement for a Cable-Stayed
Bridge,” J. Bridg. Eng., vol. 18, no. 8, pp. 748–757, Aug. 2013.
[70] Y. Yu, C. Zhang, X. Zhu, W. H. Kang, X. Mao, and B. Uy, “Design and
Experimental Investigations of a Bridge Cable Tension Monitoring,” vol. 17,
no. 11, pp. 1657–1669, 2014.
[71] M. Triantafyllou and L. Grinfogel, “Natural frequencies and modes of inclined
cables,” J. Struct. Eng., 1986.
[72] H. W. Shenton, M. J. Chajes, and G. R. Wenczel, “Indian River Inlet Bridge
Structural Health Monitoring System.” .