Advanced stereovision system for fire spreading study

Fire Safety Journal ] (]]]]) ]]]–]]]

Contents lists available at SciVerse ScienceDirect

Fire Safety Journal

0379-71

http://d

n Corr

E-m

Pleas10.10

journal homepage: www.elsevier.com/locate/firesaf

Advanced stereovision system for fire spreading study

L. Rossi a,n, T. Molinier a, M. Akhloufi b, A. Pieri a, Y. Tison a

a UMR CNRS 6134 SPE—University of Corsica, Campus Grimaldi, 20250 Corte, Franceb CRVI, 205, rue Mgr-Bourget, Levis, Quebec, Canada G6V 6Z9

a r t i c l e i n f o

Keywords:

Spreading fire

Metrological system

Stereovision

Geometrical characteristics

Surface

Volume

12/$ - see front matter & 2012 Elsevier Ltd. A

x.doi.org/10.1016/j.firesaf.2012.10.015

esponding author. Tel.: þ33 4 95 45 06 11; fa

ail address: [email protected] (L. Rossi).

e cite this article as: L. Rossi, et al., A16/j.firesaf.2012.10.015

a b s t r a c t

This paper presents research conducted in the area of stereovision in order to develop a metrological

system for the measurement of geometrical characteristics of spreading fires. In the first part, the

material and the algorithms used to obtain three dimensional coordinates of fire points are presented.

The second part describes the stereovision framework used for laboratory experiments and the

algorithms developed to estimate the fire front geometrical characteristics. With this system, a 3D global

form of the fire is obtained at each acquisition time and the position of each point of the front, their rate

of spread, their height, length, tilt angle, the width basis and the volume of the fire are estimated. In the

third part, a stereovision framework developed for the geometrical characteristics measurement of fires

at field scale is presented. Complementary information is obtained from different views of the fire.

A Human Machine Interface gives the possibility to an expert to define from the lateral view the fire area

to study. A three dimensional surface reconstruction of the back fire front is obtained and the height, the

length, the position, the rate of spread, the tilt angle and the width of the front are estimated.

& 2012 Elsevier Ltd. All rights reserved.

1. Introduction

In forest fire research, the experimental studies of the firespread across vegetal fuels are of great interest for understandingand modeling the fire behavior. Parameters like the fire frontgeometrical properties (position, rate of spread (ROS), fire height,fire length, fire inclination angle, fire base width, surface, volume)are particularly interesting to compute during a spreading experi-ment because they influence the propagation and the heattransfer of the fire [1–4]. Visual and infrared cameras are nowused as complementary metrological instruments in fire spread.Pastor et al. [5] propose a thermal image processing method forcomputing the ROS of linear fire fronts generated on flat surfaceswith known dimensions. The authors use four calibration pointslocated at the vertices of a rectangle plane with known sidelengths. These points must be visible in the image during theexperiments, which limits the approach to controlled environ-ment experiments. Martinez-de Dios et al. [6] have developed asystem using visual and infrared images for laboratory fire spreadanalysis. The cameras are positioned at two view points (frontaland lateral views). This approach uses 2D information and cannotextract 3D depth information for all the points of a nonlinear firefront. Only the length of the most advanced point of the fire frontand its inclination are estimated by the lateral view and the ROSis computed from the frontal view of the fire. Computer vision

ll rights reserved.

x: þ33 4 95 10 38 74.

dvanced stereovision syste

techniques are also applied to monitor forest fires in Martinez deDios et al. [7]. The system computes a 3D perception model of thefire. Multisensory fusion is conducted using telemetry sensorsand GPS.

Fire modeling scientists need non-destructive systems in orderto measure the geometrical characteristics of fire fronts in variousscenarios: indoors on burn tables, and outdoors in unstructuredenvironments at meso and field scales.

The aim of this paper is to present a research conducted usingstereovision in order to develop a metrological system for themeasurement of geometrical characteristics of spreading fires atlaboratory scale and at field scale. In the first part, the materialand the algorithms used to obtain three dimensional coordinatesof fire points are presented. The second part describes thestereovision framework used for laboratory experiments and thealgorithms developed to estimate the fire front geometricalcharacteristics. In the third part, a stereovision framework devel-oped for the measurement of geometrical characteristics of firesat field scale and the experimental results are presented.

2. Stereovision material and 3D coordinates of fire points

2.1. Stereovision material

The basis material of the stereovision frameworks presented inthis paper is a pre-calibrated multiple baseline stereo camerafrom Point Grey [8,9]. The camera is a color XB3 trinocular stereo

m for fire spreading study, Fire Safety J (2012), http://dx.doi.org/

www.elsevier.com/locate/firesaf

www.elsevier.com/locate/firesaf

dx.doi.org/10.1016/j.firesaf.2012.10.015



mailto:[email protected]






L. Rossi et al. / Fire Safety Journal ] (]]]]) ]]]–]]]2

system with a focal length of 3.8 mm, 661HFOV (horizontal fieldof view), a 1/3’’ Sony ICX445AQ CCD with 1280�960 imageresolution, a pixel size of 3.7 mm and a frame rate of 16 FPS. Thissystem is accurately pre-calibrated for lens distortions andcamera misalignments. The stereo pair, left and right images, isaligned to within 0.1 pixel RMS error. Also a calibration retentionmechanism permits to minimize loss of calibration due to shockand vibration. Finally, the multiple baselines choice of 12 cm or24 cm for stereo processing make it possible to efficiently choosethe best strategy for accuracy and occlusion handling. The precisepre-calibration of this optical system permits to avoid thecalibration phase during the experiments. The three multiplebaseline cameras are used in our experiments. For each acquisi-tion, three images are captured and stored in RAW format. Thethree cameras are perfectly synchronized with a maximumsynchronization time of 125 ms. Image acquisition code wasdeveloped and optimized using Cþþ programming language inorder to attain real-time performance. Color interpolation andimage rectification were done offline using a Cþþ program basedon Triclops Stereo Vision SDK. In the developed rectificationprocedure, we process images by pairs. The images from thetwo distant cameras (24 cm baseline) form the first pair andimages from the centre and right cameras (12 cm baseline) formthe second pair. Since the baseline plays an important role instereo accuracy [10], we considered the two distant cameras inmost of the experiments. (Considering all other parameters equal:

Fig. 1. Algorithm of 3D fir

Please cite this article as: L. Rossi, et al., Advanced stereovision syste10.1016/j.firesaf.2012.10.015

the largest is the baseline, the better is the stereo accuracy.) Theuse of the largest baseline permits to minimize the theoreticalerror along the Z axis during the three dimensional computationof the fire points.

2.2. Three dimensional coordinates of fire points

In order to compute the desired fire characteristics, the 3Dcoordinates of salient fire points are estimated using stereo imageprocessing. The approach involves the following steps and ispresented in [11]:

1.

e po

m f

segmentation of fire regions,
2. extraction of salient feature points, 3. automatic computation of stereo disparity over time, 4. feature selection refinement using a normalized cross correla-
tion matching strategy,
5. computation of 3D fire points using stereo correspondence.
Fig. 1 presents a view of this algorithm. In a first step, the twostereoscopic images obtained at each instant are segmented usinga color segmentation method in order to isolate the fire area inthe images. In a second step, the contours of these areas areobtained and salient feature points called ‘‘fire points’’ areselected. The third step is dedicated to the computation of thestereo disparity in order to match in a fourth step the salient

ints computation.

or fire spreading study, Fire Safety J (2012), http://dx.doi.org/





Fig. 2. Frontal fire front image (a) and 3D fire points (b).

L. Rossi et al. / Fire Safety Journal ] (]]]]) ]]]–]]] 3

feature points of the two stereoscopic images. The fifth stepcorresponds to the computation of the tridimensional coordinatesof the matching points.

Fig. 2(b) shows the extracted 3D points of the fire frontpresented in Fig. 2(a) obtained from a multiple baseline stereocamera positioned in a back position relative to the propagationdirection of the fire. The reference frame has its origin in the leftimage centre point of the back stereo camera.

Fig. 3. Stereovision cameras positioned in a front position and in a back position

relative to the direction of the fire propagation.

3. Stereovision framework for the geometrical characteristicsestimation of laboratory fire propagations

Experiments of fire propagation were conducted at laboratoryscale on a 2 m�2 m platform that has been inclined up to 201.The experimental system is composed of two stereovision cam-eras positioned, respectively, in a front position and in a backposition of the fire propagation direction (Fig. 3).

3.1. Estimation of the position, the rate of spread and the height

of fire fronts

3D processing has been developed in order to estimate theevolution of the height, the position and the ROS of any point in alinear or nonlinear fire front. These characteristics are estimatedover time directly from computed 3D fire points obtained fromthe back stereoscopic images. The details of this 3D processing aregiven in [11]. The main steps are:

1.

P1

Estimation of the base plane and of the fire front lines. The baseplane is the plane where the fire is propagating. It correspondsto the ground. As the fire is spreading along an inclinable table,the base of the fire is not at the same position over time.Therefore, it is necessary to differentiate between the points ofthe base plane and the points of the upper part of the fire front.The 3D points of the base of successive fronts are used toestimate the base plane of propagation with a least-squarestechnique. Each set of 3D points belonging to a fire front at agiven time is projected on the base plane and a 2D front line iscomputed using a B-spline interpolation (Fig. 4). Thus, theterms ‘‘front line’’ is used to define the line passing through thefire points that are situated on the base plane.

2.
Estimation of the rate of spread.The rate of spread is estimated using a method employed bythe modelers of the UMR CNRS 6134 SPE—University ofCorsica: equidistant points are selected in the first front line.For each selected point of the first front line, a correspondingpoint on the second line is selected: it is the intersection of thenormal at the selected point of the first front line and the
lease cite this article as: L. Rossi, et al., Advanced stereovision system f0.1016/j.firesaf.2012.10.015

second front line (Fig. 4). For each point of the first line, its ROSis computed as the quotient of the distance between itsposition and its corresponding point on the second line andthe lapse of time between the two image acquisitions corre-sponding to the two front lines.

3.
Estimation of the height of the fire front. The 3D points of the firefront are projected in a plane with Y¼0 corresponding to thebase plane of the front. Thus, each 3D fire point has 3 coordi-nates (X, Y, Z) where X corresponds to lateral position, Y
corresponds to the height position and Z corresponds to thedepth position. Thus, the distance between the base plane andeach point of the upper part of the fire corresponds to theheight of the point. In order to obtain continuous informationfor the height of the front, the fire front line situated on thebase plane is used. It is divided into segments using a recursivealgorithm. This algorithm divides each segment into sub-segments until one upper fire point per segment is obtained.The segment is then given the height of the point it contains. AB-spline interpolation is then applied to the points in thesegments and used in order to obtain an estimation of the firefront height. The projection plane between the obtainedsegments and the 3D position permits the computation oflocal fire inclination. Fig. 4 shows the estimated height of afire front.

From the fire 3D points, it is possible to obtain complementaryinformation like the mean height, the median height or the

or fire spreading study, Fire Safety J (2012), http://dx.doi.org/





Fig. 4. Fire front lines at two successive times.

Fig. 5. Height estimation of a nonlinear fire front.


continuous and intermittent areas of a fire front. Thus, in theparticular case of linear front, the height of the fire front isconsidered by the modelers as constant at each time whereasthe front is not uniform in reality (Fig. 5). To obtain an equivalentinformation, the B-spline line is sampled and the extracted valuesare used to compute the median height computed as the valuethat separates the sampled points in two sets containing each one50% of the data. An equivalent front is obtained. It is a rectanglehaving a width equal to the one of the fire front and a heightequal to the median height of the fire front. Fig. 6 shows anequivalent front of a fire front.

The continuous and intermittent areas of a fire front can bedetermined by using the 0.5 intermittency criteria and by con-sidering the area fire front obtained at successive times during thepropagation.


3.2. Estimation of the volume of the fire front

In order to estimate the volume of a fire front, it is necessary tocompute a complete tridimensional form of the fire front. The 3Dfire points obtained from the two synchronized stereovisioncameras are complementary. They are projected in a samereference frame and used to build a global form of the fire front.The coordinate transformations of each system with respect to asingle reference system are estimated using a registration proce-dure with a cube pattern. The details of this procedure arepresented in [12].

Each set of 3D points (belonging to the back stereo cameraimages and front stereo camera images) is projected on the baseplane and each of them is used to compute a 2D front line. Fig. 7shows the back line and the front line obtained from these two







sets of 3D points. They correspond respectively to the position ofthe front and back of the fire front. The space between these twocurves corresponds to the depth of the base of the fire front.

From the front and back lines, a mean line is computed (linewith grey color in Fig. 7) and divided into regular intervals. The3D points that are into the volume defined by the two perpendi-cular planes on the border of each interval are selected. Thesepoints are then used to compute a mean 3D point of each area(thick point in Fig. 7) that is positioned on the mean curve. A threedimension meshing is then performed in order to estimate thethree dimensional surface rendering. It is based on Delaunaytriangulation and it produces a set of triangles as presented in[12]. Fig. 8 shows the front (a) and back (b) view of a firespreading on a combustion table with an inclination angle of 201.

Fig. 6. Height of a fire front.

Fig. 7. Base of the fire front and estimatio

Fig. 8. Front (a) and bac


Fig. 9 shows the 3D surface rendering of such fire.The estimation of the volume is performed by the sum of the

volume of 3 tetrahedrons for each triangle of the computed mesh.A physical model of a fire front (with maximum dimensions:

height: 0.9; width: 1.9; depth: 0.9 m) has been constructed from ametal frame and covered with red paper. Patterns have beenadded in order to have points of interest on the form that canbe easily detected (Fig. 10(a)). The 387 estimated 3D patternpoints obtained after registration are presented in Fig. 10(b); thethree dimensional surface rendering of the fire model appears inFig. 10(c) and (d). In order to evaluate the model surfacecomputation, we built a synthetic model using paper (squareswith a surface of 0.25 m2) and a wire skeleton. This model permitsan error estimation of the developed techniques; still it is far fromreal fire models that are complex and difficult to model synthe-tically. The back surface of the physical model (presented inFig. 10(a)) is 1.83 m2 and the estimated surface is 1.9 m2 whichcorresponds to an estimation error of 3.8% of error. The volume ofthe physical model is 1.75 m3 and the estimated volume is 1.8 m3.This corresponds to a 2.7% of error. This error can be explained bythe position of the markers on the fire physical model (they arenot exactly on the edges of the shape) and the computationalaccuracy of the image processing algorithms performed at eachstep of the proposed approach.

4. Stereovision framework for the geometrical characteristicsestimation of fire propagations at field scale

Critical points were considered for the development of thesystem dedicated to field experiments. First, generally, fires donot occur at the same location and the place cannot be chosenbeforehand. Second, the dimension and the extent of the spreadcan be large. Taking into account these points, a portable vision

n of the height of the fire by slicing.

k (b) view of a fire.






Fig. 9. Global form of the fire presented in Fig. 7(b).

Fig. 10. Constructed model of fire (a); estimated 3D pattern points after registration (b); three dimensional surface rendering (c), (d).

Fig. 11. Deployment of stereo cameras.


system, electrically independent and relatively easy to deploy hasbeen built. It is composed of the same cameras describedpreviously and positioned, respectively, in a lateral position andin a back position of the fire propagation direction (Fig. 11). In thecase of outdoor experiments, the smoke emitted by the fire movesgenerally in the same direction as the fire following the directionof the wind. Images obtained by frontal stereoscopic systems arefrequently unusable because of the presence of smoke that masksthe fire areas. So, the positioning of cameras visible in front of thefire is ineffective.

Fig. 12 shows two views obtained from cameras on the ground.A procedure especially created for the registration of cameras

at field scale has been developed. It is based on the use of an easytransportable feature put successively at several positions on thefield and visible by the different stereovision systems. For each

Please cite this article as: L. Rossi, et al., Advanced stereovision system for fire spreading study, Fire Safety J (2012), http://dx.doi.org/10.1016/j.firesaf.2012.10.015





Fig. 12. Lateral view (a) and frontal view (b) on one field experiment carried out in Letia (France) in 2010.

Fig. 13. Balloon used for the registration method.

Fig. 14. Selection of the fire area points.


stereovision system, the 3D position of the centre of gravity of thefeature put at different places is compared with a theoreticalnetwork of positions generated by computer and put at a givenplace in space. The place which minimizes the distance betweenthe 3D positions of the feature and the theoretical network ofpositions gives the real pose (translation and rotation) of thefeatures in the camera frame. When the poses are estimated inthe two camera frames (back and side), the registration of the twofeatures (in one frame) gives the extrinsic parameters of theglobal system of cameras. The feature is a balloon of 1 m indiameter wrapped in a red cloth to be easily segmented in theimages. Fig. 13 shows an example of one position of the balloonrelative to the position of the circled stereo cameras.

4.1. Human Machine Interface to specify lateral fire points

Lateral images are not easy to treat automatically due to thepresence of smoke which can hide partially or completely the firefront and also because the fire front can have a complex formwith several flames that have different inclination angle andlength.

A Human Machine Interface has been developed in order topermit to experts to specify the part of the front they want toconsider for the estimation of fire characteristics. This was donein response to a request from the modelers of the UMR CNRS 6134SPE–University of Corsica and it solves partially the problem offire segmentation which is not easy in this kind of lateral imagesof fire with smoke.

The expert clicks on the presented image in order to indicatethree points that correspond to the position of the less advancedpoint of the fire front, to the position of the most advanced pointof the fire front and to the position of the highest point of the firefront. The order in which the user clicks the points does no matter


because they are automatically identified by the algorithm basedon their 2D coordinates. By clicking on the ‘‘Next’’ button, theexpert can choose not to process an image (Fig. 14).

From these 2D points, their corresponding points are searchedin the stereoscopic image and their 3D coordinates are computed.The highest 3D point is noted ‘‘A’’, the less advanced 3D point isnoted ‘‘B’’, and the most advanced 3D point is noted ‘‘C’’. Then,following the description of the parameters presented in Fig. 15,the fire length, the fire front width, the position of the mostadvanced point with respect to a reference and the fire inclinationangle are estimated.

Concerning this procedure, two points have to be noted: Onone side, the fire area is often subject to interpretation when theimage is complex. This method has the advantage of allowing theuser to define the points that define the fire front region viewedfrom the side. On the other side, the results are dependent on theselection of points by the user.

The results depend on the position of the three 2D pointsselected by the expert.

4.2. Three dimensional surface reconstruction

A 3D surface reconstruction is computed from part of thepoints obtained with the back vision system and controlled by the3D points obtained from the lateral vision system. All the pointsare merged in the same frame (the one of the back stereovisioncamera). The most advanced point and the least advanced point






Fig. 15. Description of the fire parameters to be measured: fire length (d), fire

height (h), fire front width (w), position of the most advanced point with respect to

a reference (l), fire inclination angle (y).

Fig. 16. 3D surface reconst

Fig. 17. Temporal evolution



obtained from the lateral vision system are used to limit the areaunder analysis.

Fig. 16 shows the results of the 3D surface reconstruction forthe fire that appears in Fig. 12(b).

It gives the position of the fire on the ground, the height andthe tilt angle of each point of the fire front. Fig. 17 shows thetemporal evolution of the fire front profile for the first minutes ofpropagation of the fire that appears in Fig. 12(b).

5. Conclusion

In this paper the work performed in order to develop ametrological system based on stereovision for the estimation ofgeometrical characteristics of spreading fires is presented. In thecase of a laboratory spreading, the vision system is composed oftwo stereovision cameras positioned, respectively, in the backposition and in the front position relative to the direction ofpropagation of the fire. With the proposed system, it is nowpossible to reconstruct the complete form of the fire front in threedimensions and to follow the temporal evolution of the position,

ruction of a fire front.

of the fire front profile.







the rate of spread, the height, the length, the tilt angle, and thevolume of a complex fire spreading at laboratory scale. In the caseof field spreading, the vision system is composed of two stereo-vision cameras positioned, respectively, in the back position andin the lateral position relative to the direction of fire propagation.Complementary information is obtained from the two cameras. Athree dimensional surface reconstruction of the back fire front isobtained and permits to follow the temporal evolution of the firefront position. A Human Machine Interface gives the possibility toan expert to choose from a lateral view the fire points whichdefine the fire area to study. The fire length, the fire height, thefire front width, the rate of spread, the fire inclination angle areestimated from the lateral information.

Ongoing work is being conducted on the fusion of visible andNIR data in order to overcome the problem of losing informationwhen the smoke covers the whole fire front.

Acknowledgment

This work was carried out in the scope of project PROTERINA-Csupported by the EU under the Thematic 3 of the OperationalProgram Italia/France Maritime 2007-2013, contract: G25I08000-120007.


References

[1] N.P. Cheney, J.S. Gould, The influence of fuel, weather and fire shape variableson fire-spread in grasslands, Int. J. Wildland Fire 3 (1) (1993) 31–44.

[2] N.P. Cheney, J.S. Gould, Fire growth in grassland fuels, Int. J. Wildland Fire 5(4) (1995) 237–247.

[3] J.F. Sacadura, Radiative heat transfer in fire safety science, J. Quant. Spectrosc.Radiat. Transfer 93 (1–3) (2005) 5–24.

[4] R. Viskanta, Overview of some radiative transfer issues in simulation ofunwanted fires, Int. J. Thermal Sci. 47 (2008) 1563–1570.

[5] E. Pastor, A. Agueda, J. Andrade-Cetto, M Mun ~oz, Y. Perez, E. Planas,Computing the rate of spread of linear flame fronts by thermal imageprocessing, Fire Saf. J. 41 (2006) 569–579.

[6] J.R. Martinez-de Dios, J.C. Andre, J.C. Gonc-alves, Arrue B.Ch., A. Ollero,D.X. Viegas, Laboratory fire spread analysis using visual and infrared cameras,Int. J. Wildland Fire 15 (2006) 175–186.

[7] J.R. Martinez-de Dios, B.C. Arrue, A. Ollero, L. Merino, F. Gomez-Rodriguez,Computer vision techniques for forest fire perception, Image Vis. Comput. 26(2008) 550–562.

[8] PtGrey /http://www.ptgrey.com/products/bbxb3/index.aspS (2011).[9] PtGrey Bumblebee XB3 Getting Started Manual /http://www.ptgrey.com/

support/downloads/documents/Bumblebee%20XB3%20Getting%20Started%20Manual.pdfS (2011).

[10] E. Trucco, A. Verri, Introductory Techniques for 3-D Computer Vision,Prentice Hall, 1998, p. 144.

[11] L. Rossi, T. Molinier, M. Akhloufi, Y Tison, A. Pieri, 3D vision system for themeasurement of the rate of spread and the height of fire fronts, Meas. Sci.Technol. 21 (10) (2010) 12 pp., article number 105501.

[12] T. Molinier, L. Rossi, M. Akhloufi, Y. Tison, A. Pieri, Estimation of fire volumeby stereovision’’. In: Proc. SPIE Electronic Imaging, Image Processing:Machine Vision Applications Conference IV, San Francisco, United States,Pp. 7877-10, (2011).


http://www.ptgrey.com/products/bbxb3/index.asp

http://www.ptgrey.com/support/downloads/documents/Bumblebee&percnt;20XB3&percnt;20Getting&percnt;20Started&percnt;20Manual.pdf










Documents

Advanced stereovision system for fire spreading study