FUSION OF INSAR AND GIS DATA FOR 3D BUILDING RECONSTRUCTION AND CHANGE DETECTION

Antje Thiele (1,2), Erich Cadario(2), Stefan Hinz(1)

(1) Institute of Photogrammetry and Remote Sensing (IPF), Karlsruhe Institute of Technology (KIT), Englerstr.7, 76128 Karlsruhe, Germany, Email:{antje.thiele;stefan.hinz}@kit.edu

(2) Fraunhofer IOSB, Fraunhofer Institute of Optronics, System Technologies and Image Exploitation, Gutleuthausstr.1, 76275 Ettlingen, Germany, Email:{antje.thiele;erich.cadario}@iosb.fraunhofer.de

ABSTRACT

The new generation of space-borne high resolution SAR sensors such as TerraSAR-X, SAR-Lupe or Cosmo-SkyMed provides SAR images of 1-3m spatial resolu-tion or even below in special spotlight modes. Naturally, the development of methods to automatically derive detailed cartographic information of both rural and ur-ban areas from this kind of data is a major issue driven by these missions. Since SAR is an active system and largely independent from weather conditions; it is fur-thermore an attractive imaging technology for rapidly acquiring area-wide information of regions hit by disas-ters such as floodings, landslides, or earthquakes. Thus, the development of automatic methods for damage analysis or – generally – change detection is another important issue. For such kind of tasks existing Geo-data available from Geographic Information Systems (GIS) have to be adjusted with current SAR data. In this paper an approach for 3D building reconstruction based on multi-temporal information fusion by exploit-ing GIS and InSAR data is presented. Thereby, GIS data consisting of building footprints are used as initial in-formation. The simulation exploits building hypotheses, which are based on the given 2D footprint and an as-sumed building height, and provides as result synthetic InSAR phases. An iterative adjustment between simu-lated and measured real InSAR phases enables the re-construction of the current building shape. Our approach is tested on single-pass airborne InSAR data. 1. INTRODUCTION

Since 2006, a new class of space-borne SAR sensors (e.g., TerraSAR-X, SAR-Lupe or Cosmo-SkyMed) pro-vides images of 1-3m spatial resolution or even below in special spotlight modes. One major issue of these missions is the improvement of interpretation of rural and urban areas. Especially, the development of meth-ods to automatically derive detailed cartographic infor-mation is in demand. Furthermore, due to the fact that high precision repeat-pass tracks and the new upcoming TanDEM-X mission provide high-resolution InSAR data, methods for automatic 3D information extraction are desired. Especially, since SAR is an active system and largely independent from weather conditions; it is an attractive

imaging technology for rapidly acquiring area-wide information of regions hit by disasters such as flood-ings, landslides, or earthquakes. After such events an automatic damage analysis or – generally – change de-tection is an important issue, thereby the joint exploita-tion of existing Geo-data (e.g., available from Geo-graphic Information Systems – GIS) and the current SAR data is promising. Studies concerning automatic detection of buildings based on SAR and InSAR data are presented since the mid-nineties. All of them are mainly driven by the ex-ploitation of the magnitude data (Figure 1a) alone or a combination of magnitude and interferometric height data (Figure 1b) [1,2,3,4,5]. In most cases, these ap-proaches analyze the length of the layover or shadow areas at building locations to estimate the building height. The exploitation of the interferometric heights was restricted to mean height calculation within an es-timated building footprint (Figure 1c). In [6], the shape of the interferometric phase profile at building locations was additionally analyzed, due to the fact that this shape contains valuable information. Up to now, these studies were mainly focused on isolated buildings or small building groups. In dense urban areas interaction effects between adjacent buildings have also to be considered. The interpretation as well as the utilization of such in-

a b

c d

Figure 1. SAR magnitude signature of building (a), In-SAR phase signature of building (b), optical image overlaid with 2D GIS information (c) and with 3D

building model (d)

_____________________________________________________ Proc. ‘Fringe 2009 Workshop’, Frascati, Italy, 30 November – 4 December 2009 (ESA SP-677, March 2010)

teraction effects between buildings are part of this study and need further investigations. In this paper, we de-scribe the appearance of buildings in magnitude (Fig-ure 1a) and interferometric phase data (Figure 1b). Fur-thermore our approach for building modeling and height estimation based on multi-temporal information fusion by exploiting GIS (Figure 1c) and InSAR data is pre-sented. Preliminary results (Figure 1d) are shown and the future work is pointed out, especially in view of the upcoming mission TanDEM-X. 2. INSAR BUILDING SIGNATURE

In this section the magnitude and interferometric phase signature of buildings are described by considering iso-lated buildings and interaction effects between adjacent buildings occurring with increasing building height.

2.1. Magnitude Signature

The signature of elevated objects in SAR data is charac-terized by the side looking illumination and the slant range projection, which leads to the well-known phe-nomena like foreshortening, layover, and shadow. Es-pecially, at building location due to the right-angled structure between ground and wall, multi-bounce scat-tering is observable. In detail, Figure 2a shows sche-matically the signature of an isolated building in SAR magnitude data by considering an increasing building height. The expected slant range profile for four differ-ent heights hi are presented with the marked layover li and shadow si. Considering a constant off-nadir angle θ in equations (1) and (2), with an increasing building height an increasing layover and shadow area are given at the expense of single roof backscattering. θcos⋅= ihil (1)

θcosih

is = (2)

Furthermore the corner, which will appear brighter with increasing building height, is marked white in the slant

range profiles. Overall, the full signature of a building is increasing with increasing building height. This leads to the questions: How do adjacent buildings in dense urban area interact? Are there changes in the signature observ-able? Such a situation is shown in Figure 2b by considering two similar buildings with a small distance between each other. The two slant range profiles are resulting from different height assumptions and show conse-quently differences. For the assumption of lower height, the building signatures do not affect each other. Both buildings show the classical signature – “building lay-over”, “building corner”, “building roof”, and “building shadow” – with the same amplitude. The second, the higher height assumption, shows the expected interac-tion effects between the buildings. Focusing first on the signature of the first building, “building layover”, “building corner”, and “building roof” are uninfluenced. Only “building shadow” shows contraction, due to the “building layover” of the building behind. This interac-tion applies to the second building, too. The “building layover” is subdivided, whereby the first part appears darker than the second. This is caused by the different groups of contributors. For the first part resulting from the contributors “building wall” and “building roof”, the direct reflection of the ground in front of the building is missing. The second layover part is formed by all three of them. Next to the “building layover” also the ampli-tude / brightness of “building corner” is affected, due to a smaller effective dihedral area between ground and wall. The subsequent “building roof” and “building shadow” parts of the signature are not affected by the adjacent building. An intensification of these effects is observable, if adja-cent buildings show different heights (e.g., skyscrapers in metropolis). Here, complete intermixture in the “building layover” or complete occultation in the “building shadow” would be possible. The interpreta-tion of the SAR magnitude signature alone will not be sufficient in dense built-up areas for the well known reconstruction approaches [1,2,3,4]. Of course, there exist studies [2,7] on SAR magnitude simulation of such areas, but these approaches require a high-level knowl-

layover

roofshadow

slant range

corner

�

h1

h2

h3

h4

l1

l2

l3

l4

s1

s4

s3

s2

layoverroof

shadow

slant range

corner

�

layover

corner

shadow

roof

a b

Figure 2. Building signature in SAR magnitude data by varying building height (a), Interaction effects between adjacentbuildings (b)

edge concerning the used 3D models (e.g., substruc-tures, material information). This is unrealistic for the application which we envisage. 2.2. Interferometric Phase Signature

Beside the magnitude signature of buildings also the interferometric phase signature is characterized by the phenomena caused by the side looking illumination. In general, the resulting interferometric phase value of a single range cell results, just as the magnitude, from the mixture of signal of all contributors. Due to the fact that the InSAR phase results from the backscattered signal of different objects, the InSAR height can be interpreted as a function of the heights from all objects contributing signal to the particular range cell. Consequently, in the “building layover”, heights from terrain, wall, and roof contribute to the final InSAR height. In Figure 3a, the interferometric phases of a flat-roofed building are simulated by considering only direct backscattering. In the following this simulation approach [6] is described in more detail: The InSAR phase profiles are simulated by considering an increasing building height. Due to the proportional context of the InSAR phases and the object heights, the increasing building height leads to an increasing inter-ferometric phase. The shapes of the given InSAR phase profiles are characterized by a maximum value at the beginning of the “building layover” followed by a downwards oriented ramp. This is caused by the mix-ture of different contributions, whereby the ground and “building roof” portions show constant height, but the “building wall” delivers a decreasing height portion. The lowest point in the InSAR phase profiles corre-sponds with the corner position in the magnitude data. This phase value is defined by the zero height value of ground and building wall, as well the roof height. In real InSAR data, this minimum value can be similar to local terrain phases, due to the fact that the contributions from double-bounce reflections between ground and wall

intensify this phase minimization. Behind the minimum position, the single response of the building roof leads to a constant trend in the phase profile. The subsequent shadow region shows a constant trend on zero level in the simulations, due to the fact that no signal is received for this area. In real InSAR data this area is only charac-terized by noise. In addition to the magnitude data, also the interferomet-ric phases are affected by interaction effects between adjacent buildings, which are shown in Figure 3b. A DSM-Profile (Digital Surface Model) for two buildings is given with considering four different building height assumptions. The corresponding interferometric phase profiles achieved from the simulation approach are shown in the lower part of Figure 3b. The first constel-lation marked in green, shows an example for non-interacting buildings due to the low building height. For the second constellation given in yellow, differences between the “building layover” shapes are observable. This is caused by the different contributor groups. For the first building, the layover contains a continuous mixture of ground, “building wall”, and “building roof”. In contrast, the first part of the layover of the second building is characterized only by “building wall” and “building roof”, which is effected by the “building shadow” of the first building. The bend (marked black in Fig. 3b) in the phase shape marks the point, where again all three contributors (ground, “building wall”, and “building roof”) are acting. The third constellation colored in orange, shows again differences in the lay-over shape between the buildings. The sensor closed building (left building in Fig. 3b) is not interacted, but the second shows again higher phase values in the lay-over, because only building wall and roof are scattering back. A bend like in the yellow profile is not observ-able, due to the fact that the longer shadow of the first building hampers the illumination of ground. The mini-mum position between “building layover” and “building roof” is at the same position as for the green and yellow example. Furthermore, the phase value at this position is

0 50 100 150 200 250Slant-Range [pixel]

4

3

2

1

0

-1

-2

-3

-4

Phase-Value

layover area building area

Build

ing

Hei

ght[

m]

20

11

3

0 50 100 150 200 250Slant-Range [pixel]

4

3

2

1

0

-1

-2

-3

-4

Phase-Value

layover area building area

Build

ing

Hei

ght[

m]

20

11

3

3500 35500

10

DSM-Profile

Ground Range Distance [m]

Hei

ght [

m]

0 50 100 150

0

pi/2

Slant-Range [pixel]

Phase-Profiles

3500 35500

10

DSM-Profile

Ground Range Distance [m]

Hei

ght [

m]

0 50 100 150

0

pi/2

Slant-Range [pixel]

Phase-Profiles

a b

Figure 3. Building signature in InSAR phase data by varying building height (a), Interaction effects between adjacent buildings (b)

nearly similar to the building in front. This means that the shadow of the first building ends nearly at the sensor close base point (point of intersection - between ground plan and building wall) of the second building. Focusing on the fourth – red – profile, the changes are most sig-nificant. Due to the fact that the shadow of the first building hampers the illumination of the ground and parts of the wall, the layover shapes show large differ-ences. The phase values of the start points are different caused by the different contributor groups, such as for the orange and yellow case. Furthermore, the length of the layover is different, because the lower wall part of the second building is now also occluded by the build-ing in front. Hence, the minimum of phase profile is shifted and no longer identical with the base point of the building. In addition, the single backscattering area of the second building is growing in comparison to the first building. For our future applications, these described phenomena have to be studied on real data and the interaction ef-fects have to be considered in our planed assessment step between real and simulated interferometric phases. 3. FUSION APPROACH

Concerning the discussed phenomena at building loca-tions in InSAR data, we prefer the detailed analysis of the interferometric phase information in our approach of information fusion. The approach contains the following processing steps: - Extraction of the 2D footprint of the building from

GIS data or cadastral data - Definition of an initial building height - Simulation of interferometric phases based on the

given 2D building footprint and the defined build-ing height by considering the appropriate sensor configuration and illumination geometry

- Extraction of real measured interferometric phases based on the given 2D building footprint

- Assessment between simulated and real interfer-ometric phases

- Iterative adaptation of the estimated building height up to the best assessment result

- Visualization of 3D building hypothesis and label-ing of changes

In the following we will describe some of the process-ing steps in detail. First the simulation step; our simula-tion is an adapted version to [6], where not only a single phase profile per building hypothesis is calculated but the full interferometric phase signature of the building. The simulation is based on the given 2D footprint and the assumed building height. Furthermore, the sensor and scene parameters of the InSAR data are taken into account. Based on these parameters a building ground range profile for each azimuth position is calculated. The simulation is carried out on these profiles; thereby we consider the fact that – especially at building loca-tions – a mixture of several contributions can contribute to the measured interferometric phase within a single range cell, which is visualized in Figure 4. The back-scatter signals of the object points A, B, and C are summed up into the same slant range pixel. The simulation of all the ground range profiles is per-formed as follows. The profile is split up into connected linear components of constant gradient (Figure 5, frag-ment 1-5). Afterwards, for each range cell, certain fea-tures are calculated from these segments, such as nor-mal vector, local incidence angle, range distance differ-ences and phase differences. The sum up of the calcu-lated phases is carried out according to the slant range grid of the real measured interferogram. Subsequently, the interferometric phase of each single range cell is calculated by coherently summing up all contributions (e.g., from ground, building wall and roof). At the end, we achieve for all ground range profiles one interfer-ometric phase profile in slant range geometry. Finally, all these phase profiles are connected to a final InSAR phase signature of the building. The subsequent extraction of the real measured interfer-ometric phases is realized by re-projection of the 2D building footprint into the slant range geometry of the real InSAR data. Therefore, the local terrain height is used. Due to the fact, that the real building height is

A1

A

B

C

r1A

r1B

r1C

r2A

r2Br2C

A2

equidistant ground spacing

equidistant range spacingA1

A

B

C

r1A

r1B

r1C

r2A

r2Br2C

A2

equidistant ground spacing

equidistant range spacing

A

r1A r1B r1C

DSM height profile

B

C

fragment 1

PS1 PE1

PS4

PE4

fragment 4αA

αB

αC

fragment 2

fragment 3

fragment 5

A

r1A r1B r1C

DSM height profile

B

C

fragment 1

PS1 PE1

PS4

PE4

fragment 4αA

αB

αC

fragment 2

fragment 3

fragment 5

Figure 4. Contribution of InSAR measurements at building location

Figure 5. DSM profile of a flat-roofed building based on fragments; local incidence angle of radar signal

unknown, only the position of the corner line is assess-able. Hence, based on the assumed corner position in the slant range data and an estimated maximum building height, an area from the real measured InSAR phases is extracted. The subsequent assessment step between simulated and extracted real InSAR phases is based on the correlation coefficient. The necessary registration between both real and simulated phase areas is realized by the corner posi-tion known in both. During the assessment step only the layover and the single roof backscattering part are con-sidered. The simulation and adjustment step are repeated by in-creasing building height up to a defined maximum height or another abort criterion (e.g., correlation value). Afterwards, based on the best fit between real and simu-lated phases a 3D building hypothesis is found or in case of reconstruction failure, a significant change at the building is assumed. When applying our approach as a part of a crisis information system this change can be interpreted as possible damage. 4. RESULTS

The first results of our approach are given in Figure 6. In the first column (a) an orthophoto (row 1,2) and LI-DAR image (row 3) overlaid with the GIS information

are given, where the second and third row show the same building. In the second column (b), the simulated interferometric phases are visualized by considering the ground truth building height at the simulation step. The real interferometric phases are shown in the third col-umn (c). The fourth column (d) contains the assessment plot of the iterative increasing of the building height. The first and second row building examples were mapped by the AeS-1 SAR sensor of Intermap Tech-nologies. The data of the third example were acquired by the STAR 3i system of Intermap Technologies, too. The relatively low correlation values for all buildings result from the actualities of the situation. There are in all cases interaction effects with closed vegetation ob-servable, which hamper or inhibit the appearance of layover. Furthermore, the real phase signature of the building roofs is affected by superstructures on the roof, which are not considered in the used simulation step. The highest correlation value between real and simu-lated phases is achieved for the first building. The dif-ferences between the correlation result of the second and third row are probably caused by the different sen-sor configurations (e.g., baseline configuration, height ambiguity interval, geometric resolution), because the mapped scene and the illumination direction are nearly identical.

0 5 10 15 20

0

0.2

0.4

0.6

0.8

1

building height [m]

corre

latio

n va

lue

0 5 10 15 20

0

0.2

0.4

0.6

0.8

1

building height [m]

corre

latio

n va

lue

0 5 10 15 200

0.2

0.4

0.6

0.8

1

building height [m]

corre

latio

n va

lue

a b c d

Figure 6. Optical image and LIDAR image overlaid with 2D GIS information (a), simulated InSAR phases at building location (b), real InSAR phases at building location (c), correlation plot of real and simulated phases (d)

5. CONCLUSION AND OUTLOOK

In this paper we presented an approach for fusing 2D GIS data with InSAR phase data to achieve 3D building models. The approach implied the simulation of inter-ferometric phases, the extraction of real InSAR phases, as well as their assessment. Finally, 3D building hy-potheses and correlation values between the phases were delivered for the shown three examples. In the future, we will focus on three working topics to improve our preliminary results. Firstly, we want to improve our correlation results by modifying our phase simulation and the interferogram calculation steps. Sec-ondly, we will test our approach on more complex building structures, where we have to switch from such isolated buildings to buildings showing interaction ef-fects with the adjacent buildings. Thirdly, we will refer to crisis information system, due to the fact that a de-tected change between measured and simulated phase, can be interpreted as a possible damage. And finally, considering the test data, we will test the impact of dif-ferent baseline and off-nadir angle configurations on the correlation result. Especially, with regard to transfer this concept to current (dual- or multi-pass) TerraSAR-X interferograms (Figure 7) and to data of the upcoming mission TanDEM-X. 6. REFERENCES

[1] Bolter R (2001) Buildings from SAR: Detection and Reconstruction of Buildings from Multiple

View High Resolution Interferometric SAR Data. University Graz: Ph. D. Thesis

[2] Soergel U, Thoennessen U, Stilla U (2003) Itera-tive Building Reconstruction in Multi-Aspect In-SAR Data. In: Maas HG, Vosselman G, Streilein A (eds) 3-D Reconstruction from Air-borne Laser-scanner and InSAR Data, IntArchPhRS, vol 34, part 3/W13, pp 186-192

[3] Jahangir M, Blacknell D, Moate CP, Hill RD (2007) Extracting information from shadows in SAR imagery. IEEE Proceedings of the Interna-tional Conference on Machine Vision, pp 107-112

[4] Xu F, Jin YQ (2007) Automatic Reconstruction of Building Objects From Multiaspect Meter-Resolution SAR Images. IEEE Transactions on Geoscience and Remote Sensing, vol 45, no7, pp 2336-2353

[5] Thiele A, Cadario E, Schulz K, Thoennessen U, Soergel U (2007a) Building recognition from multi-aspect high resolution InSAR data in urban area. IEEE Transactions on Geoscience and Re-mote Sensing, vol 45, no 11, pp 3583-3593

[6] Thiele A, Cadario E, Schulz K, Thoennessen U, Soergel U (2007b) InSAR Phase Profiles at Build-ing Locations. Proceeding of ISPRS Photogram-metric Image Analysis, vol. XXXVI, part 3/W49A, pp 203-208

[7] Auer, S., Hinz, S., Bamler, R. (2010) Ray Tracing Techniques for Understanding High Resolution SAR Images. IEEE Transactions on Geoscience and Remote Sensing, 2010 (in press)

a b

c

Figure 7. Berlin: optical image (source: Google Earth) overlaid with 2D GIS information (a), InSAR phases of Spot-light TerraSAR-X data (source: DLR) (b), optical image overlaid with 3D Building models (source: Google Earth) (c)