URBAN SOUND SYMPOSIUM · Sound technologies: Topography for quiet areas and quiet sides Jin Yong Jeon ... sound and visual components for enhancement of urban soundscapes” The Journal

Sound technologies: Topography for quiet areas and quiet sides

Jin Yong Jeon

Hanyang University, SEOUL, KOREA

4 April 2019

URBAN SOUND SYMPOSIUMApril 3-5, 2019 in Ghent, Belgium

Ghent University

Soundscape descriptors

• Pleasantness-Eventfulness model

Typical approach

PleasantUnpleasant

Eventful

Uneventful

Exciting

Boring Calm

Chaotic

Soundscape Design

불쾌한 쾌적한

활기찬

정온한

비활동적인

단조로운

혼란스러운

활동적인

Ö . Axelsson, M.E. Nilsson, and B. Berglund, “A principal components model of soundscape perception.”The Journal of the Acoustical Society of America,128(5),2836–2846,2010.

Nature appreciation

and Tranquility

Vibrant city life

From noise control to soundscape

• Soundscape concepts• ISO 12913-1 “acoustic environment as perceived or experienced and/or understood by a person or people, in context”

• Correlation between acoustic indicators and soundscape descriptors(Yang & Kang, 2005; Yu & Kang, 2009; Kang & Zhang, 2010; Hong & Jeon, 2013; Meng, Sun, & Kang, 2017)

• Relationship between soundscape and context (Jeon, Lee, Hong & Cabrera, 2011; Galbrun & Calarco, 2014)

• Soundscape perception model• Correlation between soundscape and landscape

(Southworth, 1969; Pheasant, Horoshenkov & Watts, 2008; Joynt and Kang, 2010; Liu, Kang, Behm, Luo & Tao, 2014)

• Urban environment (soundscape & landscape) interpretation model (Liu et al., 2013; 2014; Yu, Behm, Bill & Kang, 2017)

• Audio-visual interaction• Relationship between landscape spatial patterns (e.g., urban morphology) and soundscapes for entire cities

(Ge, Lu, Morotomi & Hokao, 2009; ; Mazaris, Kallimanis, Chatzigianidis, Papadimitiou & Pantis, 2009; Liu, Kang, Behm & Coppack, 2013; Hong & Jeon, 2017)

• Audio-visual interaction on soundscape perception (traditional 2D photographs, collage, or photoshop)(Stamps, 1993; Lange, 2001; Daniel, 2001; de Val, Atauri, & de Lucio, 2006; Hong & Jeon, 2013)

Previous studies

Related audio-visual studies (2013-2018)

• Preference• Natural elements increase the aesthetic preference

• Laboratory experiment• Photomontage method

Audio-visual interaction (Hong & Jeon, 2013)

Progressive evolvement

J.Y. Hong and J. Y. Jeon, “Designing sound and visual components for enhancement of urban soundscapes” The Journal of the Acoustical Society of America,134(3),2026–2036,2013.

• Preference• Vegetated (Ve) > concrete (Co) > wood (Ti) >

translucent acrylic (Tr) > aluminum (Al)

• Different types of noise barrier

Audio-visual interaction (Hong & Jeon, 2014)

Progressive evolvement

Hong, J. Y. and Jeon, J. Y. “The effects of audio–visual factors on perceptions of environmental noise barrier performance,” Landsc. Urban Plan., 125, 28–37, 2014.

Influence of urban morphology (Hong & Jeon, 2017)

• Urban morphological indices

More expanded recognition model

Indicators Definition Formula Range

Bldg (A) Sum of building area 𝐢

𝐧

𝐁𝐢𝐝𝐠 𝐚𝐫𝐞𝐚 0.00-144543.70

Bldg (P) Sum of building perimeter 𝐢

𝐧

𝐁𝐢𝐝𝐠 𝐩𝐞𝐫𝐢𝐦𝐭𝐞𝐫 0.00-4819.35

Bldsf (A) Sum of building surface area 𝐢

𝐧

𝐁𝐢𝐝𝐠 𝐬𝐮𝐫𝐟𝐚𝐜𝐞 𝐚𝐫𝐞𝐚 0.00-10387198

BPAF The ratio of the plan area of buildings to the total surface area𝐁𝐢𝐝𝐠(𝐀)

𝐆𝐫𝐢𝐝(𝐀)0.00-0.64

CAR The summed area of roughness elements and exposed ground divided by the total surface area of the study region𝐁𝐢𝐝𝐬𝐟(𝐀)

𝐆𝐫𝐢𝐝(𝐀)0.22-4.82

Gr (A) Sum of green area 𝐢

𝐧

𝐆𝐫𝐞𝐞𝐧 𝐚𝐫𝐞𝐚 0.00-1575.57

Gr (P) Sum of green perimeter 𝐢

𝐧

𝐏𝐞𝐫𝐢𝐦𝐞𝐭𝐞𝐫 𝐨𝐟 𝐠𝐫𝐞𝐞𝐧 𝐚𝐫𝐞𝐚 0.00-557.03

Op (A) Sum of open public area including urban squares, green and water areas 𝐢

𝐧

𝐎𝐩𝐞𝐧 𝐩𝐮𝐛𝐥𝐢𝐜 𝐚𝐫𝐞𝐚 0.00-15754.57

Op (P) Sum of open perimeter including urban squares, green and water feature areas 𝐢

𝐧

𝐏𝐞𝐫𝐢𝐦𝐞𝐭𝐞𝐫 𝐨𝐟 𝐨𝐩𝐞𝐧 𝐩𝐮𝐛𝐥𝐢𝐜 𝐚𝐫𝐞𝐚 0.00-686.20

OSR The ratio of the open area divided by the total surface area of the study region𝐎𝐩(𝐀)

𝐆𝐫𝐢𝐝(𝐀)0.00-0.70

Grd (A) Sum of exposed ground area 𝐢

𝐧

𝐄𝐱𝐩𝐨𝐬𝐞𝐝 𝐠𝐫𝐨𝐮𝐧𝐝 𝐚𝐫𝐞𝐚 630.29-174169.47

Rd (A) Sum of road area 𝐢

𝐧

𝐑𝐨𝐚𝐝 𝐚𝐫𝐞𝐚 32.07-14765.52

EGR The ratio of the exposed ground area divided by the total surface area of the study region𝐆𝐫𝐝(𝐀)

𝐆𝐫𝐢𝐝(𝐀)0.03-0.77

RAF The ratio of the road area to the study region𝐑𝐝(𝐀)

𝐆𝐫𝐢𝐝(𝐀)0.00-0.66

Wt (A) Sum of water feature area 𝐢

𝐧

𝐖𝐚𝐭𝐞𝐫 𝐟𝐞𝐚𝐭𝐮𝐫𝐞 𝐚𝐫𝐞𝐚 0.00-2106.92

Wt (P) Sum of water feature perimeter 𝐢

𝐧

𝐏𝐞𝐫𝐢𝐦𝐞𝐭𝐞𝐫 𝐨𝐟 𝐰𝐚𝐭𝐞𝐫 𝐟𝐞𝐚𝐭𝐮𝐫𝐞 𝐚𝐫𝐞𝐚 0.00-342.11

J. Y. Hong and J. Y. Jeon, “Relationship between spatiotemporal variability of soundscape and urban morphology in a multifunc tional urban area: A case study in Seoul, Korea” Build Environ 2017;126:382–95.

• Main space functions• Morphological factors



Component 1 (41.29%) 2 (19.44%) 3 (16.81%) 4 (10.54%)

Open space

Gr (A) 0.89 - 0.11 - 0.04 - 0.09

Gr (P) 0.84 - 0.16 - 0.10 - 0.09

Op (A) 0.90 - 0.29 0.10 0.04

Op (P) 0.77 - 0.40 0.08 0.31

OSR 0.89 - 0.32 0.11 0.17

Building

Bldg (A) - 0.18 0.91 - 0.10 - 0.19

Bldsf (A) - 0.30 0.87 0.06 - 0.09

Bldg (P) - 0.09 0.75 0.06 - 0.14

BRAF - 0.18 0.94 - 0.11 - 0.13

CAR - 0.36 0.81 - 0.06 - 0.01

Water feature

Wt (A) 0.04 - 0.20 0.10 0.96

Wt (P) 0.02 - 0.21 0.09 0.96

Road

Rd (A) - 0.23 - 0.18 0.90 - 0.16

Grd (A) - 0.39 - 0.17 - 0.79 - 0.30

RAF - 0.22 - 0.27 0.92 0.12

EGR - 0.42 - 0.25 - 0.85 - 0.14



• Perceived affective model• Pleasantness model was developed using LAeq, open space and water feature components

• Pleasantness model show higher R2 than eventfulness model



Pleasantness Eventfulness

Total P1 P2 P3 Total P1 P2 P3

R2 0.49 0.54 0.53 0.50 0.13 0.08 0.22 0.21

Acoustic

Laeq - 0.67** - 0.45** - 0.65** - 0.81** 0.22** 0.02 0.32** 0.17

LCeq-Aeq - 0.05 0.13 - 0.10 - 0.11 - 0.17** - 0.30* - 0.24* - 0.04

L10-90 0.12** 0.16 0.06 0.13 - 0.11 - 0.09 - 0.15 - 0.09

Sharpness - 0.03 0.27 - 0.07 - 0.13 - 0.06 - 0.09 - 0.05 - 0.04

Morphological

Open space 0.12** 0.12 0.14* - 0.01 0.05 - 0.01 0.17* - 0.01

Building 0.00 0.04 - 0.06 0.09 0.11* - 0.06 0.11 0.25**

Road 0.00 -0.02 - 0.02 0.07 0.10 0.05 0.08 0.23*

Water feature 0.26** 0.24** 0.27** 0.25** 0.10 0.09 0.10 0.14

P1 (09:00–11:00), P2 (13:00–15:00) and P3 (18:00–20:00)

Soundscape mapping (Hong & Jeon, 2017)

• Soundscape perception• Temporal variation: pleasantness > eventfulness

Visualization


Development of VR tool

• Spatial audio in VR (Hong et al., 2017)• Visual image• 360 camera (Insta 360, Samsung gear, …)

• 8k ultra-high definition, 30 fps resolution

• Head mounted display (Vive, oculus, …)

• Spatial audio recording • Stereo and surround recording

• Microphone array

• Ambisonics (well suitable)

• Binaural recording (most common used)

• Spatial audio reproduction• Perceptual reconstruction

• Binaural, transaural

• Physical reconstruction

• Stereo, multichannel, ambisonics, wave field synthesis

• Calibration with dummy torso (B&K)

Immersive soundscape evaluation tools

Characteristics of the Acoustic Environ. Recommended Techniques

Spatial

Fideli

Movements Virtual Sound

Source

Localization

Reproduction

Techniques

Recording

TechniquesListener

Pos.Head

Low

X X 0DMono loudspeaker;

stereo headphoneMono

X X 1DStereo/surround loudspeaker;

stereo headphoneStereo/surround

X X 2D

Surround sound

Loudspeakers with heightArray

Ambisonics (2D) Ambisonics

Med

X X 3D- Ambisonics; BinauralAmbisonics;

Binaural;

X X 3D+Personalized

binaural (PB)

Personalized

binaural;

Ambisonics

X 3D+Binaural/PB with

head trackingAmbisonics

High

3D+WFS; Binaural/PB with

positional & head tracking

Mono (anechoic);

Ambisonics

3D+WFS; Binaural/PB with

positional & head tracking

Mono (anechoic);

Ambisonics

Hong et al. Spatial Audio for soundscape Design: Recording and Reproduction. Applied Sciences 2017;7:627–49.

Validation of VR techniques (Hong et al., 2019)

• Compare soundscape in situ and VR environment• In situ, FOA-static binaural, FOA-tracked binaural, FOA-2D octagonal array

• Overall soundscape quality• No significant differences in 4 different environments

• Sufficient spatial aural fidelity• FOA-tracked binaural play back, FOA-octagonal speaker array

Subjective attributes Acoustic reproduction methods

FOA-static binaural FOA-tracked binaural FOA-2D octagonal array

Overall soundscape quality

Dominance of sound sources ○ ○ ○

Affective quality of soundscape ○ ○ ○

Source-related spatial attributes

Distance ▲ ▲ ▲

Directivity ▲ ○ ○

Width ○ ○ ○

Distinctiveness ▲ ○ ○

Hong et al. Quality assessment of acoustic environment reproduction methods for cinematic virtual reality in soundscape applications, Building and Environment 149. 1-4 (2019)

VR technology in indoor environment

Effect of visual information (Jeon et al., 2019)

• Head mounted display (HMD)• Sound sources: water supply and drainage noise

• The difference of acceptance limit and annoyance : 6% and 8%

J. Y. Jeon, H. I. Jo, S. M. Kim, H. S. Yang, “Subjective and objective evaluation of water-supply and drainage noises in apartment buildings by using a head-mounted display” Applied Acoustics 48:289-99 (2019).

Audio-visual interaction (Jeon & Jo, 2019)

• HMD + Head related transfer function (HRTF)• Sound sources: road traffic noise

• Experiment environment: None, HRTF, HMD, HRTF+HMD

• Effect of HRTF and HMD on subjective evaluation: 77% and 23 %

• Source-related spatial attributes: HRTF dominant effect

• Environment-related spatial attributes: HMD dominant effect

J. Y. Jeon, H. I. Jo, “Three-dimensional virtual reality-based subjective evaluation of road traffic noise heard in urban high-rise residential buildings”, Building and Environment, 148, 468-477 (2019).

Source-related spatial attributes Environment-related spatial attributes

Audio-visual interaction (Jo & Jeon, 2019)

• HMD + Head related transfer function (HRTF)• Sound sources: heavy-weight impact noise

• Experiment environment: None, HRTF, HMD, HRTF+HMD

• Annoyance: HRTF dominant, HMD (higher than 53 dBA)

• Allowance limit: HRTF odd ratio (2.90) , HMD odd ratio (1.30)

• Lowered the criterion for satisfaction by 6-7 dB

Class %ALA,Fmax [dBA]

NONE HRTF HMD HRTF+HMD

I 0 − 20% <48.0 <41.5 <48.0 <42.5

II 20 − 40% <52.0 <45.5 <52.0 <46.0

III 40 − 60% <56.0 <49.0 <55.0 <49.0

IV 60 − 80% <59.5 <52.5 <58.0 <52.0

V 80 − 100% ≥59.5 ≥52.5 ≥58.0 ≥52.0

Participant

Television

Living room

Balcony

Kitchen

A

A’

B

H. I. Jo, J. Y. Jeon, “Downstairs resident classification characteristics for upstairs walking vibration noise in an apartment building under virtual reality environment”, Building and Environment, 150, 21-32 (2019).

VR technology in outdoor environment

Main issue

• Main objective• To investigate the correlations of the overall satisfaction of urban environment with soundscape and landscape,

respectively, and examine the influence of audio-visual interactions

Introduction

STEP 1 STEP 2

Capturing & Analysis

of Urban SoundscapePsychoacoustic

Evaluation

STEP 3

Deployment of

Soundscape

Recognition

model

• 3D Audio-Visual Recording

• Analyzing psychoacoustic indicators• Psychoacoustic evaluation using VR

• Determination landscape and soundscape indicator

• Development of soundscape mapping

• Applying in the smart city

Human behavior

• Urban park soundscape• Activity: chatting, loitering, talking on the phone, stroll, …

• Group: alone, group

Recent soundscape research

Audio-visual interaction in VR environments

• Virtual reality techniques• Recording method

• 360° Camera (Insta 360) + Soundfield microphone (SPS 200)

• Reproduction method

• Head mounted display (HMD) + 3D auralization (FOA + head tracking)

Methods

3 different experiment set-ups

1) Only audio

2) Only visual

3) Combined audio and visual

H1 H2 H3

H4 H5 H6

H7 H8 H9H1

H2

H3H5

H4

H6

H7H8

H9

Locations LAeq LA10 LA50 LA90 LA10-A90 LCeq-Aeq

H1 79.4 82.6 78.3 70.1 12.5 16.5

H2 71.1 73.2 70.5 60.9 12.3 9.5

H3 68.4 71.1 67.0 61.0 10.1 13.4

H4 69.4 71.8 68.5 62.9 8.9 16.5

H5 65.7 68.8 63.3 57.3 11.6 6.6

H6 72.8 75.9 66.2 60.2 15.8 9.2

H7 65.1 67.1 64.9 61.5 5.6 7.7

H8 57.2 69.4 59.9 58.0 11.4 9.3

H9 60.1 63.0 56.8 55.2 7.8 17.9

Audio-visual interaction

• Correlation between soundscape elements and landscape elements• Soundscape and landscape components: PCA analysis

• Pleasantness show positive correlation with overall quality, regularity, naturalness

• Eventfulness show negative correlation with regularity

Discussion

Visual elements Landscape components

Vehicle Building Road Open Green People Sky Overall quality Regularity Spatial impression Naturalness

Sound

Sources

Traffic 0.72** 0.29** 0.63** -0.04 -0.07 0.09 0.01 -0.28** -0.22** 0.09 -0.32**

Human 0.07 0.16** 0.06 0.04 0.05 0.43** 0.22** 0.05 0.01 0.10 -0.09

Bird -0.22** -0.04 -0.19** 0.18** 0.25** 0.04 -0.05 0.30** 0.17** -0.01 0.23**

Wind 0.05 0.13* 0.06 0.01 0.15* 0.18** 0.22** 0.11 0.12 0.12 0.08

Music -0.12 -0.04 -0.04 -0.06 0.17** -0.01 -0.14* 0.01 0.03 -0.03 0.12**

Soundscape

components

Pleasantness -0.56** -0.21** -0.53** 0.02 0.08 -0.01 -0.01 0.50** 0.30** -0.06 0.26**

Eventfulness 0.05 0.08 -0.03 -0.12 -0.11 0.05 0.24** 0.17** -0.24** 0.11 -0.11

Soundscape satisfaction model

• Regression model using sound and visual element • Bird sound and the visual element of vehicles are major factors

• Regression model using soundscape and landscape components• Pleasantness, overall quality, regularity, and naturalness are major factors

Conclusion

Environment R2 Pleasantness Eventfulness Overall quality Regularity Spatial impression Naturalness

Visual effect Only Audio 0.45 0.66** 0.12** - - - -

Audio + Visual 0.32 0.56** -0.05** - - - -

Audio effect Only Visual 0.49 - - 0.62** 0.28** 0.16** -

Audio + Visual 0.49 - - 0.55** 0.29** 0.05 0.32**

Interaction Audio + Visual 0.51 0.21** -0.05 0.45** 0.22** 0.06 0.25**

Environment R2 Traffic Human Bird Wind Music Vehicle Building Road Open Green People Sky

Visual effect Only Audio 0.14 -0.27** 0.12* 0.12* -0.14* 0.02 - - - - - - -

Audio + Visual 0.25 -0.29** 0.03 0.31** -0.01 0.07 - - - - - - -

Audio effect Only Visual 0.15 - - - - - -0.28** 0.06 -0.05 0.17* 0.12 -0.08 0.05

Audio + Visual 0.21 - - - - - -0.36** -0.02 -0.10 0.07 0.11 0.02 -0.06

Interaction Audio + Visual 0.31 -0.02 .005 0.29** 0.02 0.04 -0.30** -0.03 -0.07 0.03 0.04 -0.01 -0.04

Conclusion

• VR technology in noise evaluation• Visual information: Head mounted display (HMD)

• Audio information

• Head related transfer function (HRTF)

• Steam audio technology: combine accurate occlusion, reflection, reverb and HRTF effects for natural sounding immersion

• Audio-visual interaction in urban environment perception• Soundscape elements

• Dominant sound sources, perceived affective quality, and so on

• Landscape elements

• Urban morphology, dominant visual elements, human behavior, and so on

• Further plan• Utilizing urban big-data with deep learning technology

• Real-time 3D soundscape mapping technology3D soundscape mapping

Thank you for your [email protected] www.researchgate.net/profile/Jin_Yong_Jeon

Documents

URBAN SOUND SYMPOSIUM · Sound technologies: Topography for quiet areas and quiet sides Jin Yong Jeon ... sound and visual components for enhancement of urban soundscapes” The Journal