20110415 Scattering in CG and CV

Scattering in CG and CV

Toru Tamaki

(Hiroshima University)

HIISeminar 2011/4/15

Fog

Underwater

Sunset

Drink

Jewely

Hair

Konnyaku

http://photosozai-database.com/modules/photo/photo.php?lid=664

Outline

• Reflection, Scattering – Diffuse / specular

reflection, BRDF – Subsurface scattering,

single/multiple scattering

• Subsurface scattering models – Diffuse approximation

• Dipole/Multipole

– Plane-parallel approximation

• Semi-analytic solution • Anisotropic scattering

– Donner’s empirical BSSRDF

• Measuring scattering – Single scattering with

liquid – Multiple scattering

(2D・3D) – 2 layer model, multi-

layer model

REFLECTION, SCATTERING OF LIGHT

Diffuse / specular reflection, BRDF

Subsurface scattering, single/multiple scattering

Diffuse reflection

• Constant radiance – Independent to

viewpoint

• Many CV assume – Stereo, multi-view

– Shape-from-shading

• Pure diffuse surface – Lambertian

• Example – Plaster statues

http://sekkouzou.shop-pro.jp/?pid=22649690

copyright©2010 石膏像ドットコム（堀石膏制作）,All Rights Reserved.

Active range finder

山崎俊太郎, DLPプロジェクタによる高速度構造化光投影を利用した高速度アクティブビジョン, http://www.dh.aist.go.jp/~shun/research/dlp/index.html.ja

Copyright © 2009, Shuntaro Yamazaki.

If not Lambertian:

古瀬達彦, 日浦慎作, 佐藤宏介, スリット光の変調による相互反射と表面下散乱に頑健な三次元形状計測, MIRU2009, pp. 214-221, 2009. Tatsuhiko Furuse, Shinsaku Hiura and Kosuke Sato, 3-D Shape Measurement Method with Modulated Slit Light Robust for Interreflection and Subsurface Scattering, PROCAMS2009, pp. 1-2, 2009.

Copyright © by Authors

http://www.eml.hiroshima-u.ac.jp/member/staff/tamaki/memo/index.php?MIRU

Specular reflection

𝜃

• Angle of reflection

– is angle of incidence

• Highlights

– More reflection near to the angle of reflection

– Phong model • Proportional to cos𝑛 𝜃

BRDF

𝑥

𝜔𝑖 𝜔𝑜

• Any refection

• Bidirectional Reflectance Distribution Function – 双方向反射分布関数

• 6 parameters – Point 𝑥, angles of

incidence 𝜔𝑖 and reflection 𝜔𝑜

• If isotropic and homogeneous – 3 parameters

Wojciech Matusik, Hanspeter Pfister, Matthew Brand, and Leonard McMillan. 2003. Efficient isotropic BRDF measurement. In Proceedings of the 14th Eurographics workshop on Rendering (EGRW '03). Eurographics Association, Aire-la-Ville, Switzerland, Switzerland, 241-247.

Isotropic BRDF measurement

Eurographics Association Aire-la-Ville, Switzerland, Switzerland ©2003

Wojciech Matusik, Hanspeter Pfister, Matthew Brand, and Leonard McMillan. 2003. Efficient isotropic BRDF measurement. In Proceedings of the 14th Eurographics workshop on Rendering (EGRW '03). Eurographics Association, Aire-la-Ville, Switzerland, Switzerland, 241-247.

Rendering with measured BRDF

Eurographics Association Aire-la-Ville, Switzerland, Switzerland ©2003

武田祐樹, 坂口嘉之, 田中弘美, 少数視点画像の反射光解析に基づくシルクライク織物の異方性反射レンダリング, 芸術科学会論文誌, Vol. 7, No. 4, pp.132-143, 2008 .

Anisotropic BRDF


Scattering

• Media

– Volumetric

– Subsurface

• Wavelengths

– Rayleigh scattering

– Mie scattering

• Number of scattering

– Single scattering

– Multiple scattering • Double scattering

• Triple scattering

• …

Volumetric scattering in CG

1987

1993

1994

Tomoyuki Nishita, Yasuhiro Miyawaki, and Eihachiro Nakamae. 1987. A shading model for atmospheric scattering considering luminous intensity distribution of light sources. SIGGRAPH Comput. Graph. 21, 4 (August 1987), 303-310. DOI=10.1145/37402.37437 http://doi.acm.org/10.1145/37402.37437

Tomoyuki Nishita, Takao Sirai, Katsumi Tadamura, and Eihachiro Nakamae. 1993. Display of the earth taking into account atmospheric scattering. In Proceedings of the 20th annual conference on Computer graphics and interactive techniques (SIGGRAPH '93). ACM, New York, NY, USA, 175-182. DOI=10.1145/166117.166140 http://doi.acm.org/10.1145/166117.166140

Tomoyuki Nishita and Eihachiro Nakamae. 1994. Method of displaying optical effects within water using accumulation buffer. In Proceedings of the 21st annual conference on Computer graphics and interactive techniques (SIGGRAPH '94). ACM, New York, NY, USA, 373-379. DOI=10.1145/192161.192261 http://doi.acm.org/10.1145/192161.192261

Copyright © 2011 by the Association for Computing Machinery, Inc. (ACM).

Tomoyuki Nishita, Yoshinori Dobashi, and Eihachiro Nakamae. 1996. Display of clouds taking into account multiple anisotropic scattering and sky light. In Proceedings of the 23rd annual conference on Computer graphics and interactive techniques (SIGGRAPH '96). ACM, New York, NY, USA, 379-386. DOI=10.1145/237170.237277 http://doi.acm.org/10.1145/237170.237277

Yonghao Yue, Kei Iwasaki, Bing-Yu Chen, Yoshinori Dobashi, and Tomoyuki Nishita. 2010. Unbiased, adaptive stochastic sampling for rendering inhomogeneous participating media. ACM Trans. Graph. 29, 6, Article 177 (December 2010), 8 pages. In ACM SIGGRAPH Asia 2010 papers (SIGGRAPH ASIA '10). DOI=10.1145/1882261.1866199 http://doi.acm.org/10.1145/1882261.1866199

Volumetric scattering in CG

1996

2010


Subsurface scattering

• Transmitted light – Non-metal

• Scattering – Inside medium

– Some come the surface

– Some absorbed

• Outgoing light – From a point

different from the incident point

Surface reflection

Henrik Wann Jensen, Stephen R. Marschner, Marc Levoy, and Pat Hanrahan. 2001. A practical model for subsurface light transport. In Proceedings of the 28th annual conference on Computer graphics and interactive techniques (SIGGRAPH '01). ACM, New York, NY, USA, 511-518.


Subsurface scattering



Surface reflection



Surface reflection



Surface reflection



Surface reflection



Single scattering

Double scattering

Multiple scattering

Multiple scattering



Single scattering



Single + Multiple scattering + Fresnel reflection



Single + Multiple scattering

Craig Donner, Jason Lawrence, Ravi Ramamoorthi, Toshiya Hachisuka, Henrik Wann Jensen, and Shree Nayar. 2009. An empirical BSSRDF model. ACM Trans. Graph. 28, 3, Article 30 (July 2009), 10 pages. In ACM SIGGRAPH 2009 papers (SIGGRAPH '09). DOI=10.1145/1531326.1531336


Single scattering



BSSRDF

𝑥𝑖

𝜔𝑖

𝜔𝑜

𝑥𝑜

• Bidirectional Scattering Surface Reflectance Distribution Function – 双方向深層散乱反射

分布関数

• 9 parameters – Points of incidence 𝑥𝑖

and outgoing 𝑥𝑜, angles of incidence 𝜔𝑖 and outgoing 𝜔𝑜

• If isometric and homogeneous – 5 parameters

SUBSURFACE SCATTERING MODELS

Diffuse approximation: Dipole/Multipole

Plane-parallel approximation: Semi-analytic solution,

Anisotropic scattering,

Donner’s empirical BSSRDF

Dipole model

• Homogeneous and semi-infinite medium

• Single + multiple scattering (isometric and diffusion)

• Sum of two terms – Each represents a

virtual point light source





Multipole approximation

• Finite thickness

– Infinite sum of Dipoles

– Multipole model

• Finite-thick layers

– The use of some Multipole models


Craig Donner and Henrik Wann Jensen. 2005. Light diffusion in multi-layered translucent materials. ACM Trans. Graph. 24, 3 (July 2005), 1032-1039. In ACM SIGGRAPH 2005 Papers (SIGGRAPH '05). DOI=10.1145/1073204.1073308



Homogeneous medium

Homogeneous medium

＋ Dye spraying




Backlight


Homogeneous medium

Homogeneous medium

＋ Dye spraying

Dipole approximation


Finite-thick medium (paper)





Finite-thick medium (paper)


Backlight Sidelight


Multipole approximation of human skin layers

Dipole approximation






Plane-parallel approximation

• Dipole

• Multipole

• Semi-infinite plane medium

– Depends on the depth 𝑧 from the surface

– Plane-parallel

Single layer, semi-infinite

Finite-thick layers

Layered homogeneous

Analytic solution

新谷幹夫, 白石路雄, 土橋宜典, 岩崎慶, 西田友是, "固有解を用いた表面下散乱の高速表示", Visual Computing / グラフィクスと CAD 合同シンポジウム 2008, 2008. http://nis-lab.is.s.u-tokyo.ac.jp/nis/abs_cgi.html#ipsj09-2

Integro-differential equation

Integro-ordinary differential equation

Discretization

Eigensolution Eigenfunction

Eigenvalue

Copyright © 画像電子学会および(社)情報処理学会

http://www.eml.hiroshima-u.ac.jp/member/staff/tamaki/memo/index.php?Visual Computing

新谷幹夫，土橋宜典，白石路雄，岩崎慶，西田友是, "多重散乱モデルの効率化と樹木レンダリングへの応用", Visual Computing / グラフィクスと CAD 合同シンポジウム 2009, 2009.

単散乱

モンテカルロ準解析解

準解析解形状補正

単散乱

準解析解

Copyright © 画像電子学会および(社)情報処理学会

Comparison between analytic solution and single scattering


Anisotropic Plane-parallel model

Nelson Max, Suguru Saito, Kazuya Watanabe, Masayuki Nakajima, "Rendering Grass Blowing in The Wind with Global Illumination", Pacific Graphics 2009, 2009. Nelson Max, Kazuya Watanabe, Suguru Saito, Masayuki Nakajima, "Plane-Parallel Radiance Transport for Rendering Grass Blowing in The Wind", Visual Computing / グラフィクスと CAD 合同シンポジウム 2009, 2009.

新谷幹夫，白石路雄, 土橋宜典，岩崎慶，西田友是, "異方性Plane-Parallel散乱モデルによる毛髪状物体の高速レンダリング", Visual Computing / グラフィクスと CAD 合同シンポジウム 2010, 2010.

Copyright ©一般社団法人画像電子学会，(社)情報処理学会および(社)映像情報メディア学会



Donner’s empirical BSSRDF

• Semi-infinite homogeneous plane medium


𝑥𝑖

𝜔𝑖

𝜔𝑜

𝑥𝑜

• Simulating 5D parameter BSSRDF

– 6 parameters

– 0.85 million patterns

• Function fitting

– Each incident / outgoing angles / points

– 36GB data






Monte Carlo Method



Single scattering only



Single scattering ＋ Dipole model



Single scattering ＋ This model



Monte Carlo Method



MEASURING SCATTERING

Single scattering with liquid

Multiple scattering (2D・3D)

2 layer model, multi-layer model

Measuring single scattering with diluted liquied

Srinivasa G. Narasimhan, Mohit Gupta, Craig Donner, Ravi Ramamoorthi, Shree K. Nayar, and Henrik Wann Jensen. 2006. Acquiring scattering properties of participating media by dilution. ACM Trans. Graph. 25, 3 (July 2006), 1003-1012. In ACM SIGGRAPH 2006 Papers (SIGGRAPH '06). DOI=10.1145/1141911.1141986


http://www.eml.hiroshima-u.ac.jp/member/staff/tamaki/memo/index.php?scattering







Rendering as changing scattering parameters




Interpolation of scattering parameters




Interpolation of scattering parameters




貴田健介，坂上文彦，佐藤淳, "表面下散乱に基づく半透明物体の厚み推定", MIRU2010, pp. 578-584, 2010.


2 layer model: two translucent materials


2 layer mode: scattering in liquid and metal reflectance

馬場葉子，向川康博，八木康史，"照明光と反射光の散乱を考慮した反射特性の推定"，CVIM, 2011-CVIM-175(41), 2011.

Copyright © by (社)情報処理学会

2 layer model: makeup and skin layers

馬場葉子，向川康博，八木康史，"化粧と肌の２層構造からなる化粧肌反射特性モデル", MIRU2010, pp. 1679-1685, 2010.



How heavy (or light) makeup is

馬場葉子，向川康博，八木康史，"化粧と肌の２層構造からなる化粧肌反射特性モデル", MIRU2010, pp. 1679-1685, 2010.



Analysis of face skin w.r.t ages and genders

馬場葉子，間下以大，向川康博，八木康史，"大規模データベースを用いた肌の反射・散乱光の統計的解析", MIRU2009, pp. 1164-1171, 2009.



馬場葉子，間下以大，向川康博，八木康史，"大規模データベースを用いた肌の反射・散乱光の統計的解析", MIRU2009, pp. 1164-1171, 2009.


Blue : men Red : women

Senior

Specular

Senior

Variance of scattering


Multiple layer model: Anisotropic BSSRDF with GMM

間下以大，向川康博，八木康史， "多層表面下散乱モデルによる皮膚の異方性散乱と不均一性の表現"，MIRU2009, pp. 1216-1222, 2009.



Separating direct / indirect reflections




Direct reflection




Multi-layer anisotropic BSSRDF




Direct reflection + Multi-layer anisotropic BSSRDF




Direct reflection + homogeneous isotropic BSSRDF




Real image




Direct reflection + Multi-layer anisotropic BSSRDF




Measuring a solid object with Dipole model

向川康博，鈴木和哉，八木康史， "一般照明下での表面下散乱の解析''，MIRU2008, pp.63-70, 2008. Y.Mukaigawa, K.Suzuki, Y.Yagi, "Analysis of Subsurface Scattering under Generic Illumination", Proc. ICPR2008, 2008. Y.Mukaigawa, K.Suzuki, Y.Yagi, ``Analysis of Subsurface Scattering based on Dipole Approximation'', IPSJ Transactions on Computer Vision and Applications, Vol.1, pp.128-138, 2009.

PP PE POM



http://www.eml.hiroshima-u.ac.jp/member/staff/tamaki/memo/index.php?ICPR

PP (Polypropylene)





PE (Polyethylene)





POM (Polyoxymethylene)





Single, double, triple, … scattering

向川康博，ラメシュラスカル，八木康史， "散乱媒体中のライトトランスポートの解析'', MIRU2010, pp. 665-672, 2010. Y.Mukaigawa, Y.Yagi, R.Raskar, "Analysis of Light Transport in Scattering Media", Proc. CVPR2010, 2010.



向川康博，ラメシュラスカル，八木康史， "散乱媒体中のライトトランスポートの解析'', MIRU2010, pp. 665-672, 2010. Y.Mukaigawa, Y.Yagi, R.Raskar, "Analysis of Light Transport in Scattering Media", Proc. CVPR2010, 2010.

Decomposition into each scattering



Outline

• Reflection, Scattering – Diffuse / specular

reflection, BRDF – Subsurface scattering,

single/multiple scattering

• Subsurface scattering models – Diffuse approximation

• Dipole/Multipole

– Plane-parallel approximation

• Semi-analytic solution • Anisotropic scattering

– Donner’s empirical BSSRDF

• Measuring scattering – Single scattering with

liquid – Multiple scattering

(2D・3D) – 2 layer model, multi-

layer model

Scattering in CG and CV

Toru Tamaki



Scattering not for CG and CV

Toru Tamaki



A lot of related research fields

• tissue optics Laser-tissue interaction 光医工学光医療工学 • 放射光 X線CTR散乱位相CT 位相Ｘ線イメージングシステム光

CT 蛍光CT赤外線散乱トモグラフィー光ヘテロダイン • 医療での人体可視化拡散光トモグラフィー無侵襲計測生体可

視化光イメージング屈折イメージング • 超音波非破壊検査超音波検査アコースティックイメージング超

音波医学医用超音波ティッシュハーモニックイメージングコントラストハーモニックイメージング

• interstellar diffusion 宇宙塵小惑星惑星間塵惑星間空間塵 Interplanetary dust cloud Cosmic dust

• 雪の中に埋もれた人を電磁波で探知 # 偏波情報によるターゲットの識別・分類レーダポーラリメトリ地中レーダー雪中レーダーレーダーリモートセンシングレーダー光学

• 地質探査物理探査地震波トモグラフィー地震波散乱地震波探査地下構造イメージング

• 中性子散乱核反応の散乱 • 光散乱

Technology

20110415 Scattering in CG and CV