VR802p

Virtual Reality 2. Human perception system NCU IPVR Lab. 1

2. Human Perception System

2.1 Introduction ………………… 22.2 Human visual system …….. 112.3 Color spaces ……………..… 622.4 Human haptic system …….. 862.5 Human auditory system …... 97


2.1 Introduction

視覺是人類最重要的知覺，而視覺中最重要的是色彩與立體感覺。

顏色是完完全全要給眼睛看的。

顏色是不存在的。

顏色完全是不同波長的電磁波刺激眼睛視網膜上的三種錐狀感光細

胞 (cone) 所造成的。三種錐狀細胞受光線刺激後會產生脈衝

(impulse) 傳輸到大腦的視覺皮質 (visual cortex) 。三種錐狀細胞各

有其不同的感光曲線，因而可以在視覺皮質中組合出多種不同的感

受，不同的感受造成不同顏色的認知。

顏色的認知是後天學習而來的，不是與身俱來。

所以顏色是大腦所產生的感知，而不是存在我們的環境中。


Three components of studying color science:(1) physics and chemistry of color,(2) physiology and psychophysics of color vision, and(3) colorimetry: the science of measuring color quantitatively.

Light is the electromagnetic radiation that stimulates our visual response. It is expressed as a spectral energy distribution L(λ),where λ is the wavelength that lies in the visible region, 380 nmto 780 nm, of electromagnetic spectrum, where 1 nm =10-9 m.

Microwaves TV Radio Electric powerInfraredUltraviolet

Cosmic Gamma xrays rays rays

400 nm 500 nm 600 nm 700 nm

10 -5 10 -3 1 nm 10 nm 10 -4 m 10 -2 m 1 m 10 2 m 10 3 m 10 6 m


White light (sun light) consists of seven visible color lights which are red, orange, yellow, green, blue, indigo, and purple.

Any combination of these colors will produce another color.


Due to the structure of the human eye, all colors are seen as variable combinations of the three so-called primary color RGB. For the purpose of standardization, the CIE designated in 1931 the following specific wavelength values to the three primary colors: blue = 435.8nm, green = 546.1nm, red = 700nm.

綠

黃靛白

紅洋紅藍

洋紅

藍紅黑

靛綠黃


The principle of subtractive light mixture


The amounts of red, green and blue needed to form any given color are called the tristimulus values and are denoted, respectively, by X, Y and Z. A color is then specified by itstrichromatic coefficients (x,y,z),

ZYXZ

zZYX

Yy

ZYXX

x

, ,

andx + y + z = 1.


Three tristimulus values construct a 3-D geometric structure ofcolor space.


Two trichromatic coefficients construct a 2-D chromaticity diagram. That is, a chromaticity diagram is used for displaying all visible color on a 2-D map.


The characteristic generally used to distinguish one color from another are brightness, hue and saturation.brightness = intensity (明暗度)hue (色調，與波長有關)saturation (飽和度，指加入多少比率的白光).We can takehue + saturation = chromaticity, then we havecolor = [brightness, chromaticity].

Pure color on the boundary of the diagram. It is said to be completely saturated. The point of equal energy corresponds to equal fractions of the three primary colors; it represents the CIEstandard for white light; and the saturation at the point is zero.

The chromaticity diagram is useful for color mixing.

The color space is a 3-D solid model and a 2-D chromaticity diagram is just a cut plane of the 3-D model; the plane is perpendicular to the intensity axis of the color model.


The structure of the human visual system in the horizontal cross section of a head. The visual neurotic fibers of two eyes are semidecussation(top view).

2.2 Human visual systemA. Structure of visual system


The structure of the human eye in the horizontal cross section of a right eye (top view).

視網膜 (第三層)

脈絡膜 (第二層)

角膜

虹膜

水晶體

鼻側

盲點視神經房水中心凹

鞏膜 (第一層)

玻璃體

太陽穴側

視軸


The relative sizes and positions of the macula and optic disc in the left eye (back view).

2o盲點

中心凹

5o

5o

7o

15o

太陽穴側鼻側


Courses of the capillaries (heavy lines) and nerve fibers (light lines) (back view).


15o

15o

invisible

invisible

Blind axis

Side view Top view

invisible

2o Optic axis

Optic axis

Optic axis


Demonstrating blind regionsLook at the below figure. Hood the book about 45 cm away, close your right eye, and fixate on the cross with your left eye. Move the book around slightly, and eventually you will find a position where the black circle disappears. At this position, the spot is images on the optic disk of the left eye. This region ofvisual space is called the blind spot for the left eye.

Look at another figure below. Fixate on the cross with your lefteye. You will find that where the break in the line falls is imaged on the blind spot, your perception will be of a continuous, uninterrupted line.


The retina of the human eye contains two types of photoreceptors called rods and cones.

錐狀細胞

桿狀細胞

細胞核

神經鍵

樹突

視紫質

粒線體


The schematic diagram of the primate retina structure.


二極細胞

神經節細胞

視網膜

傳輸至大腦桿狀細胞

錐狀細胞

視神經

光

脈衝


The rods, about 130 million in number, are relatively long and thin. They provide scotopic vision, which is visual response at the lower several orders of magnitude of illumination.

The cones, about 7 million in number, are shorter and thicker and less sensitive than the rods. They provide photopic vision, the visual response at the higher 5 to 6 orders of magnitude of illumination. The cones are also responsible for color vision.

Rod and cones 細胞中帶有對光線十分敏感的視紫質 (Rhodopsin)。視紫質碰到光線時會立刻產生化學反應，視紫質吸收光能，分解成維生素 A 醛和視蛋白而變成白色，同時產生電衝動 (impulse) 傳給兩極細胞。維生素 A 醛和視蛋白在黑暗中又會結合成視紫質，以便再次循環使用。在眼睛完全適應黑暗時，每個 rod 細胞中有 107 個視紫質分子。

Rhodopsin Vitamins A + Protein + Impulse

Vitamins A + Protein + Nutrition RhodopsinEnzyme

(酵素)

Light


Adaptation of light intensity (眼睛對光線亮度的調整能力)

眼睛對於光線亮度 (light intensity) 的感受相當敏感。眼睛感受亮度的範圍從最亮到最暗的可感受亮度差異可達 1013 倍。沒有人造的亮度感應器具有如此大的感光範圍。 [Kalawsky, 1993]。

在黑暗中逗留 1 分鐘後，眼睛對光的敏感度會提高 10 倍，逗留 20 分鐘後，敏感度會增加為 6,000 倍，逗留 40 分鐘後，敏感度會增加為 25,000 倍，這是人眼對光線敏感度的極限。


The distributions of rods and cones

The density of photoreceptors in human eye is about 200,000 per mm2, but that in the hawk (獵鷹) is over 1,000,000 per mm2; thus hawk eye is sharper than human eye.

視軸

80 o

60 o

40 o

20 o

0 o

80 o

60 o

40 o

20 o

盲點

中心凹視神經

桿狀細胞

錐狀細胞

盲點

80∘70∘60∘50∘40∘30∘20∘10∘0∘10∘20∘30∘40∘50∘60∘70∘80∘

以中心凹為中點的視角

每mm2

的桿狀及錐狀細胞數

180,000

160,000

140,000

120,000

100,000

80,000

60,000

40,000

20,000 錐狀細胞

桿狀細胞


Foveal cone inner-segment mosaic with 10μm scale bar (1μm = 10 -6 m).


The luminance efficiency function of the visual system

Why ?

Most raincoats are yellow.

For the human eye, the function is a bell-shaped curve whose characteristic depend on whether it is scotopic or photopic vision.


There are three different types of cones in the (normal) human retina with absorption spectra based on different kinds of Rhodopsin, where with

.780 and 380 nmnm maxmin

)( and ),( ),( 321 SSS

Why ?Someone has color blindness.

maxmin

400 500 600 700

10 5

10 4

10 3

10 2

10

1

敏感度

波長


Based on the three-color sensitivity, the spectral energy distribution of a "colored" light, C(λ), will produce a color sensation that can be described by spectral responses as

3. 2, 1, ,)()()( idCSC maxmin ii

If C1() and C2() are two spectral distributions that produceresponses and such thatthen the colors C1 and C2 are perceived to be identical. Hence two colors that look identical could have different spectral distributions.

3, 2,1, ),()( 21 iCC ii

The luminance, intensity, or lightness of a spatially distributed object with light distribution I(x,y,λ) is defined as

where v(λ) is the relative luminance efficiency function of the visual system.

,)(),,(),( dvyxIyxf max

min


The influence of sensitivity for visual system

Optic sensitivity of photoreceptorssensing intensity

Number of photoreceptorssensing speed

Density of photoreceptors


Brightness range of the eye.Typical to all psychometric functions, the apparent brightness is not a linear function of the luminance, rather than two logarithmic functions.

B. Properties of human vision


Lateral inhibition

Lateral inhibition refers to an architecture for neural networks in which neurons inhibit spatially neighboring neurons.

Lateral inhibition is a physiological mechanism of human vision;more accurately speaking, lateral inhibition is one of the most pervasive architectures in the human visual nervous system. The phenomena:simultaneous lightness contrast,simultaneous color contrast, Mach bands, andHermann grid illusion

are all belonging to the effect of lateral inhibition.


The luminance of an object is independent of the luminances of the surrounding objects. The brightness (or apparent brightness) of an object is the perceived luminance and depends on the luminance of the surround.

All small squares have exactly the same intensity.


The perceived color depends on the color of the surround.

The upper and lower circles have exactly the same color.


How many colors in the picture ?


Mach band effect


The Hermann grid illusion


The illusion is one of lateral inhibition effects. In which illusorydark spots seem to occupy the light intersections. One can convince himself that the darks spots indeed illusory because when he look directly at an intersection, the spot disappears. That is, dark spot appear at every intersection except the one on which he is fixated.

The perception of the dark spots can be simulated by a lateral inhibitory network because the amount of inhibition at the lightintersections is appreciably greater than that in other light regions. Each intersection is surrounded by light regions on four sides, in contrast to just two in other places.

The fact that dark spots do not appear when you look at them directly can be explained if there is less lateral inhibition in the fovea than in the periphery.


A simplified model of lateral inhibition in a 1-D neural network.Lateral inhibition occurs in a two-layer network when each neuron in the first layer excites a corresponding neuron (or set of neurons) in the second layer but inhibits the second-layer neurons lateral to the excitatory connection. In this diagram, excitatory connections (arrowheads) between layers are indicated by positive numbers, and inhibitory connections (black dots) are indicated by negative numbers. As with most neural network models, the output of a target neuron is calculated by multiplying the activation of each of its input neurons by the weight of its connection to the target neuron and summing these products over all inputs.


Lateral inhibition implemented in a 2-D network. (A) A single receptor projects to many ganglion cells in a center/surround organization. (B) Each ganglion cell receiving such connections therefore has a center/surround receptive field. The illustrated network exhibits an excitatory center and inhibitory surround, but the opposite organization (inhibitory center and excitatory surround) also exists in the retina.


(A) A lightness contrast stimulus is shown above its luminance profile through the center (dashed line). Notice that the two central squares have the same physical intensity even though they appear different in lightness.

(B) A lateral inhibitory network whose output predicts that the left square will look darker than the right square.

Lateral inhibition and lightness contrast.


當你眼睛凝視 (fixation) 圖中的色點，刻意固定眼球不動；你會發現圖中的淺藍或淺紅色圈會慢慢的消失；但你只要稍微動了一下眼球，該色圈即刻出現。這個特性是瑞士物理學家裘斯勒 (Ignaz Paul Vital Troxler) 在1804年所發現的，稱為裘斯勒褪色或效應 (Troxler's fading or Troxler's effect)。


刻意或不經意的讓你的視線在圖中左右移動，你會或多或少看到似乎有三個圓筒左右滾動。這是眼球運動所造成的錯覺 -微顫動 (microsaccade) 。


Human visual perception (just noticeable difference, JND)

The increment threshold BT as a function of reference intensity B.


Factors for color discrimination (影響色彩判定的因素)

(1)色彩的調適能力 (chromatic adaptation)目前眼睛對色彩的分辨能力會受到剛剛所看到色彩的影響。色彩飽和度的增減也會影響色彩判定的能力。

(2)亮度 (luminance) 亮度不足將會影響色彩的分辨能力。

(3)視網膜上的位置 (retinal position)，視網膜中心凹處對色彩能夠感受較大的飽和度。

(4)持續曝光時間 (duration)在持續曝光時間大於 0.02 秒的情況下，增加對眼睛刺激的時間會提高對色彩的分辨能力。

(5)物體大小 (object size) 面積太小的色彩區域不易分辨色彩。


Accommodation of depth of field (眼睛的景深適應能力)

瞳孔可以調整眼睛的進光量亦可改變視野景深 (depth of field)。

瞳孔收縮到最小時直徑只有 1.5 mm，而擴散到最大時直徑則有 8.5 mm，兩者相差 5.6 倍。進入眼睛的光線相差 30 多倍。

在瞳孔直徑為 4mm 時，注視無窮遠處，景深範圍約從眼前 3.5m至無窮遠處；注視 1m 處，景深範圍約在 0.8m 到 1.4m 之間。

當光線亮度增大，瞳孔直徑縮小為 2mm 時，注視無窮遠處的景深範圍約為 2.3m 至無窮遠處；注視 1m 處，景深範圍約在 0.7m 至1.8m 間。

radius of pupil = 4

radius of pupil = 2

depth of field

depth of field

mm

mm


景深短景深長


View of field (視界、視角)單眼瞬間視角 (instantaneous field of view) 約上下 120o，左右150o。而雙眼視角約上下120o，左右 200o，中央重疊約100o。


Binocular vision Not all animals have binocular vision.

Categories 1. Semidecussation (半交錯): The visual fields of two eyes of

some animals are heavily overlapped; their visual nerves are semi-chiastic at the optic chiasma; for example, human, dog, cat, monkey, etc. Their vision systems have stereo vision based on the disparity.

2. Decussation (交錯): The visual fields of two eyes of some animals are heavily overlapped, but their visual nerves are fully chiastic at the optic chiasma; for example, fogs, owl (貓頭鷹), etc. These animals have no stereo vision.


3. Decussation (交錯): Even if mammals, the visual fields of two eyes of some animals are less overlapped; their visual nerves are fully chiastic at the optic chiasma, for example, rabbit, goat, etc. These animals have still no stereo vision.

4. Decussation (交錯): The view fields of two eyes are non-overlapped; their visual nerves are fully chiastic at the optic chiasma, for example, fishes, birds, hawk (獵鷹), etc. These animals have no stereo vision.

The animals who have binocular vision move their eyes synchronously.


Binocular vision or stereopsis (雙眼視覺)

The depth information (range data) can be obtained by using stereoscopic (stereo for short) imaging techniques.

xc

yc

影像2

xc

yc

影像1

B

鏡片中心點

光軸

基底線

w (X, Y, Z)空間點

(x1, y1)(x2, y2)

q

- zc

- zc


Assume (X,Y,Z) is a world point, (x1, y1) and (x2, y2) are two corresponding image points on image1 and image2, respectively.

The principal of binocular vision is using the disparity (雙眼像差) to infer the depth or range information (深度資訊).

B

w (X, Y, Z)

A - zc

xc

影像2

影像1

(x2, y2)

(x1, y1)

q

(0, 0, 0)

(0, 0, 0)


The term (x1 - x2) is called the disparity.

q

ZxA

AB

Z

x

q

A

Z

x

q

1

2

1

qBZxZx

ZxqBZx

ZxqBqZxq

qBZx

Zq

qqBZx

Z

BqZx

Z

BA

Z

x

q

21

21

21

1112

2

. = 21

xx

BqZ

Suppose that we bring the first camera into coincidence with theworld coordinate system. Then by the similar triangle principle, we have


雙眼視覺的對映問題 (correspondence problem)

The problem is to match one specified point on the left image tothe point on the right image.

Left image. Right image.


錯誤的對映

Wrong range data come from wrong correspondence.


Stereogram


Stereo image pair


Partial binocular overlap (雙眼視角部份重疊)

單眼左右視角約為 155o，雙眼左右視角為 190o，中央約為 120o 重疊。

雙眼視角重疊可分為收斂 (convergent) 與發散 (divergent) 兩類。雙眼視角內側夾角重疊區域為發散重疊，雙眼視角外側夾角重疊區域為收斂重疊。

發散重疊的光學系統通常能產生較好的立體效果 (because it allows the combining optics to be tilted outwards.)

Divergent area. Convergent area.


Binocular rivalry (雙眼競賽)

幾乎每個人都有主控眼睛 (dominant eye)。主控眼睛是指能夠對景觀或物體注視較久的一眼。當兩個不同畫面同時各別呈現在兩眼前，視覺系統通常會抑制其中一個畫面而忽視其實質的資訊。偶而主控畫面 (dominant image) 會在雙眼間交替著，這種現象稱為雙眼競賽。兩眼中大小、亮度和色調不同的畫面都可能造成雙眼競賽的現象。例如，當其一眼看到較亮的景觀，則人腦即會抑制看到較暗的另一眼。

在雙眼顯示器中，顯示畫面的表現方式 (representation) 及複雜度(complexity) 會誘導雙眼競賽的效果。例如，雙眼注視彩繪影像(raster or rendered images) 會比注視線條畫面 (vector or drawing images) 更易引起雙眼競賽的效果 [Kalawsky, 1993]。

一個消除雙眼競賽的方法是減弱畫面的清晰度，也就是將畫面變模糊些。


Depth cue (深度知覺的線索)

深度知覺的線索可分為生理 (physiological) 及心理 (psychological) 兩方面。

生理方面的 (physiological) 線索有：

物距

q

q

p

p

p :q :像距

焦距f :

1f

=1p

1q+

(a) Accommodation (眼睛的適應力)眼睛看遠看近會自動調整水晶體對焦以求得最清晰的畫面。因水晶體的改變是人類獲得深度知覺的一個線索。

Retina


(b) Convergence (雙眼視線的收斂性)

雙眼看遠看近眼球會左右轉動改變視角對焦以注視目標物。


(c) Binocular disparity (雙眼像差)

雙眼視網膜上之映像在水平方向的距離。

(d) Motion parallax (移動視差)

移動中的眼睛看到近距離物體的相對移動速度比遠距離物體的相對移動速度快。


心理方面的 (psychological) 線索：

(a) Linear perspective (線性透視效果)

物體成像在視網膜上的大小改變和物體到眼睛的距離成反比。(size of the image of an object on the retinal changes is inverse proportion to its change in distance.)

(b) Shading and shadows (漸層和陰影)物體曲面的漸層顏色變化及物體造成的陰影都可以用來決定物體與觀測點的相對距離。


(c) Overlap (重疊)

物體前後部份重疊與遮蔽可以顯示物體間的前後相對位置。

(d) Size constancy (物體大小的恆常性)

物體的大小是深度知覺的一個線索。但當眼睛看到熟悉的物體時，心理上常會傾向認定為熟悉的大小而忽略視網膜上成像的大小 (tendency to perceive objects of familiar size as being the same real size, regardless of their retinal size)。如此會誤導深度知覺。

(e) Texture gradients (紋理層次)

距離愈遠，紋理將會愈小愈密集。

Temporal resolution (視覺時間的解晰度)

眼睛成像的時間性與照像底片的曝光時間性不同。眼睛成像的暫留時間必需要很短促，如此才能捕獲快速的動態畫面。此短促時間可

能是數十分之一秒，因此眼睛必需具有高感度的感光細胞及快速整

合視覺資訊的功能。


There are several color models (color spaces, or color coordinate systems) which are

CIE RGBCIE XYZCIE Yuv (UVW)CIE U*V*W*NTSC RGBNTSC YIQCIE L*a*b*CIE L*u*v*IHS

Any three coordinates (attributes) is a basis of a color space.

CIE : Commission Internationale de L'Eclairage, the International committee on color standards.

NTSC : National Television Standards (Systems) Committee.

2.3 Color spaces


R, G, B = three spectral primary sources.

Reference white : R = G = B = 1.

There is a negative tristimulus value.

Geometric structure Chromaticity diagram

(1) CIE RGB (1931)


Theoretically, the color is fully dependent on the wavelength; the three fixed RGB components acting alone cannot generate all spectrum colors (pure colors). This is an unresolved defect for color presentation.


Y = luminance, X = ?, Z = ?It is truly said that these three coordinates are physically nonrealizable.

Reference white : X = Y = Z = 1.This model is modified from RGB model such that all spectral

tristimulus values are positive.Spectral tristimulus Chromaticity diagram

(2) CIE XYZ ( )

ab

c

dist (a,b) < dist (a,c)cd (a,b) > cd (a,c)cd: color distance.


Y = luminance, u, v = chromaticities. This model is modified from XYZ model to achieve a uniform

chromaticity scale (UCS). This is the first UCS model, but it is just uniform on uv-plane. The MacAdam ellipses in the non-UCS XYZ chromaticity diagram

Chromaticity diagram

MacAdam ellipses Chromaticity diagram

(3) CIE Yuv (UVW) ( )


(4) CIE U*V*W* (1964)

This is a modified UCS system whose origin is shifted to the reference white in the u, v chromaticity plane.

This model is useful for measuring color differences quantitatively. In this model, for unsaturated colors, (i.e., for colors lying near the grays in the color solid), the difference between two color is, to a good approximation, proportional to the length of the straight line joining them. That is, the unit shifts in luminance and chrominance are uniformly perceptible.


(5) CIE L*u*v* (1976)

This model is modified from uvY model. (uo, vo, Yo) is the reference white, where uo=0.201, vo=0.461,

Yo=1. This model is similar to the U*V*W* model. The color model was mainly developed for displays, while

the next L*a*b* color model was initially designed for reflective prints.

Geometric structure Monitor gamut


(6) CIE L*a*b* (1976) L* = brightness

a* = red-green content (changes red/green balance)b* = yellow-blue content (changes green/blue balance)Neither a* nor b* corresponds to known psychophysical properties of visual perception.

This model gives a quantitative expression for Munsell system. L*a*b* is more appropriate over the small color differences

encountered near the end of a match than L*u*v* model. Xo, Yo, Zo = tristimulus values of the reference white.

Geometric structure Monitor gamut


R,G,B = three receiver primaries for reproducing colors.Reference white: R = G = B = 1.

The reference white for NTSC is different from that for the CIE system.

Geometric structure Chromaticity diagram

(7) NTSC RGB ( )


(8) NTSC YIQ (1953)

This model was developed to facilitate transmission of color images using the existing monochrome television channels without increasing the bandwidth requirement.Y = luminance

I, Q = chrominances (I: inphase, Q: quadrature)I axis encoding chrominance information along a blue-green to orange vector. Q axis encoding chrominance information along a yellow-green to magenta vector. Neither corresponds to a known psychophysical quantity.

R

G

B

Y

Q I


To achieve high-fidelity audio and video quality in consumer electronic devices, the color models have been progressively improved in the devices:YIQ ← audio and video are composed togetherA / V ← audio and video are separatedS / A ← video is further separated into two channelsColor difference / A ← video is further separated into three

channels Y Cb Cr


The relationship diagram of these color models

CIE RGB

CIE XYZ, Yxy

CIE Yuv, uvw

NTSC RGB

CIE L* a* b*

CIE L* u* v*

CIE U* V* W*

NTSC YIQ NTSC YCbCr

CIE Yu’v’

19311964

19311964

1975 1964

1953

19531976

1976


Transformation formulas for color models.

.

990.0010.0000.0

011.0813.0177.0

200.0310.0490.0

1

B

G

R

Z

Y

X

RGBCIEXYZCIE

.

896.0118.0058.0

028.0000.2985.0

288.0533.0910.1

2

Z

Y

X

B

G

R

XYZCIERGBNTSC

.

117.1066.0000.0

114.0587.029.0

201.0174.0607.0

3

B

G

R

Z

Y

X

RGBNTSCXYZCIE

.

311.0523.0212.0

321.0275.0596.0

114.0587.0299.0

4

B

G

R

Q

I

Y

RGBNTSCYIQNTSC

.

701.1104.1000.1

647.0273.0000.1

621.0956.0000.1

5

Q

I

Y

B

G

R

NTSC YIQNTSC RGB

.

128.1059.0001.0

159.0753.0114.0

151.0146.0167.1

6

B

G

R

B

G

R

RGBNTSCCIE RGB


Transformation formulas for color models (continued).

.

2 / )3(-

(2/3)

8

ZYXW

YV

XU

XYZCIEUVW

.

627.0827.0145.0

144.0587.0299.0

133.0116.0405.0

7

B

G

R

W

V

U

RGBNTSCUVW

.

)315( /6

)315( /4

9

ZYXYv

ZYXXu

YY

XYZCIEYuv

.

])/(-)/([200

])/(-)/([500

16-)/100(25

10

oo

oo

o

3131

3131

31

ZZYYb

YYXXa

YYL

XYZCIEbaL

//*

//*

/*

***

.

)5.1(13

)(13

16-)/100(25

12

o

o

o

**

**

3/1*

***

vvLv

uuLu

YYL

YuvvuL

.

)(13

)(13

17-)/100(25

11

o

o

o

**

**

3/1*

***

vvWV

uuWU

YYW

YuvWVU


RGB to YCbCr transform

YCbCr to RGB transform

B

G

R

Cr

Cb

Y

081.0419.0500.0

499.0331.0168.0

114.0587.0299.0

Cr

Cb

Y

B

G

R

0015.07753.1000.1

7137.03443.0000.1

4017.10009.0000.1

Transformation between NTSC RGB and YCbCr

Another format with less computation

Y = 0.299 (R-G) + G + 0.114 (B-G)Cb = 0.564 (B-Y)Cr = 0.713 (R-Y)

9 , 6 +

4 , 6 +


IHS model (perceptual color model) I : intensity (the more black in it, the darker (less light) it is),

H: hue,S: saturation (the more white mixed in the color, the less saturated it is).Hue and saturation taken together are called chromaticity.

Intensity Hue Saturation


intensity = gray level: [0, 255]hue = angle : [0o, 360o] [0, 255]saturation = percentage : [0,1] [0, 255]

Geometric structureWhite

G R

BSaturation

Intensity

Hue ( )

Black

A horizontal cross section of the HIS

color model.


Conversion from CIE RGB to IHS

BGRI 31

BGRminBGR

S , , 3

1

BGBRGR

BRGRcosH

2 2121

1

where H = 360o - H , if (B/I) > (G/I).

IHS color model is a generalized cylindrical coordinate form of a color model. Many IHS forms have been derived for known color models.


Conversion from IHS to RGB

. 1 ,)60(

1

31

,131

then ,1200 If

o

oo

brgHcos

cosHSrSb

H

. 1 ,)60(

1

31

,131

120

then ,240120 If

o

o

oo

grbHcos

cosHSgSr

HH

H

. 1 ,)60(

1

31

,131

240

then ,360240 If

o

o

oo

bgrHcos

cosHSbSg

HH

H

Where R, G, B are in the range [0, 1] and r + g + b = 1; we takeR = 3I r, G = 3I g, and B = 3I b.


IHS of PAL YIQ

IHS of L*a*b*

If a color model is a uniform chromaticity scale (UCS) system, then its IHS form is also a uniform chromaticity scale system.

).( , ,

1.0515.0615.0

437.0289.0147.0

11.0587.0299.0

122 IQtanHQISYI

B

G

R

Q

I

Y

NTSCPAL

).( , , ***** 122)()( abtanHbaSLI


The classical theory states that any color can be reproduced by mixing an appropriate set of three primary colors.

Three primary colors: Red, Green, and Blue. Three complementary colors: Cyan, Magenta, and Yellow. In application, the primary colors are mainly used for describing color

light, and the complementary colors are mainly used for describing color printing.

Color reproduction


R G B

C M Y

complement

R = W - C C = W – R

G = W - M M = W – G

B = W - Y Y = W – B

where "R = W - C" means that cyan is absorbed from white light and red is reflected to the viewer's eye.

B

G

R

Y

M

C

ei

255

255

255

,..

The relationship between RGB and CMY

W (White) = R + G + BK (black) = C + M + YC = G+B R = M+YM = R+B G = C+YY = R+G B = C+M

R G B

C M Y


Laws of color matching(1) Any color can be matched by mixing at most three colored lights.

(2) The luminance of a color mixture is equal to the sum of the luminances of its components.

(3) A color match at one luminance level holds over a wide range of luminance, where color = luminance + chrominance.

(4) Color addition:

(5) Color subtraction:

(6) Transitive law:

(7) Color matches:

These are also called Grassman's laws (1853).

DDCCDCDC 2 21 12 21 12211 and

DCDCDDCC 11222121 and

CCCCCC 313221 and

CCCC

CCCC

CCCC

332211

332211

332211


Color matching by human inspection


人們要為虛擬環境構建觸覺設備，則必需充分了解觸覺感受器感受程度及範圍。

接觸 (touch) 是用來描述皮膚上的力學接觸 (mechanical contact)。接觸，更準確的字眼是“皮膚的感覺” (cutaneous sensitivity)，是一個複雜的議題，因為這其中還牽涉到非力學的刺激；例如，冷熱。

A. Structure of human skin 皮膚 (skin) 覆蓋在身體的表面，以一個成年男人而言，皮膚的表面

積約為 1.8 m2，重量約為 4.8 kg，約佔總體重的 8 %。

皮膚是由表皮 (epidermis) 及緊貼表皮的真皮 (dermis) 所構成的。皮膚內含有汗腺、皮脂腺、毛根、毛、血管、淋巴管、末梢神經等、以及有助於皮膚功能的皮下組織 (subcutaneous tissue)。

手掌、腳掌、手指、腳趾的表皮厚度約在 0.1 cm 以上，其他部份的厚度約為 0.01 ~ 0.02 cm。

2.4 Human haptic system


皮膚構造


Layer structure of skin


皮膚的觸覺

皮膚的觸覺感受器 (tactile receptor) 包含:髮根神經叢 (hair root plexuses)，自由神經末梢 (free nerve endings)，觸覺碟 (tactile discs)，觸覺小體 (corpuscle of touch) 和type II 皮膚力學感受器 (type II cutaneous mechanoreceptors)：

髮根神經叢可以偵測皮膚外表的移動；

自由神經末梢可以偵測痛覺及連續觸覺；

觸覺碟是用以協助判定皮膚上的觸覺位置；

觸覺小體是用以確認皮膚上的接觸位置，觸覺小體在手指頭及手掌

上最多；

type II 皮膚力學感受器是在真皮的深層裡，主要是感受較重和連續

的刺激。


不同區域的皮膚觸覺是由不同的脊髓神經所控制，這也是造成不同部位的皮膚有著不同觸感的原因之一。


觸覺神經系統

神經系統分為中樞神經與末梢神經。中樞神經是腦與脊髓，而末梢

神經是連接中樞神經與身體各部位末梢的情報連絡路線，分為腦神

經及脊髓神經。

主管皮膚觸覺的中樞神經是脊髓，而其末梢神經即為脊髓神經。


脊髓全長成人約為 44cm，重約 25g，生長在脊椎骨體的背側孔穴

中。脊髓外側是白質，內部為 H 形的灰白質。白質是由神經細胞的軸

索構成，灰白質則由神經元所構成。H 形灰白質的前側叫前角

(anterior horn)，後側叫後角 (posterior horn)。前角的神經元為運動性

的，連接運動神經，而後角為感覺性的神經元，連接感覺神經。


脊髓可以當做腦部與身體末梢的連絡媒體 (media)。它可以接收身

體末梢傳來的訊號，傳送至腦部；然後再將腦部所下達的指令傳達

到身體末梢。其次是當身體面臨緊急狀況時，脊髓會發揮中樞的功

能，直接引起反射運動，而省略對腦部的連絡。(脊髓也是內臟與

血管的自律反射中樞。)


作用在皮膚上的力學刺激 (mechanical stimuli) 有三種形態：

(1) 持續刺激 (step function)。皮膚在一段時間內的垂直移位

(displacement)；例如，有一小點壓迫在皮膚上一段時間。

(2) 瞬間刺激 (impulse function)。指皮膚在極短時間內的垂直位移。

(3) 週期性刺激 (periodic function)。是指皮膚受到規律性的短暫重複刺激。

皮膚具有觸覺門檻值 (touch threshold or vibrotactile threshold)；也就是說皮膚上的刺激要大於此門檻值才會有觸感。

B. Properties of human skin


觸覺門檻值與刺激的形態、位置及、時間有關。

Hill (1967) 曾導出觸覺門檻值與刺激脈衝持續時間的關係

其中 At 是觸覺力量門檻值 (threshold amplitude)，單位是μm (10 -6

m)，t 是刺激脈衝持續時間，單位是 ms (10 -3 s)。

觸覺門檻值與刺激脈衝持續時間的範例，

也就是說刺激的持續時間愈小，則刺激力量要愈大才會有感覺。

上述關係僅適用於手指的刺激，刺激面積為直徑 0.06 cm 的圓形範圍。

)5.1(e1

2ttA

2.472.723.764.117.06∞At

2.52.01.51.00.50t


觸覺門檻值是皮膚刺激的一大特性。

第二個皮膚刺激的特性是持續刺激的時間愈久，觸覺的敏感性愈低，甚至於低到完全沒有感覺。

另一個特性是當皮膚上的刺激移去後，刺激的感覺仍然殘留在皮膚上。

Sensorial adaptation for a “slow adapting” receptor, for three stimulus intensities.


2.5 Human auditory systemA. Ear structure

耳在結構分為：外耳 (external ear)，中耳 (middle ear)及內耳(internal ear) 三部份。


外耳包括耳廓 (pinna) 及外耳道 (ear canal)。耳廓是用來收集聲

波，經由約 2.5cm 長的外耳道傳輸至鼓膜。

中耳包括鼓膜 (tympanic membrane)、鼓室、耳小骨 (auditory ossicle) 及耳咽管 (eustachian tube)。鼓膜是凹面朝外呈圓椎狀張

開的半透明薄膜。鼓膜會受聲波的共鳴而振動。鼓膜中心附著鎚

骨柄。耳小骨包含鎚骨 (malleus)、砧骨 (incus) 及鐙骨 (stapes) 三部份。鎚骨及砧骨會擴大鼓膜的振動並傳到鐙骨。

內耳包含三個主要結構：耳蝸 (cochlea)、耳石器及半規管

(semicircular canal)。鐙骨的振動經前庭窗 (oval window) 傳入呈

螺旋狀，充滿淋巴液的耳蝸前庭階 (scala vestibuli)、鼓階 (scalatympani)，最後被第二鼓膜吸收；而振動的訊號則會經耳蝸中的柯

氏器轉成電訊，經耳蝸神經 (聽覺神經) 傳送到大腦。

鼓膜的表面積大約是前庭窗的 22 倍大，因此中耳的功能就像是一

個小小的擴大機將鼓膜的振動訊號擴大、傳入耳蝸中。


Transmission of sound


聲波的傳送及轉換


人體平衡器 - 半規管 (semicircular canal)


聽覺通往大腦的途徑平衡感通往大腦的途徑


B. Properties of auditory system 人類聽覺系統是一個相當敏感的感覺器官。它對於頻率的感受範圍

非常廣，同時能分辨 0.2% 的差異。而對於聲音振幅的容忍範圍大約在 110 分貝 (dB, decibel) 以內。

0 分貝：可聽聞的極限30 分貝：非常安靜

140 分貝：耳朵受傷

合成一個似真的聽覺環境，最重要的是回音 (echo) 或稱聲音反射(reverberation) 的問題。凝聽的位置 (position) 和方向 (orientation) 也扮演著重要的角色。

在一個吵雜的環境中，個人的聽覺系統能夠凝聽某一特定地方或較遠處的聲音而忽略其它處或較近處的聲音。在分析這種狀況，我們要找出雙耳同時接收到多種音源的表示方式 (representation)。當一個凝聽者移動頭部，則音源僅能做與頭部位移和轉動相關的改變。這也就是音源的改變是一個頭部相關的轉移函數 (head related transfer function (HRTF))。


與音源相關的兩個機制 (mechanism) 是聲音大小的差異 (intensity difference) 和到達時間的差異 (temporal or phase difference)。

聲音大小在雙耳間的差異最大可達 20 分貝。而聲波到達雙耳的時差在 10 ms 以上者可以被偵測出來；到達雙耳的時差在 650 ms 以上則可以準確的判斷音源的方向。

人耳可以非常準確地判斷音源的方位，即使是有強烈的擾亂回音(conflicting echoes) 也是一樣。

聽覺系統具有優先效果 (precedence effect)，也就是在一個會產生回聲的室內 (reverberant room)，人耳會拒絕由室內所產生的回聲。當人耳聽到兩個時差在 1~5 ms 之內的聲音，則第二個聲音將不被接受。

人類聽覺系統非常的複雜。一個最令人印象深刻的能力是它可以從來自不同方位的諸多音源中獨立出某一特色音源。

而某一聲音遮蔽另一聲音的程度與相對頻率 (relative frequency) 、大小聲及、音源位置有關。頻率愈高，遮蔽的效果愈明顯。


聲音到達雙耳的時差及大小聲是人耳判斷音源方位的主要線索。假設人耳與音源的相對方位如下圖所示：

則音源到達雙耳的距離相差 d, d = r sin + r ,

雙耳時差為 t , ,

其中 c 是聲波在空氣中傳播的速度。

)( sincr

cd

t


雙耳時差與聲波頻率無關。

在諸多聲音中要能聽到某特定的聲音則其頻率要能與別的聲音有所區

別。當同時發生的兩個聲音頻率太相近，則會聽成另一個聲音。這種

現象稱為歐姆音響定則 (Ohm's acoustic law)。

人耳對於在 1KHZ 至 3KHZ 頻率範圍內的音調變化 (sound pitch variation) 極為敏感。平均人耳可以感覺到的最小頻率變化是 0.3 %；

也就是說 1000 至 1003 的變化可以被偵測出來。受過專業訓練的人

可以分辨更細微的音調變化。在較低頻的聲音中（從 32HZ 到

64HZ)，人耳可以感受到的最小頻率變化約為 1 %；而在較高頻的範

圍中 (16 KHZ 至 20 KHZ)，人耳對於音調變化的感覺就變了非常的

差。正常的聽覺系統約能區別 2000 個音調的變化。

假設在頭顱半徑為 8.75cm，

聲波速度為 344 m/sec 的情

形下，方位角與雙耳時差的

關係如右圖所示：

Documents

VR802p