7
J. Shanghai Jiaotong Univ. (Sci.), 2014, 19(1): 28-34 DOI: 10.1007/s12204-014-1473-y Aesthetic Visual Style Assessment on Dunhuang Murals YANG Bing 1 ( ), XU Duan-qing 1(), TANG Da-wei 1 () YANG Xin 2 ( ), ZHAO Lei 1 ( ) (1. College of Computer Science, Zhejiang University, Hangzhou 310027, China; 2. College of Computer Science, Dalian University of Technology, Dalian 116024, Liaoning, China) © Shanghai Jiaotong University and Springer-Verlag Berlin Heidelberg 2014 Abstract: Dunhuang murals are gems of Chinese traditional art. This paper demonstrates a simple, yet powerful method to automatically identify the aesthetic visual style that lies in Dunhuang murals. Based on the art knowledge on Dunhuang murals, the method explicitly predicts some of possible image attributes that a human might use to understand the aesthetic visual style of a mural. These cues fall into three broad types: composition attributes related to mural layout or configuration; color attributes related to color types depicted; brightness attributes related to bright conditions. We show that a classifier trained on these attributes can provide an efficient way to predict the aesthetic visual style of Dunhuang murals. Key words: Dunhuang murals, aesthetic visual style, feature descriptors CLC number: TP 391.41 Document code: A 0 Introduction Aesthetic visual style assessment is to evaluate which school one painting belongs to. Aesthetic visual style as a whole can be complex, which includes not only the contents of the visual object itself in the paintings, but also the presentation technique or the feeling the paint- ing wants to tell audiences. Automatic assessment on aesthetic visual style of pictures is a very challenging and far from solved problem. The visual objects to be evaluated in this paper are especially Dunhuang mu- rals, aesthetic visual style research on them could help people fully understand the unique Dunhuang murals. Comparing with other Chinese paintings, Dunhuang murals have their own characteristics. From a special viewpoint, Dunhuang murals are the fusion of Eastern and Western culture that affected by the Western-style, Tibetan life and painting style from the Central Plains region. Moreover, these murals painted in different dy- nasties almost have their aesthetic styles [1-2] . Even in the adjacent era, or even in a short dynasty, Dunhuang murals also have their unique characteristics. In these murals, more charts are used to portray religious be- liefs, religious activities or Jataka stories, and the in- coherence of space is taken to express the changes of Received date: 2012-04-27 Foundation item: the National Basic Research Program (973) of China (No. 2012CB725305) and the Na- tional Key Technology R&D Program of China (No. 2012BAH03F02) E-mail: [email protected] time. In this paper, we aim at getting more information to assess the aesthetic visual style of Dunhuang mu- rals. We try to build a connection between human perception on Dunhuang murals and computational vi- sual features extracted from these murals. Through- out comprehensive studies on literatures [1-2] that re- lated to Dunhuang murals and Chinese art history, we have found that the aesthetic visual style of Dunhuang murals contains mainly three components: depiction by line, emotional expression by color, and the light and dark of brightness. Therefore, in our method, compo- sition, color and brightness information are combined together to describe the aesthetic visual style of Dun- huang murals. 1 Related Work For an image, including its points, lines, composition, light, or colors, all components are integrated together to represent the aesthetic visual style, while the aes- thetic visual style could be reflected by each component respectively. There are a large number of works related to analy- sis of the aesthetic visual style. Based on Na¨ ıve Bayes classifier, Keren [3] proposed an algorithm under the es- timation that for one artist, his works had the same art style. Li and Wang [4] applied 2D multiresolution hidden Markov models combined with multi-levels De- bauchies wavelet coefficient feature to identify the au- thors of Chinese ancient paintings.

Aesthetic visual style assessment on Dunhuang murals

  • Upload
    lei

  • View
    217

  • Download
    3

Embed Size (px)

Citation preview

Page 1: Aesthetic visual style assessment on Dunhuang murals

J. Shanghai Jiaotong Univ. (Sci.), 2014, 19(1): 28-34

DOI: 10.1007/s12204-014-1473-y

Aesthetic Visual Style Assessment on Dunhuang Murals

YANG Bing1 (� �), XU Duan-qing1∗ (���), TANG Da-wei1 (���)YANG Xin2 (� �), ZHAO Lei1 (� �)

(1. College of Computer Science, Zhejiang University, Hangzhou 310027, China;2. College of Computer Science, Dalian University of Technology, Dalian 116024, Liaoning, China)

© Shanghai Jiaotong University and Springer-Verlag Berlin Heidelberg 2014

Abstract: Dunhuang murals are gems of Chinese traditional art. This paper demonstrates a simple, yet powerfulmethod to automatically identify the aesthetic visual style that lies in Dunhuang murals. Based on the artknowledge on Dunhuang murals, the method explicitly predicts some of possible image attributes that a humanmight use to understand the aesthetic visual style of a mural. These cues fall into three broad types: � compositionattributes related to mural layout or configuration;� color attributes related to color types depicted;� brightnessattributes related to bright conditions. We show that a classifier trained on these attributes can provide an efficientway to predict the aesthetic visual style of Dunhuang murals.Key words: Dunhuang murals, aesthetic visual style, feature descriptorsCLC number: TP 391.41 Document code: A

0 Introduction

Aesthetic visual style assessment is to evaluate whichschool one painting belongs to. Aesthetic visual styleas a whole can be complex, which includes not only thecontents of the visual object itself in the paintings, butalso the presentation technique or the feeling the paint-ing wants to tell audiences. Automatic assessment onaesthetic visual style of pictures is a very challengingand far from solved problem. The visual objects to beevaluated in this paper are especially Dunhuang mu-rals, aesthetic visual style research on them could helppeople fully understand the unique Dunhuang murals.

Comparing with other Chinese paintings, Dunhuangmurals have their own characteristics. From a specialviewpoint, Dunhuang murals are the fusion of Easternand Western culture that affected by the Western-style,Tibetan life and painting style from the Central Plainsregion. Moreover, these murals painted in different dy-nasties almost have their aesthetic styles[1-2]. Even inthe adjacent era, or even in a short dynasty, Dunhuangmurals also have their unique characteristics. In thesemurals, more charts are used to portray religious be-liefs, religious activities or Jataka stories, and the in-coherence of space is taken to express the changes of

Received date: 2012-04-27Foundation item: the National Basic Research Program

(973) of China (No. 2012CB725305) and the Na-tional Key Technology R&D Program of China(No. 2012BAH03F02)

∗E-mail: [email protected]

time.In this paper, we aim at getting more information

to assess the aesthetic visual style of Dunhuang mu-rals. We try to build a connection between humanperception on Dunhuang murals and computational vi-sual features extracted from these murals. Through-out comprehensive studies on literatures[1-2] that re-lated to Dunhuang murals and Chinese art history, wehave found that the aesthetic visual style of Dunhuangmurals contains mainly three components: depiction byline, emotional expression by color, and the light anddark of brightness. Therefore, in our method, compo-sition, color and brightness information are combinedtogether to describe the aesthetic visual style of Dun-huang murals.

1 Related Work

For an image, including its points, lines, composition,light, or colors, all components are integrated togetherto represent the aesthetic visual style, while the aes-thetic visual style could be reflected by each componentrespectively.

There are a large number of works related to analy-sis of the aesthetic visual style. Based on Naıve Bayesclassifier, Keren[3] proposed an algorithm under the es-timation that for one artist, his works had the sameart style. Li and Wang[4] applied 2D multiresolutionhidden Markov models combined with multi-levels De-bauchies wavelet coefficient feature to identify the au-thors of Chinese ancient paintings.

Page 2: Aesthetic visual style assessment on Dunhuang murals

J. Shanghai Jiaotong Univ. (Sci.), 2014, 19(1): 28-34 29

Like aesthetic visual style assessment, aestheticvisual quality assessment also belongs to the categoryof subjective problem. The two topics both need totransfer human cognition knowledge to mathematics orcomputer models. Therefore, we give a brief review onaesthetic visual quality assessment.

There are many frameworks for aesthetic quality as-sessment. Ke et al.[5] found three distinguishing factorsthat lie between professional photos and snapshots (e.g,simplicity, realism and basic photographic technique)and designed high level semantic features to measurethe perceptual differences. Datta et al.[6] extracted 56visual features of photographic images to discriminatebetween aesthetically pleasing and displeasing images.Moorthy et al.[7] proposed and evaluated a set of lowlevel features that are combined in a hierarchical way inorder to construct a computational model of aestheticappeal of consumer videos.

In this paper, we try to construct a scheme to bridgethe human vision with the computer. We just con-sider global features in our method. This is because,according to Heihrich Wolfflin’s comments[8], the aes-thetic visual style is in general represented by the wholeimage. Furthermore, local features may be inaccurateto describe, so these features are ignored in this pa-per. Visual characteristics such as color, brightness andcomposition concepts constitute the main body of ex-pert analysis in this paper.

2 Describable Attributes

Before designing features to assess a mural’s aes-thetic visual style, we have studied a lot of researchpapers related to Dunhuang murals[9-11]. Finally, wefound that three distinguishing factors play importantroles in determining the aesthetic visual style of murals:composition attributes, color attributes and brightnessattributes. In general, we propose 20 features whichare concatenated together to construct the feature setFea = {fi, fj |i = 1, 2, · · · , 5, 9, 10, · · · , 20, j = 6, 7, 8}that used in aesthetic visual style assessment. Notethat, these features are not randomly selected, but areproposed based on knowledge and experiences in Dun-huang murals and human perception.2.1 Composition Attributes

The composition of Dunhuang murals forms a uniqueaesthetic visual style that is characterized by dynasticchanges. Our compositional attributes address ques-tions related to the arrangement of locations and re-lationship of objects in a mural where artist organizesindividual local components to represent his ideologicalcontents.2.1.1 Blur Effect

There is inevitably certain extent damage to Millen-nium due to natural or man-made destruction duringover one thousand years. The most direct appearance

is the blur effect of the mural itself. For one mural,the ambiguity may reduce its own ornamental quality,when considering a series of murals created in differ-ent dynasties, and the blur effect could be seen as stylechanging cues along with the era.

To evaluate the blur effect in Dunhuang murals, weapply the method proposed by Brainard et al.[12], whichwe had found to achieve best results. We model ablurred mural Ib as the result of a Gaussian smooth-ing filter Gσ applied to an otherwise sharp mural Is, asIb = Gσ ∗ Is, where the symbol ∗ means convolution.The parameter σ of Gaussian filter and the sharp muralIs are both unknown. We assume that the frequencydistribution for all sharp murals Is is approximatelyequal, we then have the parameter σ of Gaussian filterto represent the degree of blurring.

We can estimate the maximum frequency of the mu-ral Ib by taking its two dimensional Fourier transformsand looking for the highest frequency whose power isgreater than some threshold as: Ffft = F (Ib), C ={(x, y)|Ffft(x, y) > θ}, where F means Fourier trans-form and θ is set to be 4 in our experiments, x and ydenote the coordinates in pixel.

If the highest frequency is small, it can be consideredto be blurred by a large σ. So the blurring featureis inverse-proportioned to the smoothing parameter σ,which can be measured as

f1 = max{

2W

(w−

⌊W

2

⌋),

2H

(h−

⌊H

2

⌋)}∝ 1

σ, (1)

where w and h are variables, W and H represent thewidth and height of the image.2.1.2 Standard Deviation

In probability and statistics theory, standard devi-ation is taken to measure the variability or diversity,and to measure the degree of the data deviate from thearithmetic mean value. We use the standard deviationto probe the amount of variation in shading with eachregion because it may reveal painter-specific shade vari-ations, as following:

f2 =1

W ∗ H

√√√√ W∑x=1

H∑y=1

(I(x, y) − I)2, (2)

where I(x, y) is intensity value at pixel (x, y), and I isthe mean value of the mural.2.1.3 Rule of Thirds

The rule of thirds[13] is a composition rule in visualart such as painting and photography. The rule meansthat, using imaginary lines to cut the image horizon-tally and vertically each into three parts, there are nineparts with same area and four intersection points inthe image after cutting. Consequently, the intersectionpoints may be found as the important parts of the com-position instead of the center point. To some extent,

Page 3: Aesthetic visual style assessment on Dunhuang murals

30 J. Shanghai Jiaotong Univ. (Sci.), 2014, 19(1): 28-34

the rule of thirds can be considered as an approximationto the “golden ration”.

In this paper, we combine the rule of thirds withfollowing edge distribution to represent the compositionof images, aiming at detecting the compositional styleof Dunhuang murals developed in different dynasties.2.1.4 Edge Distribution

The stroke style of Dunhuang murals has been con-stantly changing in the long period. For example, inthe Northern Zhou Dynasty murals, the Buddhist andJataka usually used white as grounding color, and lineswere drawn with a smooth outline. When it came tothe Tang Dynasty murals, there appeared the “Eigh-teen Descriptions” and the “Bump Law”[2]. From theabove discussion, it is clear that the edge distributioncould be selected as a feature since we don’t considerthe brushwork in this paper.

Edge distribution 1 To measure the spatial dis-tribution of edges, we calculate the ratio of area thatthe rule of thirds cuts. One issue is that, artists may notlimit in accord with the four intersections point the ruleof thirds cuts. To overcome this problem, we choose

two region thresholds:( 5

12W—

34W ,

512

H—34H

)and(1

4W—

34W ,

14H—

34H

). Then the image energy of the

chosen region is computed as:

f3 =1

W ∗ H

3W/4∑x=5W/12

3W/4∑y=5H/12

I(x, y), (3)

f4 =1

W ∗ H

3W/4∑x=W/4

3W/4∑y=H/4

I(x, y). (4)

Edge distribution 2 For one mural, we calculatethe area of the smallest bounding box that encloses acertain ratio of the edge energy. Through trials on thetraining set, the ratio is selected to be 81% (90% in eachdirection). So the second feature for edge distributionis to calculate the area ratio of the bounding box overthe area of the whole image, i.e.,

f5 =Wb ∗ Hb

W ∗ H, (5)

where Wb and Hb are the width and height of thebounding box, respectively.2.1.5 Wavelet-Based Texture

The use of texture is a composition attribute in paint-ings. In this paper we use Daubechies wavelet trans-form to measure the spatial smoothness in the images.We perform a wavelet transform on all three color bandsIh, Is, Iv, the corresponding feature are formed as f6,f7 and f8, where Ih, Is and Iv mean the h, s and vvectors in HSV color space respectively.

2.2 Color AttributesAs long as the substitution of dynasties and the

change of social environment, the color of Dunhuangmurals alters continuously, color murals were increas-ingly more abundant, color were represented lively, thefigures were full life-like, and the strokes were delicate.2.2.1 Spatial Variations of Color

This feature tries to identify the differences in colorpalette used by murals in different dynasties. To mea-sure the spatial variation of color, we apply the follow-ing method which is similar to Florin’s method[14].

For an input mural, its R, G and B channels are nor-malized by division by mural intensity. At each pixel,we could determine the orientation of the plane thatbest fits a 5 × 5 neighborhood centered on the pixel ofinterest in the R, G and B domains respectively. Thus,at each pixel, we obtains three normals: nR, nG andnB. The average of the areas constructing facets ofthe pyramid determined by these normals is taken asa measurement of the spatial variation of color aroundthe pixel.

f9 = S(I), (6)

where S(·) is the function of the above processes, andI is the vector of mural image.

It is intuition that for murals created in different dy-nasties, the color palate is distinguishable. Therefore,the spatial variation of color is different from each other.2.2.2 Average Value of Hue and S

HSV color model template corresponds to the colorpalette of the artists. The artists obtain different col-ors from a solid color by changing the concentration anddepth of the color relative to the solid color. So, usingHSV color model could clearly represent the artists’ in-ner idea about color. We choose the average value ofHue and S channels as two features:

f10 =1

W ∗ H

W∑x=1

H∑y=1

Ih(x, y), (7)

f11 =1

W ∗ H

W∑x=1

H∑y=1

Is(x, y), (8)

where Ih(x, y) and Is(x, y) are the values at pixel (x, y)respectively.2.2.3 The Proportion of Warm Color and Cold Color

Artists in the Tang Dynasty were good at using ofthe harmonic of contrasting colors, mainly through thecontrast that a small amount of red soil, azurite andmalachite green colors were included to make the colorof murals more intensity and quite dynamic. Thus, weselect the ratio of warm color to cold color as one featureto depict characteristics of dynasty styles.

In RGB color space, there’s no explicit definition thatwhether each color in the natural world belongs to the

Page 4: Aesthetic visual style assessment on Dunhuang murals

J. Shanghai Jiaotong Univ. (Sci.), 2014, 19(1): 28-34 31

warm color or cold color. To solve this problem, in thispaper, we define a distance that evaluates the differencebetween the color of one pixel and the color of red,yellow, blue and green. If the minimum distance arisesin the color of red or yellow case, then the color of thispixel is treated as the warm color, vice versa:

f12 =√

I(x, y) − c, (9)

where c means the value of red, yellow, blue, or greencolor. Throughout all pixels of the mural, we calculatethe number of warm color pixels and cold color onesrespectively, and then the ratio of warm color pixels tocold color ones is chosen as one measurement of aes-thetic visual style.2.3 Brightness Attributes

It is intuitively rational that brightness affects peo-ple’s impression on a mural. Artists use a series oftechniques to represent bright condition of a scene.2.3.1 Average Brightness Value

The average brightness value is computed accordingto:

f13 =1

W ∗ H

W∑x=1

H∑y=1

Iv(x, y), (10)

where Iv(x, y) is the intensity value at pixel (x, y) of Vvector in HSV colour space.2.3.2 Uniformity of the Luminance

To capture the brightness, we measure the uniformityof illumination of the mural. For one input image I, Iis the vector of mural image, we do the followings stepby step.

(1) Get the log of I → the logged image is I′.(2) Undertake fast Fourier transformation (FFT) of

I′ → Ifft.(3) Generate a binary image Ibin, the same size as

the FFT matrix, which is 0 all over, except for identical8 pixel × 8 pixel squares in each corner.

(4) Multiply Ibin by the FFT image Ifft → themasked spectral image Imasked.

(5) Apply the inverse Fourier transform to Imasked →Iifft;

(6) Transfer exp(Iifft) → Iillu.

f14 = Iillu(I), (11)

where after a series of processes Iillu, we could attain avalue Iillu and assign it to f14.

Finally, we get the uniformity of luminance of themural according to Eq. (11).2.3.3 Arithmetic Average Brightness

The arithmetic average brightness is calculated as fol-lows:

f15 =1

W ∗ H

W∑x=1

H∑y=1

Bri(x, y), (12)

where Bri(x, y) is the value at pixel (x, y) of B, B =(IR +IG +IB)/3, the IR, IG, IB are the R, G, B chan-nel vectors of the mural image, and IR(x, y), IG(x, y),IB(x, y) are the values at pixel (x, y) of IR, IG and IB

respectively.2.3.4 Brightness Contrast

Without brightness contrast, it would be difficultto discriminate the exact place. We add the bright-ness contrast feature as follows. First, we compute thebrightness difference of one pixel in image as:

f16 =∑

j∈N(j)

|Bri(x, y) − Bri(x′, y′)|, (13)

where |Bri(x, y) − Bri(x′, y′)| represents the absolutedifference between the Bri(x, y) for pixel (x, y) andthe Bri(x′, y′) for its neighboring pixel (x′, y′), denote(x′, y′) as j, and N(j) is the neighborhood around pixelj.2.3.5 Logarithmic Average Brightness

To some extent, the logarithmic average brightnesscould describe the brightness of the mural. It is calcu-lated as:

f17 =255

W ∗ H

W∑x=1

H∑y=1

lg(Bri(x, y)

255+ ε

), (14)

where ε is a small positive constant to prevent fromcomputing lg 0.2.3.6 Lab Color Descriptor

A Lab color space[14] is a color-opponent space withdimension L for lightness, and dimensions a and b forthe color-dimensions, based on nonlinearly compressedcommission international de I’eclairage (CIE) XY Zcolor space coordinates. The nonlinear relations for L,a, b are intended to mimic the nonlinear response of theeye. The Lab color descriptor is computed as:

f18 =1

180(W ∗ H)

W∑x=1

H∑y=1

IL(x, y), (15)

f19,20 =1

200(W ∗ H)

W∑x=1

H∑y=1

IQ(x, y) + 80, (16)

Q ∈ (a, b),

where IL(x, y), IQ(x, y) are the values at pixel (x, y) ofvectors IL and IQ, IL is the vector of L channel, IQ isvector of a, b channel respectively.

In summary, 20 features are extracted from a muralto represent its aesthetic visual style globally. Thesefeatures are based on the analysis of Dunhuang morals,including some art rules, and they were evaluatedthrough experiments in next section.

Page 5: Aesthetic visual style assessment on Dunhuang murals

32 J. Shanghai Jiaotong Univ. (Sci.), 2014, 19(1): 28-34

3 Experimental Results

Aesthetic visual style assessment is a subjective prob-lem. Therefore, how to construct a mathematics andcomputer model to describe the aesthetic style visualof one mural is a non-trivial task. In this section, wewill detail our experimental results.3.1 Dataset

To evaluate the classification performance, we chosenthree dynasties Dunhuang murals as typical aestheticvisual styles to be evaluated: Northern Zhou, Tang andYuan. The reason why we selected these dynasties isthat, in Dunhuang mural history, art researchers alwaysdivided murals into three categories: the early, middleand final types based on dynastic style of mural. Con-sequently, we took these three dynasties murals as therepresentative of each period.

Although the number of Dunhuang murals is large,digital murals are not easy to obtain, thus the actualsamples in our experiment are not so big. We haveconstructed a dataset consisting of 450 murals with 150murals for each dynasty. To objectively and correctlymeasure our method, the content of murals included:joss, plant, mountain and water, building and others.We adopted the “leave-N-out” cross validation methodfor experiment. We repeated the algorithm for 10 times:for each class, N (N = 30, 40, or 50) murals were ran-domly selected as train murals and the rest are testmurals. Each time we perform an independent experi-ment for training and testing.3.2 Classification Performance

In this section, we will show our experimental resultsaccording to classification performance. Table 1 demon-strates the classification results and we could find thatfor the classification of Tang from Northern Zhou, our

method works very well and the classification accuracyis high. But for the classification of Tang from Yuanand Northern Zhou from Yuan, the accuracy is a littlelower. The reason for this can be explained as follows.The Northern Zhou dynasty murals emphasized the po-tential of air movement toward the formation of flowlines which shows a kind of beautiful emotion, whilein the Tang dynasty, colors were lively used and thebrushwork line was thick. When it came to the Yuandynasty, it almost reached the end of the Dunhuangmural history. Artists tended to use uniform, energeticand flexible curves, and don’t give re-color. Thereforethe murals in this period looked like somewhat solid.In a word, in the Northern Zhou period the rhythmof curve was stressed, murals belonged to freehand im-ages rather than realistic ones. Murals in Tang showeda rich, soft, vibrant artistic charm. However, in theYuan dynasty, the characteristics style of former twoperiods had been absorbed. The style in the Yuan dy-nasty had both the Tang dynasty’s round, i.e., a senseof flexibility, and the characteristics of the NorthernZhou dynasty.

So, compared with the difference between Tang andNorthern, the differences between Tang and Yuan orNorthern Zhou and Yuan are not so clear. As seenin Table 1, this results in a little lower classificationaccuracy in our experiment. Note that, the number inthe last line of Table 1 shows multi-label classificationaccuracy of three typical dynasties Dunhuang murals.

In order to comprehensively evaluate the benefits ofour proposed method, we also test the method[15] onour established dataset. Table 2 shows the classifica-tion results when using that method. It is easy to con-clude that our method is superior to method proposedin Ref. [14] on Dunhuang murals dataset.

Table 1 Classification performances for Dunhuang murals of different dynasties

DynastiesAccuracy

N = 30 N = 40 N = 50

Tang, Northern Zhou 0.969 2±0.031 0 0.971 3±0.019 5 0.982 7±0.028 2

Tang, Yuan 0.788 5±0.026 2 0.793 3±0.047 6 0.816 4±0.036 9

Northern Zhou, Yuan 0.832 6±0.019 8 0.859 2±0.034 2 0.890 1±0.022 9

Tang, Northern Zhou, Yuan 0.737 4±0.043 1 0.770 2±0.035 7 0.789 6±0.045 5

Table 2 Classification performances for Dunhuang murals using the method in Ref. [15]

DynastiesAccuracy

N= 30 N= 40 N= 50

Tang, Northern Zhou 0.964 5±0.012 7 0.970 2±0.011 4 0.979 7±0.038 7

Tang, Yuan 0.760 3±0.010 2 0.770 7±0.013 5 0.793 2±0.010 3

Northern Zhou, Yuan 0.828 6±0.043 1 0.846 3±0.011 8 0.881 2±0.029 9

Tang, Northern Zhou, Yuan 0.715 4±0.027 5 0.742 5±0.024 0 0.762 2±0.018 4

Page 6: Aesthetic visual style assessment on Dunhuang murals

J. Shanghai Jiaotong Univ. (Sci.), 2014, 19(1): 28-34 33

Table 3 lists the classification performances on themulti-label case (i.e., Tang, Northern Zhou and Yuan)based on different categories of features. From Table2, the experimental results show that, among the threetypes attributes, e.g., composition attributes, color at-tributes and brightness attributes, the composition fea-tures perform better than others, and color and bright-ness features seem as competitors with each other. Fur-thermore, the improvement by combining all featuresproves that our proposed features are complementary,as shown in the last line of Table 2.

We also tested the performance of each individ-ual feature by using support vector machine (SVM).The most five distinctive features are listed below:{f2, f9, f15, f18, f20}. These results help us understandmore about which feature is more powerful to describeaesthetic visual style of Dunhuang murals. We try toexplain these results in the following ways based onsome art knowledge.

(1) f2, standard deviation: a measurement of the dis-persion of a set of data from its mean. If an image is

supposed to be uniform throughout, the standard devi-ation should be small that indicates that the pixel inten-sities do not stray very far from the mean. The muralsin different dynasties may relate to different standarddeviation values, as shown in Fig. 1, the vertical coor-dinate is the standard deviation value, and horizontalcoordinate corresponds to index of test image.

(2) f9, spatial variation of color: this feature triesto describe the color palette of Dunhuang murals. Itis intuitively rational that the variation of the wholeimage color affects people’s impression on a painting.

(3) f15, arithmetic average brightness: the most pop-ular brightness editing algorithm is based on arithmeticmean model. This brightness measure has the biggestdifference with luminance. Figure 2 depicts the his-tograms of the arithmetic average brightness value forthree dynasties, and the difference among three dynas-ties is obvious, the vertical coordinate is the arithmeticaverage brightness value, and horizontal coordinate cor-responds to index of test image.

(4) f18, f20, L, b average values in Lab color space: in

Table 3 Classification performances on the multi-label case using different categories of features

DynastiesAccuracy

N= 30 N= 40 N= 50

Composition attributes 0.735 1±0.011 8 0.740 8±0.027 9 0.752 9±0.016 5

Color attributes 0.626 7±0.029 8 0.624 7±0.059 4 0.642 1±0.017 4

Bright attributes 0.640 9±0.031 9 0.651 0±0.027 2 0.662 4±0.013 7All attributes 0.737 4±0.043 1 0.770 2±0.035 7 0.789 6±0.045 5

Fig. 1 The histograms of the standard deviation value for three dynasties

Fig. 2 The histograms of the arithmetic average brightness value for three dynasties

Page 7: Aesthetic visual style assessment on Dunhuang murals

34 J. Shanghai Jiaotong Univ. (Sci.), 2014, 19(1): 28-34

Lab color space, its b component aspires to perceptualuniformity, and its L component closely matches hu-man perception of lightness. As mentioned above, wehave pointed that, the relations for L, a, b are intendedto mimic the response of the eye and also used for iden-tify the aesthetic visual style among murals of differentdynasties.

However, there are some misclassified murals asshown in Fig. 3. From the Fig. 3, we argue that, somemurals are so atypical in their aesthetic visual style thatthey are not easy to be distinguished even by the hu-man eye. Thus, it is a non-trivial work for computer toassess the aesthetic visual style of these murals.

Fig. 3 The misclassified murals in the experiment

4 Conclusion

The aesthetic visual style assessment is a problem ofsubjective cognition. As one of important art works,Dunhuang murals have their sole characteristics; espe-cially the aesthetic visual style develops with the dy-nasties going by.

To solve this problem, we extract a group ofperception-related global feature to construct a frame-work to describe the aesthetic visual style of Dunhuangmurals. With the analysis of on Dunhuang murals, theaesthetic visual style of these murals is represented ac-cording to three components: composition attributes,color attributes, and brightness attributes. These at-tributes are then all integrated together to evaluatewhich dynastic aesthetic visual style the mural belongsto. Finally, we use support vector machine as the classi-fier. The experimental results show that, the proposedmethod produces good classification accuracy when us-ing the extracted features. The performance of individ-ual extracted feature is also evaluated, and this will helpus learn more about intrinsic art knowledge of Dun-huang murals.

There is great room for future work in several di-rections. First, the same aesthetic visual styles shouldhave somewhat similarity. We could try to find the sim-ilarity measurement between same styles. Second, moreart-related analysis should be explored by the futureresearch. Third, search a more efficient classificationmethod to improve accuracy of our method. Finally,the best paper[15] of 2011 International Conference onComputer Vision (ICCV 2011) inspires us to use theidea of “relative attributes” to recognize different aes-thetic visual styles, and this work is in process.

References

[1] Guo Xiao-li. The enlightenment of Dunhuang frescocolor research in Sui-Tang Dynasties to modern colordesign [D]. Beijing, China: Beijing Institute of FashionTechnology, 2010 (in Chinese).

[2] Huang Jun. The studies for Dunhuang fresco char-acter style in the Tang Dynasty [D]. Beijing, China:China Academy of Art, 2007 (in Chinese).

[3] Keren D. Recognizing image “style” and activities invideo using local features and naive Bayes [J]. PatternRecognition Letters, 2003, 24(16): 2913-2922.

[4] Li J, Wang J Z. Studying digital imagery of ancientpaintings by mixtures of stochastic models [J]. IEEETransactions on Image Processing, 2004, 13(3): 340-353.

[5] Ke Y, Tang X, Jing F. The design of high-level fea-tures for photo quality assessment [C]//2006 IEEE In-ternational Conference on Computer Vision and Pat-tern Recognition (CVPR’06). New York, USA: IEEEComputer Society, 2006: 419-426.

[6] Datta R, Joshi D, Li J, et al. Studying aesthetics inphotographic images using a computational approach[C]//The 9th European Conference on Computer Vi-sion (ECCV’06). Graz, Austria: Springer, 2006: 288-301.

[7] Moorthy A K, Obrador P, Oliver N. Towardscomputational models of visual aesthetic appeal ofconsumer videos [C]//The 11th European Confer-ence on Computer Vision (ECCV’10). Crete, Greece:Springer, 2010: 1-14.

[8] Heihrich W. Principles of art history [M]. Pan Yao-chang trans. Shenyang, China: Liaoning People’s Pub-lishing House, 1987.

[9] Liu Xing. Rendering 3D mountain and rock models inChinese painting style [D]. Shanghai, China: Schoolof Software, Shanghai Jiao Tong University, 2007 (inChinese).

[10] Wang Xiang-hai, Qin Xiao-bin, Xin Ling. Advancesin non-photorealistic rendering [J]. Computer Science,2010, 37(9): 20-27 (in Chinese).

[11] Qian Xiao-yan. Study on non-photorealistic renderingtheory and method of artistic style image [D]. Nanjing,China: Nanjing University of Science and Technology,2007 (in Chinese).

[12] Brainard D H. Color appearance and color differencespecification [M]. Washington D C, USA: Optical So-ciety of America, 2003: 191-216.

[13] Dhar S, Ordonez V, Berg T L. High level de-scribable attributes for predicting aesthetic and in-terestingness [C]//2011 IEEE International Confer-ence on Computer Vision and Pattern Recognition(CVPR’11). Colorado Springs, USA: IEEE ComputerSociety, 2011: 1657-1664.

[14] Gehler P, Nowozin S. On feature combination formulticlass object classification [C]//2009 IEEE Inter-national Conference on Computer Vision (ICCV’09).Kyoto, Japan: IEEE, 2009: 221-228.

[15] Parikh D, Grauman K. Relative attributes[C]//2011 IEEE International Conference on Com-puter Vision (ICCV’11). Barcelona, Spain: IEEE,2011: 503-510.