Theme-Based Comprehensive Evaluation in New Product

University of WollongongResearch Online

Faculty of Engineering and Information Sciences -Papers: Part A Faculty of Engineering and Information Sciences

2011

Theme-Based Comprehensive Evaluation in NewProduct Development Using Fuzzy HierarchicalCriteria Group Decision-Making MethodJie LuUniversity of Technology Sydney, [email protected]

Jun MaUniversity of Wollongong, [email protected]

Guangquan ZhangUniversity of Technology Sydney, [email protected]

Yijun ZhuLille University of Science and Technology

Xianyi ZengENSAIT

See next page for additional authors

Research Online is the open access institutional repository for the University of Wollongong. For further information contact the UOW Library:[email protected]

Publication DetailsLu, J., Ma, J., Zhang, G., Zhu, Y., Zeng, X. & Koehl, L. (2011). Theme-Based Comprehensive Evaluation in New Product DevelopmentUsing Fuzzy Hierarchical Criteria Group Decision-Making Method. IEEE Transactions on Industrial Electronics, 58 (6), 2236-2246.

http://ro.uow.edu.au/



http://ro.uow.edu.au

http://ro.uow.edu.au/eispapers

http://ro.uow.edu.au/eispapers

http://ro.uow.edu.au/eis

Theme-Based Comprehensive Evaluation in New Product DevelopmentUsing Fuzzy Hierarchical Criteria Group Decision-Making Method

AbstractOne of the features of the digital ecosystem is the integration of human cognition and socio-economic themesinto the process of new product development (NPD). In a socio-economic theme-based NPD, ranking a set ofproduct prototypes that have been designed always requires the participation of multiple evaluators andconsideration of multiple evaluation criteria. Using the well-being theme-based garment NPD as abackground, this paper first presents a fuzzy hierarchical criteria group decision-making (FHCGDM) methodwhich can effectively calculate final ranking results through fusing all assessment data from human beings andmachines. It then presents a garment NPD comprehensive evaluation model with hierarchical criteria underthe well-being theme through identifying a set of marketing tactics from a consumer acceptance survey. Itfurther provides an establishment process for an NPD evaluation model under the digital ecosystemframework. Finally, a garment NPD case study further demonstrates the proposed well-being NPDcomprehensive evaluation model and the FHCGDM method. The advantages of the proposed evaluationmethod include successfully handling criteria in a hierarchical structure, automatically processing bothobjective measurements from machines and subjective assessments from human evaluators, and using themost suitable type of fuzzy numbers to describe linguistic terms.

Keywordscriteria, hierarchical, fuzzy, development, product, evaluation, comprehensive, method, theme-based,decision-making, group

DisciplinesEngineering | Science and Technology Studies

Publication DetailsLu, J., Ma, J., Zhang, G., Zhu, Y., Zeng, X. & Koehl, L. (2011). Theme-Based Comprehensive Evaluation inNew Product Development Using Fuzzy Hierarchical Criteria Group Decision-Making Method. IEEETransactions on Industrial Electronics, 58 (6), 2236-2246.

AuthorsJie Lu, Jun Ma, Guangquan Zhang, Yijun Zhu, Xianyi Zeng, and Ludovic Koehl

This journal article is available at Research Online: http://ro.uow.edu.au/eispapers/6754

http://ro.uow.edu.au/eispapers/6754

2236 IEEE TRANSACTIONS ON INDUSTRIAL ELECTRONICS, VOL. 58, NO. 6, JUNE 2011

Theme-Based Comprehensive Evaluation in NewProduct Development Using Fuzzy Hierarchical

Criteria Group Decision-Making MethodJie Lu, Member, IEEE, Jun Ma, Guangquan Zhang, Member, IEEE,

Yijun Zhu, Xianyi Zeng, and Ludovic Koehl

Abstract—One of the features of the digital ecosystem is the in-tegration of human cognition and socio-economic themes into theprocess of new product development (NPD). In a socio-economictheme-based NPD, ranking a set of product prototypes that havebeen designed always requires the participation of multiple evalu-ators and consideration of multiple evaluation criteria. Using thewell-being theme-based garment NPD as a background, this paperfirst presents a fuzzy hierarchical criteria group decision-making(FHCGDM) method which can effectively calculate final rankingresults through fusing all assessment data from human beings andmachines. It then presents a garment NPD comprehensive evalu-ation model with hierarchical criteria under the well-being themethrough identifying a set of marketing tactics from a consumeracceptance survey. It further provides an establishment process foran NPD evaluation model under the digital ecosystem framework.Finally, a garment NPD case study further demonstrates theproposed well-being NPD comprehensive evaluation model andthe FHCGDM method. The advantages of the proposed evaluationmethod include successfully handling criteria in a hierarchicalstructure, automatically processing both objective measurementsfrom machines and subjective assessments from human evalua-tors, and using the most suitable type of fuzzy numbers to describelinguistic terms.

Index Terms—Decision support systems, digital ecosystems,fuzzy sets, multicriteria decision making (MCDM), new productdevelopment (NPD), product evaluation.

I. INTRODUCTION

THE digital ecosystem is defined as an open, looselycoupled, domain clustered, demand-driven, and self-

organizing agents’ environment, in which each species is proac-tive and responsive for its own benefit or profit [8], [11]. Theunderlying technology for digital ecosystems is composed of

Manuscript received October 14, 2009; revised April 8, 2010; acceptedMay 12, 2010. Date of publication December 3, 2010; date of current versionMay 13, 2011. This work was supported in part by the Australian ResearchCouncil under discovery Grants DP0557154 and DP0880739.

J. Lu, J. Ma, and G. Zhang are with the Decision Systems & e-ServiceIntelligence Laboratory, Centre for Quantum Computation and Intelligent Sys-tems, Faculty of Engineering and Information Technology, University of Tech-nology, Sydney, Sydney, N.S.W. 2007, Australia (e-mail: [email protected];[email protected]; [email protected]).

Y. Zhu is with the Université des Sciences et Technologies de Lille, 59650Villeneuve d’Ascq, France (e-mail: [email protected]).

X. Zeng and L. Koehl are with the French Engineer School, Ecole NationaleSupérieure des Arts et Industries Textiles, 59100 Roubaix, France (e-mail:[email protected]; [email protected]).

Color versions of one or more of the figures in this paper are available onlineat http://ieeexplore.ieee.org.

Digital Object Identifier 10.1109/TIE.2010.2096171

web services architecture, self-organizing intelligent agents,ontology-based knowledge sharing and intelligence-based de-cision support, and recommendation and evaluation systems. Itis at the intersection of industry, business, human endeavors,social science, and cutting edge Internet technologies, and is anapplication-driven research.

According to the social and technical requirements presentedin the privilege technical report of the European Commissionfor FP7 Framework Program [10], future research activities ondigital ecosystem development will deal with multidisciplinarysubjects, pay more attention to the integration of informationwith socio-economic systems, and improve the user-centereddesign of products and services through integration with humanactions and cognition. The enhancement of interactions be-tween human actions, socio-economic concepts, and technicalstructures has become an important trend for the develop-ment of new digital ecosystems. The implementation of thesesystems in different industrial sectors can effectively changetraditional methods of design, management, and production.

In this context, we aim to integrate human actions andcomplex socio-economic themes, such as well-being andsustainable development, into the process of new product de-velopment (NPD) in order to adapt its design to various com-petitive markets. Under the digital ecosystem framework, thisprocess of NPD consists of four main phases: theme generation,product design and detail engineering, product and processdevelopment and evaluation/testing, and production and marketlaunch [1], [12], [30]. This process becomes more and moredigitalized in current digital ecosystems. In a garment NPDprocess, theme-related evaluation/testing can be directly linkedwith marketing analysis through a set of digital processes.The evaluation/testing aims to identify which features must beincorporated into the product, what benefits the product willprovide, and how consumers will react to the product [1], [17].This phase includes choosing and ranking prototype productsby a set of criteria under a design theme. It will test the themethrough asking a sample of prospective consumers and relevantdesigners on what they think of the idea, as well as by obtainingmeasuring data from particular machines. With the supportof digital ecosystems, more NPDs will start using advancedtechniques to optimize the evaluation process and analyze theevaluation results.

Literatures and practices indicate that NPD evaluation is akind of preference-based dynamic decision, and many human

0278-0046/$26.00 © 2010 IEEE

LU et al.: THEME-BASED COMPREHENSIVE EVALUATION IN NPD USING FHCGDM METHOD 2237

cognition, technical standard, and complex socio-economicfactors need to be considered [20], [29]. Therefore, the firstis to determine the evaluation criteria and their relevance tothe implementation of a design theme. A garment NPD com-prehensive evaluation always involves multiple criteria in ahierarchical structure. These criteria have different roles andrelevance to a design theme. In this paper, the design theme fora garment NPD is “well-being”—i.e., to assess which garmentproduct best matches the features of a well-being design andowns the well-being experience in wearing. This design themeneeds to assess the criteria which include factors not onlyrelated to garment function properties but also to fashion stylesand marketing demands.

Well-being of garment product quality is a complex conceptrelating consumer preference to product design; in particular,it takes the idea of quality as a philosophical and sociologicalnotion which is evaluated through the senses and perception. Awell-being-based garment NPD evaluation is often organized ingroups, such as a set of garment designers and/or salespersons,because an individual may not have sufficient knowledge toappropriately assess a product. Moreover, it often requiresgathering, processing, and assessing data from manufacturingdevices. In practice, human evaluators often express their as-sessment in subjective expressions, in particular, the linguisticterms [19], [24]; for example, to express the relevance of acriterion to a design theme, the terms “relevant (or important)”and “very relevant (or very important)” can be used, whilerelated machines provide data in objective data in which 10 m2

and −20 ◦C could be used. Therefore, a challenge here is how toeffectively fuse both subjective assessments and objective data.

Since a garment NPD evaluation exhibits typical features ofmulticriteria decision making (MCDM) and group decision-making (GDM) problems, the aforementioned issues requireextending MCDM and combining it with GDM methods withthe capability to deal with the following: 1) hierarchical criteria(for example, a product can be evaluated by multiple aspects,in which each aspect can be evaluated by multiple criteria, andeach criterion can be evaluated by multiple subcriteria); 2) ob-jective and subjective data fusion; and 3) linguistic informationprocessing to the well-being design theme.

This paper first explores what constitutes a well-being prod-uct and what features represent the concept of a well-beingdesign from the consumer’s point of view. A marketing surveywas conducted to discover consumers’ perceptions and require-ments on the well-being theme. The collected data were usedto determine product evaluation criteria and identify their rel-evance grades (weights). This paper proposes a garment NPDcomprehensive evaluation model which has a hierarchical struc-ture of evaluation factors under the well-being theme. In orderto handle linguistic terms used in the NPD evaluation model, afuzzy hierarchical criteria group decision-making (FHCGDM)method is developed. The method combines MCDM with GDMmethods and proposes hierarchical operators to fuse the dataobtained from both human evaluators and machines. It appliesfuzzy set techniques to handle three kinds of linguistic termsin evaluating garment products. A fuzzy hierarchical criteriagroup decision-support system (FHCGDSS) implements theproposed method and is used to support real garment NPD

evaluations. Furthermore, through analyzing the evaluation dataobtained from evaluators and machines under each criterion,we establish a relationship between human senses and machinemeasurement values. This relationship is very important, as itcan be directly used in the detailed engineering step of futureproduct design.

The main contributions of this paper include the following:1) the proposal of a garment NPD evaluation model under thetheme of well-being design; 2) the proposal of an establishmentprocess for an NPD evaluation model, which can be usedin many kinds of other products and/or with other themes;3) the development of an FHCGDM method and a specializedsoftware tool for garment NPD evaluation; and 4) the devel-opment of an application to directly support garment NPDevaluations and the establishment of a corresponding relationbetween human-sense and machine measurements.

Following the introduction, preliminaries and related workare presented in Section II. Section III describes the FHCGDMmethod for NPD evaluation. Section IV discusses the conceptof well-being design and presents a garment NPD comprehen-sive evaluation model under the concept and its establishmentprocess. An application of garment NPD using the proposedmodel and method is illustrated in Section V. Conclusions arehighlighted in Section VI.

II. PRELIMINARIES AND RELATED WORK

A. Related Work

Multicriteria decision making is widely used in the evalua-tion of a finite number of predetermined alternatives, which areassociated with a level of achievement of the criteria. Based onthe criteria, a selection or ranking of the alternatives is made[21]. GDM, including group decision-making models [20] andaggregation approaches [1], [18], has been extensively studiedin the literature. Since it was proposed by Zadeh [33], fuzzy settheory has been widely used to deal with linguistic variablesin various decision and evaluation applications, particularly inMCDM and GDM [2], [21], [28], [34]. For example, Changand Chen [9] developed a fuzzy MCDM method for technol-ogy transfer strategy selection in biotechnology. Furthermore,Carlsson and Fuller [7] reviewed the developments in fuzzyMCDM methods and identified some important applications.Dubois et al. [13] proposed two uncertainty modeling frame-works, namely, the transferable belief model and the qualita-tive possibility theory, to deal with multiexperts multicriteriaassessment problems. Stover and Hall [25] further providedan overview of the fuzzy-logic architecture and discussed itsapplication in data fusion, indicating that highly complex sys-tems with a large number of inputs will benefit from the useof qualitative linguistic rules if the control task is properlypartitioned. Goodridge and Kay [16] presented a modular fuzzycontrol architecture and an inference engine that can be used tomultilayer fuzzy behavior fusion for real-time reactive controlsystems. A more recent result by Sun et al. [26] reports thefuzzy-logic-based control of an induction motor for a dc gridpower-leveling system. Bouafia, Krim, and Gaubert [3] particu-larly presented their results on the evaluation and selection of afuzzy-logic-based switching state. Yu and Kaynak [32] further


reported their survey on how to use soft computing-basedintelligent systems to provide alternative means for engineeringand product development. Our recent developments on fuzzyMCDM proposed a fuzzy multicriteria group decision method[21], [22] and a software tool called Decider [23]. However,theme-based product evaluation in NPD with fuzzy hierarchicalcriteria in a group is a new exploration.

B. Preliminaries

For the convenience of describing the proposed FHCGDMmethod, we first introduce some basic notions and relatedtheorems of fuzzy sets and fuzzy numbers.

Definition 1: A fuzzy set A in a universe of discourse Xis characterized by a membership function μ

A(x) from X to

the interval [0, 1]. The function value μA(x) is termed the

membership degree of x belonging to A.Definition 2: The λ-cut of a fuzzy set a is defined as

aλ ={x;μ

a(x) ≥ λ, x ∈ X

}. (1)

If aλ is a nonempty-bounded closed interval in X , it can bedenoted by aλ = [aL

λ , aRλ ]; aL

λ and aRλ are the lower and upper

endpoints of the closed interval, respectively.Definition 3: A fuzzy number is a convex and normalized

fuzzy set on R if, for any λ ∈ [0, 1], aλ is a finite closed interval.Definition 4: A triangular fuzzy number a is defined by

a triplet (aL0 , a, aR

0 ), and the membership function μa(x) is

defined as

μa(x) =

⎧⎪⎪⎪⎪⎨⎪⎪⎪⎪⎩

0 x < aL0

x−aL0

a−aL0

aL0 ≤ x < a

aR0 −x

aR0 −a

a ≤ x ≤ aR0

0 aR0 < x

(2)

where a = aL1 = aR

1 .Definition 5: If a is a fuzzy number and 0 < aL

λ ≤ aRλ ≤ 1

for any λ ∈ (0, 1], then a is called a normalized positive fuzzynumber; if a triangular fuzzy number a is a normalized positivefuzzy number, then a is called a normalized positive triangularfuzzy number [31].

Let F ∗(R) and F ∗T (R) be the sets of all normalized positive

fuzzy number and all normalized positive triangular fuzzynumbers on R, respectively.

Definition 6: If a is a fuzzy number and aLλ > 0 for any λ ∈

(0, 1], then a is called a positive fuzzy number. Let F ∗+(R) be

the set of all finite positive fuzzy numbers on R.For any a, b ∈ F ∗

+(R) and 0 < α ∈ R

a + b =⋃

λ∈(0,1]

λ[aL

λ + bLλ , aR

λ + bRλ

]

αa =⋃

λ∈(0,1]

λ[αaL

λ , αaRλ

]

a × b =⋃

λ∈(0,1]

λ[aL

λ × bLλ , aR

λ × bRλ

].

Definition 7: Let a, b ∈ F ∗+(R), then the quasi-distance

function of a and b is defined as

d(a, b) =

⎛⎝ 1∫

0

12

[(aL

λ − bLλ

)2+

(aR

λ − bRλ

)2]dλ

⎞⎠

12

. (3)

III. FHCGDM METHOD

This section illustrates the proposed FHCGDM method forthe garment NPD evaluation problem.

A. Overview

Evaluating a set of new products under the well-being themeneeds to embrace both subjective assessments from humanevaluators (such as designers) as well as objective measure-ments from machines. As the well-being concept is an intrinsicelement of a human being’s psychological feelings, the machinemeasurements need to be converted to subjective assessments toreflect human experiences in a qualitative way. To implementthe conversion, designers and engineers should identify anacceptable range for each kind of machine measurement anddefine a preference value. The ranges are determined by thewell-being theme, and the preference value is determined bythe nature of the criterion relevant to the well-being theme.

After establishing the conversion from machine objectivemeasurements to subjective assessments, the method uses nor-malized positive fuzzy numbers to express linguistic termsused for subjective assessments on evaluation criteria, relevancegrades of criteria, and impacts of evaluators.

The evaluation result for each aspect of the well-being themeis obtained by aggregating assessments on its related criteriaand indicators. The obtained results for new designs are used fortwo purposes. One is to compare different designs to select theone which best matches the features of the well-being theme;the other is to guide the manufacturing procedure to produce abetter well-being product. In order to determine which designis better suited to meet the demand of the well-being concept,the results obtained will be compared with two predefined well-being acceptance grades, i.e., the best and the worst acceptancegrades. The nearer an evaluation result is to the best acceptancegrade, the more satisfactorily the design meets the requirementsof the well-being theme.

In summary, the presented evaluation method includes threestages: 1) converting the machine measurements into subjectiveassessments; 2) evaluating new designs following the hierarchyof criteria; and 3) calculating the acceptance grades of new de-signs based on the evaluation results. In the following sections,we will provide more detail of these three stages.

B. Symbol Notations

For convenience, this section lists the symbols used in themethod illustrations.

Let Cn (n = 1, 2, 3) be the sets of aspects, criteria, and theindicators defined in the garment NPD evaluation model inFig. 1, respectively. From Fig. 1, we can see that C1 includes


Fig. 1. Hierarchical garment NPD evaluation model under the well-being theme.

two aspects, i.e., “fashion style” and “functional properties”; C2

includes nine criteria, i.e., “protection,” “warmth,” “dynamic,”“coolness,” “fabric hand,” “smell,” “sound,” “wash and care,”and “durability”; and C3 includes 30 indicators. (Note: dupli-cated occurring indicators in C3 are used for clarifying thehierarchy structure.) An aspect (or criterion) is supported bysome criteria (or indicators). The supporting relationship is

displayed by the connection lines. If a factor (i.e., an aspect,a criterion, or an indicator) is not supported by other factors,the factor is called a leaf node; otherwise, it is called a nonleafnode. For instance, the criterion “smell” is a leaf node, and allindicators are leaf nodes.

Based on consumer survey and development practices, eachevaluation factor/criterion is given a relevance grade which is


TABLE ILINGUISTIC TERMS FOR RELEVANCE GRADES

TABLE IILINGUISTIC TERMS FOR IMPACT FACTORS

TABLE IIILINGUISTIC TERMS FOR SUBJECTIVE ASSESSMENTS

chosen from the linguistic term set shown in Table I, where ai

is a normalized positive fuzzy number, i = 1, 2, . . . , 7.Let P = {P1, P2, . . . , Pm}(m ≥ 2) be the set of human

evaluators, and M = {M1,M2, . . . ,Mk} (k ≥ 2) be the set ofmachines. Each Pi (or Mj) is associated with an impact factorwi which indicates its influence on the evaluation result. Theimpact factors are linguistic terms taken from Table II, wheresi is a normalized positive fuzzy number, i = 1, 2, . . . , 7.

Let Ts ⊆ F ∗(R) be the linguistic terms used for expressingsubjective assessments as shown in Table III.

C. Machine Measurement Conversions

Machine measurements as evaluations of some factors ofthe well-being theme must be converted into subjective as-sessments before being used in the evaluation method. Basedon the different natures of these factors, a preference valueis first defined for each reasonable measurement range. Thepreference value has three possible meanings. The first repre-sents the value having the most enhancements for the featuresof the well-being theme. The second meaning indicates theopposite situation to the first. The third meaning emphasizesthat the preference value is the intermediate state between theprevious two.

According to the definition of the preference value in agiven measurement range, the feedback of consumer surveys,and previous development practices, a conversion function is

determined. Without loss of generality, this paper uses fc for theconversion function with respect to factor c, which is definedfrom the range Uc = [a, b] to TS . For any x ∈ Uc, fc(x) is theobtained subjective assessment from the measurement x.

Machine measurements are often obtained from real man-ufacturing procedures. These objective measurements do notprovide information on consumer experiences. The aforemen-tioned conversion functions attempt to establish links betweenthem.

D. New Design Evaluation

After obtaining subjective assessments from the machinemeasurements, the evaluation method combines and extendsthe conventional GDM and MCDM techniques to evaluate newdesigns under a hierarchy of factors with subjective assessments(linguistic terms). Multiple-source information fusion is a maincomponent of the presented evaluation method for new designs.The implemented aggregation for factors in the hierarchy isillustrated as follows.

Suppose that c is a leaf node which is evaluated by human be-ing evaluators only. Let Pc = {e1, e2, . . . , el} ⊆ P be the set ofhuman evaluators, and their assessments on c are v1, v2, . . . , vl,respectively. vi is taken from Table III. Then, the evaluationresult v on c is obtained by

v =

l∑i=1

wivi

l∑i=1

(wi)R0

(4)

where (wi)R0 is the right end of the 0-cut of wi and wi is the

impact factor of ei and is taken from Table II. If c is evaluatedby both a machine and human being evaluators, let Pc ={e1, e2, . . . , el} ⊆ P be the set of human being evaluators, andtheir assessments on c are v1, v2, . . . , vl, respectively, and themachine measurement value is x, then the evaluation result von c is given by

v =

l∑i=1

wivi + wfc(x)

l∑i=1

(wi)R0 + (w)R

0

(5)

where w is the impact factor of the machine and fc(x) is thesubjective assessment derived from x.

Suppose that c is a nonleaf node, {c1, c2, . . . , cq} is the setof factors supporting c, and v1, v2, . . . , vq are the obtainedevaluations on c1, c2, . . . , cq, respectively. Then, the evaluationresult v on c is calculated by

v =

q∑i=1

rivi

l∑i=1

(ri)R0

(6)

where ri is the relevance grade of ci with respect to the well-being theme and (ri)R

0 is the right end of the 0-cut of ri.


Based on the evaluations on all aspects in C1, the finalevaluation for a new design is

v =

t∑i=1

rivi

t∑i=1

(ri)R0

(7)

where vi is the evaluation on an aspect ci.

E. Acceptance Grade Calculation

The evaluation result of a new design given in (7) is a primaryparameter to calculate the acceptance grade of the design. Anacceptance grade is a real number which is calculated by (8),where v(0) and v(1) represent the worst and the best evaluationexpectations and d is the distance between two fuzzy numbersdefined in (3)

av =12

(d

(v, v(0)

)+

(1 − d

(v, v(1)

))). (8)

v(0) and v(1) are given by the following: v(0), v(1) ∈ F ∗(R),and

v(0)(x) ={

1, x = 00, otherwise.

v(1)(x) ={

1, x = 10, otherwise.

The acceptance grade av is then used to rank new designs. Ifa new design has a higher acceptance grade, the design is saidto better match the features of the well-being theme.

F. Further Discussions

The presented NPD comprehensive evaluation method underthe well-being theme is not only used to rank new designsbut also used to adjust new designs. Real manufacturing ofa product can only be based on machine settings. A smalldifference in the machine settings may lead to a completelydifferent consumer experience. If there are no links betweenmachine settings and consumer experiences, it is hard to knowwhether a product is really following the design theme andfulfilling consumer demand or not. This job will be completedautomatically in the current digital ecosystem. The presentedmethod provides an alternative approach to narrow the gap byconverting machine measurements into subjective assessmentsand fusing information from human beings and machines. Theevaluation result that is obtained for each new design is asubjective assessment representing the experiences of humanbeings. Through adjusting the machine settings and observingthe obtained evaluation result, engineers can find the settingsthat most satisfy the requirements of a product theme and, inturn, can adjust machine settings to produce a product whichcan best enhance the consumer’s experience.

Many other NPD evaluation problems have similar featuresto the well-being garment NPD, which include the following:1) evaluation criteria are within a hierarchy; 2) there is a groupof evaluators and/or a set of machines or both; 3) there is a

set of products (can be any); and 4) evaluators can use anylinguistic terms and/or numbers to express their evaluationvalues. Consequently, the presented FHCGDM method can bedirectly applied to those problems after necessary modificationsin related parameters: n, m, k, and so on. The determinationof parameters is dependent on a particular NPD evaluationmodel. In fact, the presented method can also be used onother MCDM and GDM problems because it is based on andcombines MCDM and GDM methods.

The presented method is implemented as a component inour FHCGDSS software. Section V will give a real example ofusing the presented method to evaluate a set of garment designsunder the well-being theme.

IV. GARMENT NPD EVALUATION MODEL

UNDER THE WELL-BEING THEME

This section presents a garment NPD comprehensive eval-uation model under the well-being theme in detail. The es-tablishment process of this model is also presented in thissection which can help bridge the communication gap betweenconsumers and designers.

A. Well-Being Theme in Garment Development

Well-being as a theme is a many-sided and complex concept.In broad terms, well-being refers to a well-lived life, a life richin meaning and personal growth, a life that reflects the fact ofone’s humanness and one’s membership of a community, and,finally, a life built from some sort of conscious thought andreflection as to its content and purpose [5]. A failed product canmean a potential risk to human life and well-being, the loss ofpublic confidence and funding, or the potential loss of marketshare and competitive advantage [4]. Every engineer, in turn,expects that each new product or service will, in some way, addto our health, comfort, and material well-being [6]. The well-being that individuals derive from clothing will depend to anoverwhelming extent on the social norms governing fashion.The expense of purchasing sufficient material for body warmthis trivial compared with the expense of buying clothes thatconform to accepted standards within society [14].

In almost all European textile companies, garment NPDfocuses on innovative products with modern concepts andadvantage techniques, which is a new trade in the current digitalecosystem. A highly innovative product is defined as a productthat offers new or unique benefits or solutions for market needsand involves great designs and production challenges. Well-being is one of the solutions. In general, an NPD cycle consistsof four stages: development of new product concept and test,simulation of product design and analysis of the productionprocess, product and process development and evaluation/testing, and production and market launch [1], [12], [15].Except for technical feasibility, strategic fit, financial perfor-mance and market opportunity, consumer acceptance, includingmarket requirement, consumer satisfaction and product qualityhave become the most important issues considered in the NPDprocess [27].

Well-being of garment product quality is a complex concept,particularly since it takes the idea of quality as a psychological


Fig. 2. Establishment process for a garment NPD evaluation model.

notion and is evaluated through the senses; its perception isnot usually expressed quantitatively [15], [27]. The design ofa well-being garment product has to satisfy the requirements ofindividuality. These requirements include functional propertiessuch as fulfilling high standards of comfort in wearing andfashion style requirements, such as the feeling of being veryrelaxed as if on a holiday. To meet these requirements, it isnecessary to pay special attention to obtaining high-qualityexpression and description of influencing factors and criteriato both fashion themes and functional properties. The conceptof well-being is described by a set of theme words, whichform the criteria to evaluate a product’s well-being features.However, some theme words are more relevant to the conceptof well-being than others; therefore, a well-being concept willbe described by these theme words and their relevance grades(weights). We have to consider the difference between garmentdesigners and consumers in understanding well-being products.An effective way is to combine both viewpoints for evaluatingthe fashion themes and functional property features of a productunder the well-being concept. As machines provide a technicalmeans to assist garment designers in the evaluation to improveengineering standards and accuracy, some features of the prod-ucts will also be measured by some machines. For example,machines can measure the flexibility and surface friction of aproduct.

B. Establishment Process of a Garment Product EvaluationModel Under the Well-Being Theme

A garment NPD comprehensive evaluation should be a for-mative activity, aimed at improving consumer satisfaction andmarketing acceptance and enhancing product quality. It usuallyemploys survey results from garment experts and consumerssimultaneously. The main difference between the two groups ofevaluators is that the experts will assess new prototype products,and consumers can normally only give feedback on productsalready in the market.

It is presumed to take place at the stage in the NPD cyclewhere a major resource commitment is required to “prove” theproduct. This occurs when the product concept moves from pre-liminary development and testing to scale-up, where piloting/prototyping or small-scale manufacturing needs to be evaluated.

We developed the following process/steps to establish a gar-ment NPD evaluation model for this task, as shown in Fig. 2.

1) To establish an initial garment NPD evaluation modelwith a hierarchical structure of factors under the themeof well-being.

Through the literature review and garment expert inter-views, an initial garment NPD evaluation model was firstestablished which has two sets of criteria: one relatingto fashion styles and the other to functional properties.Fig. 1 shows all the details.

2) To identify the factors and their relevance grades to thewell-being design theme under the proposed garmentNPD evaluation model.

A questionnaire was developed to include all questionsrelated to the factors and indicators identified in the initialevaluation model. Consumer surveys were conducted inselected shops using this questionnaire. The survey re-sults were then used to further identify these factors andtheir relationships. The finalized factors well-reflect theconsumers’ viewpoints for all identified features of well-being garment products.

3) To determine the relevance grade (weight) of each factorwith respect to both fashion styles and functional proper-ties in NPD evaluation under the well-being concept fromthe consumer’s viewpoint.

The consumer survey results also identify the weightsof factors to the two aspects and indicators to thesefactors. These weights reflect the importance grade, influ-ence grade, and closeness of each subfactor to its superiorfactor. They are described by linguistic terms.

4) To finalize the garment NPD evaluation model.Through the above steps, the initial evaluation model

is revised by adding weights of factors, and a well-beingtheme-based garment NPD evaluation model is finalized.It has two main aspects (fashion style and function prop-erty), each of which has multiple evaluation criteria; eachcriterion has also a set of indicators (subcriteria). Allthese factors have the weights shown in Fig. 1. The gar-ment NPD evaluation model can be used to evaluate newproducts. It can also be used to support the comparison ofevaluation results between consumers and designers.


C. Garment NPD Comprehensive Evaluation Model Underthe Well-Being Theme

Fig. 1 shows a garment NPD comprehensive evaluationmodel under the theme of well-being. It has two sets ofcriteria relating to fashion styles and functional propertieswithin three levels, namely, aspects, criteria, and indicators.The fashion styles are described by terms such as “protection,”“dynamism,” “warmth,” or “coolness.” Each has a number ofsubfactors, such as “health,” “sport,” “serenity,” “pleasure,”“holiday,” and so on. The functional properties include “fab-ric hand,” “smell,” “sound,” “wash and care,” and “durabil-ity.” Some properties also include detailed criteria and havemultiple values. For example, fabric hand has subfactors of“extensibility,” “density,” “compressibility,” “flexibility,” “sur-face friction,” “resilience,” “surface contour,” and “thermal-wetsensation.”

We give the following explanations for some important con-cepts and criteria used in this model.

1) Protection means the condition of being protected.2) Dynamism means sportive or active feeling and image in

social and personal life.3) Warmth concerns the state, sensation, or quality of pro-

ducing or having a moderate grade of heat, friendliness,kindness, or affection.

4) Coolness is the feeling of being comfortable or moder-ately cold.

5) Fabric handle is a consumer’s instinct to use the sense oftouch when choosing a garment, which is to describe andassess the fabric quality and its suitability for a specificend use. The way that the fabric feels is described asfabric handle. Some of these indicators can also be testedby machines.

6) Smell is about the odor of the garments containing asubstance or preparation intended for long-term releaseonto various surfaces of the human body, notably theskin, claiming one or more specific properties such asfragrance, maintenance in good condition, or control ofbody odor.

7) Sound refers to the noise of the garment, which is madeby body movement.

8) Wash and care label is the instruction for washing andcaring for the garment.

9) Durability is the capability of being able to wear theclothing for a long time.

In the model, each criterion presents a concept of well-beingand is described by a set of indicators.

For example, dynamism is described by three indicators:sport, pleasure, and relaxation. Different evaluators may havedifferent feelings of well-being and different preferences for itsfeatures. Fig. 1 also shows the weights of these aspects, criteria,and indicators. These weights come from the average valuesof consumer feedback. We can see that protection, dynamism,and coolness are all marked “absolutely related (AR),” and theaverage value of warmth’s weight from consumers is “highlyrelated (HR).” We also find that some indicators have relevanceto more than one criterion but with different relevance grades.For example, relaxation is an indicator relevant to protection,

Fig. 3. Objective measure settings for machine measurement.

warmth, dynamism, and coolness but with weights R, MR, R,MR, respectively. These relevance grades can be changed whena product emphasizes a particular theme or style.

V. EXPERIMENTAL STUDY

This section illustrates the presented evaluation methodthrough a real application. A garment company has eight newproduct designs to be evaluated by five human evaluators underthe well-being theme-based garment NPD evaluation modelshown in Fig. 1. Evaluators are asked to give their assess-ments of each question (factor) in the form of agreement inthe questionnaires. For example, if an evaluator agrees withthe statement that the product looks very relaxed, he/she cananswer “Very High” to the question (statement). The companywill summarize all the evaluators’ answers for all questions.The summarized results are described by linguistic terms inTable III. Moreover, 11 devices are used to measure the criteriasuch as “smell,” “sound,” “durability,” and indicators of thecriterion “fabric handle.” Based on the hierarchy of factorsin Fig. 1 and the evaluators’ answers, the company is ableto evaluate these new garment products. The relevance gradeof each factor is obtained from the consumer survey and isexpressed by a linguistic term in Table I.

According to the aforementioned settings, the garment NPDevaluation problem is solved in the FHCGDSS software [23]. Inthe real experiment, we designed a virtual device to substitutethose 11 real devices for simplifying inputs. This will not affectthe final evaluation result. An example of settings for machinemeasurement is given in Fig. 3. Figs. 4 and 5 compare theevaluation results by a human evaluator and by a machine. Thecomparison indicates that the new product E is the one whichmost satisfies the consumer and designer experience underthe well-being theme; therefore, the machine’s current settingsfor product C is not good enough to match this consumer’sexperience. However, by comparing Figs. 6 and 7, we knowthat product C is the most accepted product by all five humanevaluators. Hence, the current settings for product C need not


Fig. 4. Final evaluation result for indicator “pleasure” under criterion“warmth.”

Fig. 5. Machine evaluation result for the eight product designs.

Fig. 6. Evaluation result by all evaluators.

be adjusted too much because it is already the best one in mostevaluators’ experience.

Compared with other MCDM and GDM methods, the pre-sented method can deal with both subjective and objective

information simultaneously, which is more suitable to NPDevaluation. The presented method can deal with criteria ina multilevel hierarchy, which can help decision makers toobserve interesting aspect related to NPD evaluation and adjustmanufacturing technologies as necessary.

Based on human and machine evaluation results for eachproduct or material on the related criteria, we can establish aset of relationships between human senses and machine mea-surement values. Therefore, for example, a satisfactory levelof “flexibility” by fabric hand can be expressed and describedby a set of machine measurement values. These relationshipscan be directly used in the detailed engineering step of futureproduct design, which is an important part of digital ecosystemdevelopment. Through these efforts, the product evaluationprocess becomes more and more digitalized in current digitalecosystems.

VI. CONCLUSION AND FURTHER STUDY

With the development of digital ecosystems, optimized NPDintegrating human cognition and socio-economic conceptsbecomes more important in practice and technically imple-mentable. Success in human-centered NPD can be critical fora company to maintain its competitive advantage. The compre-hensive evaluation of proposed products is an important step inthe NPD process. It involves a complex situation in which somequalitative criteria are within a hierarchy, multiple membershave different opinions, and the judgments from evaluators areoften in vague rather than crisp numbers, and the data frommachines have different ranges. An FHCGDM method hasbeen presented in this paper and applied to evaluate garmentproducts under the theme of well-being. Based on this method,an FHCGDSS is developed to solve hierarchical criteria NPDevaluation problems in practice. Both the developed FHCGDMmethod and FHCGDSS tool can be easily applied to NPD ofdifferent products and with different themes. The contributionof this paper includes the following: 1) the proposal of agarment NPD evaluation model under the theme of well-beingdesign and its establishment means; 2) the development of anFHCGDM method for garment NPD evaluation; and 3) thedevelopment of an application to directly support garment NPDevaluation in practice.

Our further study includes establishing ontologies for well-being-related concepts to support automatic evaluation sys-tem establishment. We will particularly work on matchingexperiments for human evaluation with machine assessment.Based on the method proposed in this paper, a web-basedonline human-centered NPD evaluation system will be alsodeveloped.

REFERENCES

[1] P. Belliveau, A. Griffin, and S. Somermeyer, PDMA ToolBook 2 for NewProduct Development. New York: Wiley, 2004.

[2] E. Bim, “Fuzzy optimization for rotor constant identification of an indirectFOC induction motor drive,” IEEE Trans. Ind. Electron., vol. 48, no. 6,pp. 1293–1295, Dec. 2001.

[3] A. Bouafia, F. Krim, and J.-P. Gaubert, “Fuzzy-logic-based switchingstate selection for direct power control of three-phase PWM rectifier,”IEEE Trans. Ind. Electron., vol. 56, no. 6, pp. 1984–1992, Jun. 2009.


[4] M. Brown, P. Leavitt, and S. Wright, New Product Development: AGuide for Your Journey to Best-Practice Processes. Houston, TX: Amer.Productiv. Quality Center, 2004.

[5] H. J. Bruton, On the Search for Well-Being. Ann Arbor, MI: Univ.Michigan Press, 1997.

[6] J. G. Burke, The New Technology and Human Values. Belmont, CA:Wadsworth, 1966.

[7] C. Carlsson and R. Fuller, “Fuzzy multiple criteria decision making:Recent developments,” Fuzzy Sets Syst., vol. 78, no. 2, pp. 139–153,Mar. 1996.

[8] E. Chang and M. West, “Digital ecosystems and comparison to existingcollaboration environment,” WSEAS Trans. Environ. Develop., vol. 2,no. 11, pp. 1396–1404, 2006.

[9] P. Chang and Y. Chen, “A fuzzy multicriteria decision making method fortechnology transfer strategy selection in biotechnology,” Fuzzy Sets Syst.,vol. 63, no. 2, pp. 131–139, Apr. 1994.

[10] P. Dini, N. Rathbone, M. Vidal, P. Hernandez, P. Ferronato, G. Briscoe,and S. Hendryx, The digital ecosystems research vision: 2010 and beyond,Eur. Comm., Brussels, Belgium. [Online]. Available: http://www.digital-ecosystems.org/events/2005.05/de_position_paper_vf.pdf

[11] H. Dong, F. K. Hussain, and E. Chang, “A framework for discovering andclassifying ubiquitous services in digital health ecosystems,” J. Comput.Syst. Sci., vol. 77, no. 4, pp. 687–704, 2011, to be published.

[12] H. H. Driva, S. K. Pawar, and U. Menon, “Performance evaluation of newproduct development from a company perspective,” Integr. Manuf. Syst.,vol. 12, no. 5, pp. 368–378, 2001.

[13] D. Dubois, M. Grabisch, H. Prade, and P. Smets, “Using the transferablebelief model and a qualitative possibility theory approach on an illustrativeexample: The assessment of the value of a candidate,” Int. J. Intell. Syst.,vol. 16, no. 11, pp. 1245–1272, 2001.

[14] N. W. Edward, What Has Happened to the Quality of Life in the AdvancedIndustrialized Nation?. Northampton, MA: Edward Elgar Publ., 2004.

[15] J. Geršak, “Development of the system for qualitative prediction of gar-ments appearance quality,” Int. J. Clothing Sci. Technol., vol. 14, no. 3/4,pp. 169–180, 2002.

[16] S. G. Goodridge, M. G. Kay, and R. C. Luo, “Multilayered fuzzy behaviorfusion for real-time reactive control of systems with multiple sensors,”IEEE Trans. Ind. Electron., vol. 43, no. 3, pp. 387–394, Jun. 1996.

[17] S. Grubic, J. M. Aller, B. Lu, and T. G. Habetler, “A survey on testingand monitoring methods for stator insulation systems of low-voltage in-duction machines focusing on turn insulation problems,” IEEE Trans. Ind.Electron., vol. 55, no. 12, pp. 4127–4136, Dec. 2008.

[18] F. Herrera and E. Herrera-Viedma, “Choice functions and mechanisms forlinguistic preference relations,” Eur. J. Oper. Res., vol. 120, no. 1, pp. 144–161, Jan. 2000.

[19] N. Karacapilidis and C. Pappis, “Computer-supported collaborativeargumentation and fuzzy similarity measures in multiple criteriadecision-making,” Comput. Oper. Res., vol. 27, no. 7/8, pp. 653–671,Jun. 2000.

[20] J. M. Liberatore and C. M. Stylianou, “Expert support systems for newproduct development decision making: A modeling framework and appli-cations,” Manage. Sci., vol. 41, no. 8, pp. 1296–1316, Aug. 1995.

[21] J. Lu, G. Zhang, D. Ruan, and F. Wu, Multi-Objective Group DecisionMaking: Methods, Software and Applications With Fuzzy Set Technology.London, U.K.: Imperial College Press, 2007.

[22] J. Lu, G. Zhang, and D. Ruan, “Intelligent multi-criteria fuzzygroup decision-making for situation assessments,” Soft Comput.—FusionFound., Methodologies Appl., vol. 36, no. 1, pp. 18–26, 2008.

[23] J. Ma, J. Lu, and G. Zhang, “Decider: A fuzzy multi-criteria group de-cision support system,” Knowl.-Based Syst., vol. 23, no. 1, pp. 23–31,Feb. 2010.

[24] Marimin, M. Umano, I. Hatono, and H. Tamura, “Linguistic labels forexpressing fuzzy preference relations in fuzzy group decision making,”IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 28, no. 2, pp. 205–218,Apr. 1998.

[25] J. A. Stover and D. L. Hall, “A fuzzy-logic architecture for autonomousmultisensor data fusion,” IEEE Trans. Ind. Electron., vol. 43, no. 3,pp. 403–410, Jun. 1996.

[26] X.-D. Sun, K.-H. Koh, B.-G. Yu, and M. Matsui, “Fuzzy-logic-based V/fcontrol of an induction motor for a DC grid power-leveling system usingflywheel energy storage equipment,” IEEE Trans. Ind. Electron., vol. 56,no. 8, pp. 3161–3168, Aug. 2009.

[27] J. P. Venter and C. C. Van Waveren, “New product development withdynamic decision support,” in Proc. PICMET , 2007, pp. 21–42.

[28] J. Wang and Y. Lin, “A fuzzy multicriteria group decision making ap-proach to select configuration items for software development,” FuzzySets Syst., vol. 134, no. 3, pp. 343–363, Mar. 2003.

[29] L. Xu, Z. Li, S. Li, and F. Tang, “A decision support system for productdesign in concurrent engineering,” Decis. Support Syst., vol. 42, no. 4,pp. 2029–2042, Jan. 2007.

[30] H.-M. Yeh and K.-S. Chen, “Development of a digital-convolution-basedprocess emulator for three-dimensional microstructure fabrication usingelectron-beam lithography,” IEEE Trans. Ind. Electron., vol. 56, no. 4,pp. 926–936, Apr. 2009.

[31] K. Yoon and C. Hwang, Multiple Attribute Decision Making—An Intro-duction. Thousand Oaks, CA: Sage, 1995.

[32] X. Yu and O. Kaynak, “Sliding-mode control with soft computing: Asurvey,” IEEE Trans. Ind. Electron., vol. 56, no. 9, pp. 3275–3285,Sep. 2009.

[33] L. A. Zadeh, “Fuzzy sets,” Inf. Control, vol. 8, no. 3, pp. 338–353, 1965.[34] G. Zhang and J. Lu, “An integrated group decision making method with

fuzzy preference for alternatives and individual judgments for selectioncriteria,” Group Decis. Negotiation, vol. 12, no. 6, pp. 501–515, 2003.

Jie Lu (M’07) received the Ph.D. degree from theCurtin University of Technology, Perth, Australia,in 2000.

She is a Professor and the Director of the DecisionSystems and e-Service Intelligence Research Labo-ratory with the Faculty of Engineering and Informa-tion Technology, University of Technology, Sydney,Sydney, Australia. She has published five researchbooks and more than 250 papers in refereed jour-nals and conference proceedings. Her main researchinterests lie in the area of intelligent information

systems, decision making modeling, decision support system tools, uncer-tain information processing, e-Government, and e-Service intelligence andpersonalization.

Dr. Lu served as the Editor-In-Chief for Knowledge-Based Systems and asa Guest Editor of Special Issues for six international journals. He has alsodelivered four keynote speeches at international conferences. She has won fourAustralian Research Council discovery grants and many other research grants.

Jun Ma received the B.S. and M.S. degreesin applied mathematics from Southwest JiaotongUniversity, Chengdu, China, in 1996 and 1999,respectively.

He is currently a Research Associate with theFaculty of Engineering and Information Technol-ogy, University of Technology, Sydney, Sydney,Australia. He has about 40 publications in inter-national journals and conferences. His research in-terests lie in automated and approximate reasoningwith linguistic information and their application in

decision making.

Guangquan Zhang (M’10) received the Ph.D. de-gree in applied mathematics from the Curtin Univer-sity of Technology, Perth, Australia.

He is an Associate Professor with the Faculty ofEngineering and Information Technology, Univer-sity of Technology, Sydney, Sydney, Australia. From1979 to 1997, he was a Lecturer, Associate Professor,and Professor with the Department of Mathematics,Hebei University, Hebei, China. He has publishedfour monographs, four reference books, and over250 papers including more than 120 refereed journal

articles. His main research interests lie in the area of multiobjective, bilevel, andgroup decision making, decision support system tools, fuzzy measure, fuzzyoptimization, and uncertain information processing.

Dr. Zhang has served and continues to serve as a Guest Editor of SpecialIssues for three international journals. He has won four Australian ResearchCouncil discovery grants and many other research grants.


Yijun Zhu was born in Shanghai, China, in 1982.She received the B.S. degree in knitting engineeringand apparel from Donghua University, Shanghai,China, in 2003, the “Ingénieur” degree from theFrench Engineer School, Ecole Nationale Supérieuredes Arts et Industries Textiles (ENSAIT), Roubaix,France, in 2006, the M.S. degree in industrial engi-neering from the Université des Sciences et Tech-nologies de Lille, Villeneuve-d’Ascq, France, in2006, the M.S. degree in textile engineering fromDonghua University, in 2007, and the M.S. degree

in international industrial marketing and innovation from the Université desSciences et Technologies de Lille, in 2007, where she is currently workingtoward the Ph.D. degree in industrial engineering.

Her research interests include textile engineering, data analysis, sensoryevaluation, consumer behavior, brand management, and new textile and apparelproduct development.

Xianyi Zeng received the B.Eng. degree fromthe Department of Science and Technology,Tsinghua University, Beijing, China, in 1986, andthe Ph.D. degree from the Centre d’Automatique,Université des Sciences et Technologies de Lille,Villeneuve d’Ascq, France, in 1992.

He is currently a Full Professor with the FrenchEngineer School, Ecole Nationale Supérieure desArts et Industries Textiles (ENSAIT), Roubaix,France. Since 2000, he has led one research teamin ENSAIT. He has published two scientific books,

more than 160 papers at reviewed international journals, and internationalconference proceedings. His research interests include: 1) the development ofintelligent systems for the design of advanced materials and 2) modeling andanalysis of human perception and cognition on industrial products.

Dr. Zeng is currently an Associate Editor of International Journal ofComputational Intelligence System and Journal of Fiber Bioengineering andInformatics, a Guest Editor of Special Issues for three international journals,and delivered five plenary presentations at international conferences. He hasorganized more than ten international conferences and workshops since 1996.Since 2000, he has been the leader of a great number of research projects,including one European project, two national cooperative projects funded bythe French government, three bilateral research cooperation projects, and morethan 20 industrial projects.

Ludovic Koehl was born in Paris, France, onAugust 12, 1969. He received the B.S. degreefrom the French Engineer School, Ecole NationaleSupérieure des Arts et Industries Textiles (ENSAIT),Roubaix, France, in 1994, and the Ph.D. degreein automation from the Université des Sciences etTechnologies de Lille, Villeneuve-d’Ascq, France,in 1998.

Since 2010, he has been a Professor with theENSAIT. Since 1999, he has been involved in agreat number of projects dealing with optimization

of the quality and comfort of textiles by integrating physical measures andhuman knowledge. Since 1998, he has published 30 papers and attended60 international conferences. His research interests include pattern recognition,data mining, and computer modeling and their applications in textile industry.

Documents

Theme-Based Comprehensive Evaluation in New Product