Fuzzy Entropy Based Neuro-Wavelet Identifier-Cum-Quantifier for Discrimination of Gases/Odors

1548 IEEE SENSORS JOURNAL, VOL. 11, NO. 7, JULY 2011

Fuzzy Entropy Based Neuro-WaveletIdentifier-Cum-Quantifier for

Discrimination of Gases/OdorsRavi Kumar, R. R. Das, V. N. Mishra, and R. Dwivedi

Abstract—In this paper, a new approach to design of an odor/gasidentifier-cum-quantifier is presented. Dynamic response curvesof an oxygen-plasma treated thick-film tin oxide sensor array ex-posed to four different gases were subjected to continuous wavelettransform (CWT). Appropriate wavelet coefficients were selectedusing multiscale principal component analysis (MSPCA). Fuzzyentropy and fuzzy subsethood values were calculated for the indi-vidual odor/gas and for the particular concentration band of eachodor/gas, respectively. The quantitative information was encodedin the fuzzy subsethood values of the particular concentrationbands in the output feature space, whereas the fuzzy entropyvalues were used to normalize the training data set consistingof MSPCA selected wavelet coefficients. A feedforward neuralnetwork was trained with a backpropagation algorithm withthe training data containing the wavelet coefficients normalizedwith fuzzy entropies of individual odors/gases. The target dataset was made up of the fuzzy subsethood values of the particularconcentration band. The proposed network achieved identifica-tion and quantification of odors/gases with a 100% success rate.Also, fuzzy entropy based normalization helped to achieve 100%identification/quantification with a reduced number of sensors inthe array.

Index Terms—Backpropagation algorithm, fuzzy entropy, multi-scale principal component analysis, oxygen plasma, tin oxide sen-sors, wavelet transform.

I. INTRODUCTION

O XYGEN-plasma-treated thick-film tin oxide sensorsare capable of sensing at room temperature, whereas a

temperature upwards of 300 C is required for a conventionalthick-film sensor to attain a significant level of sensitivitytowards various gases. Thus, oxygen-plasma treatment ofthick-film tin oxide sensors eliminates the need to fabricate anintegrated heater along with the sensor array [1].

Due to their room temperature operation, design ruggedness,and ease of fabrication, oxygen-plasma-treated thick-film sen-sors have a clear edge over conventional thick-film sensors.In spite of the aforementioned advantages, a very serious

Manuscript received September 06, 2010; accepted November 13, 2010. Dateof publication December 03, 2010; date of current version May 18, 2011. Theassociate editor coordinating the review of this paper and approving it for pub-lication was Prof. Kiseon Kim.

The authors are with the Department of Electronics Engineering, Instituteof Technology, Banaras Hindu University (IT-BHU), Varanasi, Uttar Pradesh,India 221005 (e-mail: [email protected]; [email protected];[email protected]; [email protected]; [email protected])

Color versions of one or more of the figures in this paper are available onlineat http://ieeexplore.ieee.org.

Digital Object Identifier 10.1109/JSEN.2010.2096209

Fig. 1. Schematic diagram of thick-film tin oxide sensor array.

limitation of oxygen-plasma-treated sensors is their poor selec-tivity [1]. If the limits imposed by their poor selectivity couldbe overcome, oxygen-plasma-treated sensors would surelybe preferred over the conventional thick-film based sensingdevices. The task of chemical sensing with the help of a sensorarray can be roughly divided into two parts, viz. identificationand quantification. Where on one hand poor selectivity hampersidentification, the tendency of sensor response to saturate athigher concentrations of test gas is an obstacle in the path ofproper quantification. Computational techniques employedare generally different for identification and quantification ofodors/gases. Gas identification is typically treated as a patternclassification problem, whereas a quantification problem istypically viewed as a function approximation task [2]. Com-putationally, pattern classification and function approximationproblems belong to the same genre. Pattern classification isa special case of function approximation [3]. Also, one kindof problem can be expressed in terms of the other. Therefore,simultaneous identification and quantification of odors/gasesis possible, provided that proper computational techniques areapplied. In the present work, we have attempted simultaneousidentification and quantification of four different odors/gasesby applying the theory of fuzzy-entropy to wavelet coeffi-cients. In the past, researchers have used wavelet transformof sensor response for effective feature extraction [4]–[6].In the present work, the wavelet coefficients were obtainedby applying wavelet decomposition to the response–recoverycurves (dynamic response) of individual sensors of the arrayto four different odors/gases. This paper is organized into thefollowing five sections. The first section describes in brief thefabrication procedure and experimental setup for the realizationof the sensor array. The second section describes the theories

1530-437X/$26.00 © 2010 IEEE

KUMAR et al.: FUZZY ENTROPY BASED NEURO-WAVELET IDENTIFIER-CUM-QUANTIFIER 1549

Fig. 2. Recovery plot of the array for different concentrations of (a) LPG; (b) CCl ; (c) CO; and (d) C H OH.

of wavelet transform and fuzzy entropy. The third section is onproblem formulation. The discussions of results and conclu-sions derived have been discussed in the final two sections.

II. EXPERIMENTAL

Tin oxide based sensors were fabricated at our laboratoryby the thick-film screen printing technique [1]. The gas-sensi-tive tin oxide paste was doped with 1% (by weight) Pt, CuO,ZnO, Pd, and Cd for the fabrication in the array. Gas-sensitivelayers of tin oxide doped with aforementioned dopants wereprinted and fired onto alumina substrate to result in an arrayof six sensors, with the sixth sensor being the undoped SnO2sensor. The schematic structure of the fabricated array is shownin Fig. 1. Oxygen plasma was generated at low pressure by ap-plying R.F. power at 13.56 MHz using the SAMCO plasma de-position system (Model no. 10). The response of the array was

measured in a test chamber of volume 2894.7 ml, with an inlet toinject the test gas. The resistance variations were measured fordifferent concentrations of the test gases viz. LPG, CCl , CO,and C H OH.

The measurement of resistance variation with time gave thedynamic response of the sensor array in the form of response–re-covery time plots. Fig. 2(a)–(d) shows the response–recoverytime analysis of the oxygen-plasma-treated array for differentgases. The dynamic response of the array was measured by mea-suring the resistance variations of all the sensors of the arraywith time. Fig. 2(a)–(d) [1] shows the dynamic response of thesensor array in the form of recovery time analysis plots. Whenthe test gas is adsorbed on the sensor surface, the conductivityof the sensor first increases and then reduces back to its orig-inal value. The time required for the test gas to reduce the con-ductivity of the sensor to its background conductance is known


Fig. 3. Flowchart of MSPCA approach.

as recovery time. It has been observed that as the concentra-tion of the test gases is increased, the recovery time increases,and at higher concentrations, the recovery time shows saturatingtendencies.

III. WAVELET TRANSFORM AND FUZZY MEASURES

A. Continuous Wavelet Transform and MSPCA

The continuous wavelet transform (CWT) has been de-vised with an aim to give good time–frequency resolution,thereby providing simultaneous frequency and temporal infor-mation [7]. The process of wavelet transform yields waveletcoefficients, which are evaluated for all values of (scale).Wavelet coefficients of a signal contain detailed frequencyand temporal information about it. They provide specific “fin-gerprints” that are characteristic of the signal. That is whythey can be used to extract features from time-varying signalseffectively. Varying degrees of success achieved have been re-ported in the past by employing wavelet coefficients for featureextraction tasks [8]. Since wavelet coefficients of a signal arecalculated for many scales, a lot of redundancy remains in thedata set thus generated. It necessitates the selection of the bestcoefficients from different scales for maximizing information

content. This can be accomplished by using multiscale principalcomponent analysis (MSPCA). MSPCA is especially useful forthe modeling of data coming from nonstationary signals [9]. Aflowchart of MSPCA is given in Fig. 3. According to Fig. 3,raw data is subjected to wavelet decomposition that yields aset of “detail coefficients” and “approximationcoefficient” at each of the data samples. Here, representsthe number of scales. Again, PCA is computed for each scale,and threshold values are defined depending upon the nature ofthe problem. Thresholding at each scale gives a set of waveletcoefficients, which are used to reconstruct the wavelet.

B. Fuzzy Entropy and Fuzzy Subsethood

Fuzzy sets can be considered as a generalization of the con-ventional crisp sets. Fuzziness is a measure of the degree towhich an event occurs [10], [11]. Each element of a fuzzy sethas a degree of membership assigned to it in accordance with amembership function.

The size or cardinality of a fuzzy set is given by , asfollows:

(1)


Fig. 4. Block diagram representation of simultaneous identification and quantification scheme.

where is the membership value of the th element of-valued fuzzy set .According to the fuzzy set theory

(2)

and

(3)

Fuzzy entropy measures the degree of fuzziness of a fuzzy setand is denoted by

(4)

Fuzzy subsethood measures the degree of belongingness of afuzzy set to its superset and is denoted by

Degree (5)

A fuzzy set can be a subset of another fuzzy set iff:for all .

The fuzzysubsethood theorem is given by

(6)

IV. PROBLEM FORMULATION

The dynamic response curves of Fig. 2(a)–(d) were sampledat regular time intervals to give a raw data set. The raw data weresubjected to the following operations:

1. wavelet transform and MSPCA to select a set of most ap-propriate wavelet coefficients;

2. calculation of fuzzy entropy of four gases for qualitativeanalysis;

3. calculation of fuzzy subsethood of individual concentra-tion bands of a gas for quantitative analysis.

The aforementioned three techniques were combined in a wayas shown in Fig. 4.

The input training vectors consisted of MSPCA selectedwavelet coefficients and fuzzy-entropies of the individual gases,whereas the target vectors fed to the neural classifier were madeup of the fuzzy subsethood values of concentration bands be-longing to a particular gas. There are four concentration bandsin each gas designated by b-1 (25 ppm), b-2 (50 ppm), b-3 (75ppm), and b-4 (100 ppm). Thus, there are a total of 16 fuzzysubsethood values, which represent the degree of belongingnessof a particular concentration band to its gas class.

A. Wavelet Transform and Coefficient Selection

The dynamic response curves of Fig. 2(a)–(d) were sampledat regular intervals of time to give raw data. There were 64 sam-ples taken from each of the four curves, yielding a vector of 256


samples. There were 16 samples each taken from each concen-tration band. Five out of six sensors of the sensor array weresampled since Cd doped sensor did not shown any significantresponse recovery. The raw data were subjected to continuouswavelet transform. The total number of scales selected was 32.The “db 4” wavelet family was chosen for analysis as it is themost widely used [12]. Wavelet decomposition yielded waveletcoefficients equal to the number of samples for each scale. Thus,a total of 32 sets of wavelet coefficients were obtained for eachcurve. The most appropriate coefficients were selected throughMSPCA as described in Fig. 3. The coefficients were used asinputs to neural classifiers so that well-defined decision bound-aries could be drawn.

B. Fuzzy Entropy Calculation

Fuzzy memberships were assigned to each wavelet coeffi-cient corresponding to a sensor response sample for all gasesas follows.

Centroid S was calculated for each set (where denotes agas and S will have elements, where denotes the number ofsensors in the array) as follows:

(7)

where , , , and represent the centroids for thecoefficients of sensor 1 to , respectively. In our case, the cen-troids were calculated by taking the mean of wavelet coefficientsof individual sensor response samples for gases.

The Euclidean distance of the wavelet coefficient vectorcorresponding to sample for gas can be obtained as given inthe following equation:

(8)

where is the wavelet coefficient corresponding to sensorresponse for sensor 1, gas , and sample , and so on. Eachcoefficient corresponding to sample is assigned a membership

in the output feature space, in the fuzzy set for th gasat sample by using triangular membership function as

(9)

where is the Euclidean distance of the wavelet coefficientcorresponding to sensor response vector at sample for gas .

and are the modulo of the maximum and min-imum values respectively of for a particular gas . Fuzzyentropy of a gas can thus be calculated as

(10)

C. Normalization of Data by Fuzzy Entropy

The fuzzy entropy of a particular data set is representativeof the degree of fuzziness associated with it. Since the calcula-

tion of fuzzy entropy involves operations only on the elementsof a particular data set, fuzzy entropy comes out to be repre-senting the underlying statistical properties of that class. Thishas served as the primary motivation to normalize the input data,made up of wavelet coefficients by dividing each pattern by itscorresponding set of fuzzy entropy. This has ensured that classinformation has been incorporated beforehand in the input dataso that subsequent neural network classification could proceedmore efficiently.

If is the wavelet coefficient of sample of gas , thenormalized value is given by

(11)

where is fuzzy entropy of gas .

D. Calculation of Fuzzy Subsethood

The wavelet coefficient corresponding to sensor responsesamples obtained from different gases were assigned member-ships in different concentration bands of those gases.

Centroid was calculated for each concentration bandof where denotes the gas and will have elements asgiven in the following equation:

(12)

Here, , , , and represent the centroids forthe response of sensors 1, 2, 3 and , respectively, in a concentra-tion band . The centroids were calculated by taking the simplemean of the wavelet coefficient corresponding to samples be-longing to a particular concentration band. The Euclidean dis-tance of the sensor response vector at sample of a gas

and concentration band of that particular gas can be ob-tained in the sensor response ratio vector space as given in thefollowing equation:

(13)

where is the wavelet coefficient corresponding to sensorresponse for sensor 1, gas , band , and sample , and so on.

Each wavelet coefficient corresponding to sample of bandis assigned a membership in the output feature space,

in the fuzzy set again by using a triangular membershipfunction as

(14)

where is the Euclidean distance of the sensor responsevector at sample for band of gas . andare the modulo of the maximum and minimum values, respec-tively, of for a particular band of gas . It is clear that theaforementioned fuzzy set is a subset of fuzzy set . Thedegree of belongingness of to changes, as changes for


Fig. 5. Fuzzy subsethood coefficients with number of sensors in the array.

TABLE ISENSORS SELECTED FOR REDUCED DIMENSIONALITY OF THE DATA SET

a particular . Thus, all the elements of can be mapped to asingle value which is the fuzzy subsethood value.

Degree (15)

(16)

Fuzzy-subsethood representation helped in achieving class sep-arability of concentration bands of individual gases. Since thecalculation of fuzzy subsethood coefficients depends upon thenumber of sensors in the array, a plot of fuzzy-subsethood coef-ficients with the number of sensors is shown in Fig. 5.

The legends b1, b2, etc., in Fig. 5 represent respective concen-tration bands of four gases, where b1, b2, b3, and b4 correspondto concentrations 25, 50, 75, and 100 ppm, respectively.

E. Backpropagation Algorithm Trained NeuralNetwork Classifier

The wavelet coefficients selected through MSPCA and nor-malized by fuzzy entropy were fed as inputs to a neural networkclassifier trained with backpropagation algorithm. The input

layer of the neural network consisted of five neurons while thenumber of neurons in the output layer was four. Each neuron ofthe output layer represented a gas class. The number of neuronsin the hidden layer was optimized by experimentation. The net-work when presented with an input pattern was trained to givean output in the form of firing of a neuron which representedthe gas class to which the pattern belonged. In addition to it, theneuron was made to fire at the fuzzy subsethood value of theparticular concentration band of the gas to which the patternbelonged. Thus, both qualitative and quantitative informationabout the input pattern was obtained. The network was firsttrained with a five-dimensional data set obtained from fivesensors of the array. Subsequent analysis involved trainingwith reduced number of sensors. The number of sensors wasreduced from initially five to four, three, two, and one, and theclassification performance was observed in each case.

V. EXPERIMENTAL RESULTS AND DISCUSSION

A. Classification With Five Sensors

The input data vector consists of five elements representingfive different sensors of the array where each element is anMSPCA-selected wavelet coefficient normalized by fuzzy en-tropy. The data were divided into training and testing sets. Atotal of 75% of the data were kept for training, whereas the re-maining 25% were assigned to the testing set. Thus, the trainingand testing phase input data matrices were having a size of5 192 and 5 64, respectively. Similarly, the output data con-sisted of target vectors for training and testing matrices with asize of 4 192 and 4 64, respectively. The simulations werecarried out on Matlab platform, and several versions of the back-propagation algorithm available in the Matlab neural networktoolbox [13] were tried out. The Trainlm methodology has beenfound to give satisfactory results. Trainlm employs the Leven-berg–Marquardt optimization method to update weights and bi-ases during training [14]. To eliminate the possibility of overfit-ting the -fold, the cross-validation scheme [15] withwas used. Trainlm is found to train the network best when the


TABLE IIPERCENTAGE CLASSIFICATION USING FUZZY ENTROPY NORMALIZATION

number of neurons in the hidden layer was four. Hence, the op-timal architecture of the proposed classifier is 5:4:4 trained withthat Trainlm methodology of a backpropagation algorithm.

TABLE IIIPERCENTAGE CLASSIFICATION WITHOUT USING FUZZY ENTROPY

B. Classification With Reduced Number of Sensors

Samples from a lesser number of sensors were taken and pre-processed similarly as explained in Section III. The selection of


sensors was done by calculating the coefficient of dominancefor each sensor, which is given by

(17)

where is the coefficient of dominance for sensor . is theinterclass variance of the sensor response, and is theintraclass variance of the sensor response for the gas.The interclass variance of the sensor response is calculatedas

(18)

where is the sensor response for the gas at theobservation, and is the mean value of all the responses of the

sensor.Similarly, the intraclass variance of the sensor response

for the gas is given by

(19)

The coefficient of dominance for each sensor was calculatedusing (17) and, thus, dominant sensors carrying the highestamount of information were identified. Table I enlists theselected senors carrying the highest information for the reduceddimensionality of the data set. A network with an optimizednumber of neurons in the hidden layer (in this case, 4) wastrained with the Trainlm methodology.

Table II summarizes the percentage classification and quan-tification performance of the network using fuzzy entropy whenthe number of sensors used to extract the training samples werevaried from five to one, while Table III represents the same whenfuzzy entropy was not employed. It is clear that the use of fuzzyentropy has resulted in a classifier that gives more than 90% suc-cess rate with only three sensor responses being used to extracttraining samples. Hence, fuzzy entropy normalization is the keyto proper identification with reduced number of sensors.

VI. CONCLUSION

Advanced fuzzy measures like fuzzy entropy and fuzzy sub-sethood can serve as better representatives of qualitative andquantitative information of odors/gases. Fuzzy entropy is ob-served to work better for codifying qualitative information ofa gas, while fuzzy subsethood gives a better representation todifferent concentration levels of a particular gas. In additionto it, MSPCA based wavelet coefficient selection generates anoptimal training matrix from a highly complex and jumbledraw data. By a proper combination of the aforementioned tech-niques, simultaneous identification and quantification of fourdifferent gases have been achieved in this work with a 100%success rate. The proposed identifier-cum-quantifier can accom-plish the task with a reduced number of sensors in the array andgives a high success rate.

REFERENCES

[1] A. Chaturvedi, V. N. Mishra, R. Dwivedi, and S. K. Srivastava, “Re-sponse of oxygen plasma treated {\hbox{CCl}} tin oxide sensor arrayfor LPG, CCl , CO, and C H OH,” Microelectron. J., vol. 30, pp.259–264, 1999.

[2] R. Gutierrez-Osuna, “Pattern analysis for machine olfaction: A review,”IEEE Sensors J., vol. 2, pp. 189–202, 2002.

[3] R. Gutierrez-Osuna, “Signal processing and pattern recognition foran electronic nose,” Ph.D. dissertation, North Carolina State Univ.,Raleigh, 1998.

[4] J. R. Huang, G. Y. Li, Z. Y. Huang, X. J. Huang, and J. H. Liu, “Temper-ature modulation and artificial neural network evaluation for improvingthe CO selectivity of �� gas sensor,” Sens. Actuators B, vol. 114,pp. 1059–1063, 2006.

[5] M. H. Hammond, K. J. Johnson, S. L. Rose-Pehrsson, J. Ziegler, H.Walker, K. Caudy, D. Gary, and D. Tillet, “A novel chemical detectorusing cermet sensors and pattern recognition methods for toxic indus-trial chemicals,” Sens. Actuators B, vol. 116, pp. 135–144, 2006.

[6] H. Ge and J. Liu, “Identification of gas mixtures by a distributed sup-port vector machine network and wavelet decomposition from tempera-ture modulated semiconductor gas sensor,” Sens. Actuators B, vol. 117,pp. 408–414, 2006.

[7] S. G. Mallat, A Wavelet Tour of Signal Processing. New York: Aca-demic, 1998, ch. 1.

[8] S. Pittner and S. V. Kamarthi, “Feature extraction from wavelet coeffi-cients for pattern recognition tasks,” IEEE Trans. Pattern Anal. Mach.Intell., vol. 21, no. 1, pp. 83–88, Jan. 1999.

[9] B. R. Bakshi, “Multi-scale PCA with application to multivariate statis-tical process monitoring,” AICHE J., vol. 44, pp. 1596–1610, 1998.

[10] L. A. Zadeh, “Fuzzy sets,” Inf. Control, vol. 8, pp. 338–353, 1965.[11] B. Kosko, Neural Networks and Fuzzy Systems: A Dynamical Systems

Approach to Machine Intelligence. Delhi, India: Prentice-Hall, 1992,ch. 7.

[12] I. Daubechies, “The wavelet transform, time-frequency localization,and signal analysis,” IEEE Trans. Inf. Theory, vol. 36, no. 5, pp.961–1005, 1990.

[13] MATLAB® Neural Network Toolbox Documentation. Natick, MA,MathWorks, Inc..

[14] U. Seiffert, “Training of large-scale feed-forward neural networks,”presented at the Int. Joint Conf. Neural Networks, Vancouver, BC,Canada, Jul. 16–21, 2006.

[15] S. Hykin, Neural Networks a Comprehensive Foundation. Delhi,India: Prentice-Hall, 1999, ch. 4.

Ravi Kumar received the B.E. degree in electronicsengineering from Jiwaji University, Gwalior, India.He is currently working towards the Ph.D. degree inthe Department of Electronics Engineering, Instituteof Technology, Banaras Hindu University (IT-BHU),Varanasi, India.

His research interests are neural and fuzzy sys-tems, e-nose, and VLSI design.

R. R. Das received the M.Sc.Eng. degree in electronics and the Ph.D. de-gree from the Institute of Technology, Banaras Hindu University (IT-BHU),Varanasi, India, in 1975 and 1999, respectively.

He is currently working as a Professor in the Department of Electronics En-gineering, IT-BHU. His research interests include neural networks and digitalsystems.

V. N. Mishra received the Ph.D. degree in electronics engineering from theUniversity of Roorkee India in 1998.

He is currently working as a Reader in the Department of Electronics Engi-neering, Institute of Technology, Banaras Hindu University (IT-BHU), Varanasi,India. His research interests include design and fabrication of IC compatible gassensors, neural networks, and PR techniques.

R. Dwivedi received the Ph.D. degree in electronics engineering from the In-stitute of Technology, Banaras Hindu University (IT-BHU), Varanasi, India, in1978.

He is currently working as a Reader in the Department of Electronics En-gineering, IT-BHU. He has over 200 research publications in various interna-tional/ national journals and in the proceedings of symposia. His research inter-ests include silicon based and thick-film sensors, MOS devices, photovoltaics,and LSI/VLSI.

Documents

Fuzzy Entropy Based Neuro-Wavelet Identifier-Cum-Quantifier for Discrimination of Gases/Odors