1. OUTLINE Introduction Adaptive Polynomial Filters using Truncated Volterra Series Expansion Adaptive Lattice Polynomial Filters Adaptive Bilinear Filters Classes of Application Sandip Joardar, MEE, 1st Sem, 1st Yr, JADAVPURSunday, March 31, 2013 UNIVERSITY 2

2. Sandip Joardar, MEE, 1st Sem, 1st Yr, JADAVPURSunday, March 31, 2013 UNIVERSITY 3 3. NECESSITY Polynomial Filters are required for non linear systems where the linear filters generally fail to meet the performance criteria. Adaptive Polynomial Filters are required under those circumstances where:-i. The a priori information about the statistical characteristics of the data to be processed is unavailable.ii.In Non stationary environment.iii. Presence of Model Uncertainties. Sandip Joardar, MEE, 1st Sem, 1st Yr, JADAVPURSunday, March 31, 2013 UNIVERSITY4 4. APPLICATIONS Speech Signal Processing Noise Cancellation in EEG signals Foetal ECG extraction and noise cancellation And more Sandip Joardar, MEE, 1st Sem, 1st Yr, JADAVPURSunday, March 31, 2013 UNIVERSITY 5 5. Sandip Joardar, MEE, 1st Sem, 1st Yr, JADAVPURSunday, March 31, 2013 UNIVERSITY 6 6. VOLTERRA SERIES: INTRODUCTION Formulated by Vito Volterra in work dating from 1887. Volterra series is used to model weakly non-linear dynamical systems. Volterra series is used to model a wide range of nonparametric models. Differs from the Taylors series in its ability to possess memory. Sandip Joardar, MEE, 1st Sem, 1st Yr, JADAVPURSunday, March 31, 2013 UNIVERSITY 7 7. VOLTERRA SERIES: A MathematicalAnalysis The Volterra Series representation for a continuous time system whose output response y(t) on being excited with an input signal x(t) is given as follows. Sandip Joardar, MEE, 1st Sem, 1st Yr, JADAVPURSunday, March 31, 2013 UNIVERSITY 8 8. VOLTERRA KERNELS The Volterra Kernel is the where n is the order of the truncated Volterra series used to represent the TI, causal, non linear system. Volterra kernels are nothing but functionals, i.e., map from the Euclidean Space to the undelying scalar field. Sandip Joardar, MEE, 1st Sem, 1st Yr, JADAVPURSunday, March 31, 2013 UNIVERSITY 9 9. DISCRETE VOLTERRA SERIES With the advent of Digital Signal Processing more emphasis has been laid on the Discrete Volterra Series expansion for representing non linear systems. Sandip Joardar, MEE, 1st Sem, 1st Yr, JADAVPURSunday, March 31, 2013 UNIVERSITY 10 10. DISCRETE VOLTERRA SERIES:MATHEMATICAL FORMULATION Letandbe the input and output of a discrete time, causal, non linear system respectively. Then we can representby the discrete Volterra series expansion using as given below. Sandip Joardar, MEE, 1st Sem, 1st Yr, JADAVPURSunday, March 31, 2013 UNIVERSITY 11 11. TRUNCATED DISCRETE VOLTERRASERIES Since an infinite series expansion is not practically implementable we generally resort to the truncated Discrete Volterra series expansion. Order = p, Number of delay elements = (N-1) Sandip Joardar, MEE, 1st Sem, 1st Yr, JADAVPURSunday, March 31, 2013 UNIVERSITY 12 12. GRAPHICAL REPRESENTATION: DISCRETEVOLTERRA SERIES 2nd order Sandip Joardar, MEE, 1st Sem, 1st Yr, JADAVPURSunday, March 31, 2013 UNIVERSITY 13 13. NECESSITY OF ADAPTIVE ALGORITHMS FORVOLTERRA FILTER IMPLEMENTATION The accuracy of Volterra kernel estimation is a major problem in practical applications. Kernels depend on the order of the truncated Volterra series. The adaptive algorithms are, therefore, widely used for the kernel estimation. Sandip Joardar, MEE, 1st Sem, 1st Yr, JADAVPURSunday, March 31, 2013 UNIVERSITY 14 14. ADAPTIVE VOLTERRA FILTER Sandip Joardar, MEE, 1st Sem, 1st Yr, JADAVPURSunday, March 31, 2013 UNIVERSITY 15 15. ADAPTIVE VOLTERRA FILTER:MATHEMATICAL FORMULATION The adaptation algorithm in this case would try to estimate the desired response signal using a truncated second order Volterra (SOV) series expansion given by the equation as follows. Sandip Joardar, MEE, 1st Sem, 1st Yr, JADAVPURSunday, March 31, 2013 UNIVERSITY 16 16. ADAPTIVE VOLTERRA FILTER:MATHEMATICAL FORMULATION (contd.) and are the coefficients of the Adaptive Filter that are iteratively updated at each time so as to minimize some convex function of the error signal defined as follows. Sandip Joardar, MEE, 1st Sem, 1st Yr, JADAVPURSunday, March 31, 2013 UNIVERSITY 17 17. LMS ADAPTIVE VOLTERRA FILTER Sandip Joardar, MEE, 1st Sem, 1st Yr, JADAVPURSunday, March 31, 2013 UNIVERSITY 18 18. LMS ADAPTIVE VOLTERRA FILTER(contd.) The Least Mean Square (LMS) adaptation algorithm is based on the Stochastic Gradient approach. The cost function, also referred to as the index of performance, is defined as the mean square error (MSE). The basic function of the algorithm is to estimate the vector of filter coefficients so as to minimize the MSE. Sandip Joardar, MEE, 1st Sem, 1st Yr, JADAVPURSunday, March 31, 2013 UNIVERSITY 19 19. LMS ADAPTIVE VOLTERRA FILTER(contd.) The update equation of the LMS algorithm. Filter Co-efficient vector H(n) and input vector x(n) Sandip Joardar, MEE, 1st Sem, 1st Yr, JADAVPURSunday, March 31, 2013 UNIVERSITY 20 20. LMS ADAPTIVE VOLTERRA FILTER(contd.) is the step size of the learning parameter of the adaptation algorithm that determines the speed of the convergence. The LMS algorithm for SOV filter can be represented as given below. Initialization:H(0) can be arbitrarily chosen. Update:for n 0 and n Mn = (n+1);end where M is the number of samples of the input signal x(n). Sandip Joardar, MEE, 1st Sem, 1st Yr, JADAVPURSunday, March 31, 2013 UNIVERSITY 21 21. LIMITATIONS OF THE LMSALGORITHM Areas of concern: Large Eigen value spread. Tracking Performancein a non stationary environment. Sandip Joardar, MEE, 1st Sem, 1st Yr, JADAVPURSunday, March 31, 2013 UNIVERSITY22 22. RLS ADAPTIVE VOLTERRA FILTER Sandip Joardar, MEE, 1st Sem, 1st Yr, JADAVPURSunday, March 31, 2013 UNIVERSITY 23 23. RLS ADAPTIVE VOLTERRA FILTER(contd.) Based on the method of Least Squares. Least Squares Error (LSE) given as follows.where Adaptation algorithm minimizes the following cost function called sum of weighted error squares. Sandip Joardar, MEE, 1st Sem, 1st Yr, JADAVPURSunday, March 31, 2013 UNIVERSITY 24 24. RLS ADAPTIVE VOLTERRA FILTER (updatealgorithm) Sandip Joardar, MEE, 1st Sem, 1st Yr, JADAVPURSunday, March 31, 2013 UNIVERSITY 25 25. LIMITATIONS OF THE RLSALGORITHM An operations count will show that the LMS algorithm has a computational complexity that is proportional to N2, i.e., O(N2), multiplications per time instant, whereas the complexity of the RLS algorithm is O(N4) multiplications per time instant. But, on the other hand, the RLS algorithm is more robust to the statistical variations of the input signal. Sandip Joardar, MEE, 1st Sem, 1st Yr, JADAVPURSunday, March 31, 2013 UNIVERSITY 26 26. Sandip Joardar, MEE, 1st Sem, 1st Yr, JADAVPURSunday, March 31, 2013 UNIVERSITY 27 27. ADAPTIVE LATTICE POLYNOMIALFILTERS: NECESSITY A very efficient method of implementing the Gram-Schmidt Orthogonalization algorithm. Showsfaster and less input-signal dependent convergence behaviour than their direct form counterparts. Adaptive Lattice Polynomial filters are fairly modular and, hence, theyare suitable for VLSI implementation. Sandip Joardar, MEE, 1st Sem, 1st Yr, JADAVPURSunday, March 31, 2013 UNIVERSITY28 28. ADAPTIVE LATTICE POLYNOMIAL FILTERS:MATHEMATICAL FORMULATION (contd.) Let us design a lattice predictor of three stages. Sandip Joardar, MEE, 1st Sem, 1st Yr, JADAVPURSunday, March 31, 2013 UNIVERSITY 29 29. ADAPTIVE LATTICE POLYNOMIAL FILTERS:MATHEMATICAL FORMULATION (contd.) By Gram-Schmidt Orthogonalization technique we try to develop the orthogonal basis for the input vector. ,, andrepresent an orthogonal basis for,,and, respectively.Sandip Joardar, MEE, 1st Sem, 1st Yr, JADAVPURSunday, March 31, 2013UNIVERSITY30 30. ADAPTIVE LATTICE POLYNOMIAL FILTERS:MATHEMATICAL FORMULATION (contd.) Thus, the output response of the adaptive system is given by the following equation. where, is the coefficients vector. andis the backward prediction error vector. Sandip Joardar, MEE, 1st Sem, 1st Yr, JADAVPURSunday, March 31, 2013 UNIVERSITY 31 31. ADAPTIVE LATTICE POLYNOMIAL FILTERS Sandip Joardar, MEE, 1st Sem, 1st Yr, JADAVPURSunday, March 31, 2013 UNIVERSITY 32 32. ADAPTIVE LATTICE POLYNOMIAL FILTERS:MATHEMATICAL FORMULATION (contd.) The output of the adaptive system is given by The error signal is given as follows The error signal for the i-th stage of the latticepredictor, Sandip Joardar, MEE, 1st Sem, 1st Yr, JADAVPURSunday, March 31, 2013 UNIVERSITY 33 33. ADAPTIVE LATTICE POLYNOMIAL FILTERS:LMS ADAPTATION ALGORITHMThe relevant update equations are: Sandip Joardar, MEE, 1st Sem, 1st Yr, JADAVPURSunday, March 31, 2013 UNIVERSITY 34 34. ADAPTIVE LATTICE POLYNOMIAL FILTERS:ORDER UPDATE RECURSIONFrom Levinson Durbin algorithm, Sandip Joardar, MEE, 1st Sem, 1st Yr, JADAVPURSunday, March 31, 2013 UNIVERSITY 35 35. LATTICE PREDICTOR FOR SOV SYSTEM Sandip Joardar, MEE, 1st Sem, 1st Yr, JADAVPURSunday, March 31, 2013 UNIVERSITY 36 36. ADAPTIVE LATTICE POLYNOMIALFILTERS: SHORTCOMINGS Complexity O(N3) compared to O(N2) complexity of their direct form versions. (N is the number of delay elements). This complexity is still greater in case of RLS algorithm. There is no guarantee that the decoupling property of the multi-stage lattice predictor is preserved in a non-stationary environment. Sandip Joardar, MEE, 1st Sem, 1st Yr, JADAVPURSunday, March 31, 2013 UNIVERSITY 37 37. Sandip Joardar, MEE, 1st Sem, 1st Yr, JADAVPURSunday, March 31, 2013 UNIVERSITY 38 38. ADAPTIVE BILINEAR FILTERS:NECESSITY The Adaptive Volterra filter requires large number of multi dimensional coefficients so as to accurately model the non linear system. For Adaptive SOV filter the computational burden increases exponentially as the order of the non linearity of the concerned system increases. Cannot model systems with strong non linearity. Sandip Joardar, MEE, 1st Sem, 1st Yr, JADAVPURSunday, March 31, 2013 UNIVERSITY 39 39. ADAPTIVE FILTER USING RECURSIVENON LINEAR DIFFERNCE EQUATIONS Input output relationship is governed by a recursive non linear difference equation of the type given as follows. is an ith order polynomial in the quantities within the parenthesis. Sandip Joardar, MEE, 1st Sem, 1st Yr, JADAVPURSunday, March 31, 2013 UNIVERSITY 40 40. ADAPTIVE BILINEAR FILTERS:MATHEMATICAL FORMULATION For a one dimensional input output case, its relationship is given by the Bilinear polynomial as follows. Sandip Joardar, MEE, 1st Sem, 1st Yr, JADAVPURSunday, March 31, 2013 UNIVERSITY 41 41. ADAPTIVE BILINEAR FILTERS:GRAPHICAL REPRESENTATION The block diagram for the case when N = 3 is shown in the figure below. Sandip Joardar, MEE, 1st Sem, 1st Yr, JADAVPURSunday, March 31, 2013 UNIVERSITY 42 42. ADAPTIVE BILINEAR FILTERS:MATHEMATICAL FORMULATION (contd.)where, Sandip Joardar, MEE, 1st Sem, 1st Yr, JADAVPURSunday, March 31, 2013 UNIVERSITY 43 43. ADAPTIVE BILINEAR FILTERS: TWOAPPROACHES Sandip Joardar, MEE, 1st Sem, 1st Yr, JADAVPURSunday, March 31, 2013 UNIVERSITY 44 44. ADAPTIVE BILINEAR FILTERS:EQUATION ERROR APPROACH Minimization of the mean square error. The feedback to the adaptive system is the output of the unknown Bilinear system. The update equations are given as follows. Sandip Joardar, MEE, 1st Sem, 1st Yr, JADAVPURSunday, March 31, 2013 UNIVERSITY 45 45. ADAPTIVE BILINEAR FILTERS:OUTPUT ERROR APPROACH The feedback to the adaptive system is its own output. The cost function in this case is given as follows. The output error approach performs better than the equation error approach in a noisy environment. Sandip Joardar, MEE, 1st Sem, 1st Yr, JADAVPURSunday, March 31, 2013 UNIVERSITY 46 46. ADAPTIVE BILINEAR FILTERS: STABILITY The concept of stability of such systems is frequently known to be input signal dependent. Such systems can be driven to instability just by changing their input signal characteristics. Adaptation algorithm has to guarantee continuous and global stability, otherwise, continual tracking of the adaptive filter coefficients has to be done. Sandip Joardar, MEE, 1st Sem, 1st Yr, JADAVPURSunday, March 31, 2013 UNIVERSITY 47 47. CLASS I: SYSTEM IDENTIFICATIONAdaptive Polynomial Filter (-) (+)System Unknown System InputPlantOutputSandip Joardar, MEE, 1st Sem, 1st Yr, JADAVPUR Sunday, March 31, 2013 UNIVERSITY49 48. CLASS II: INVERSE MODELLINGSystemUnknown AdaptiveOutputSystem Noisy Polynomial Input Plant Filter (-)(+)DelaySandip Joardar, MEE, 1st Sem, 1st Yr, JADAVPUR Sunday, March 31, 2013 UNIVERSITY 50 49. CLASS III: PREDICTIONSystemOutput 2(+)Adaptive System Delay Polynomial (-)Random Output 1 Filter Signal Sandip Joardar, MEE, 1st Sem, 1st Yr, JADAVPUR Sunday, March 31, 2013UNIVERSITY51 50. CLASS IV: INTERFERENCECANCELLATIONPrimarySignal(+) AdaptiveSystemReference Polynomial(-) Output Signal Filter Sandip Joardar, MEE, 1st Sem, 1st Yr, JADAVPURSunday, March 31, 2013 UNIVERSITY 52 51. References [1] Haykin S., Adaptive Filter Theory, Fourth Edition. [2] Treichler John R., Jr. Johnson C. Richard, Larimore Michael G., Theory and Design of Adaptive Filters. [3] Proakis John G., Manolakis Dimitris G., Digital Signal Processing. [4] V.J. Mathews, Adaptive Polynomial Filters, IEEE Signal Processing Magazine, July 1991, pp 10-25. [5] Singh Th. Suka Deba, Chatterjee Amitava, A comparative study of adaptation algorithms for nonlinear system identification based on second order Volterra and bilinear polynomial filters, Elsevier Measurement, 2011. [6] Koh. T. and E.J. Powers. Second-order Volterra filtering and its application to nonlinear system identification. IEEE Transactions on Acoustics, Speech, and Signal Processing, Vol. ASSP-33, No. 6, pp 1445-1455, December 1985. [7] Kenefic, R. J. and D. D. Weiner. Application of the Volterra functional expansion in the detection of nonlinear functions of Gaussian processes, IEEE Transactions on Communications. Vol. COM-31, No.3, pp 407-412, March 1983. Sandip Joardar, MEE, 1st Sem, 1st Yr, JADAVPURSunday, March 31, 2013 UNIVERSITY 53 52. References (contd.) [8] Zhang H., Volterra Series: Introduction and Application, ECEN 665(ESS): RF communication Circuits and Systems. [9] Abrudan T., Volterra Series and Non linear Adaptive Filters, S-88.221 Postgraduate Seminar on Signal Processing 1, Espoo, 30.10.2003 p. 1/23. [10] Boyd S., Chua L.O., Desoer C.A., Analytical Foundation of Volterra Series, IMAJournalof Mathematical Control & Information (1984) I, 243 282. [11] Niknejad Ali M., EECS 242: Volterra/Wiener representation of Non-Linear Systems, AdvancedCommunication Integrated Circuits, University of California, Berkeley.[12] Lesiak Casimir M., Krener Arthur J., THE EXISTENCE AND UNIQUENESS OF VOLTERRA SERIES FOR NONLINEAR SYSTEMS. [13]Zhang J., Zhao H., A novel adaptive bilinear filter based on pipelined architecture, Elsevier Digital Signal Processing, 2010. [14]Georgeta B., Botoca C., Nonlinearities Identification using The LMS Volterra Filter, 2005 WSEAS Int. Conf. on DYNAMICAL SYSTEMS and CONTROL, Venice, Italy, November 2-4, 2005 (pp148-153). Sandip Joardar, MEE, 1st Sem, 1st Yr, JADAVPURSunday, March 31, 2013 UNIVERSITY54 53. References (contd.) [15] Kreyszig E., Adavanced Engineering Mathematics, 8th Edition. [16] Ling F., Proakis J.G., A generalized multichannel least squares lattice algorithm based on sequential processing stages, IEEE Trans. Acoust., Speech Signal Proc., Vol. ASSP-32, No. 2, pp 381-390, April 1984. [17]Zarzycki J., Nonlinear Prediction Ladder Filters for Higher Order StochasticSequences, Springer-Verlag, Berlin, 1985. [18]Mumolo E., Carini A., A stability condition for adaptive recursive second order polynomial filters, Signal Processing 54(1996) 85 90, Elsevier. [19]Moore J.B., Global convergence of output error recursions in colored noise, IEEE Trans, Automatic Control, Vol. AC-27, No. 6, pp. 1189-1199, December 1982. [20]Lee J., Mathews J.V., A Stability Condition for Certain Bilinear Systems, IEEE Trans. Signal Pros.,Vol. - 42, No. 7, pp. 1871 1873, July 1994. [21]http://www.google.com [22]http://en.wikipedia.org Sandip Joardar, MEE, 1st Sem, 1st Yr, JADAVPURSunday, March 31, 2013 UNIVERSITY55