Neural network-based prediction and optimization of estradiol release from ethylene–vinyl acetate...

Computers and Chemical Engineering 28 (2004) 2407–2419

Neural network-based prediction and optimization of estradiolrelease from ethylene–vinyl acetate membranes

Laurent Simon∗, Maria Fernandesa Otto H. York Department of Chemical Engineering, New Jersey Institute of Technology, Newark, NJ 07102, USA

Received 11 December 2003; received in revised form 24 May 2004; accepted 9 June 2004

Available online 7 July 2004

Abstract

Drug-delivery systems, with predictable delivery rates, were designed using an artificial neural network-based optimization algorithm. Atwo-chamber diffusion cell was used to study the permeation of estradiol through ethylene–vinyl acetate copolymer membrane. The explanatoryvariables were the vinyl acetate (VA) content of the membrane, poly(ethylene glycol) (PEG)—solvent-composition, and membrane thickness.After deriving a neural network model to predict estradiol delivery rates as a function of these input variables, a constrained optimizationprocedure was applied to estimate the membrane/vehicle properties necessary to achieve a prescribed dosage. The results compared adequatelywell with experimental data with 71% of the data agreeing within one standard deviation. Input sensitivity analysis showed that at specificVA levels, drug delivery was more sensitive to changes in PEG compositions. The non-uniqueness of the inversion method and the accuracyof the procedure were investigated using neural network-based two-dimensional contour plots. The methodology proposed could be used todesign customized polymer-based drug-delivery systems that meet specific end-user requirements.© 2004 Elsevier Ltd. All rights reserved.

Keywords:Neural networks; Constrained optimizations; Drug delivery rate

1. Introduction

A major challenge in the pharmaceutical industry isthat current drug-delivery methods, such as tablets, injec-tions and sprays, are inefficient in administering long-termconsistent drug release to a target site. Certain drugs loseeffectiveness over a period of time, requiring multiple in-jections to maintain the necessary therapeutic level. Theplasma drug concentration may fall below the minimumeffective concentration (MEC) or rise above the minimumtoxic concentration (MTC) in a dosing cycle. This workinvestigates a new mathematical formulation that ensuresaccurate, cost-effective polymer-based delivery of drugsto target sites. Although optimal experimental designs(e.g. sequential simplex delivery method or response sur-face experiments) are helpful tools in the development ofdrug-delivery devices, they are costly and provide onlylimited predictive capabilities. Therefore, it is imperative to

∗ Corresponding author. Tel.:+1 973 596 5263; fax:+1 973 596 8436.E-mail address:laurent.simon@njit.edu (L. Simon).

investigate the use and benefits of new mathematical frame-works to enhance understanding of controlled drug release.

One of the outcomes of the Genome project is the influxof new drugs being developed and evaluated to treat humandiseases. Polymer-based delivery devices are currently beingstudied for delivery of these new drugs. Such systems can beengineered to release the medicinal agent to the target sitein a controlled fashion. Recent advances in polymer sciencealso contribute to the continued use of polymer-based drugdelivery systems. These developments include nanoporousmaterials with photosensitive pores (Flanagan, 2003) andtemperature-sensitive polymer membranes (Grassi, Yuk, &Cho, 1999). Innovative material, currently under develop-ment, has the potential to pave the way for sophisticateddrug/polymer systems that have the ability to respond tochanges in the environment (Bettini, Colombo, & Peppas,1995; Siegel & Pitt, 1995). Biodegradable polymers, basedon polyactic acid and polyanhydrides, are employed forthe long time-release of drugs from implanted patches andother drug-delivery systems (Kocova El Arini & Leuen-berger, 1995). Hydrophilic water-soluble polymers are pre-ferred for oral administration of the drug.Koizumi, Ritthidej,

0098-1354/$ – see front matter © 2004 Elsevier Ltd. All rights reserved.doi:10.1016/j.compchemeng.2004.06.002

2408 L. Simon, M. Fernandes / Computers and Chemical Engineering 28 (2004) 2407–2419

and Phaechamud (2001)proposed a mechanistic model fordrug release from chitosan coated tablets. In their applica-tions, spherical core tablets, containing a model drug pro-pranolol hydrochloride, were investigated. The rate of drugrelease was controlled by diffusion through the inner spaceand the polymer coating of the pills.

In view of this rapid development and growing complex-ity of drug delivery systems, a new methodology is proposedto integrate experimental data, prior system knowledge, andartificial neural networks (ANNs), to assist in the develop-ment of novel drug-release device technologies.

Controlled drug-release-based technology has been stud-ied extensively (Langer, 1998) and exploited by the phar-maceutical industry with major breakthroughs. However,even when the release of drugs from a polymer matrix oc-curs mainly by diffusion, it is still difficult to achieve thedesired release profile. One reason is that the release rateis a function of several factors, such as the characteristicsof the excipients added to the polymeric drug device tocontrol the drug release pattern (Shah & Zatz, 1992) andthe conditions of the external environment (e.g. pH andtemperature). In some cases, a mathematical model can bederived using Fick’s second law; however, the appropriatedrug formulation is not explicit in such models. Current es-timation procedures are iterative and time-consuming sincethe effect of pertinent contributing factors on the drug de-livery rate is often evaluated in a qualitative manner, whichprecludes the use of any mathematical theory of optimiza-tion. As a result, numerous experiments are necessary toapproximate the drug formulation required to achieve thedesired release profile. Further refinement of these pro-cedures is often based on trial and error approaches orexperimental optimization schemes (e.g. sequential simplexmethods). These techniques are cost-prohibitive and oftenresult in sub-optimum drug-delivery devices.Berkland,King, Kim, and Pack (2002)agree that the ability to tai-lor drug release profiles is difficult. In their investigation,using poly(d,l-lactide-co-glycolide) (PLG) microspheres,they identified (by testing different systems) appropriatemixtures of uniform microspheres that provided constantrelease of rhodamine and piroxicam for 8 and 14 days,respectively (Berkland et al., 2002).

This work is based on the use of ANN models to predictthe drug-release rates as a function of key membrane andsolvent properties. The reason to use an artificial neuralnetwork-based model is that the transport of pharmaceuticsthrough membranes is complex and influenced by a hostof factors such as additives, polymer functionality, poros-ity, film casting conditions, temperature, and even the typeof membrane. Since the relationship between these factorsand drug delivery rate is not well defined, it is custom-ary to write the diffusion coefficient as a nonlinear func-tion of the systems property (e.g. polymer compositions,excipients, etc.). For instance,Siepmann, Lecomte, andBodmeier (1999)investigated the effect of the compositionof diffusion-controlled release devices (type and amount

of plasticizer, type of polymer) on the drug diffusivity, andultimately on the release kinetics. Their goal was to achievedesired release profiles by manipulating the drug diffusioncoefficient. Since only a few researchers have studied theeffect of plasticizer on the drug diffusion in a quantitativemanner,Siepmann, Lecomte, and Bodmeier (1999)usedexponential functions to describe the relationships betweenthe diffusion coefficient of theophylline and the plasticizerlevel of various polymer–plasticizer systems. Since thesynergistic effect of the properties studied in the currentwork on drug-release rate has not been described fromfirst-principle modeling, an ANN function was used.

The ANN function was then inverted to estimate the nec-essary solvent/membrane properties required to achieve adesired delivery rate. The use of a new Mathematica® (Wol-fram Research, Inc.) add-on package is also described. Sincethe package provides symbolic representation of ANN mod-els, contour plots can easily be drawn based on the gener-ated model instead of polynomial approximations, routinelyused for this type of analysis. A new method for ANN-basedinput sensitivity analysis is also provided.

This paper is divided into three sections: (1) A Materialsand methods section underlying the experimental proce-dure and concepts involved; (2) A Results and discussionssection addressing: (a) Release kinetics due to the effectsof vinyl acetate (VA) contents of ethylene–vinyl acetate(EVA) copolymer membrane, membrane thickness, andpoly(ethylene glycol) (PEG) (solvent) composition; (b) De-velopment of an ANN model to predict release rates as afunction of the variables mentioned above; (c) Design ofdrug-delivery devices to meet specific end-user require-ments based on an inversion of the ANN model; and (d)Local input sensitivity analysis; (3) Conclusion.

2. Materials and methods

2.1. Materials

Estradiol and PEG 400 were purchased from Sigma–Aldrich (St. Louis, MO). The 3M Company generouslydonated the ethylene–vinyl acetate membranes.

2.2. Determination of estradiol saturation concentration

A drug suspension in different PEG 400 solution concen-trations was placed in the donor cell to guarantee that thedrug concentration, following a zero-order kinetics, wouldremain constant in the solvent. The drug equilibrium sol-ubilities, in different concentrations of PEG 400 solutionsand at a temperature of 37◦C, were determined by addingan excess amount of estradiol to 3 ml of PEG 400-salinesolutions: 20, 30 and 40%. Samples were produced in du-plicates and placed in a water bath at 37◦C for 3 days withoccasional mixing. The saturated solutions were centrifugedand the supernatant liquid collected. The concentration was

L. Simon, M. Fernandes / Computers and Chemical Engineering 28 (2004) 2407–2419 2409

determined by measuring the absorbance at 290 nm using aUV spectrophotometer.

2.3. Drug release through EVA membranes

A two-chamber diffusion cell was used to study thepermeation of estradiol through EVA membranes. Eachhalf-cell contained 3 ml of solution with an effective diffu-sional area of 0.785 cm2. The EVA membrane was clampedbetween the donor and receiver cells and the system waskept at a temperature of 37◦C by placing the apparatus in atemperature-controlled water bath. A pump was used to cir-culate water through the device. Saturated drug-in-solventsolutions, to be used in the experiments, were prepared,in advance, to allow some time for equilibration. Threemilliliters of the equilibrated drug/PEG solution were placedin the donor cell compartment. The receptor cell compart-ment was filled with 3 ml of the same concentration ofPEG solution. At specific time intervals, the receiver cellcontent was removed for analysis and replaced with thesame volume of fresh PEG solution. The concentrations ofthe collected samples were determined by measuring thesample’s absorbance in a spectrophotometer at 290 nm.

2.4. Computational methods

2.4.1. Drug permeation through the EVA membraneThe following equation was used to calculate the cumu-

lative amount of drug released per unit area (Shin & Lee,2002):

Q = P(Cd − Cr)t, (1)

whereP is the permeability coefficient,t the time,Cd andCrare the drug concentrations in the donor and receiver cells,respectively. Since the drug in the donor compartment isabove the drug saturation concentration (Ceq) and the drugconcentration in the receptor solution is maintained undersink condition (i.e.Cr � Ceq), Eq. (1)becomes:

Q = PCeqt (2)

The drug permeation rate per unit area is then:

Q

t= PCeq (3)

Its value was estimated by plottingCd cumV/Aas a functionof time. The diffusion area is denoted byA; V is the receivercell volume (3 ml) andCd cum the cumulative receptor cellconcentration.

2.4.2. Artificial neural network prediction of thepermeation rate

A feed-forward ANN model was developed to predict thedrug permeation rate as a function of vinyl acetate contents(VA) of the ethylene–vinyl acetate (EVA) copolymer mem-brane, membrane thickness (Th), and PEG 400 composi-tions (PEG). When dealing with noisy and incomplete data,

ANN models offer a better filtering capacity and generallyperform better than empirical models. Artificial neural net-work models have emerged as an accepted tool for modelingnonlinear processes. Different architectures of these modelshave been implemented in the literature (Eikens, Karim, &Simon, 2001; Gao & Loney, 2001; Simon & Karim, 2001;Simon, Karim, & Schreiweis, 1998). Artificial neural net-works consist of a set of weights,w, a set of basis functions,f(·), and a set of parameters,p (biases, centers, etc.). Thegeneral equation can be written as:

y = fNN(x, w, p) (4)

wherey denotes the NN estimation (i.e., the permeation rateQ/t), x is the input vector (i.e., VA, Th, and PEG) andfNN isthe network function. Developing a supervised ANN modelinvolves three phases: A training phase in which parametersw and p are adjusted in order to minimize a performancecriterion, such as the squared error between the experimen-tal or measured output and the output predicted by the net-work. This phase requires a training set consisting of knowninput and output patterns. Examples, representative of theproblem, are propagated through a series of layers. The net-work learns the underlying relationship between input andoutput variables. A recall phase, in which the network isexposed to the input patterns seen in the training phase, isthen used. Adjustments, such as the number of the layers,can be made to enhance the robustness and reliability of thesystem. The weights and biases are computed. This step isfollowed by a testing phase, in which new input patterns(not part of the training set) are fed through the networkwith fixed weights and biases. The performance of the sys-tem is obtained by comparing experimental and predictednetwork outputs.Fig. 1 shows the structure of a neural net-work with one hidden layer and two nodes (h1 andh2). Thechoice of the number of nodes is explained in the Resultsand Discussion section. A “bias” unit is used to help thetraining phase. The bias term is equivalent to ay-interceptin a linear regression model. The components ofw arew2h1,w2h2, w3h1, w3h2, w4h1, andw4h2. The components ofp arep1h1, p1h2, andp2o. Optimum values of the weights (wopt)

1

VA

PEG

Th

σΣ

σΣΣ

1

[Q/t ]hat

p1h1

p1h2w2h1

w2h2

w3h1

w3h2

w4h1 w4h2

h1

h2

wh1o

wh2o

p2o

Fig. 1. Neural network architecture using one hidden layer and two nodes.Weight wihj connects the ith input to the jth hidden neuron; weight whio

connects output from the ith hidden layer to the output; p1hj connectsthe unit bias to the jth hidden node; p2o connects the unit bias to theoutput node. The weighted inputs are summed up and passed through anonlinear activation function sigma.

2410 L. Simon, M. Fernandes / Computers and Chemical Engineering 28 (2004) 2407–2419

and biases (popt) are obtained and used when solving theinverse problem (Section 2.4.4).

2.4.3. Input sensitivity analysisNeural network-based sensitivity analysis is usually im-

plemented to assess the causal importance of individual in-put variables. Different applied methods have been reportedin the literature (Caldwell, 1995; Caldwell, 1996; Howes& Crook, 1999; Karim, Hodge, & Simon, 2003; Simon &Karim, 2002). The technique used by Simon and Karim(2002) can be summarized as follows: (1) find an optimumANN model to represent the input–output mapping; (2) per-turb the variable of interest of the input vector while keep-ing the remaining components at their mean values; and (3)compute the output of the network. This method was basedon previous work by Principe, Euliano, and Lefebvre (2000),who used the following expression to calculate the sensitiv-ity of input k on the network output:

Sk =∑p

p=1

∑Oi=1(yip − yip)2

σ2k

(5)

where yip is the ith output of the optimum network for thepth pattern before input perturbation (the other input vari-ables are kept at their main values), yip the ith output of theoptimum network for the pth pattern after input pertubation,O stands for the number of network output (O = 1), P is thenumber of patterns, and σ2

k represents the variance of theinput perturbation. Although this method yields successfulresults, Eq. (5) can be viewed as a discretized form of theequation for input sensitivity (i.e. squared sensitivity) and isuseful for cases in which the input–output mapping functionis too complex. One of the advantages of using Eq. (5) isthat irrelevant input variables (variables that do not affect theoutcome of a process) can be eliminated (Simon & Karim,2002). As a result, data collection is reduced and a bet-ter insight into the underlying relationships between inputand output variables can be gained. However, in processes,such as drug delivery, researchers engaged in data analysisand interpretation may be interested in knowing whethera small change in an input variable results in a relativelylarge change in the outcome (e.g. drug-release rate) for val-ues of the remaining input variables other than the averagequantities.

The approach used in the present work makes use of thedefinition of input sensitivity and can help researchers totake into account the other input values. The effect of aninput χ (component of a input vector χ) on the output ζ suchthat ζ = f(χ) is the partial derivative of the function f withrespect to the independent variable of interest:

S = ∂f

∂χ

∣∣∣∣χ0

(6)

where χ0 is the remaining components. The methodologyproposed in this work uses Eq. (6) to perform the neuralnetwork-based input sensitivity analysis. As a result, an

ANN model with a minimum number of nodes and hid-den layers, still capable of explaining changes in estradiolpermeation based on vinyl acetate (VA) content of the mem-brane, poly(ethylene glycol) (PEG)—solvent-composition,and membrane thickness will be sought. Contrary to pre-vious work, an explicit expression will be given for thenetwork model to show such dependence.

2.4.4. Solution of the inverse problemThe inverse problem consists of finding the VA content of

the membrane, the membrane thickness, and the percentageof PEG (i.e. the input vector xi) necessary to achieve a pre-scribed drug permeation rate (yset i). A constrained optimiza-tion scheme, implemented in Mathematica® (Wolfram Re-search, Inc.), was used to solve the problem. “NMinimize” isa function that provides four different methods for global op-timization. The “Random search” option was selected. Theoptimization problem consists of finding the optimal inputvector parameters xi min such that the objective function Fdefined as:

F(xi) = [yset i − fNN(xi, wopt, popt)]2 (7)

subject to

xi ∈ Ψ (8)

is minimized:

xi min = Min F(xi) (9)

where xi min is a member of an admissible solution set Ψ .The search space Ψ is made up of the data used in the train-ing and testing sets. The constraint defined by Eq. (8) forcesthe algorithm to explore only solutions that can be testedimmediately under the experimental conditions studied. Theweights wopt and biases popt correspond to the optimum net-work parameters and remain fixed when solving the inverseproblem.

Because of the intrinsic non-uniqueness of inversionmethods, two-dimensional contour plots were employed toanalyze the characteristics of the optimized ANN model andto estimate the accuracy of the procedure outlined above.These contours are often based on polynomial equationsfrom a multivariate regression analysis. However, since aparsimonious network structure is derived, it is possibleto use the network model directly to draw the contours.The two approaches to design customized polymer-baseddrug-delivery systems that meet specific end-user require-ments were compared.

3. Results and discussion

3.1. Release kinetics

Twenty-four samples were generated by varying PEG 400volume fractions (20, 30, 40, and 50%), VA contents of the

L. Simon, M. Fernandes / Computers and Chemical Engineering 28 (2004) 2407–2419 2411

0.00

2.00

4.00

6.00

8.00

10.00

12.00

14.00

0 5 10 15 20

Vinyl acetate content (%)

Dru

g r

elea

se r

ate

(ug

/cm

^2.h

)

PEG = 20%PEG = 30%PEG = 40%PEG = 50%PEG = 50% (NN)PEG = 40% (NN)PEG = 30% (NN)PEG = 20% (NN)

Membrane thickness:0.00508 cm

Fig. 2. Experimental (—) and predicted (- - -) drug-release rate per area at a membrane thickness of 0.00508 cm. PEG compositions are 20, 30, 40, 50%;VA contents are 9 and 19%.

membrane (9 and 19%), and membrane thickness (0.00508,0.00762, 0.01016 cm). The release rates, corresponding tothe various combinations, are shown in Figs. 2–6. Theseresults suggest that the permeation rate rose with an in-crease in VA content, and PEG composition and a decreasein membrane thickness. These findings agree with researchled by Shin and Byun (1996). Through their investiga-tions of the permeability of ethinylestradiol through EVAcopolymer membranes, they reported that an increase in VAcomonomer content increased the drug-release rate and per-meability coefficient (Shin & Byun, 1996). This result maybe due to an increase in the diffusivity of the drug. Theirconclusion was reached after considering previous workswhich had proved that the introduction of VA comonomers

0.00

1.00

2.00

3.00

4.00

5.00

6.00

7.00

0 5 10 15 20

Vinyl acetate content (%)

Dru

g r

elea

se r

ate

(ug

/cm

^2.h

)

PEG = 20%PEG = 30%PEG = 40%PEG = 50%PEG = 50% (NN)PEG = 40% (NN)PEG = 30% (NN)PEG = 20% (NN)

Membrane thickness: 0.00762 cm

Fig. 3. Experimental (—) and predicted (- - -) drug-release rate per area at a membrane thickness of 0.00762 cm. PEG compositions are 20, 30, 40, 50%;VA contents are 9 and 19%.

to a highly crystalline polyethylene decreased the crys-tallinity of the system, which, in turn, increased the diffu-sivity of polymers (see Shin & Byun, 1996 for a completereference).

It should be noted that if the diffusion coefficients areof the same order of magnitude for the EVA copolymermembranes studied, the increase in the permeation rate maybe the result of an increase in the partition of the drug intothe EVA membrane (Shin & Byun, 1996).

In this work, estradiol equilibrium solubility was foundto increase with the addition of PEG 400: 51, 180, 600,1000 �g/cm3 for 20, 30, 40, and 50% PEG, respectively,which resulted in an increase of the drug-release rate. Thistrend can be predicted by the analysis of Eq. (3).

2412 L. Simon, M. Fernandes / Computers and Chemical Engineering 28 (2004) 2407–2419

0.00

0.50

1.00

1.50

2.00

2.50

3.00

3.50

4.00

4.50

0 5 10 15 20

Vinyl acetate content (%)

Dru

g r

elea

se r

ate

(ug

/cm

^2.h

)

PEG = 20%PEG = 30%PEG = 40%PEG = 50%PEG = 50% (NN)PEG = 40% (NN)PEG = 30% (NN)PEG = 20% (NN)

Membrane thickness: 0.01016 cm

Fig. 4. Experimental (—) and predicted (- - -) drug-release rate per area at a membrane thickness of 0.01016 cm. PEG compositions are 20, 30, 40, 50%;VA contents are 9 and 19%.

Shin and Byun noted that the permeation rate was in-versely proportional to the membrane thickness (Shin &Byun, 1996). From Figs. 5 and 6, it is evident that such atrend exists (although not quite linear for all PEG and VAcompositions) and can be used to control drug permeation.It is known that an increase in membrane thickness impliesan increase in the distance the drug has to travel through themembrane, thereby reducing the diffusion of the drug.

0.00

1.00

2.00

3.00

4.00

5.00

6.00

7.00

0 0.002 0.004 0.006 0.008 0.01 0.012

Menbrane thickness (cm)

Dru

g r

elea

se r

ate

(ug

/cm

^2.h

)

PEG = 20%PEG = 30%PEG = 40%PEG = 50%PEG = 50% (NN)PEG = 40% (NN)PEG = 30% (NN)PEG = 20% (NN)

Vinyl acetate: 9%

Fig. 5. Experimental (—) and predicted (- - -) drug-release rate per area at a VA content of 9%. PEG compositions are 20, 30, 40, 50%; membranethicknesses are 0.00508, 0, 0.00762, 0.01016 cm.

3.2. Artificial neural network model

The data generated for the kinetics study were usedfor the training and testing of the ANN. These sets werepre-processed by initially randomizing the rows of datacorresponding to different input and output vectors. Thisstep is important because similar input data, grouped to-gether, may considerably affect the training phase given

L. Simon, M. Fernandes / Computers and Chemical Engineering 28 (2004) 2407–2419 2413

0.00

2.00

4.00

6.00

8.00

10.00

12.00

14.00

0 0.002 0.004 0.006 0.008 0.01 0.012

Membrane thickness (cm)

Dru

g r

elea

se r

ate

(ug

/cm

^2.h

)

PEG = 20%PEG = 30%PEG = 40%PEG = 50%PEG = 50% (NN)PEG = 40% (NN)PEG = 30% (NN)PEG = 20% (NN)

Vinyl acetate: 19%

Fig. 6. Experimental (—) and predicted (- - -) drug-release rate per area at a VA content of 19%. PEG compositions are 20, 30, 40, 50%; membranethicknesses are 0.00508, 0, 0.00762, 0.01016 cm.

that they are unlikely to be fully representative of the rangeof possible values.

The input and output patterns were scaled to lie in theinterval [0.1, 0.9] for efficient processing of the networkbecause of the sigmoid activation function, later used in thetraining phase. Seventy-five percent of the data were usedfor tutorial purposes (training set) and the remaining (testingset) were used to test the performance of the model learnedduring the training phase.

When dealing with a large data set, several algorithms,such as “optimal brain damage” (Cun, Denker, & Solla,1990) and “optimal brain surgeon” (Hassibi & Stork, 1993)can be employed to decide on the optimum number of nodesto use in the hidden layer. However, with a relatively smalldata set, the choices become limited. Since it is consideredgood statistical practice that the number of adjustable pa-rameters be less than the number of data points, two hiddennodes were chosen. A feed-forward network, consisting ofone hidden layer, two neurons and a sigmoid-type activationfunction, was built with three inputs and one output (oftendenoted as a 3:2:1 network). The maximum number of it-erations to train the network was fixed at thirty. The NeuralNetworks—“Train and analyze neural networks to fit yourdata” (Jonas Sjöberg, 2002)—add-on Mathematica® pack-age from Wolfram Research, Inc., was used in this study.This work is the first attempt to take advantage of the capa-bility of this software in engineering controlled drug-releasedevices.

Artificial neural network models are usually consideredimplicit black-box predictors. Subramanian, Yajnik, and

Murthy (2003) investigated the use of these models asalternatives to multiple regression analysis in optimizingformulation parameters. They observed that ANN modelswere more accurate than polynomial equations (commonlyused to generate contour plots) in predicting percentagedrug entrapment (PDE). These network models can alsohandle more input variables and process a larger numberof experimental data (Subramanian et al., 2003). However,in their work, data analysis such as relationship betweenindependent variables and the PDE, and second-order maineffects of both drug/lipid ratio and volume of hydration,was performed using a reduced polynomial equation.

This work, by taking advantage of Mathematica®’s mixednumeric–symbolic environment and the Neural Networkadd-on package, extends the usefulness of ANN modelsbeyond that of black-box predictors. An explicit mathe-matical representation of the input–output mapping is firstderived and later used for developing a new method for im-plementing neural network-based input sensitivity analysis,and solving drug formulation problems using contour plotsand function minimization.

The steps involved in building the ANN models are:

• Data pre-processing. The data (randomized) was initiallyread from a text file using the Mathematica® command“ReadList” . They were then normalized from 0.1 to 0.9.

• Seventy-five percent of the data was used for training, andthe rest for testing.

• A feed-forward network with two neurons in the hiddenlayer was initialized using the Mathematica® command:“ InitializeFeedForwardNet” .

2414 L. Simon, M. Fernandes / Computers and Chemical Engineering 28 (2004) 2407–2419

0.1 0.2 0.3 0.4 0.5Data

0.1

0.2

0.3

0.4

0.5

Model

Fig. 7. Neural network predicted output (q/t) vs. the normalized output(q/t). The clustering of points around line y = x indicates a good fit. Thetraining set was used.

• The network was trained using the Mathematica® com-mand “NeuralFit” . A symbolic function was generated atthe end of the training procedure.

• The network was tested by using the symbolic functionwith the testing set as argument.

• If the performance of the testing set was not satisfactory,the training phase was repeated.

Learning took place after only 14 iterations, at which pointthe minimum squared error no longer diminished—therefore,preventing overtraining. The optimized neural network

Table 1Estimation of the VA content, PEG composition and membrane thickness using a specified permeation rate per unit area

Sample VA (%) TH (cm) PEG (%) Q/t (�g/cm2 h) S.D. (�g/cm2 h) VAopt (%) THopt (cm) PEGopt (%) (Q/t)opt (�g/cm2 h)

1 9 0.00508 30 1.11 0.104 9 0.00762 30 1.062 9 0.00508 40 3.18 0.236 9 0.00508 40 3.183 19 0.00508 20 2.04 0.205 9 0.00762 40 3.194 19 0.00508 40 6.47 0.299 19 0.00508 40 6.475 9 0.00762 40 3.19 0.207 9 0.00508 40 3.186 19 0.01016 20 1.03 0.017 19 0.01016 20 1.037 19 0.00762 30 2.76 0.247 9 0.01016 50 2.018 19 0.00762 20 1.77 0.188 9 0.00508 30 1.119 9 0.01016 50 2.01 0.120 9 0.00762 40 3.19

10 19 0.00762 40 3.81 0.195 19 0.00508 30 4.1811 19 0.00762 50 5.79 0.412 19 0.00762 50 5.7912 9 0.00762 20 0.72 0.095 9 0.00762 20 0.7213 9 0.00762 30 1.06 0.097 19 0.01016 20 1.0314 19 0.01016 40 2.00 0.178 9 0.00762 40 3.1915 9 0.00762 50 4.29 0.543 9 0.00762 50 4.2916 9 0.00508 20 1.00 0.150 9 0.00508 20 1.0017 9 0.00508 50 5.54 0.592 9 0.00508 50 5.5418 9 0.01016 40 1.44 0.158 19 0.01016 30 1.08

The training set was used.

0.2 0.4 0.6 0.8Data

0.2

0.4

0.6

0.8

Model

Fig. 8. Neural network predicted output (q/t) vs. the normalized output(q/t). The clustering of points around line y = x indicates a good fit. Thetesting set was used.

function is:

q

t= 0.10 + 1386.56

1 + e23.04−2.06peg+1.58th−15.34va

+ 0.58

1 + e1.55−4.38peg+1.98th−16.28va(10)

where peg, th, va, and q/t are the normalized PEG volumefaction, membrane thickness, vinyl acetate content, and drugpermeation rate per unit area, respectively. The results ofthe training phase are summarized in Fig. 7. The clusteringof points around the regression line y = x shows that the

L. Simon, M. Fernandes / Computers and Chemical Engineering 28 (2004) 2407–2419 2415

Table 2Estimation of the VA content, PEG composition and membrane thickness using a specified permeation rate per unit area

Sample VA (%) TH (cm) PEG (%) Q/t (�g/cm2 h) S.D. (�g/cm2 h) VAopt (%) THopt (cm) PEGopt (%) (Q/t)opt (�g/cm2 h)

19 19 0.00508 30 4.18 0.155 9 0.00762 50 4.2920 9 0.01016 30 0.99 0.102 9 0.00508 20 1.0021 19 0.01016 50 3.91 0.257 19 0.00508 30 4.1822 9 0.01016 20 0.56 0.026 9 0.01016 20 0.5623 19 0.01016 30 1.08 0.127 19 0.01016 20 1.0324 19 0.00508 50 11.48 0.479 19 0.00508 50 11.48

The testing set was used.

model is accurate in predicting the system output. The net-work performed equally well when unlabeled testing inputdata (not part of the training set) were fed into the network(Fig. 8). These results are also shown in Figs. 2–6, in whichthe network predictions and experimental data are plotted.Eq. (10) will be used in the rest of the work.

3.3. Inversion of the artificial neural network model

Although artificial neural networks have been imple-mented to predict drug-release profiles (Lim, Quek, &Peh, 2003), additional research is needed to investigate the

Fig. 9. PEG sensitivity analysis at different vinyl acetate contents and membrane thicknesses.

feasibility of ANNs in designing effective pharmaceuticalformulations based on user-defined drug-release rate tar-gets. Recent contributions include the work of Takahara,Takayama, and Nagai (1997) and Takayama, Fujikawa,Obata, and Morishita (2003).

Using the methodology outlined in Section 2.4.4, the de-normalized results summarized in Table 1 were obtainedfor the training data. The first column “Sample” numbersthe data points; the second, third, and fourth columns de-note the input set, and the fifth column represents the ex-perimental output (Q/t): permeation rate per diffusion area.The sixth column is the standard deviation of the output.

2416 L. Simon, M. Fernandes / Computers and Chemical Engineering 28 (2004) 2407–2419

Fig. 10. Membrane thickness sensitivity analysis at different vinyl acetate contents and PEG compositions.

Using the scaled target output (yset i = q/t), Eq. (10), andthe optimization procedure outlined by Eqs. (7)-(9), an op-timum scaled input vector was calculated xi min = [vaopt,thopt, pegopt]. Columns seven through nine represent the de-normalized optimum input vector [VAopt, THopt, PEGopt].For comparison, the last column, which corresponds to theactual output of [VAopt, THopt, PEGopt], is added. The boldvalues correspond to cases in which the methodology pre-dicted the experimental values (taking into account standarddeviations). Eleven out of 18 such cases (61%) were ob-served. The underlined values correspond to cases where theoutput estimated by the inverse problem corresponds exactlyto the experimental value. These results can be explained bythe fact that several combinations of the inputs gave a simi-lar output. A more complex network architecture (additionalnodes) would probably predict 100% of the input vector.This would cause the network to memorize the input andoutput patterns in the training sets, but testing would proveunsuccessful. This point becomes clearer when we examineTable 2, corresponding to the testing set. One-hundred per-cent of the input data were predicted correctly (taking intoaccount standard deviations).

This method gives satisfactory results, although the in-verse problem performed better on the testing than on thetraining sets. This example illustrates a very important point.Although overparametrization and overtraining would even-tually lead to a more accurate prediction of the trainingdata, the network may lose the ability to predict unseen pat-terns. Tutorial (training) data are important insofar as theyhelp to achieve a model that best describes the underly-ing process. Such an empirical model can then be used,in various applications, to perform system analysis, designand performance evaluations. With a large number of hid-den units, it is possible to model spurious relationships,which do not reflect true dependencies among the variables.This will lead to meaningless results and erroneous processdecisions.

The methodology proposed could be used to designaccurate polymer-based drug-delivery systems to meet spe-cific end-user requirements. The predictive capability of themethodology allows for a drug-delivery device performanceassessment without having recourse to a large number ofexperiments. Such a black-box modeling approach, al-though it does not substitute for a first-principle physical

L. Simon, M. Fernandes / Computers and Chemical Engineering 28 (2004) 2407–2419 2417

knowledge of the system, can provide a starting point forfuture investigation and discussions.

Contour plots were drawn to solve the inverse problemand evaluate the results of the procedure outlined above. Thecommand “contour (Z)” from Matlab (The MathWorks, Nat-ick, MA) was used to draw a contour plot of matrix Z withcontour lines and their values chosen automatically. Afterusing the functions “ linspace” and “meshgrid” to create aset of points at values that spanned the data range, Eq. (10)(denormalized form) was used at 9 and 19% vinyl acetatecontent. The result is shown in Fig. 9. The contour plotsagree, very well, with the best solutions of the minimiza-tion procedure. Consider, for instance, the first two pointsof Table 1. The neural network prediction for the input vec-tors:[9, 0.00762, 30], and [9, 0.00508, 40], representing thepercentage of vinyl acetate, membrane thickness, and PEGcompositions, are 1.06 and 3.18 �g/cm2, respectively. Fromthe “%VA = 9” plot (Fig. 9), the non-uniqueness of thesetwo problems is evident. A release rate of 1.1 �g/cm2 canbe obtained by using several combinations of PEG con-centrations and membrane thicknesses, including the onesgiven by the minimization method. The same is true for

Fig. 11. Contour plots of the influence of PEG compositions, and membrane thickness on the drug-release rate per area at different VA contents.

a release rate of 3.2 mg/cm2. Both the contour plots andthe minimization procedure can assist in the design of cus-tomized polymer-based drug-delivery systems that meet spe-cific end-user requirements. However, for pharmaceuticalapplications, the contour plots provide the clinician with sev-eral viable alternatives that can be considered. Therefore,least-squares optimization routines are not necessary, unlessone is dealing with a high dimensional input feature space.

3.4. Local input sensitivity analysis

Previously published methods calculate average sensi-tivity analysis based on the ANN output. The proposedmethodology used the ANN model directly and allowedcalculation of input sensitivities when the other inputs werekept at levels other than their mean values. To compare, forinstance, how sensitive the drug-release rate was to changesin PEG and TH, derivatives of Eq. (10) with respect tothe variables: PEG, and TH, respectively, were calculated.The sensitivity function, for a particular input (Sin), wasevaluated for four different combinations of the other inputvariables. Four functions for PEG sensitivity (SPEG) were

2418 L. Simon, M. Fernandes / Computers and Chemical Engineering 28 (2004) 2407–2419

obtained: SPEG at low vinyl acetate composition and lowmembrane thickness (LOW VA–LOW TH), SPEG at highvinyl acetate composition and high membrane thickness(HIGH VA–HIGH TH), SPEG at low vinyl acetate com-position and high membrane thickness (LOW VA–HIGHTH), and SPEG at high vinyl acetate composition and lowmembrane thickness (HIGH VA–LOW TH). The resultsare shown in Fig. 10. As expected, SPEG is higher forHIGH VA–LOW TH. Also, an increase in PEG composi-tion is accompanied by an increase in the drug-release rate(positive correlation). When compared to STH at differentcombinations of VA and PEG (Fig. 11), the drug-releaserate is overall more sensitive to a change in PEG composi-tions for the levels investigated. Also, a negative correlationexists between drug-release rate and membrane thickness.From Fig. 11, the system responds better to a change inmembrane thickness for at HIGH VA–HIGH PEG. Thisfinding is in good agreement with Figs. 5 and 6 which showa more drastic change in the drug-release rate at 19% VAand 50% PEG.

4. Conclusion

Franz diffusion cells were used to study the perme-ation of estradiol through ethylene–vinyl acetate copolymermembrane. Twenty-four samples were obtained by vary-ing PEG 400 volume fractions (20, 30, 40, and 50%), VAcontents of the membrane (9 and 19%), and membranethickness (TH) (0.00508, 0, 0.00762, 0.01016 cm). The re-sults show that the permeation rate rose with an increasein VA content and PEG composition and a decrease inmembrane thickness. An artificial neural network, withtwo hidden nodes, was developed and validated to predictestradiol release rates (per unit area) as a function of theexplanatory variables. A high correlation between experi-mental and predicted drug delivery rates was achieved forthe training (75% of the data) and testing sets (25% of thedata). A constrained optimization procedure, implementedin Mathematica® (Wolfram Research, Inc.), was applied tocalculate the optimum VA content, PEG volume fraction,and membrane thickness for a given delivery rate. Becauseof the non-unique character of this inverse problem, it wasnecessary to define an admissible solution space Ψ , madeup of data used in the training and testing phases. Sixty-onepercent of the training data agreed with experimental resultswhile 100% of the testing data were correctly predicted.Input sensitivity analysis showed that at specific VA lev-els, drug delivery was more sensitive to changes in PEGcompositions than to changes in membrane thickness. Thenon-uniqueness of the inversion method and the accuracyof the procedure were studied using neural network-basedtwo-dimensional contour plots. Using the methodology pro-posed, accurate drug-delivery devices can be tailored to meetspecific drug-release requirements with the least amount ofexperiments.

References

Berkland, C., King, M., Kim, K., & Pack, D. W. (2002). Precise controlof PLG microsphere size provides enhanced control of drug-releaserate. Journal of Controlled Release, 82(1), 137–147.

Bettini, R., Colombo, P., & Peppas, N. A. (1995). Solubility effects on drugtransport through pH-sensitive, swelling-controlled release systems:Transport of theophylline and metoclopramide monohydrochloride.Journal of Controlled Release, 37(1–2), 105–111.

Caldwell, R. B. (1995). Three methods of neural network sensitivityanalysis for input variable reduction: A case study in forecasting theS&P 500 index (Part 1). NeuroVe$t Journal, 3(6), 22–25.

Caldwell, R. B. (1996). Three methods of neural network sensitivityanalysis for input variable reduction: A case study in forecasting theS&P 500 index (part 2). NeuroVe$t Journal, 4(1), 16–22.

Cun, Y. L., Denker, J. S., Solla, S. A. (1990). Optimal brain damage.In D. S. Touretzky (Ed.), Advances in neural information processingsystems(Part 2). San Mateo, CA: Morgan Kaufmann Publishers, Inc.

Eikens, B., Karim, M. N., Simon, L. (2001). Combining neural networkswith first principle models for bioprocess modeling. In I. M. Mu-jtaba, M. A. Hussain (Eds.), Applications of neural networks andother learning technologies in process engineering. London: ImperialCollege Press.

Flanagan, N. (2003). Fast-dissolving technologies, pulsing tablets, andtransdermal means of drug transport. Genetic Engineering News, 23(4),30–31.

Gao, L., & Loney, N. W. (2001). Evolutionary polymorphic neural networkin chemical process modeling: Computers. Computers and ChemicalEngineering, 25(11–12), 1403–1410.

Grassi, M., Yuk, S. H., & Cho, S. H. (1999). Modeling of solute transportacross a temperature-sensitive polymer membrane. Journal of Mem-brane Science, 152(2), 241–249.

Hassibi, B., Stork, D. G. (1993). Second order derivatives for networkpruning: Optimal brain surgeon. In D. S. Lippman, J. E. Moody, D. S.Touretzky (Eds.), Advances in neural information processing systems(Part 5). San Mateo, CA: Morgan Kaufmann Publishers, Inc.

Howes, P., & Crook, N. T. (1999). Using input parameter influences tosupport the decisions of feed-forward neural networks. Neurocomput-ing, 24(1–3), 191–206.

Karim, M. N., Hodge, D., & Simon, L. (2003). Data-based modelingand analysis of bioprocesses: Some real experiences. BiotechnologyProgress, 19(5), 1591–1605.

Kocova El Arini, S., & Leuenberger, H. (1995). Modeling of drug releasefrom polymer matrices: Effect of drug loading. International Journalof Pharmaceutics, 121(2), 141–148.

Koizumi, T., Ritthidej, G. C., & Phaechamud, T. (2001). Mechanisticmodeling of drug release from chitosan coated tablets. Journal ofControlled Release, 70(3), 277–284.

Langer, R. (1998). Drug delivery and targeting. Nature, 392(6679 Suppl.),5–10.

Lim, C. P., Quek, S. S., & Peh, K. K. (2003). Prediction of drug releaseprofiles using an intelligent learning system: an experimental study intransdermal iontophoresis. Journal of Pharmaceutical and BiomedicalAnalysis, 31(1), 159–168.

Principe, J. C., Euliano, N. R., Lefebvre, W. C. (2000). Neural andadaptive systems: fundamentals through simulations. New York: JohnWiley and Sons.

Shah, P. S., & Zatz, J. L. (1992). Plasticization of cellulose esters used inthe coating of sustained release solid dosage forms. Drug Developmentand Industrial Pharmacy, 18(18), 1759–1772.

Shin, S. C., & Byun, S. Y. (1996). Controlled release of ethinylestra-diol from ethylene–vinyl acetate membrane. International Journal ofPharmaceutics, 137(1), 95–102.

Shin, S.-C., & Lee, H.-J. (2002). Controlled release of triprolidine us-ing ethylene–vinyl acetate membrane and matrix systems. EuropeanJournal of Pharmaceutics and Biopharmaceutics, 54(2), 201–206.

L. Simon, M. Fernandes / Computers and Chemical Engineering 28 (2004) 2407–2419 2419

Siegel, R. A., & Pitt, C. G. (1995). A strategy for oscillatory drug release:General scheme and simplified theory. Journal of Controlled Release,33(1), 173–188.

Siepmann, J., Lecomte, F., & Bodmeier, R. (1999). Diffusion-controlleddrug delivery systems: calculation of the required composition toachieve desired release profiles. Journal of Controlled Release, 60(2),379–389.

Simon, L., & Karim, M. N. (2001). Probabilistic neural networks usingBayasian decision strategies and a modified Gompertz model forgrowth phase classification in the batch culture of Bacillus subtilis.Biochemical Engineering Journal, 7(1), 41–48.

Simon, L., Karim, M. N., & Schreiweis, A. (1998). Prediction and clas-sification of different phases in a fermentation using neural networks.Biotechnology Techniques, 12(4), 301–304.

Simon, L., & Karim, M. N. (2002). Control of starvation-induced apoptosisin CHO cell cultures. Biotechnology and Bioengineering, 78(6), 645–657.

Subramanian, N., Yajnik, A., & Murthy, R. S. R. (2003). Artificial NeuralNetwork as an alternative to multiple regression analysis in optimizingformulation parameters of cytarabine liposomes. AAPS PharmSciTech,5(1), 1–9 (Article 4).

Takahara, J., Takayama, K., & Nagai, T. (1997). Multi-objective simulta-neous optimization technique based on an artificial neural network insustained release formulations. Journal of Controlled Release, 49(1),11–20.

Takayama, K., Fujikawa, M., Obata, Y., & Morishita, M. (2003). Neuralnetwork-based optimization of drug formulations. Advanced DrugDelivery Reviews, 55(9), 1217–1231.