Sethuraman Sankaran and Nicholas Zabaras

Materials Process Design and Control LaboratoryMaterials Process Design and Control Laboratory

CCOORRNNEELLLL U N I V E R S I T Y


Sethuraman Sankaran and Nicholas Zabaras

Materials Process Design and Control LaboratorySibley School of Mechanical and Aerospace Engineering

188 Frank H. T. Rhodes HallCornell University

Ithaca, NY 14853-3801

Email: [email protected], [email protected]: http://mpdc.mae.cornell.edu/

An Information-theoretic Tool forProperty Prediction Of Random

Microstructures




RESEARCH SPONSORS

U.S. AIR FORCE PARTNERS

Materials Process Design Branch, AFRL

Computational Mathematics Program, AFOSR

CORNELL THEORY CENTER

ARMY RESEARCH OFFICE

Mechanical Behavior of Materials Program

NATIONAL SCIENCE FOUNDATION (NSF)

Design and Integration Engineering Program




An overviewAn overview

Mathematical representation of random microstructures

Extraction of higher order features from limited microstructural

information : the MAXENT approach

MAXENT optimization schemes

Evaluation of homogenized elastic properties from microstructures

Effect of varying information content on property statistics

Numerical examples

Summary and future work




Idea Behind Information Theoretic ApproachIdea Behind Information Theoretic Approach

Statistical Mechanics

InformationTheory

Rigorously quantifying and modeling

uncertainty, linking scales using criterion

derived from information theory, and

use information theoretic tools to predict parameters in the face

of incomplete Information etc

Linkage?

Information Theory

Basic Questions:1. Microstructures are realizations of a random field. Is there a principle by which the underlying pdf itself can be obtained.2. If so, how can the known information about microstructure be incorporated in the solution.3. How do we obtain actual statistics of properties of the microstructure characterized at macro scale.




Representation of random microstructures

Indicator functions used to represent microstructure at different regions in the physical domain Indicator functions take values over a binary alphabet Statistical features of microstructure are mathematically tractable in terms of expected values over indicator functions

if

if

Two-phase material

if

if

n-phase material

Define IIii as the set comprising

(Ii(x1), Ii(x2), … Ii(xn)). IIi i

represents a random field of indicator functions over the domain. Microstructures are hierarchically characterized over a set of random variables of this field




Defining correlation vectors using indicator functions

Two-point probability functions

Lineal Path Functions

n-point probability functions




Microstructure Reconstruction Schemes




Reconstruction of microstructures

• Correlation features of desired microstructures is provided.• Aim to reconstruct microstructures that satisfy these ensemble statistical properties.• Ill-posed problem with many distributions that satisfy given ensemble properties.

Pb-Sn microstructuresHigh strength steel

microstructures obtainedby thermal processing

Media with short range interactions




Current schemes for microstructure reconstructionCurrent schemes for microstructure reconstruction

D. Cule and S. Torquato ’99 Reconstruction of porous media using Stochastic Optimization

C. Manwart, S. Torquato and R. Hilfer ’00, Reconstruction of sandstone structures using stochastic optimization

N. Zabaras et.al. ’05 Reconstruction of microstructures using SVM’s

T.C.Baroni et al. ’02, Reconstruction of microstructures using contrast imaging techniques

A.P. Roberts ’97, Reconstruction of porous media using image mapping techniques from 2d planar images.

Input: Given statistical correlation or lineal path functions

Obtain: microstructures that satisfy the given properties Start from a random configuration over the specified problem domain such that the volume fraction information is satisfied.

Randomly choose two locations (pixels) and define a move by interchangingthe intensities of the two pixels.

If the error norm defined as the deviation of the correlation features fromtarget features reduces, accept the move, otherwise reject it.

Stochastic Optimization ProcedureStochastic Optimization Procedure




A MAXENT viewpoint




Information Theoretic Scheme: the MAXENT principleInformation Theoretic Scheme: the MAXENT principle

Input: Given statistical correlation or lineal path functions

Obtain: microstructures that satisfy the given properties

Constraints are viewed as expectations of features over a random field. Problem is viewed as finding that distribution whose ensemble properties match those that are given.

Since, problem is ill-posed, we choose the distribution that has the maximum entropy.

Additional statistical information is available using this scheme.




The MAXENT PrincipleThe MAXENT Principle

The principle of maximum entropy (MAXENT) states that amongst the probability distributions that satisfy our incomplete information about the system, the probability distribution that maximizes entropy is the least-biased estimate that can be made. It agrees with everything that is known but carefully avoids anything that is unknown.

E.T. Jaynes 1957

A MAXENT viewpoint

Trivial case: no information is available about microstructure.

From MAXENT, the equiprobable case is the case with maximum

entropy for an unconstrained problem. This agrees with intuition

as to the most unbiased case

Information about volume fraction given.

The MAXENT distribution is one wherein we sample from the

volume fraction distribution itself at all material points

Correlation between material pointsto be taken into account. Result is

not trivial and needs to be numerically computed

Higher order information provided




MAXENT as a feature matching toolMAXENT as a feature matching tool

D. Pietra et al. ‘96, MAXENT principle for language processing.

Features of language extracted and MAXENT principle is used to develop a language translator

Zhu et al. ‘98, MAXENT principle for texture processing

Texture features from images in the form of histograms is extracted andMAXENT principle used to reconstruct texture images

Sobczyk ’03

MAXENT used for obtaining distributions of grain sizes from macro constraints in the form of expected grain size.

Koutsourelakis ‘05,

MAXENT for generation of random media. Correlation features of random media used as constraints to generate samples of random media.




MAXENT for microstructure reconstructionMAXENT for microstructure reconstruction

• MAXENT is essentially a way of MAXENT is essentially a way of generating a PDFgenerating a PDF on a hypothesis space which, on a hypothesis space which, given a measure of entropy, is guaranteed to incorporate only known given a measure of entropy, is guaranteed to incorporate only known constraints.constraints.

• MAXENT MAXENT cannotcannot be derived from Bayes theorem. It is fundamentally different, be derived from Bayes theorem. It is fundamentally different, as Bayes theorem concerns itself with inferring a-posteriori probability once the as Bayes theorem concerns itself with inferring a-posteriori probability once the likelihood and likelihood and a-priori probability are knowna-priori probability are known, while MAXENT is a guiding , while MAXENT is a guiding principle to principle to construct the a-priori PDFconstruct the a-priori PDF..

• We associate the PDF with a microstructure image and generate samples of the We associate the PDF with a microstructure image and generate samples of the image. image.

• MAXENT produces images with features (information) that are consistent with MAXENT produces images with features (information) that are consistent with the known constraints. Another way of stating this is that the known constraints. Another way of stating this is that MAXENT produces MAXENT produces the most uniform distribution consistent with the datathe most uniform distribution consistent with the data..




MAXENT optimization schemes




Find

Subject to

Lagrange Multiplier optimization

Lagrange Multiplier optimization

feature constraints

features of image I

MAXENT as an optimization problemMAXENT as an optimization problem

Partition Function




Equivalent log-linear modelEquivalent log-linear model

Find that maximizes

Equivalent log-likelihood problem

Kuhn-Tucker theorem: The that maximizes the dual function L also maximizes the system entropy and satisfies the constraints posed by

the problem

Direct modelsDirect models Log-linear modelsLog-linear models

ConcaveConcave ConcaveConcave

Constrained (simplex)Constrained (simplex) UnconstrainedUnconstrained

““Count and normalize” Count and normalize”

(closed form solution)(closed form solution)Iterative methodsIterative methods

A A ComparisonComparison




Optimization SchemesOptimization Schemes

• Generalized Iterative ScalingGeneralized Iterative Scaling

• Improved Iterative ScalingImproved Iterative Scaling

• Gradient AscentGradient Ascent

• Newton/Quasi-Newton MethodsNewton/Quasi-Newton Methods– Conjugate GradientConjugate Gradient– BFGSBFGS– ……

Start from a equal to 0. This is equivalent to uniform distribution Start from a equal to 0. This is equivalent to uniform distribution over sample space.over sample space.

Evaluate gradient at this point.Evaluate gradient at this point. Perform a line search on a direction based on the gradient Perform a line search on a direction based on the gradient

information.information. Evaluate the gradient information at the next point and continue the Evaluate the gradient information at the next point and continue the

procedure till it is within tolerance limit.procedure till it is within tolerance limit.




Gradient EvaluationGradient Evaluation

• Objective function and its gradients: Objective function and its gradients:

• Infeasible to compute at all points in one conjugate gradient iterationInfeasible to compute at all points in one conjugate gradient iteration

• Use sampling techniques to sample from the distribution evaluated Use sampling techniques to sample from the distribution evaluated at the previous point. (Gibbs Sampler)at the previous point. (Gibbs Sampler)

stochastic function

stochastic function




Sampling techniquesSampling techniques

• Sample from an exponential distribution using the Gibbs algorithmSample from an exponential distribution using the Gibbs algorithm

Choose a random point.Choose a random point. Evaluate the effective “energy” for various phases at that point Evaluate the effective “energy” for various phases at that point

using the updation algorithm to estimate “energy”.using the updation algorithm to estimate “energy”. Draw a sample from the given distribution and replace the pixel Draw a sample from the given distribution and replace the pixel

value at the material point.value at the material point.Continue the procedure till a sufficiently large number of Continue the procedure till a sufficiently large number of

samples are drawn. samples are drawn.




Updation SchemeUpdation Scheme

r

zone of influence

Rozman,Utz ‘01Rozman,Utz ‘01

Two point Correlation FunctionTwo point Correlation Function Lineal Path FunctionLineal Path Function

A scheme to update correlation function of an image when the A scheme to update correlation function of an image when the phase of a single pixel is changedphase of a single pixel is changed

r

zone of influence (regionwhere correlation function is affected)

Material point whose intensity is changed




Line search and conjugate directionsLine search and conjugate directions

Brent’s parabolic interpolation used for line search.Brent’s parabolic interpolation used for line search.

Stabilization in conjugate gradient machinery (Schraudolph ’02)Stabilization in conjugate gradient machinery (Schraudolph ’02) Add a correction term so that as line search becomes increasingly Add a correction term so that as line search becomes increasingly

inaccurate, its effect on the conjugate direction is also subdued.inaccurate, its effect on the conjugate direction is also subdued.

Stabilization term




Optimization Schemes

Convergence analysis with stabilization Convergence analysis w/o stabilization

Noise in function evaluation increases as step size for the next minima increases. This ensures that the impact on the next evaluation is reduced.




0 20 40 60 80 100 120 140 160 180 2000

10

20

30

40

50

60

70

80

90

100

Iteration

Ent

ropy

(bits

)

Entropy variation during MAXENT algorithmic scheme




Evaluation of effective elastic properties




Effective elastic property of microstructures

Variational Principle: Subject to applied loads and other boundary conditions, minimize the energy stored in the microstructure.

Pixel based mesh with a single phase inside each pixel (E. Garboczi, NIST ’98). Each pixel attributed the property of that particular phase.

Homogenization: The effective homogenized property of the

microstructure is obtained by equating energy of microstructure with that of a

specimen with uniform properties




Consolidated Algorithm

Experimental images

Analytical Correlation functions

Extract features and rephrase as mathematical constraints

Pose as a MAXENT problem and use gradient-based schemes for obtaining

solution

Use Gibbs sampling algorithm for sampling from

underlying distribution

Generate samples and interrogate using FEM

Obtain property statistics and use them for further analysis




Numerical Examples




Example 1




Reconstruction of 1d hard disks

Reconstruct one-dimensional hard disk microstructures based on two differentkinds of information: (a) two-point correlation functions (b) two point correlation and Lineal path function. Obtain elastic property statistics and compare for the two schemes.Input: Analytical two-point and lineal path functions (Torquato et.al. ’99)




150 160 170 180 190 200 210 220 230 240 2500

2

4

6

8

10

12

14

16

18

20

Effective young's modulus(GPa)

No.

of

sam

ples

Microstructures based on two-point correlation function

MAXENT distribution




140 160 180 200 220 240 2600

5

10

15

20

25

30

35

Effective young's modulus (GPa)

No.

of

Sam

ple

s

Microstructures based on two-point and lineal path function

MAXENT distribution




150 160 170 180 190 200 210 220 230 240 2500

2

4

6

8

10

12

14

16

18

20

Effective young's modulus(GPa)

No.

of

sam

ples

140 160 180 200 220 240 2600

5

10

15

20

25

30

35

Effective young's modulus (GPa)

No.

of

Sam

ple

s

Comparison of property statistics between two schemes




Example 2




Porous Media with short range order

To generate microstructures of porous media which exhibit short range orders of given

specific structure. (S2 is the two point correlation function, k and ro depend depend on characteristic length scales chosen)

Problem Parameters

correlation length ro= 32

oscillation parameter

ao= 8

2

o

ka

Input: Analytical two-point correlation functions (Torquato et.al. ’99)




Property statistics for media with short range order

MAXENT distribution




Example 3




Reconstruction using heterogeneous graded materials

Heterogeneous Graded Materials

Given a description of the gradation of phase-distribution in a graded material, reconstruct microstructures compatible with the given information, estimate statistics of

microstructure properties from this set.

Applications Tools with desirable properties at tips. Artificial joints for implants in humans

Input: Analytical volume fraction information throughout sample (Koutsourelakis ’04)




Samples of bilinearly graded heterogeneous materialsSamples of bilinearly graded heterogeneous materials

at smooth resolution levels




203 210 217 224 231 238 245 252 259 266 273 280 287 2940

5

10

15

20

25

30

35

Effective Young's Modulus(GPa)

No.

of

sam

ples

Effective elastic properties for a tungsten-silver bilinear graded material at 25oC

Elastic properties of bilinear graded materialsElastic properties of bilinear graded materials




Conclusions and future work




ConclusionsConclusions

Microstructures were characterized stochastically and scheme for

obtaining samples based on a MAXENT and time efficient update

scheme implemented.

Gradient based schemes and property of system entropy were

analyzed in detail.

Elastic properties were obtained using FEM and property statistics

developed

Schemes were discussed for numerical microstructures and effect of

incorporation of higher information on property statistics studied.




Future WorkFuture Work

Extend the method for polycrystal materials incorporating

information in the form of odf’s.

Couple the scheme with pixel based methods for obtaining plastic

properties.

Extend the method to physical deformation processes taking into

account the evolution of microstructure.




ReferencesReferences

1. E.T. Jaynes, Information Theory and Statistical Mechanics I, Physical Review 106(4)(1957) 620—630.

2. D. Cule and S. Torquato, Generating random media from limited microstructural information via stochastic optimization, Journal of Applied Physics 86(6)(1999) 3428—3437

3.P.S. Koutsourelakis, A general framework for simulating random multi-phase media, NSF Workshop-Probability and Materials: From Nano to Macro scale (2005)

4. K. Sobczyk, Reconstruction of random material microstructures: patterns of Maximum Entropy, Probabilistic Engineering Mechanics 18(2003) 279—287

5. S.C.Zhu et al, Filters, Random Fields and Maximum Entropy (FRAME): Towards a Unified Theory for Texture Modeling, IJCV 27(1998) 107-126

6. A.Berger et.al., A maximum entropy approach to natural language modeling, (1996), Computational Linguistics 22 (1996),39-71

Documents

Sethuraman Sankaran and Nicholas Zabaras