FRIEDRICH-ALEXANDER-UNIVERSITÄTERLANGEN-NÜRNBERG

TECHNISCHE FAKULTÄT • DEPARTMENT INFORMATIK

Lehrstuhl für Informatik 10 (Systemsimulation)

Time-dependent Earth mantle convection simulations withnon-trivial viscosity

Iniyan Kalaimani

Masterarbeit

Time-dependent Earth mantle convection simulations withnon-trivial viscosity

Iniyan KalaimaniMasterarbeit

Aufgabensteller: Prof. Dr. Ulrich RüdeBetreuer: Dr.-Ing. Dominik Bartuschat,

Simon Bauer, M.Sc(Ludwig-Maximillians-Universität,München)

Bearbeitungszeitraum: 15.11.2017 – 15.06.2018

Erklärung:

Ich versichere, dass ich die Arbeit ohne fremde Hilfe und ohne Benutzung anderer als der an-gegebenen Quellen angefertigt habe und dass die Arbeit in gleicher oder ähnlicher Form nochkeiner anderen Prüfungsbehörde vorgelegen hat und von dieser als Teil einer Prüfungsleistungangenommen wurde. Alle Ausführungen, die wörtlich oder sinngemäß übernommen wurden, sindals solche gekennzeichnet.

Der Universität Erlangen-Nürnberg, vertreten durch den Lehrstuhl für Systemsimulation (In-formatik 10), wird für Zwecke der Forschung und Lehre ein einfaches, kostenloses, zeitlich undörtlich unbeschränktes Nutzungsrecht an den Arbeitsergebnissen der Masterarbeit einschließlichetwaiger Schutzrechte und Urheberrechte eingeräumt.

Erlangen, den 15. Juni 2018 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Abstract

The competition in the development of high-performance computing facilities lead us to anera of massively parallel supercomputers, and at the same time, the demands to solve real-worldproblems has also increased with the progress of sophisticated modelling softwares. One suchcrucial real-world problem still on the desk of many researchers from different fields is to interpretour planet Earth’s behaviour and one such sophisticated modelling software framework that isestablishing its presence in the competition, to solve highly complicated large-scale problemsefficiently in a massively parallel setup, is Hierarchical Hybrid Grids (HHG) framework. Toprove its correctness and capability, HHG has to achieve a standard benchmark concerning theproblem to be solved.

This thesis presents the results obtained during the validation of the HHG framework andthe necessary implementations to achieve the chosen standard benchmark on Earth’s mantleconvection, which is responsible for many geological activities, e.g. Earthquakes. The rheologyof Earth’s mantle is highly non-linear with respect to many properties and is characterised bylarge thermal heterogeneities. A time-dependent simulation with a time-variable temperature-dependent viscosity model is used in this work as a benchmark study. This work is one of thebasis to conduct investigations on further complicated mantle convection problems.

vii

Contents

Abstract vii

1 Introduction 1

2 Description of the model problem 32.1 Governing equations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32.2 Non-dimensionalisation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

3 Numerical Methods 73.1 Finite Element formulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

3.1.1 Pressure-stabilised Petrov–Galerkin (PSPG) Stabilisation . . . . . . . . . 93.2 Finite Volume formulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

3.2.1 Mass-corrected conservative coupling . . . . . . . . . . . . . . . . . . . . . 113.3 Non-rotating reference frame for the mantle . . . . . . . . . . . . . . . . . . . . . 123.4 Hierarchical Hybrid Grids . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

3.4.1 Grid Structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133.4.2 Mesh stencils . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153.4.3 Parallelisation concept . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

3.5 Projected Coordinates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183.6 Multigrid Solver . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

3.6.1 Solver Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 213.6.2 Coarse-grid solver . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

4 Time-dependent benchmark 234.1 Initial temperature . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 234.2 Free-slip boundary condition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 234.3 Variable viscosity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 244.4 Test cases and results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

4.4.1 Test cases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 244.4.2 Results and discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

5 Conclusion and Outlook 31

A Non-dimensionalisation 32

References 34

Corrigenda 39

ix

Chapter 1

Introduction

Our planet Earth, according to the geoscientific community can be defined either by mechanicalor chemical properties. Mechanically, the internal layering of Earth can be defined as the litho-sphere, asthenosphere, mesospheric mantle, outer core and inner core, ranging approximatelyaround 6371 km from the surface of Earth to its centre. The lithosphere is the cool and rigidoutermost shell of the Earth and is composed of several major and minor Tectonic plates. Itsmechanical strength is greater than the underlying asthenosphere, which is fluid like visco-elasticsolid, more ductile compared to lithosphere. The mantle extends from few tens of kilometresbelow the surface down to the core-mantle boundary at about 3,490 km depth. It behaves likea highly viscous convecting fluid with respect to geological time scales, taking about 100 millionyears to complete its overturn. A mantle overturn is considered as the movement of the lessdense layer of the mantle at Core-Mantle-Boundary (CMB) up to the surface and fall back downagain as a highly dense layer.

The heat flux from the Earth’s core and the internal heat generated due to the decay ofradioactive material in the mantle rocks are convected via mantle, asthenosphere and finallycooled down by the conduction in lithosphere drives the entire process of mantle convection.The clear demarcation of what is happening in asthenosphere and mantle layers can be explainedalong with the help of rheological factors and chemical properties, which is beyond the scope ofthis work.

In simple words the geologic activity of Earth can be put forth as, our planet cooling to space.The most massive portion of the Earth, the mantle, moves so slowly and cools so gradually overthe lifetime of solar system, sets the pace of cooling for the whole Earth. The study of dynamicsof mantle, its circulation and evolution, is critical to understand how the entire planet functions.The phenomenon of mantle convection and its motion, wherein hot buoyant material rises andcold heavy material sinks, governs several processes from plate tectonics to crustal evolution,mountain building to volcanism, accumulation of stresses leading to earthquakes and hence thegeodynamo.

The purpose of this thesis is to assist the studies on the mantle convection via numericalsimulations, confined to temperature-dependent viscosity with the Hierarchical Hybrid Grids(HHG) framework. The simulations are performed on a real unit sphere representing Earth,which is scaled down by its radius and initiated with spherical harmonic modes of thermalconvection. As a progressive step, to solve problems with real time data, enhancing the HHG

1

framework to achieve standard benchmarks is vital and this thesis is one of the stepping stones tothe benchmark study on Earth mantle convection. The discretised Earth model is solved with ageometric Multigrid method that uses Uzawa-type smoothing algorithm and is intended to studyand account the challenges faced during the process of enhancement and validation of HHG.

The thesis is structured in 5 chapters. This chapter gives a short introduction about theEarth mantle convection and the motivation to study their behaviour with respect to differentproperties. In chapter 2, the governing equations responsible for mantle convection and how theyare handled in this work are briefly described. Chapter 3 gives an overview of numerical methodsused to treat these governing equations, introduces HHG and recalls its recent developments tosolve large-scale Earth mantle convection simulations. The test cases used to study a standardmantle convection benchmark and the results obtained are discussed in chapter 4. Finally, itends with a brief conclusion with respect to the results obtained and motivates the possibilitiesfor future studies of mantle convection modelling.

2

Chapter 2

Description of the model problem

2.1 Governing equationsThe dynamics of the Earth mantle is governed by the conservation of mass, momentum and

energy. The conservation equations for an incompressible fluid considering Boussinesq approx-imation, including the effects of self-gravitation are given by the system of partial differentialequations,

∇ · u = 0, in Ω× t , (2.1)

−∇ · [µ(T, z)(∇u+∇Tu)]− (δρg0 + ρ0δg)er +∇p = 0, in Ω× t , (2.2)

ρ0Cp(∂T

∂t+ u · ∇T ) = k∆T + ρH, in Ω× t , (2.3)

where,

u velocity in a non-rotating mantlereference frame

er unit vector aligned in the directionof gravitational forces

p dynamic pressure T temperatureµ dynamic viscosity of mantle H internal heat generation rateρ0 density of mantle g0 radial gravitational accelerationk thermal conductivity of mantle δρ perturbations to densityCp specific heat capacity of mantle δg perturbations to radial gravitational

acceleration

and Ω ⊂ R3 represents the Earth’s mantle.

The dynamic viscosity of mantle potentially depends on the temperature and/or the depthwhich is a domain-specific function of x := (x1, x2, x3) ε Ω. As mentioned in [29], the densityperturbations δρ for a simple equation of state can be linked via linear thermal expansion andcan be expressed as

δρ = −ρ0α(T − T0), (2.4)

where α is the coefficient of thermal expansion, T0 and ρ0 are the surface temperature anddensity which are taken as reference values. The perturbation to radial gravitational

3

acceleration δg is given as

δg = −∇rφ, and ∇2φ = 4πGρ, (2.5)

where φ is the perturbation to gravitational potential as explained in [27] and G is theGravitational constant. Here δg represents the effect of self-gravitation.

An identical mathematical form of (2.2) is generated by introducing reduced pressure P inrelation to the dynamic pressure p,

P = p− ρ0φ, (2.6)

which indicates that the self-gravitation does not affect the flow velocity and heat transfersimilar to [27]. The Boussinesq approximated momentum equation for the incompressible fluidwith reduced pressure is

−∇P +∇ · [µ(T, z)(∇u+∇Tu)] + [ρ0α(T − T0)]g0er = 0, in Ω× t . (2.7)

Since the gravitational force is considered to be constant and uniform in the experimentalenvironment, it is practically independent of the local changes in density.However, self-gravitationis essential to explain the Earth’s gravitational field from mantle density anomalies (e.g., asin [29]), it is not in the scope of this work.

2.2 Non-dimensionalisationThe governing equations can be non-dimensionalised as in the work of [30] using the following

normalisation with planetary radius R as the length scale:

x = Rx′, t =R2

κt′, u =

κ

Ru′,

T = δTT ′ + T0, P =µrκ

R2P ′, γ =

HR2

κCpδT, µ = µrµ

′,

(2.8)

where κ = k/(ρ0Cp) is the thermal diffusivity, µr is the reference viscosity, δT is the temperaturedifference across the mantle with relative thickness D between the core-mantle boundary (CMB)and the surface. The variables with a prime are non-dimensional. Cf. Appendix A for detailedsteps.

The non-dimensionalised conservation equations obtained after substituting (2.8) in (2.1), (2.7)and (2.3) and dropping the primes are

∇ · u = 0, in Ω× t , (2.9)

−∇ · [µ(∇u+∇Tu)] + ξRaT er +∇P = 0, in Ω× t , (2.10)∂T

∂t+∇ · (uT −∇T ) = γ, in Ω× t , (2.11)

where Ra is the Rayleigh number which controls the vigour of convection and γ is the non-dimensional internal heat generation rate. These are given by the relations

Ra =αρ0g0δTd

3

κµr, γ =

R2

κCpδTH. (2.12)

4

For simplicity, we assume an entirely basal heating, i.e. internal heat generation rate is ne-glected, γ = 0 in (2.12). Using R as a standard length scale for non-dimensionalisation as in [30],determines the relative top and bottom radii as rt = 1, rb = 0.55 throughout, which gives D =0.45. This helps to intuitively understand the results and the current formulations can also beemployed for different values of d in different problems accordingly, by introducing the factorξ = (R/d3) in (2.10) along with the usual definition of Ra which is based on the depth of themantle.

5

Chapter 3

Numerical Methods

In this chapter, methods used to discretise the system of partial differential equations (2.1)- (2.3) are discussed in the first 2 sections. In addition to that, the careful treatment of mass-balance equation as proposed in [25], to avoid spurious source and sink terms in the non-linearcoupling between the incompressible flow and transport, demanded by the buoyancy-drivenBoussinesq approximation will be recalled. An overview of the hierarchical hybrid frameworkwill be recalled from the works in [11, 18, 19] and is given in section 3.4. In section 3.5, movingtowards the realistic approach, coordinates are projected to the actual spherical domain for abetter approximation. And lastly, the choice of solvers for the considered problem in the thesisis explained.

For simplicity, this thesis is restricted to first-order time-stepping. The system of equationsare decoupled into Stokes equation part and Energy equation part which are only linked throughthe equation of state. The instantaneous constraint is obtained by approximating ∂T/∂t ≈(Tn+1 − Tn)/δtn in (2.3). This gives the form of a generalised Stokes equation

∇ · un = 0, in Ω, (3.1)

−∇ · [µ(∇un +∇Tun)] +∇P = −ξRaTn er, in Ω, (3.2)

and a transport-type, semi-discretised with Explicit Euler method giving

Tn+1 = Tn − ∂tn ∇ · (unTn −∇Tn), in Ω. (3.3)

These equations can be solved in a time-stepping fashion, where the Stokes part is solved first,advancing in time for n = 0, 1, 2, . . . to calculate T with previous u and p and then solve Stokesagain with updated T and so on. The flow part is now discretised in terms of stabilised equal-order finite-elements and the transport part in terms of vertex-centred finite-volumes. To notethat, (2.11), (3.3) are in its divergence form or popularly, in the context of continuum mechanicsconservative form. This is handled differently with HHG and its importance is emphasised inSection 3.2.

3.1 Finite Element formulationThe computational domain Ω ⊂ R3 is subdivided into a conforming tetrahedral mesh T−2

with a mesh size H, satisfying the usual shape regularity assumptions [15]. Here T−2 is a

7

basic input grid, usually coarse and decomposed into different primitive types (vertices, edges,faces and volumes) after refining Ω two times, such that each primitive type owns at least oneinterior node. This initial refinement is necessary to enable the grid to use the array-based datastructures in HHG framework [cf. Section 3.4]. A hierarchy of grids T := T`, ` = 0, 1, 2, ..., L isconstructed based on the initial triangulation T−2 by successive uniform refinement for L levels.This T`, ` ∈ N0 shall be referred to as Primal mesh as shown in Figure 3.1. To point out,this uniform refinement strategy guarantees that all these meshes also satisfy a uniform shaperegularity. The level size of mesh is given by h` = 2−(`+2)H.

The primal mesh is then discretised by introducing linear conforming Lagrangian finite elementspace as the basic function space on T`, i.e., a function space of piecewise linear and globallycontinuous function is defined by

S`(Ω) := q` ∈ H 1(Ω) : q`|T ∈ P1 (T ), ∀T ∈T`, (3.4)

where T ∈ T` is any finite element of a given mesh T` ∈ T .For simplicity, to describe the discretisation, a homogeneous Dirichlet boundary condition

u = 0 which satisfies the compatibility condition∫∂Ωu · n ds = 0 is considered. Also, to

make the pressure well-defined, its mean-value condition is incorporated in the space Q`. Theconforming finite-dimensional spaces, which satisfy boundary conditions for velocity and pressureare then given by

V ` := [S`(Ω) ∩H 10 (Ω)]3, Q` := [S`(Ω) ∩ L2

0(Ω)], (3.5)

where L20(Ω) := q ∈ L2(Ω) : 〈q, 1〉Ω = 0 and 〈·, ·〉Ω denotes the inner product in L2(Ω).

Considering equal-order scheme, coefficient functions shall be defined formally as members ofS`(Ω). For the sake of brevity, temperature T is also interpreted as a member of S`(Ω), whichshall be clarified in 3.2.1.

The nodal values of the viscosity coefficient µn` ∈ S` are first evaluated and subsequently

considering the following stabilised weak formulation to discretise the Stokes problem: find(u`, p`) ∈ V ` ×Q` such that

a(u`,v`) + b(v`, p`) = f(v`) ∀v` ∈ V `,

b(u`, q`) − c`(q`, p`) = g`(q`) ∀q` ∈ Q`,(3.6)

where the bilinear and the linear forms are defined as

a(u,v) :=

∫Ω

µ(∇u+∇Tu) : (∇v +∇Tv) dx, b(u, q) := −∫

Ω

∇ · u q dx and

f(v) :=

∫Ω

f · vdx, ∀u,v ∈ [H 10 (Ω)]3 and q ∈ L2

0(Ω). Here, f = −ξRaT er.

Moreover, the bilinear form c`(·, ·) :=∑

T∈T` cT (·, ·) and the linear functional g` :=∑

T∈T` gT (·)are required to stabilise the finite element pairing V `×Q` as the uniform LBB condition, [1,13,22]is violated.

8

3.1.1 Pressure-stabilised Petrov–Galerkin (PSPG) StabilisationAmong several popular approaches widely used in CFD codes to stabilise the equal-order finite-

element spaces, the standard PSPG-scheme is considered in this thesis and stabilisation termscT and gT for an element T ∈ T` are chosen as

cT (q`, p`) := δT

∫T∈T`

∇q` · ∇p`, gT (p`) := −δT∫T∈T`

f · ∇q`, (3.7)

where δT := αTh2T /µ

nT for some sufficiently large constant αT > 0 and µn

T := |T |−1∫Tµn` dx

denotes the element-wise average of the viscosity. The diameter of an element T is definedas hT = (

∫T∈T` dx)1/3. The stabilisation parameter αT has to be chosen carefully in order to

avoid unwanted effects of over-stabilisation, i.e., constraints imposed on the degrees of freedombecome unnecessarily strong, which would otherwise contribute to a better approximation of thesolution. This is clearly demonstrated in [21] with an appropriate numerical example and it isrecommended that αT = 1/12 would be a good choice in practice.

The weak formulation for discretising the stabilised Stokes equation of the form (3.9) can bedefined as: find (u`, p`) ∈ V ` ×Q` such that∫

Ω

µn`

(∇un

` +∇Tun`

):(∇vn` +∇Tvn`

)dx −

∫Ω

pn`∇ · v` dx −∫Ω

∇ · un` q` dx −

∑T∈Th

∫T

α∇pn` · ∇q` dx = −Ra∫

Ω

Tn` er · v` dx.

(3.8)

Remark 1. For the equal-order linear ansatz spaces, the presence of these additional stabilisationterms destroys local mass-conservation, and thus a correction for conserving the local mass isneeded to compensate the effects introduced here.

Now, the following bilinear form is introduced to simplify notation

S`(u, p;v, q) := a(u`,v`) + b(v`, p`) + b(u`, q`) − c`(q`, p`),

to write the discrete problem (3.6) as: find (u`, p`) ∈ V ` ×Q` such that

S`(u`, p`;v`, q`) = f(v`) + g`(q`) ∀(v`, q`) ∈ (V ` ×Q`). (3.9)

Let the degrees of freedom for velocity and pressure be defined as dim V ` = nl,u and dim Q` =nl,p, respectively. Then, the following isomorphisms u` ↔ u ∈ Rnl,u and p` ↔ p ∈ Rnl,p can bedefined to represent the algebraic form of the variational formulation (3.6) as

K(up

):=

(A BT

B −C

)(up

)=

(fg

), (3.10)

with the system matrix K ∈ R(nl,u+nl,p)×(nl,u+nl,p). Note, the matrix A ∈ R(nl,u×nl,u) consistsof 3× 3 block structure.

9

3.2 Finite Volume formulationA barycentric grid B` is constructed on the primal mesh T` which is a common practice in finite

volume methods and is called Dual mesh as shown in Figure 3.1. Let T be any element withnodes xi, i = 1, 2, 3, 4, of the tetrahedral primal mesh. Then, as given in [24], the barycentricdual mesh is defined as

BTi := x ∈ T : λTj (x) < λTk (x), j 6= k, j, k = 1, 2, 3, 4,

where λTi (x) denote the barycentric coordinates of x ∈ T ∈ Ti with respect to nodes xi andTi ⊆ T` is the nodal patch associated with xi. Setting Bi := ∪T∈TiBT

i results in the partition ofΩ into non-overlapping polygonal control volumes which shall be referred to as the set

B`(Ω) := Bi, i = 1, 2, . . . , N.

On this barycentric dual mesh, a piecewise constant function space is defined as,

R`(Ω) := r` ∈ L2(Ω) : r`|B ∈ P0 (B), ∀B ∈ B`.

Figure 3.1: left: Unstructured 2D primal mesh (black) T` with its associated dual mesh (blue)B`, center: A triangular patch with a set of T ∈ Ti ⊆ T` for node i with BT

i ∈ B`, right: onedual cell BT

i ∈ B` of a structurally refined Tetrahedron patch [21,25].

A first order upwind scheme is employed to propagate temperature in the transport equation,which is temporally discretised by Explicit Euler method. The length of time-step δt > 0 iscalculated at every time-step such that the Courant–Friedrichs–Lewy (CFL) condition is satisfiedfor a stable upwinding. Based on the discrete velocity solution obtained with the initial boundarycondition (4.1) and using Petrov-Galerkin scheme, the variational form of the transport equationcan be written as: given (u`, p`) ∈ (V ` ×Q`) and Tn

` ∈ R`, find Tn+1` ∈ R` such that∑

Bi∈B`

∫Bi

Tn+1` r` dx =

∑Bi∈B`

∫Bi

Tn` r` dx − δtn A(Tn

` , r`), ∀r` ∈ R` (3.11)

and the bilinear form is defined as

A(Tn` , r`) :=

∑Bi∈B`

∫∂Bi\∂Ω

(j(un

` , pn` )〈Tn

` 〉up −∇Tn` · ni

)r` ds,

where j(un` , p

n` ) = un

` ·ni is the mass flux across the control volume boundary oriented face-wisein the direction of normal vector ni. 〈Tn

` 〉up represents the upwind value defined piecewise forany control facet of node x0 as shown in Figure 3.2.

10

3.2.1 Mass-corrected conservative couplingTo note that, the temperature T is interpreted as member of two different spaces. In (3.8), it

is a weighted sum of all elements that share a node whereas in (3.11), it is an average over thecontrol volume Bi. Given the nodal basis φi of S` and the piecewise constant basis χi of R`, anatural bijection between these two spaces can be defined due to the fact dim R` = dim S` = NS .Natural transfer operators are thus given by,

π` :

NQ∑i=1

qiχi →NQ∑i=1

qiφi and π` :

NQ∑i=1

qiφi →NQ∑i=1

qiχi.

To avoid the introduction of unwanted damping effects caused by the transfer between primaland dual mesh, a crucial property π`π` = π`π` = I should hold by construction. The use of π`

in f(v`) in (3.6) couples the momentum and transport equation by introducing mass-lumping,which enables the solution of transport equation to satisfy the conservation property. The weakformulation for discretising the stabilised Stokes equation of the form (3.9) is now realised as:find (u`, p`) ∈ V ` ×Q` such that∫

Ω

µn`

(∇un

` +∇Tun`

):(∇vn` +∇Tvn`

)dx −

∫Ω

pn`∇ · v` dx −∫Ω

∇ · un` q` dx −

∑T∈Th

∫T

α∇pn` · ∇q` dx = −Ra∫

Ω

π`Tn` er · v` dx.

(3.12)

Figure 3.2: left: A Tetrahedron as dual sub cells and right: a control facet associated with anode x0 and its associated normal [25].

The issue mentioned in Remark 1 is addressed by correcting the mass flux j(un` , p

n` ) as proposed

in [25], which eliminate the compressibility effects introduced by the stabilisation. The proposedflux term reads as

j(un` , p

n` )|∂Bi

= (un` − α∇pn` )|∂Bi

· ni.

It has been proven in [21, Theorem 3.2, Lemma 4.1] that if (un` , p

n` ) satisfies (3.6), then the

integral∫∂Bi

j(un` , p

n` )|∂Bids vanishes, which is a notable property of this flux that makes it

conservative with respect to the control volumes. By this, the local conservation in discretesense is ensured, even though discrete velocities may not strongly satisfy ∇ · un

` = 0. This canbe rectified by a post-processing procedure as discussed in [21, Section 3.3].

11

3.3 Non-rotating reference frame for the mantleIt is crucial to introduce a significant factor called pure rotation, which is also referred to

as net rotation or degree-1 toroidal motion. While numerically solving the governing equationsin spherical models of mantle convection, a component of pure rotation may contribute to thesolutions of flow velocities for the whole mantle, which should be defined in a non-rotatingreference frame, as explained in [6,28]. The cause for the pure rotation is understood to be mostlikely related to numerical procedures with no specific physical meanings.

It is evident from the governing equations that neither the dynamics nor the heat transfer isaffected by the pure rotation of the whole mantle. However, the pure rotation, if existing, mayintroduce arbitrariness to the flow velocities and affects time stepping for convection problems.Therefore, it is desired that any pure rotation is explicitly removed from the numerical approxi-mation of flow velocity [28]. This has been taken care of in this work with HHG as described inthe following.

An efficient procedure to remove the pure rotation from numerical solutions was introducedin [29]. The pure rotation motion is given by r × ω for a layer of mantle Ω′ ⊂ Ω with top andbottom radii rt and rb and with a uniform density ρ0. The angular velocity ω can be determinedfrom the angular momentum L as follows,

L = Iω =

∫Ω′ρ0r × u dΩ, (3.13)

where I = 8πρ0(rt5 − rb5)/15 is the moment of inertia and r and u are the position and flow

velocity vectors, respectively. The angular velocity ω is thus given as

ω(u) =15

8π(rt5 − rb5)

∫Ω′r × u dΩ. (3.14)

Equation (3.14) can be used for both dimensional and non-dimensional calculations.

This pure rotation motion is then removed from the numerical solutions of flow velocities inevery iterative step in the following manner,

u = u− r × ω(u). (3.15)

where u is the flow velocity obtained after removal of the pure rotation motion and is used indetermining the timestep and thereby updating the temperature.

Pure rotation can also be determined by other methods, e.g., as used in CitcomS [6,28]. Purerotation of any sub-layer of the mantle can be determined by applying equation (3.14) to thatrespective sub-layer with respect to the non-rotating reference frame of the whole mantle (e.g.,net rotation of the lithosphere). In particular, net rotation of the lithosphere has implicationsfor the reference frame of plate motion [6, 28] and seismic anisotropy [7].

The other possible way to remove pure rotation is directly through the Uzawa algorithm.This has been done in [23] for the original version of Citcom with periodic boundary conditions.However, it has been stated in [29] that based on past experiences, pure rotation motion for thewhole mantle from the flow solver of CitcomS is often negligibly small for isochemical convection.

12

3.4 Hierarchical Hybrid GridsHierarchical Hybrid Grids [10,11] is a software framework, written in C++ and offers efficient

parallel implementation of finite element geometric multigrid solvers. Its goal is to combinethe flexibility of unstructured grids and structured grids with which superior performance canbe achieved on modern architectures. HHG is different from conventional FE software in thesense that avoids the overhead of indirect memory access by providing storage-efficient Matrix-free implementation. It was originally been developed as a multigrid library for scalar ellipticPDEs on clusters with a few thousand cores and had evolved as a massively parallel softwareframework to run on modern supercomputers efficiently with almost a million parallel threads [9].Its excellent node performance and good scalability on different characteristic types of modernsupercomputers have been demonstrated for the Stokes system in [19].

Matrix-free techniques are of renewed interest to develop numerical techniques that reducememory traffic. Keeping that as a motive, HHG is designed in such a way that a stencil-likeapproach, as in finite difference algorithms, can be obtained with regular refinement and gridpartitioning. It employs nested grid hierarchies and hierarchically organised data structuresto support storage- and compute-efficient matrix-free implementation of geometric multigridalgorithms. The main concepts of HHG are described in the context of considered Earth mantleconvection model problem as follows.

3.4.1 Grid StructureThe computational domain, Earth’s mantle is considered as a thick spherical shell with a

relative depth D = 0.45 as mentioned in Section 2.2, neglecting flattening at the poles byrotation and dynamically free deforming surfaces. The initial mesh T−2 is generated using anicosahedral approach, where the twelve vertices of a regular icosahedron are mapped onto a unitsphere resulting in 20 triangles [5]. Next, they are logically combined into 10 diamonds whoseedges are subdivided into nt nodes. Each triangle is then subdivided into (nt−1)2 sub-triangles,such that the subdivisions of edges and triangles coincide. Finally, each new node on triangularfaces and on edges is projected onto a spherical surface, resulting in meshes as depicted in Figure3.4. Now, taking nr spherical surfaces with different radii and connecting all vertices with theircorresponding nodes in the adjacent layers, a grid for a thick spherical shell is obtained. Eachelement is now a triangular prismoid. This mesh consists of 20 · (nr − 1) · (nt − 1)2 prismoids,each split into three conforming tetrahedral elements which are referred to as macro elementsT ∈ T−2.

Figure 3.3: Sample triangulations T of distinctively coloured 10 diamonds with nt = 3, 5, 10 [26].

13

Figure 3.4: A triangle of a diamond refined tangentially and radially with nt = 5 and nr = 4 [26].

This input coarse mesh T−2 is completely unstructured and is nlevel times uniformly refinedrepeatedly following the rules mentioned in [12] to obtain a nested hierarchy of meshes T , leadingto a total of 60 · (nr − 1) · (nt − 1)2 · 23nlevel fine grid tetrahedra. To minimise communication,HHG uses a concept to decompose a macro element into four geometric primitives namely,vertices, edges, faces and volumes. Each macro element on its initial refinement yields eight sub-elements, each of which belongs to one of the three different sub-classes, as described in Figure3.6. Further refinement does not yield new class of sub elements. On a second refinement ofthese 8 sub elements, each primitive gets at-least one interior node. To recall, this stage of gridis referred to as triangulation T0 as mentioned in Section 3.1. More precisely, T`+1 is obtainedfrom T`, ` = −2,−1, 0, 1, 2 . . . , L and nlevel = L− 2.

Figure 3.5: 3 sub classes of a tetrahedron. Gray class possess same structure as the parent whilegreen and blue are different and further refinement of these two classes does not yield a new classof sub elements [4].

With such a structured block refinement, node stencils with an identical structure for eachinner node of a macro element primitive can be obtained. Moreover, in the case of affine element

14

mapping, stencils for PDEs with constant coefficients (say, viscosity coefficients in our case) donot depend on the node location within the same primitive, as they possess same stencil entries.For example, all volume primitive stencils of a macro element are the same and are given by asymmetric 15-point stencil as shown in Figure 3.6.

3.4.2 Mesh stencils

Figure 3.6: left: 15-point stencil of a low order volume primitive node at T0, right: with itscardinal directions [4].

The stencil weight for a fixed node say i connected to another node j by an edge T` canbe defined by a`(φi, φj), which is always a non-zero. With an advantage of uniform refine-ment in our case, these stencil weights can be easily identified by their cardinal directionsw ∈ W = bc, be, bnw, · · · ,ms,mse, · · · , tse, with |W| = 15 as mentioned before. The firstcharacter denotes the three planes, bottom, middle and top, and the second character denotesthe cardinal direction within the plane as shown in Figure 3.6. With this, the stencil for anynode i can be represented as a vector,

s = (a`(φi, φjbc), · · · , a`(φi, φjmc), · · · , a`(φi, φjtse)) in R15, with jmc = i.

Here s will have constant values within a uniformly refined macro element. Thus it can becomputed and stored initially as a reference stencil and accessed anytime. For different levels, asimple scaling of this reference stencil gives the values of stencils of their respective levels. If s is anon-constant function, as in the case of projected coordinates, which is briefly discussed in Section3.5, the values of entries differ and have to be computed on-the-fly, which is computationallyexpensive. Moreover, in the case of constant function, not only the element matrix but also thestencil is symmetric, i.e., opposite stencil entries like tse and bnw are identical.

15

3.4.3 Parallelisation conceptThe geometry primitives of the coarse input grid (macro elements) act as a container for a

subset of unknowns corresponding to the uniformly refined grids. These sets of unknowns arestored in array-like data structures as follows: with an example of a macro element, the coarse-grid nodes are stored in a vertex primitive data structure, and nodes along the edges that connectthe vertices are stored in an edge primitive data structure. Nodes on the triangular faces of amacro element are stored in a face primitive data structure, and all remaining points within thecoarse-grid macro element are stored in a volume primitive data structure. All computations orlattice updates, i.e. all multigrid operations such as smoothing, prolongation, interpolation andresidual calculations and operator applications are carried out locally on each primitive itself.

Figure 3.7: left: A simplified representation of a vertex, edge and face primitive of T ∈ T0,right: Two neighbouring tetrahedra with their ghost layers [26].

On further refinement levels, the generated nodes along with the nodes from coarse levelsenter into their respective containers corresponding to that particular level. This results in acontiguous memory layout that conforms inherently to the refinement hierarchy. Additionalappropriate layers, the so called ghost layers as shown in Figures 3.8 and 3.7, are generated forunknowns corresponding to primitives whose stencil extends to their neighbours, mimic all nodesfrom the neighbouring primitives that are needed for the computation. This set-up reduces theoverhead significantly as the unknowns are accessed without in-direct addressing.

16

Figure 3.8: Ghost layer of a face primitive with 15-point stencil (green) of a node (red) [18].

The HHG framework employs distributed memory parallelism based on the message passinginterface (MPI). The macro elements are distributed among the processors. The ghost layers areupdated systematically by local copy operations for primitives that share the same memory andby message passing if they reside on different processors. The sequence of primitive updates areaccording to their ascending geometrical order: First, all unknowns at the vertices of a macroelement are updated in parallel, then at the edges, faces and finally interior volume of a macroelement. The respective ghost layers are updated before performing computations on a primitiveof a different kind, which are of block Jacobi-type smoothing. One way communication is appliedby allowing only the higher geometrical order primitive to carry over the copy operations toghost layers. To be precise, all dependencies between primitives of the same kind are ignored(for example, 3 edges connected by a vertex corner) to improve parallel efficiency and reduce thecommunication, making the computations follow the block Gauss-Seidel smoothing fashion.

Figure 3.9: Illustration of parallelisation concept [2].

17

3.5 Projected CoordinatesIn the hierarchy of meshes T , the tetrahedral elements T ∈ T` are affinely mapped by a function

ΨT from the reference tetrahedron T with nodes at (0,0,0), (1,0,0), (0,1,0) and (0,0,1), as shownin Figure 3.10. Here, the additional nodes generated from the refinement process of coarse inputgrid do not lie on the original spherical surface. This represents a polyhedral approximation ofthe computational domain. Let’s call it ΩH := ∪T∈T`T . It is a physically non-correct domain,resulting in a reduced accuracy. Resolving this with further refinement levels does not improvethe quality of geometry approximation rather makes the process computationally expensive.

Figure 3.10: A 2D representation of a Macro element mappings with discrete indices (i, j) clippedon an index plane k. ΨT maps the reference element T (left) to the macro element T ∈ T` (center),followed by φT which maps T to the blended element T ∈ T` [4].

Hence, the need arises to project these fine grid nodes on to the respective sub-radial sphericalsurfaces resulting in a non-polyhedral domain which can represent the geometry with betteraccuracy. This is done by a global mapping Φ, assuming it is globally continuous and reflectingthe domain of interest, i.e. it satisfies Φ(ΩH) = Ω, as shown in Figure 3.11. A blending functionΦ|T := ΦT is defined element-wise that maps elements of all levels, T ∈ T`, to their respectivesub-radial layers such that they form spherical layers. Examples of blending functions of threetest geometries are given in [4], out of which the below-mentioned function is adopted for thecurrent work,

ΦT (p) =p

|p|R(p)

where p is a point on some kth-plane within a prismoid P, which lies parallel to its top andbottom triangular faces as shown in Figure 3.11. The corresponding scaling factor R(p) is thedistance of point obtained by the intersection of kth−plane with radial grid linesR which connectthe vertices of the respective prismoid. A detailed description of how the nodes are projected fordifferent types of primitives can be found in [26].

Now, a second sequence of hierarchy of meshes T are obtained such that Ω` := ∪T∈T`T , which

is able to resolve Ω. The meshes in T and T possess the same vertices from the coarse input grid,and so they have the same connectivity, which means they are topologically equivalent. However,on refining the grids, the midpoints of edges of different levels are not the same anymore. Thisdistorts the sub-elements and makes them dissimilar to the three sub-element classes of theuniformly refined tetrahedron. As a consequence, the fine-grid stencil weights can vary from one

18

grid point to another. Either these stencils are computed once and then stored, consuming extramemory of O(23`), which creates extensive memory traffic or they are computed on-the-fly eachtime costing computation of O(23`) local contributions.

Figure 3.11: Projection of point p on some kth-plane f within a prismoid to p′ of sphericalsub-radial surface of a node q on some radial grid line spanning 0− r − d [26].

The On-the-fly approach leads to significantly longer runtime for large-scale simulations. Oneavailable possibility to improve the performance is by novel two-scale approach as proposedin [4], where the components of the exact stencil s associated with a volume primitive nodeare replaced by surrogate polynomials of moderate order. To be precise, instead of computingthe stencil sw(i, j, k) on-the-fly, the surrogate Psw(i, j, k) is evaluated, where Psw denotes theproposed surrogate polynomial associated with the cardinal direction w ∈ W defined either byinterpolation or alternatively by a discrete L2-best fit. It has been proved in [4] that second-orderpolynomials already guarantee high enough accuracy for typically large system sizes and thusreduce the flop count significantly. However, the scope of the current work is restricted to theclassical on-the-fly approach.

Figure 3.12: left: Earth model with non-projected and right: projected coordinates with plumesarising from the core mantle boundary. The triangular faces of tetrahedra elements are morepronounced at the boundary surfaces of the non-projected model.

19

3.6 Multigrid SolverAmong the bi-directionally coupled equations (3.11) and (3.12), Stokes-type equation is the

most expensive regarding memory consumption and computational cost, mainly due to the con-tribution of the rapidly growing number of interior nodes in the volume primitive with O(23`). Incomparison to the energy equation, wherein every explicit timestep just an operator applicationis needed, the Stokes part needs iterative solves which makes it significantly expensive. Choosingan asymptotically optimal solver, such as multigrid method would be an optimal choice whosecomplexity grows linearly with the problem size. A solver, in which the multigrid algorithm actsas a main solver part with a specialised smoother to solve the considered saddle point modelproblem, has been chosen against historically evolved various iterative solution techniques, inparticular, popular Krylov subspace methods.

Motivated by the investigation results based on quantitative performance analysis of threedifferent Stokes solvers in [18], all-at-once multigrid method with Uzawa-type algorithm is chosenas a solver here. It was shown that, when compared to Pressure Schur complement CG algorithmand preconditioned MINRES method, an all-at-once multigrid method provides the best overallperformance with respect to memory consumption and time-to-solution. Also, a study has beendone, based on "parallel textbook multigrid efficiency" [20], to assess the efficiency of paralleliterative solvers for a given hardware architecture in absolute terms, Work unit (WU). It isdefined as the cost of one application of the discrete operator for a given problem, and theanalysis of iterative solvers can be conducted in terms of WU.

Uzawa algorithm, which is based on Richardson iteration, requires the solution of a symmetricpositive definite system of linear equations (SPDLE) at its each step. But in our case, thealgebraic system (3.10) is a symmetric and indefinite problem, where A is symmetric positivedefinite and C is symmetric positive semi-definite. The work of [17] demonstrates the variant ofthe Uzawa algorithm for such indefinite linear systems, where the computation of SPDLE canbe replaced by an approximate solution produced by an arbitrary iterative method. This leadto its variant called Inexact uzawa algorithm which is convergent with a rate close to that ofexact algorithm with relatively modest requirements on the accuracy of the approximate solution.Using this along with a suitable preconditioner for improved performance, in (k+ 1)th iteration,we solve the system (

A 0

B −S

)(uk+1 − uk

pk+1 − pk

)=

(fg

)−K

(uk

pk

), (3.16)

for (uk+1,pk+1)T , where A and S denote preconditioners for A and the Schur-complement S :=

BA−1BT + C, respectively.

Here the velocity is smoothened in the first step and the current velocity is used to updatepressure in the second step, i.e.

uk+1 = uk + A−1(f −Auk −BTpk

),

pk+1 = pk + S−1 (Buk+1 − Cpk − g) .(3.17)

20

3.6.1 Solver ParametersA Variable V-cycle which has an advantage of lower coarse-grid cost is chosen instead of

W-cycle, as it is too costly for the parallel case due to the extensive work on the coarse-grid.In Variable V-cycle(Vvar ) the number of smoothing steps vary geometrically depending on thelevel. Here, In addition to 3 pre-smoothing and post-soothing steps used for a normal V-cycle,two additional pre-smoothing steps are employed on every successive coarse level, which is thendenoted as Vvar (3 , 3 ). Velocity part A is smoothened with pseudo-symmetric hybrid blockGauss-Seidel(SHGS) smoother, where a forward hybrid Gauss-Seidel (FHGS) is executed firstfollowed by a backward hybrid Gauss-Seidel (BHGS). Only the ordering within each primitive isreversed but not the ordering of primitive hierarchy which is referred to as pseudo here. For thepressure, FHGS is applied to the stabilisation matrix C, with under-relaxation ω = 0.3. Thesesmoothers are then applied within a Vvar (3 , 3 ).

3.6.2 Coarse-grid solverThe coarse-grid can be solved with either sparse direct solvers or other computationally efficient

iterative solvers. To avoid the dependency on external libraries, a suitable Krylov subspacecoarse-grid solver is employed. Since the linear system is symmetric and indefinite, a MINRESmethod preconditioned in block diagonal fashion as shown below is chosen, which is given by(

A 0

0 S

).

In large-scale simulations, the coarse-grid may still have O(109) DOFs, which becomes anotherbottleneck to be taken care of. However, in this work, the coarse-grid is considerably small andwas handled well with test systems used.

21

Chapter 4

Time-dependent benchmark

The work of Zhong et al. [29] is used as the benchmark for this thesis. The Rayleigh number Ra =7.6818× 104 is used for all the computations in this thesis, similar to the work of C. Bursteddeet al. [14] for a straightforward understanding, instead of ξRa as mentioned in Section 2.2. Inthis similar benchmark study carried out lately, based on [29], the length scale used for non-dimensionalisation is Earth’s radius R and not the mantle thickness d. So when comparing thenumerical results with [29], the results obtained in the current implementation are scaled in termsof relative depth of mantle thickness, i.e., time values are scaled by 1/D2 and the velocity by D.

4.1 Initial temperatureTo initiate the convective process, a special harmonic perturbation as a function of spherical

coordinates interpolated between the core and the surface temperature is chosen and is given as,

T (r, θ, φ) =rb(rt − r)r(rt − rb)

+ [εc cos(mφ) + εs sin(mφ)] · plm(θ) sin

[π(r − rb)(rt − rb)

], (4.1)

where r, θ, φ are the usual spherical coordinates: radial, colatitude and longitude. The parametersεc and εs are the magnitudes of perturbation for cosine and sine terms, respectively. Each ofthese parameters is set to a value of 0.01. The normalised associated Legendre polynomial plm(θ)is related to the associated Legendre polynomial Plm as follows,

plm(θ) =

√(2l + 1)(l −m)!

2π(1 + δm0)(l +m)!Plm(θ). (4.2)

For the chosen spherical harmonic degree l = 3 and orderm = 2, plm(θ) =√

6.5625/π cos θ sin2 θ.The boundary conditions for the temperature are set to T = 1 at CMB (i.e., at r = 0.55) andT = 0 at the surface (i.e., at r = 1).

4.2 Free-slip boundary conditionFree slip boundary conditions are considered at the top (surface) and bottom (CMB) bound-

aries in this work. As opposed to the implementation of free-slip conditions on planes whichare aligned along the coordinate axes, realisation of the same is not straightforward when it

23

comes to curved boundaries. The idea of free-slip condition is to not allow any flow through theboundary and to eliminate the stress along a tangential vector t or in simple words, to maintaina homogeneous tangential stress, i.e.,

u · n = 0, σ(u, p) · t = 0.

This is realised in a point-wise fashion by the condition (I −nnT ) σ(u, p)n = 0, as describedin [18]. In this approach, the normal vector is considered as a separate entity and is counted asanother degree of freedom. Normal vectors are constructed in such a way that mass conservationis not interfered by itself, which is an important requirement. Such a global mass conservingdefinition of the normal vector is given for the node xi by

ni := ‖ 〈∇φi, 1〉Ω ‖−12 〈∇φi, 1〉Ω ,

where φi denotes the basis corresponding to the ith node on the boundary. Thinking in the termsof implementation, in order to take advantage of single matrix-vector multiplication, the aboveintegral has to be understood component wise. Following the above mentioned approach, thenormals can be computed using a local stencil application of BT onto the vector 1, where B isan algebraic representation of the linear integral form 〈·, ·〉.

4.3 Variable viscosityFor the current work, all material properties except viscosity are assumed to be constant.

Viscosity is considered to be dependent on temperature and computed according to the linearisedArrhenius law. A non-dimensional viscosity of this form is expressed as

µ = exp[E(0.5− T )], (4.3)

where E is an activation parameter controlling the magnitude of viscosity variations and isequivalent to non-dimensional activation energy. A viscosity at reference temperature Tr = 0.5is used as reference viscosity µr in the definition of Ra in (2.12). In addition, the ratio ofmaximum to minimum viscosity across the mantle thickness is defined as viscosity contrast ∆µ =µmax/µmin. It should be regarded that, since the internal heat sources are absent, the computedtemperature fields should obey the discrete maximum principle throughout the simulation. Thisis necessary to perform the simulations in line with the physics of the underlying convectionproblem and the current implementation obeys this principle.

4.4 Test cases and results

4.4.1 Test casesThe test cases presented below are used to study the behaviour of the state variables, velocity

and temperature respectively by means of root-mean-square (RMS) velocity < Urms > andVolume-averaged temperature < T >, with respect to different refinement levels and velocityupdates at a chosen interval of timesteps are studied for both iso-viscous (∆µ = 1) and variableviscous (∆µ = 20) conditions.

24

Table 4.1: Test cases for different refinement levels denoted by "R" and for different velocityupdate intervals denoted by "V" for a maximum refinement level `.

Setup Case nt nr ` no. of macro elements no. of radialelements

velocity updateinterval

Setup-1 R 05 03 04 1920 32 1Setup-2 R 07 04 04 6480 48 1

Setup-3 V 05 03 04 1920 32 1Setup-4 V 05 03 04 1920 32 5Setup-5 V 05 03 04 1920 32 10

The benchmark study in [29] were carried out on a spherical domain with 32 elements in theradial direction. So the current implementation with HHG in this thesis is expected to producethe same results with similar grid size as the benchmark.

The state variables for all the test cases are computed as a function of time as,

< T > =3

4π(r3t − r3

b )

∫Ω

T dΩ, (4.4)

< Urms > =

[3

4π(r3t − r3

b )

∫Ω

u2 dΩ

]1/2

, (4.5)

where rb = 0.55, rt = 1 and Ω represents the volume of whole mantle. The integrals for < T >and < Urms > are calculated as < T ,1 > and < u2,1 >. T and u are vectors containing valuesof temperature and magnitude of velocity at all nodes.

Figure 4.1: Temperature field at steady-state for Setup-1. Contours of plumes at non-dimensi-onalised temperatures 0 (blue) and 0.5 (gray) are shown. left: iso-viscous model, right: Variableviscosity model a contrast of 20.

The resulting steady-state temperature field shows four well defined plume-like upwellings asin Figure 4.1. This is a tetrahedral symmetric pattern for the viscosity range tested here andconforms to the benchmark pattern.

25

4.4.2 Results and discussionThe behaviour of RMS velocity with respect to time presented in Figure 4.2 exhibits similar

pattern to that obtained in the benchmark for all test cases.

0.2 0.4 0.6 0.8 100

20

40

60

Time

<V

rm

s>

Setup-1Setup-2

0.2 0.4 0.6 0.8 100

0.2

0.4

0.6

0.8

Time

<T>

Setup-1Setup-2

Figure 4.2: top: Time dependence of RMS velocity and bottom: volume-averaged temperaturefor different refinement levels of macro elements, with benchmark steady-state values (Black lines)for iso-viscous (Solid lines) and variable viscous conditions (Dashdotted lines).

26

However, the steady-state values differ by a certain amount in comparison to the referenceresults. Similar behaviour is observed for the volume-averaged temperature. Comparing theresults for different grid refinement levels, we can observe that the fine grid results are bettercompared to the coarse grids in the behaviour of state variables as expected. However, thesteady-state values are still away from the benchmark.

0.2 0.4 0.6 0.8 100

20

40

60

Time

<V

rm

s>

Setup-3Setup-4Setup-5

0.2 0.4 0.6 0.8 100

0.2

0.4

0.6

0.8

Time

<T>

Setup-3Setup-4Setup-5

Figure 4.3: top: Time dependence of RMS velocity and bottom: volume-averaged temperaturefor different velocity update intervals, with benchmark steady-state values (Black lines) for iso-viscous (Solid lines) and variable viscous conditions (Dashdotted lines).

27

In order to delve into the details of this behaviour, simulations based on different velocityupdate timestep intervals, keeping the mesh size fixed, has been performed and the results forSetup-1 is presented in Figure 4.3. In this scenario, the steady-state values for different intervalsetups show similar discrepancy with respect to the benchmark results.

0.2 0.4 0.6 0.8 10

0

0.5

1

1.5

·10−4

Time

r×ω

(u)

Setup-1Setup-2

0.2 0.4 0.6 0.8 10

0

0.5

1

1.5

·10−4

Time

r×ω

(u)

Setup-3Setup-4Setup-5

Figure 4.4: top: Pure rotation in time for different refinement levels of macro elements andbottom: the same for different velocity update intervals for iso-viscous (Solid lines) and variableviscous conditions (Dashdotted lines).

28

In both the scenarios, the curves appear to deviate from the reference results at two notablepositions: first, at the global maximum which is achieved after the onset of initial perturbationsand second at the initial drop of the curve. The deviations are observed only in state variableswhile both of these maintain their behaviour in time. Figure 4.4 shows the values of pure rotationthat was eliminated from the flow velocities of all the test cases. We can observe that their valuesare in the order of 10−4 which is insignificant for the deviations caused. Finer refinements mightyield towards better accuracy, but with higher computational cost, against the benchmark resultwhich was obtained with 32 elements in the radial direction.

29

Chapter 5

Conclusion and Outlook

An evolution of the massively parallel Hierarchical Hybrid Grids framework has been discussedbriefly from the inception to its modern-day applications, one of them is to solve highly non-linearlarge scale geodynamical problems. Besides the versatile nature of HHG, the aspects adopted inthis work are confined to solve Earth mantle convection problems. The results discussed provethe current capability of HHG to solve transient mantle convection problems to a limited extentin comparison with the benchmark study carried out by Zhong et al. [29] and C. Burstedde etal. [14].

Elimination of pure rotation has been successfully implemented to define the flow velocitiesin a non-rotating frame of reference for the whole mantle. Even though the arbitrariness in-troduced by pure rotation is negligibly small in this work, it can be highly influential [29] whencompositional heterogeneities are considered in future works. The exact reason for the deviationscaused is still under investigation, both on the understanding of physics behind and also on theframework level. On the numerical level, further studies concerning the stabilisation parametercould also be one of the possible directions that might lead to achieving the benchmark.

Once the above-discussed benchmark is achieved, naturally, the next step would be to extendthe validations of HHG for solving the same problem with increased non-linearity in viscosity.One possible parameter is the depth of mantle in addition to temperature dependence, wherethe jumps in viscosity while moving from lithosphere to asthenosphere or vice versa are morepronounced. Besides, variable diffusion coefficients can be considered and can be solved with theefficient stencil scaling approach [3], which can accelerate matrix-free finite element implemen-tations to handle parallel computations with more than a hundred billion degrees of freedom.Towards a more realistic approach for testing mantle convection models, modelling past statesof Earth’s mantle and checking them against geological records, like continental-scale uplift andsubsidence, is one of the effective methods. The work in [16] shows that the retrodictions (i.e.,reconstructions of past states of Earth’s mantle obtained using present information) of the mantleflow can be extended significantly from a history of Earth’s surface velocity field which is ob-tained from past plate motion reconstructions for comparable time periods of mantle overturn.This could be another interesting benchmark study to validate HHG in the aspect of mantleconvection.

31

Appendix A

Non-dimensionalisation

Following the new parameters introduced in (2.8), we can define the derivatives of space andtime variable as,

dx = R dx′ → dx

dx′=

1

R; ∇()⇒ 1

R∇(), ∆()⇒ 1

R2∇()

∂t =R2

κ∂t′ → ∂t

∂t′=R2

κ

where variables with a prime(′) are respective quantities in dimensionless paradigm and similarlyintroducing tilde( ) for their derivatives,

∇ · u⇒ κ

R∇ · u′, ∇u⇒ κ

R· 1

R∇u′, ∇P ⇒ κ

R2· 1

R· µr ∇P ′,

∆T ⇒ 1

R· 1

R∇T ′, ∂T

∂t⇒ κ δT

R2

∂T ′

∂t′.

Substituting in (2.1), (2.7), (2.3) gives,

∇ · u′ = 0, in Ω′ × t ′,

− κ

R2

1

Rµr∇ · [µ′(∇u′ + ∇Tu′)] − αρ0g0δTT

′er +κ

R2

1

Rµr∇P ′ = 0, in Ω′ × t ′,

κ

R2δT

∂T ′

∂t′+κ

R

1

RδT (u′∇T ′) =

δT

R2(

k

ρ0Cp∆T ) +

H

Cp, in Ω′ × t ′,

where Ω′and t′ are scaled domain and time respectively. Collecting known constants togetherfinally gives,

∇ · u′ = 0, in Ω′ × t ′,

−∇ · [µ′(∇u′ + ∇Tu′)] + ∇P ′ =αρ0g0δTR

3

µrκT ′er, in Ω′ × t ′,

∂T ′

∂t′+ u′∇T ′ − ∆T =

R2

κδTCpH, in Ω′ × t ′.

Dropping primes, tildes and replacing known constants in mo mentum equation with Ra, con-verted in terms of mantle thickness d by ξ, and in transport equation with γ gives equations(2.9), (2.10), (2.11) respectively.

32

Physical Parameters

Typical average properties to calculate Rayleigh number of Earth’s mantle are based on Section7.07.3.1, "Convective Instability and the Rayleigh Number" of [8].

g0 Acceleration due to gravity 10 m/s−2

R Radius of Earth 6378 km

d Mantle thickness 2870 km

ρ0 Density 4000 kg m−3/s

α Thermal expansion coefficient 3× 10−5 K−1/s

δT Temperature contrast 3000 K/s

µr Dynamic viscosity 1022 Pa s

κ Thermal diffusivity 10−6 m2/s

33

Bibliography

[1] Ivo Babuška. The finite element method with lagrangian multipliers. Numerische Mathe-matik, 20(3):179–192, 1973.

[2] Simon Bauer, Hans-Peter Bunge, Daniel Drzisga, Björn Gmeiner, Markus Huber, LorenzJohn, Marcus Mohr, Ulrich Rüde, Holger Stengel, Christian Waluga, et al. Hybrid paral-lel multigrid methods for geodynamical simulations. In Software for Exascale Computing-SPPEXA 2013-2015, pages 211–235. Springer, 2016.

[3] Simon Bauer, Daniel Drzisga, Marcus Mohr, Ulrich Ruede, Christian Waluga, and BarbaraWohlmuth. A stencil scaling approach for accelerating matrix-free finite element implemen-tations. arXiv preprint arXiv:1709.06793, 2017.

[4] Simon Bauer, Marcus Mohr, Ulrich Rüde, Jens Weismüller, Markus Wittmann, and BarbaraWohlmuth. A two-scale approach for efficient on-the-fly operator assembly in massivelyparallel high performance multigrid codes. Applied Numerical Mathematics, 122:14–38, 2017.

[5] John R Baumgardner and Paul O Frederickson. Icosahedral discretization of the two-sphere.SIAM Journal on Numerical Analysis, 22(6):1107–1115, 1985.

[6] Thorsten W Becker. On the effect of temperature and strain-rate dependent viscosity onglobal mantle flow, net rotation, and plate-driving forces. Geophysical Journal International,167(2):943–957, 2006.

[7] TW Becker. Azimuthal seismic anisotropy constrains net rotation of the lithosphere. Geo-physical Research Letters, 35(5), 2008.

[8] D Bercovici, P Tackley, and Y Ricard. 7.07-the generation of plate tectonics from mantledynamics. Treatise on Geophysics. Elsevier, Oxford, pages 271–318, 2015.

[9] Benjamin Bergen, Tobias Gradl, Ulrich Rüde, and Frank Hülsemann. A massively parallelmultigrid method for finite elements. Computing in science & engineering, 8(6):56–62, 2006.

[10] Benjamin Karl Bergen. Hierarchical Hybrid Grids: Data Structures and Core Algorithmsfor Efficient Finite Element Simulations on Supercomputers: Hierarchische Hybride Gitter:Datenstrukturen und Algorithmen Zur Effizienten Simulation Mit Finiten Elementen AufHöchstleistungsrechnern. SCS Publishing House, 2005.

[11] Benjamin Karl Bergen and Frank Hülsemann. Hierarchical hybrid grids: data structures andcore algorithms for multigrid. Numerical linear algebra with applications, 11(2-3):279–291,2004.

34

[12] Jürgen Bey. Tetrahedral grid refinementverfeinerung von tetraeder-gittern. Computing,55(4):355–378, 1995.

[13] Franco Brezzi. On the existence, uniqueness and approximation of saddle-point problemsarising from lagrangian multipliers. Revue française d’automatique, informatique, rechercheopérationnelle. Analyse numérique, 8(R2):129–151, 1974.

[14] Carsten Burstedde, Georg Stadler, Laura Alisic, Lucas C Wilcox, Eh Tan, Michael Gurnis,and Omar Ghattas. Large-scale adaptive mantle convection simulation. Geophysical JournalInternational, 192(3):889–906, 2013.

[15] Philippe G Ciarlet. The finite element method for elliptic problems. Classics in appliedmathematics, 40:1–511, 2002.

[16] Lorenzo Colli, Hans-Peter Bunge, and Bernhard SA Schuberth. On retrodictions of globalmantle flow with assimilated surface velocities. Geophysical Research Letters, 42(20):8341–8348, 2015.

[17] Howard C Elman and Gene H Golub. Inexact and preconditioned uzawa algorithms forsaddle point problems. SIAM Journal on Numerical Analysis, 31(6):1645–1661, 1994.

[18] Björn Gmeiner, Markus Huber, Lorenz John, Ulrich Rüde, and Barbara Wohlmuth. A quan-titative performance study for stokes solvers at the extreme scale. Journal of ComputationalScience, 17:509–521, 2016.

[19] Björn Gmeiner, Ulrich Rüde, Holger Stengel, Christian Waluga, and Barbara Wohlmuth.Performance and scalability of hierarchical hybrid multigrid solvers for stokes systems. SIAMJournal on Scientific Computing, 37(2):C143–C168, 2015.

[20] Björn Gmeiner, Ulrich Rüde, Holger Stengel, Christian Waluga, and Barbara Wohlmuth.Towards textbook efficiency for parallel multigrid. Numerical Mathematics: Theory, Meth-ods and Applications, 8(1):22–46, 2015.

[21] Björn Gmeiner, Christian Waluga, and B Wohlmuth. Local mass-corrections for continuouspressure approximations of incompressible flow. SIAM Journal on Numerical Analysis,52(6):2931–2956, 2014.

[22] Olga A Ladyzhenskaya. The mathematical theory of viscous incompressible flow, volume 12.Gordon & Breach New York, 1969.

[23] L-N Moresi and VS Solomatov. Numerical investigation of 2d convection with extremelylarge viscosity variations. Physics of Fluids, 7(9):2154–2162, 1995.

[24] Lutz Angermann Peter Knabner. Numerical methods for elliptic and parabolic partial dif-ferential equations. Springer, 2003.

[25] Christian Waluga, Barbara Wohlmuth, and Ulrich Rüde. Mass-corrections for the conserva-tive coupling of flow and transport on collocated meshes. Journal of Computational Physics,305:319–332, 2016.

[26] Jens Weismueller. Development and application of high performance software for mantleconvection modeling. 2016.

35

[27] Shuxia Zhang and Ulrich Christensen. Some effects of lateral viscosity variations on geoidand surface velocities induced by density anomalies in the mantle. Geophysical JournalInternational, 114(3):531–547, 1993.

[28] Shijie Zhong. Role of ocean-continent contrast and continental keels on plate motion,net rotation of lithosphere, and the geoid. Journal of Geophysical Research: Solid Earth,106(B1):703–712, 2001.

[29] Shijie Zhong, Allen McNamara, Eh Tan, Louis Moresi, and Michael Gurnis. A benchmarkstudy on mantle convection in a 3-d spherical shell using citcoms. Geochemistry, Geophysics,Geosystems, 9(10), 2008.

[30] Shijie Zhong, Maria T Zuber, Louis Moresi, and Michael Gurnis. Role of temperature-dependent viscosity and surface plates in spherical shell models of mantle convection. Journalof Geophysical Research: Solid Earth, 105(B5):11063–11082, 2000.

36

List of Figures

3.1 left: Unstructured 2D primal mesh (black) T` with its associated dual mesh (blue)B`, center: A triangular patch with a set of T ∈ Ti ⊆ T` for node i with BT

i ∈ B`,right: one dual cell BT

i ∈ B` of a structurally refined Tetrahedron patch [21,25]. 103.2 left: A Tetrahedron as dual sub cells and right: a control facet associated with

a node x0 and its associated normal [25]. . . . . . . . . . . . . . . . . . . . . . . 113.3 Sample triangulations T of distinctively coloured 10 diamonds with nt = 3, 5, 10 [26]. 133.4 A triangle of a diamond refined tangentially and radially with nt = 5 and nr = 4 [26]. 143.5 3 sub classes of a tetrahedron. Gray class possess same structure as the parent

while green and blue are different and further refinement of these two classes doesnot yield a new class of sub elements [4]. . . . . . . . . . . . . . . . . . . . . . . . 14

3.6 left: 15-point stencil of a low order volume primitive node at T0, right: with itscardinal directions [4]. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

3.7 left: A simplified representation of a vertex, edge and face primitive of T ∈ T0,right: Two neighbouring tetrahedra with their ghost layers [26]. . . . . . . . . . 16

3.8 Ghost layer of a face primitive with 15-point stencil (green) of a node (red) [18]. 173.9 Illustration of parallelisation concept [2]. . . . . . . . . . . . . . . . . . . . . . . . 173.10 A 2D representation of a Macro element mappings with discrete indices (i, j)

clipped on an index plane k. ΨT maps the reference element T (left) to the macroelement T ∈ T` (center), followed by φT which maps T to the blended elementT ∈ T` [4]. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

3.11 Projection of point p on some kth-plane f within a prismoid to p′ of sphericalsub-radial surface of a node q on some radial grid line spanning 0− r − d [26]. . . 19

3.12 left: Earth model with non-projected and right: projected coordinates withplumes arising from the core mantle boundary. The triangular faces of tetrahe-dra elements are more pronounced at the boundary surfaces of the non-projectedmodel. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

4.1 Temperature field at steady-state for Setup-1. Contours of plumes at non-dimensi-onalised temperatures 0 (blue) and 0.5 (gray) are shown. left: iso-viscous model,right: Variable viscosity model a contrast of 20. . . . . . . . . . . . . . . . . . . 25

4.2 top: Time dependence of RMS velocity and bottom: volume-averaged tempera-ture for different refinement levels of macro elements, with benchmark steady-statevalues (Black lines) for iso-viscous (Solid lines) and variable viscous conditions(Dashdotted lines). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

4.3 top: Time dependence of RMS velocity and bottom: volume-averaged temper-ature for different velocity update intervals, with benchmark steady-state values(Black lines) for iso-viscous (Solid lines) and variable viscous conditions (Dash-dotted lines). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

37

4.4 top: Pure rotation in time for different refinement levels of macro elements andbottom: the same for different velocity update intervals for iso-viscous (Solidlines) and variable viscous conditions (Dashdotted lines). . . . . . . . . . . . . . . 28

38

Corrigenda

Following revisions should be considered in the original document.

Page 9:Equation (3.7) should read as,

cT (q`, p`) := αT

∫T∈T`

∇q` · ∇p`, gT (p`) := −αT

∫T∈T`

f · ∇q`,

where αT = βT h2T for some sufficiently large constant βT > 0. The diameter of an element

T is defined as hT = (∫T∈T` dx)1/3. The stabilisation parameter βT has to be chosen carefully in

order to avoid unwanted effects of over-stabilisation, i.e., constraints imposed on the degrees offreedom become unnecessarily strong. This is clearly demonstrated in [21] with an appropriatenumerical example and it is recommended that βT = 1/12 would be a good choice in practice.

The results produced in the original document were based on the corrected version of equation(3.7) above.

39