http://www.jamstec.go.jp

The Earth Simulator Current Status

Ken’ichi Itakura(Earth Simulator Center, JAMSTEC)

SC13Nov. 2013

Nov 2013 SC13 NEC BOOTH PRESENTATION 1

Research Sector(700)

Development and

Promotion Sector(300)

ManagementSector (100)

Research Institute for Global Change

Institute for Research on Earth Evolution Institute of Biogeosciences

Advanced Research and Technology Promotion Department

Planning Dept. Administration Dept. Finance and Contracts Dept.

Safety and Environment Management Office Audit and Compliance Office

Kochi Institute for Core Sample Research

Laboratory System Research Support Department

Leading Project

Center for Deep Earth ExplorationEarth Simulator CenterMarine Technology

Center

Mutsu Institute for Oceanography

Data Research Center for Marine-Earth Sciences

JAMSTEC OrganizationJapan Agency for Marine-Earth Science and Technology

Nov 2013 SC13 NEC BOOTH PRESENTATION 2

Yokohama

Earth Simulator Site

Tokyo

Yokosuka Headquarter

Location of Earth Simulator Facilities

Nov 2013 SC13 NEC BOOTH PRESENTATION 3

Earth SimulatorMarch, 2002 ～ March.2009 (1/2 : ～ Sep.2008)

Peak Performance ： 40 T Flops

Main Memory ： 10 T Bytes

Earth Simulator（ES2）

March, 2009 ～

Peak Performance ： 131 T Flops

Main Memory ： 20 T Bytes

Earth Simulator

Nov 2013 SC13 NEC BOOTH PRESENTATION 4

Operation Network

Maintenance Network

User Network

Labs, User Terminals

Processor nodes (PNs)Login server

Operation Servers

Storage server

FC Switch

４Gbps Fiber ChannelSAN (Storage Area Network)

Usable Capacity 1.5PBRAID6 HDD

SX-9/E 160 processor nodes (PNs)（including interactive 2nodes)

Total peak performance 131TFLOPSTotal main memory 20TBData storage system 500TBInter nodes network (IXS) 64GB/sec

(bidirectional) /node

10GbE(partially link aggregation 40GbE)

NQS2 batch job system on PNsAgent request supportUse statistical and resource information managementAutomatic power saving management

FC SwitchFC Switch

ES2 System outline

Maximum power consumption 3000kVANov 2013 SC13 NEC BOOTH PRESENTATION 5

2002Jun

2002Nov

2003Jun

2003Nov

2004Jun

2004Nov

2005Jun

2005Nov

2006Jun

2006Nov

2007Jun

2007Nov

2008Jun

2008Nov

2009Jun

2009Nov

2010Jun

2010Nov

2011Jun

2011Nov

2012Jun

2012Nov

系列1 1 1 1 1 1 3 4 7 10 14 20 30 49 73 22 31 37 54 68 94 145 217

ES

９4th

145th

217th

ES222nd

217th in November 2012 (18th in Japan)

Peak performance ratio is 93.38% (2nd in the world)

TOP500 Rank

Nov 2013 SC13 NEC BOOTH PRESENTATION 6

2013 Jun342nd

2013 Nov472nd

System TOP500（Linpack） #cores G‐FFT Effective Ratio #cores

JAMSTEC  ES2 (SX‐9)

Rank 217

122  TF 1,280 Rank 3

11.9  TF 9.1 % 1,280

RIKEN AICS(K Computer)

Rank  3

10,510  TF 705,024 Rank 1

205.9  TF 1.94  % 663,552

Ratio x 86 x 550 x 17.3 x 4.7 x 519

HPC Challenge Awards Class1Best performance on a base or optimized run submitted to the HPC Challenge website. The benchmarks to be judged are: Global HPL, Global RandomAccess, EP STREAM (Triad) per system and Global FFT.

http://www.hpcchallenge.org/

HPC Challenge Benchmark

Nov 2013 SC13 NEC BOOTH PRESENTATION 7

Projects FY2013

Total resource 1,300,000 node・hourL batch nodes 156 * 24 hour * (365 – 17.7 maintenance days)

Contract Research Projects 30%

Proposed Research Projects 30%JAMSTEC Research Projects 40%

Earth Science30% (24 Projects)

Program for Risk Information on Climate Change (SOUSEI) 27% (3 themes)

Funded researchandIndustrial Use

Research projects organized by JAMSTECand Collaboration Projects

The Strategic Industrial Use3%

Nov 2013 SC13 NEC BOOTH PRESENTATION 8

515 Users137 Organizations (January 2013)64 Universities, 15 Labs, 41 Companies, 17 Overseas

0

50

100

150

200

250

2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012

海外

民間企業

公的機関

大学UniversitiesLabsCompaniesOverseas

Users

Nov 2013 SC13 NEC BOOTH PRESENTATION 9

October 31th, 2013

Over 90% operation was

achieved.

Operation Results

Nov 2013 SC13 NEC BOOTH PRESENTATION 10

Post-ES2 system

Nov 2013 SC13 NEC BOOTH PRESENTATION 11

The Earth Simulator RoadmapFY 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018 2019 2020

ES

ES2

FutureSystem

6 years operation from March 2009 to February 2015

Half system operation, data migration

Operation until March 2009

3rd system  (5 years?)

4th systemFS target

Nov 2013 SC13 NEC BOOTH PRESENTATION 12

Positioning of ES in JAMSTEC long-term vision

Verification and accuracy improvement of modeling

Observational data of the real world

One employment of observation and a simulation is a strong point of JAMSTEC.

long-term vision (Next 15 years)(i) Analysis and prediction of global environment change (ii) Plate dynamics and disaster prevention research of an earthquake and tsunami(iii) Evolution of a life, and the history of a sea life (iv) Resource engineering of a material and energy

Nov 2013 SC13 NEC BOOTH PRESENTATION 13

Science Targets for the post-ES2• Global environment change

– Submesoscale (ensemble calculations)– Extreme Events Prediction, Guerrilla rainstorm, urban heat island– Nesting/downscaling– data assimilation

• Earthquake, Tsunami– Seismic tomography, plate tectonics, database of earthquake scenarios– Information of the optimal evacuation route/place from tsunami

• Geo-marine-bio science– Life model building which took in the information on a meta-genome

• High technology development– High resolution analysis of VIV（ Vortex Induced Vibration) problem with long cable– The on-the-ocean operation real-time supporting system of a vessel– Planning of the energy-saving efficient operation schedule of a vessel

New

New

Evolution of a life, and the history of a sea life

Resource engineering of a material and energy

Nov 2013 SC13 NEC BOOTH PRESENTATION 14

Fig.1 Calculation Power and Memory Capacity Fig.2 Architecture classification

There are many users who are thinking the B/F performance as important and this can be called common view from the position of promoting earth science.

In order to advance main researches of the earth science field in post-ES2, (1) 10 times or more faster calculation performance,(2) and 5 times or 10 bigger memory capacityare need on the assumption that resource allocation and execution efficiency comparable as ES2.

Relative Peak Performance

(v.s. ES2)

Relative MemoryCapacity(v.s. ES2)

High Peak Performance

B/FThroughput

Request from JAMSTEC Researchers

Nov 2013 SC13 NEC BOOTH PRESENTATION 15

Key point of post-ES2• Peak Performance: 1.3～13PFlops (10 times or more faster than ES2)

• Memory Capacity: 100～200TB (5 times or 10 times bigger than ES2)

• Storage Capacity: 10～20PB (5 times or 10 times bigger than ES2)

• High data bandwidth between CPU and main memory (High B/F)

• Cooperation with other small systems

• Shared memory based storage

• Cooperation job scheduler system

The most important problem is a benchmark program for an

assessment system. Communication with a main user is required.

Nov 2013 SC13 NEC BOOTH PRESENTATION 16

Summary• ES2 current operation

• 6 years operation from March 2009 to February 2015

• Top 500 rank :217th and HPCC G-FFT: 3rd in November 2012

• Calculation resource is centralized on earth science.

• Future Plan(1) Feasibility Study of exa-scale system in 2018 or later.

(2) Post-ES2 system in March 2015.

• Peak Performance: 1.3～13PFlops (10 times or more faster than ES2)

• Memory Capacity: 100～200TB (5 times or 10 times bigger than ES2)

• High data bandwidth between CPU and main memory (High B/F)

• Cooperation with other small systems

• The main user required benchmark programs for an assessment system are need.

Nov 2013 SC13 NEC BOOTH PRESENTATION 17

Thank you for your kind attention !Thank you for your kind attention !

JAMSTECJAMSTEC

Japan Agency for Marine-Earth Science and TechnologyJapan Agency for Marine-Earth Science and Technology

Visit JAMSTEC at booth #2545

Nov 2013 SC13 NEC BOOTH PRESENTATION 18