Embed Size (px)
Citation preview
Dan StanzioneExecutive Director, Texas Advanced Computing Center
March [email protected]
An Update on the Texas Advanced Computing Center
2
The Texas Advanced Computing Center: A World Leader in High Performance Computing
1,000,000x performance increase in UT computing capability in 10 years.
Ranger: 62,976 Processor Cores, 123TB RAM, 579 TeraFlops, Fastest Open Science Machine in the World, 2008
Lonestar: 23,000 processors, 44TB RAM, Shared Mem and GPU subsystems, #25 in the world 2011
Stampede: #7 in the world, Somewhere around half a million processor cores with Intel Sandy Bridge and Intel MIC, Dell: >10 Petaflops.
PS Computation 1Mx – network 100x
3
TACC’s Stampede supports and Incredible Amount of Research
• Through 2 years of production operations, Stampede is remarkably successful.– Over *1.5 Billion* processor hours delivered.– Over 5 million successful jobs
– The Stampede system alone provides time to nearly 2,000 funded projects, at UT and around 400 other institutions.
– About 10,000 students, faculty, and staff use our systems by logging in directly.
– About another 30,000 use the systems indirectly through our web portals.
• Formal requests from the community from XSEDE run ~500% available hours
4
TACC Snapshot Today• Personnel
– ~135 Full time staff (~50 PhD), 15 students
• Funding– Roughly 85% externally funded, through 7 major projects and
more than 20 smaller grants.– TACC Represents (mostly NSF and UT) cumulative investment of
more than $300M in buildings, systems, and staffing
• Facilities– Datacenter upgrade to 10MW completed in 2012.– New office facility to be completed by end of 2015 (accomodating
~150 staff total).
• Services– HPC, HTC, Visualization, Large scale data storage, curation and
analysis, Web portals and Gateways, API design, consulting, etc.
5
[2014 in Review]
6
TACC 2014
• More than *two million* production jobs on Stampede, Lonestar, and Maverick
• 40+ PB of data stored• More than 3,000 tickets! • Countless web hits and API jobs• I’ve lost count of how many trainings and
workshops, but a whole lot (35+?) . • Great service to our 10,000+ users.
7
TACC 2014 (2)
• Successfully transitioned leadership on all our projects, including PI changes on XSEDE, Stampede, Lonestar, iPlant, etc.
• Hired over *35* new people this past year; we are about 25% larger than last year. – Including several new *leaders*.
• John West to lead Strategic Initiatives• Suzanne Pierce to lead Decision Support Systems• (Niall Gaffney joined to lead Data Intensive Computing in
2013).
8
TACC 2014 (3)
• Major new team awards: – Chameleon -- NSF Cloud Testbed– Jetstream – NSF Production Science Cloud– Wrangler Operations and Maintenance Award– Conoco Phillips (now with year 2!)
• Numerous single/small team investigator ones.• ~$25M in projects awarded in 2014. • Received HIPAA/FISMA authority to operate,
which opens up new potential awards.
9
TACC 2014 (4)
• Proposals and New Collaborations: – Submitted 10 proposals over $10M in 2014– In an effort to better leverage TACC on campus, three of the
above were collaborations with UT PIs (one with ICES I hope to announce this week).
– In a fairly radical change for us, three were *not* to the NSF. – Industrial funding topped $1.5M ($250k 3 years ago).
• Awards:– IDC HPC Innovation Excellence Award: Gabriel Sawakuchi,
MD Anderson, Radiation Physics app on Lonestar. – HPCWire Editor’s Choice Award for Top Supercomputing
Achievement: NREL and TACC for work on Stampede in converting Biomass to fuels.
10
TACC 2014 (5)• New systems in production:
– Stockyard (global filesystem)– Maverick (interactive visualization and data)– Rustler (Hadoop)– Ranch upgrades (tape archive).
• New building designed and approved. • First foray into “Advanced Reasoning Systems” , • Started operations on the Digital Preservation Network• Launched a new extensible portal for the Arabadopsis
Informatics Portal• New website rolled out/New industrial partners• Launched new undergraduate research program, and new
high school programming camp starts this summer.
11
[New Things to Come]
12
New Building Update
13
New Building Update
• Design phase completed late last year. – Full design and costs received final approval at
November regents meeting, groundbreaking in December.
• Groundbreaking February 2015• Expect to occupy by early 2016• 65 new offices, new visualization facility, build
and test facility, auditorium, and training spaces.
14
New Systems in 2015• Wrangler, our first “data intensive supercomputer”, is
in early user mode, with full production coming this summer.
• Lonestar-5 will be deployed in late May– All Texas users– 30,000+ cores– First Cray at TACC in 12+ years.
• Chameleon deployment in April – Partnership with U. of Chicago/Argonne– Our first machine focused on computer science research– A testbed for Future cloud systems
15
New Systems in 2015 (2)• Jetstream – Deployment in September
– NSF-funded ($6.5M), in partnership with Indiana and U. of Arizona
– A production cloud for science• Virtual machines• An available library of images• Leveraging software we built for iPlant
• Soon to be announced machine learning-focused testbed system with Microsoft (July)
• Soon to be announced HVDC system with HP and NTT Facilities (will run on solar part-time) (November).
16
Future Projects
• Stampede upgrade late 2015/early 2016– Pending final NSF approval– Add 500 “Knights Landing” Xeon Phi nodes.
• Stampede-2 – Based on performance, we were invited to submit
a renewal proposal for Stampede without an open competition.
– Proposal was submitted March 18th ($30M).
17
Focus this year: As you might gather from new awards…
• Data– Wrangler gives us unique capabilities– Hoping to expand through nascent “National Data Service” with
NCSA and SDSC.
• Cloud Computing– With Chameleon, Rodeo, and Jetstream, not to mention tens of
thousands of national users on Stampede, we are the place for cloud research and a huge cloud resource for science.
– Invited Software Institute proposal in Science Gateways (SDSC lead, IU, Purdue, TACC)
• Life Sciences – Completing HIPAA/FISMA/FedRamp compliance
• Leveraging our (undercapitalized) strengths in software, and portals (who is aware of drugdiscovery.tacc.utexas.edu ?).
18
We build more software than you think…
19
TACC is a Software House
• While we weren’t looking: – TACC has 19 staff engaged in monitoring,
deploying, and administering systems– TACC has nearly 60 staff in some form of software
work, including 18 web portal designers (also code tuning, scalable algorithms, informatics, etc). There are opportunities in being a great place for scientific software.
20
Today, TACC Provides a Comprehensive Computational Science Ecosystem
• Computation: Stampede, Lonestar• Visualization: Maverick, Stallion.• Storage and Archive: Corral, Ranch
(160 PB) • Cloud: Rodeo, Chameleon, Jetstream
• Data Driven Computing: Wrangler, Rustler
• Reasoning Systems: Yellowhat• Connectivity: 100Gb/s to Internet2• Tools, APIs, Algorithms, Consulting,
and Team
21
The Ecosystem isn’t Just Hardware…
• Leveraging iPlant, we are one of the lead partners in a new $9M Arabadopsis Informatics Portal award.
• We received our first funding from USDA, and added several new awards from NIH in things as diverse as “Dynamic changes in the chick developing heart in response to altered hemodynamics”
• We launched a number of project in visualization, from fundamental research in techniques, to end user portals
• New awards in parallel programming models, and in enhancing our performance monitoring frameworks.
22
Why Do We Keep Growing?
• Bigger Systems for Bigger Science– Success in our core mission of being able to tackle the
most challenging problems in science and engineering.
• More Kinds of System – We lead not just in HPC, but now deploy systems for
“Big Data”, cloud computing, and new experimental technologies.
• More Kinds of Users– We now provide an array of software, systems and
services for all sorts of disciplines that are coming to advanced computing, particularly in BioMedical Research
23
Recent Science Highlights
"It is a really exciting time for the field of cosmology. We are now ready to collect, simulate and analyze the next level of precision data...there's more to high performance computing science than we have yet accomplished.”
Astronomer and Nobel Laureate Saul Perlmutter,Supercomputing '13 keynote address
Heavy Metal in the Early Cosmos
• Researchers UT-Austin used Stampede to perform ab initio simulations refining how the first galaxies are formed, and how metals in stellar nurseries influenced characteristics of first stars in the galaxy.
• These simulations are making predictions that can be validated in 2018 by the James Webb Space Telescope (and in fact determine how JWST is used).
24
Stampede Recent Science Highlights
“Technip has one of the largest computer clusters among engineering companies, but the simulations would take weeks to complete”, said Jang Kim, Chief Technical Advisor at Technip. “Stampede has allowed us to run simulations in one day or even overnight. We are then able to get these safer designs out into use faster.”
Design of Offshore Oil Platforms
• An industrial partner, Technip, used Stampede to run full-scale simulations using a numerical wave basin to design offshore floating oil platforms.
• Technip’s business is to design, construct and install offshore platforms for major oil companies such as BP, Shell, Chevron, and ExxonMobil.
• Modeling has replaced wave tank tests that take up to a year to perform.
25
Recent Science Highlights
Journal of Chemical Information and Modeling
“Identification of a Novel Inhibitor of Dengue Virus Protease through Use of a Virtual Screening Drug Discovery Web Portal”
Usha Viswanathan †, Suzanne M. Tomlinson †, John M. Fonner ‡, Stephen A. Mock ‡, and Stanley J. Watowich
TACC Drug Discovery Portal• In collaboration with Stanley Watowich at
UTMB, TACC developed a web portal to provide a graphical interface for conducting a screen for identifying small molecules that bind to a target protein. – Users can screen against libraries of ligands up
to 642k in size. – Basic screening is done, and results are
collated and best matches returned in a simple download.
• 70 researchers have used more than 5 million hours on Lonestar through the portal
• “We report the discovery of a novel small-molecule inhibitor of the dengue virus (DENV) protease (NS2B-NS3pro) using a newly constructed Web-based portal (DrugDiscovery@TACC) for structure-based virtual screening.”
VDJ Server
• Web portal for Immune Repertoire Analysis– Uses the Corral data storage
system and Lonestar supercomputer for data analysis
– Provides API access to all the tools and services for advanced users
• The UI is designed to foster collaboration, metadata capture, and reproducibility
• Led by Lindsay Cowell from UT Southwestern
• www.vdjserver.org
Stroke Imaging Repository Consortium
• TACC is working with Steve Warach at UTSW Seton (soon UT-Austin Medical) to store and organize the STIR/EPITHET datasets
• Seeks to predict outcomes of reperfusion therapy for Accurate and timely diagnoses and treatment in patients with acute ischemic stroke.
• Identifying quantitative image-based biomarkers
• Using the Corral data storage system and Lonestar supercomputer for data analysis
28
Recent Science Highlights
The gene sequencing data size would easily be 1000-fold larger than the microarray data in the reported study, which means the need to use TACC's Stampede supercomputing cluster for number crunching is even more eminent.”
- Stephen Wong,
Houston Methodist Research Institute
A link between Alzheimer’s and Cancer• A team led by Houston Methodist Hospital
(with reesearchers in Harvard, Taiwan, and Italy) used Stampede to find a link between Alzheimer’s and GBM, one of the most aggressive forms of brain cancer.
• This systems biology approach has uncovered linked signaling pathways, and identified 15 gene ontology terms relating the diseases.
"This work of Dr. Wong's is quite exciting in that it shows connections between two of the most intractable diseases in society. And while our focus is on cancer, the great hope is that as we make these connections we can leverage that to find new targets and opportunities that can provide meaningful intervention for either disease."
- Dan Gallahan,NIH, deputy director,
National Cancer Institute
