Upload
phungcong
View
216
Download
0
Embed Size (px)
Citation preview
1|P a g e
KeyInformationandNumbers–CERNDataCentre Asof1stDec2017_V1
GeneralInformationWebsites
Latestnumbers:http://monit-grafana-displays.cern.ch:3000/dashboard/db/it-overviewHowtousetheDataCentrevisitpoint:http://www.cern.ch/information-technology/about/computer-centre/visits/visitpoint
Keyfactsandnumbers
MeyrinDataCentre
WignerExtension TOTAL
Servers 11300 3500 14800=>15K Processorcores 170000 56000 226000=>230K
Disks 61800 29700
91500=>90K(units)(about200PBcapacitybutdataiscopiedtwiceondisks)
TapeCartridges 28800=>29K(units)(about400PBcapacity)
Shortoverview
TheCERNdatacentreistheheartofCERN’sentirescientific,administrative,andcomputinginfrastructure.Allservices,includingemail,scientificdatamanagementandvideoconferencinguseequipmentbasedinthedatacentre.AremoteextensionoftheCERNdatacentreishostedattheWignerResearchCentreforPhysicsinBudapest,Hungary,1800kmaway.ItprovidestheextracomputingpowerrequiredtocoverCERN’sneeds.The230000processorcoresand15000serversrun24/7.On29June2017,theCERNDataCentrepassedthemilestoneof200petabytesofdatapermanentlyarchivedinitstapelibraries.NewrecordinNovember2017:12.3petabytesofdatawerewrittenontapethatmonth.Usuallymorethan2PBarebeingaccessed(readandwrite)everyday.TheLHCdataareaggregatedintheCERNDataCentre,whereinitialdatareconstructionisperformed,andacopyisarchivedtolong-termtapestorage.Anothercopyissenttoseverallarge-scaledatacentresaroundtheworld.Subsequentlyhundredsofthousandsof
2|P a g e
computersfromaroundtheworldcomeintoaction:harnessedinadistributedcomputingservice,theyformtheWorldwideLHCComputingGrid(WLCG).
STORAGE:Websites:
www.cern.ch/eosMagnetictapesareusedasthemainlong-termstoragemedium.TheyarestoredintapelibrarieswhereroboticarmswhichWehavemanytaperobotstoensureefficientstorageandretrievalofdata.
- TheamountofdatarecordedontapeatCERNissteadilyincreasingovertime,withthistrendacceleratingovertime(+40%datastoredin2016comparedtowhathadbeenaccumulatedbytheendof2015).
- In2016,morethan49petabytesofLHCdatawererecordedattheCERNDataCentre(withastriking11petabytesinthemonthofJulyalone).Intotal,73PBofdatawererecordedontapes(fromLHCandnonLHCexperiments,etc.).
- Asoftoday,about28PBhavebeenstoredontapesin2017(fromallsources).- Accessingtapedataisrelativelyslow,about1-3minutesfromtapebeinglocated,mounted,
readanddatasent.Oftenphysicistsneedtoaccessthelatestdataimmediately,soitisalsomadeavailableondiskservers,whereaccesstimeissignificantlyfaster.
- Therearecurrently90,000disksintheCERNdatacentrewithacapacityofabout200PB- http://home.cern/about/updates/2017/07/cern-data-centre-passes-200-petabyte-
milestone:On29June2017,theCERNDataCentrepassedthemilestoneof200petabytesofdatapermanentlyarchivedinitstapelibraries.Wheredothesedatacomefrom?ParticlescollideintheLargeHadronCollider(LHC)detectorsapproximately1billiontimespersecond,generatingaboutonepetabyteofcollisiondatapersecond.However,suchquantitiesofdataareimpossibleforcurrentcomputingsystemstorecordandtheyarehencefilteredbytheexperiments,keepingonlythemost“interesting”ones.ThefilteredLHCdataarethenaggregatedintheCERNDataCentre,whereinitialdatareconstructionisperformed,andwhereacopyisarchivedtolong-termtapestorage.Evenafterthedrasticdatareductionperformedbytheexperiments,theCERNDataCentreprocessesonaverageonepetabyteofdataperday.
- Asoftoday,thereare214PBontapesandthetotalcapacityontapesreaches400PB(thiscanbeextendedasmuchasneededthough).
- Asoftoday,wehave145PBondisksandatotalcapacityof200PBondisks.However,asdisksarenotreliable(athousandtimeslessthantapes,30disksfaileachweekinthedata
3|P a g e
centre),wealwayscopytwicethedataondisksothatifadiskfailswedonotlosedata=>soonly72.5PBof‘real’dataisondisksatthemomentandthe‘real’capacityisabout100PB.
- TheCERNstoragesystem,EOS,wascreatedfortheextremeLHCcomputingrequirements.In2016,EOSinstancesatCERNapproachedonebillionfiles,matchingtheexceptionalperformancesoftheLHCmachineandexperiments.
ELECTRICITY:- 2.7megawattcomputingpowerconsumption(coolingnotincludedandrepresentsroughly
oneadditionalmegawatt)fromamaximumof3.5megawatt- DatacentreprotectedbyUPS(UninterruptablePowerSupply)=>Allowstostartdiesel
generatorsforcriticalsystemsandprovidestimetoshutdownnon-criticalsystems- Theelectricalinfrastructureisavitalelementofthedatacentre.Strategiesforincreasing
powerefficiencyarepermanentlyinvestigatedtobeabletomaximisethecomputingpowerservingtheCERN’sinfrastructureandscientificprogrammewhilststayingwithinthe3.5megawattelectricalcapacityenvelopeavailableinthedatacentre.
- Inthecaseofamajorelectricalcut,UninterruptiblePowerSupplies(UPS)providetimeforallthenon-criticalsystemstobeproperlyshutdownandacombinationofUPSsystemsanddieselgeneratorsensurethatthecriticalserviceskeepworking.
COOLING:- About1.4megawattisdedicatedtothecoolingofthedatacentre.- MainMachineroom:chilledairviasilverducts,intofalsefloorandthenintoclosedserver
aisles.Wealsousewater-cooledracksinsomeareas(basement:‘vault’).Efficientcoolingisakeyelementofthedatacentre.Whentheoutsidetemperatureislow,aircanbeusedtocooltheservers,otherwisechillersystemsareusedtocoolthedatacentreair.Coldairisdistributedviathesilverductsonthesidesofthedatacentreroom.Itthengoesunderthefalsefloorandintotheclosedserveraislesthroughtheperforatedfloortiles,tobedrawnfinallythroughtheserverstocoolthem.Someserversarewater-cooledusingactiveorpassiveheatexchangersintheirreardoors,providingahighercoolingcapacityperrack.
WLCG–WorldwideLHCComputingGridWebsites:
www.cern.ch/wlcgwww.cern.ch/wlcg-public
Keyfactsandnumbers:●Morethan170datacentresin42countrieswithmorethan600,000cpucores●CERNprovidesabout20%oftheWLCGresources●Allowsmorethan10,000physiciststoaccessLHCdata●>250,000jobsrunconcurrentlyontheGrid●In2016,globaltransferrateshaveregularlyexceeded35GB/s
Shortoverview:
4|P a g e
Today,theWorldwideLHCComputingGrid(WLCG)combinesthecomputingresourcesofmorethan600,000coresfromover170sitesin42countries,producingamassivedistributedcomputinginfrastructurethatprovidesmorethan10000physicistsaroundtheworldwithnearreal-timeaccesstoLHCdata,andthepowertoprocessit.TheWorldwideLHCComputingGridrunsover2,000,000tasksperdayandin2016,globaltransferrateshaveregularlyexceeded35GB/s.Thesenumberswillincreaseastimegoesonandascomputingresourcesandnewtechnologiesbecomeevermoreavailableacrosstheworld.CERNprovidesabout20%oftheresourcesofWLCG.
NETWORK:Webpage:
https://netstat.cern.ch/monitoring/network-statistics/ Keyfactsandnumbers:
- 35,000kmofopticalfibreneededtotransferlargeamountofLHCdatafromtheLHCexperimentstotheMeyrindatacentre.
- NetworkconnectionsforLHCdata:
- Three100GigabitpersecondcircuitsconnecttheMeyrinDataCentreanditsextensionin
Hungary(cf.below).
5|P a g e
- In2016,thedatatransferratesaroundtheglobealsoreachednewpeakrates–between30
and40gigabytespersecondcontinuousrates,aroundafactoroftwohigherthanhadbeentypicalduringRun1.
DataPreservation:Cf.computingchapteroftheCERNannualreport2016:https://e-publishing.cern.ch/index.php/Annual_Report/article/view/496/354
OpenSource:
Cf.computingchapteroftheCERNannualreport2016:https://e-publishing.cern.ch/index.php/Annual_Report/article/view/496/354
LHC@home:www.cern.ch/lhcathomeVolunteercomputingfortheLHC
CERNopenlab:www.cern.ch/openlabCERNopenlabisauniquepublic-privatepartnershipthatacceleratesthedevelopmentofcutting-edgesolutionsfortheworldwideLHCcommunityandwiderscientificresearch.ThroughCERNopenlab,CERNcollaborateswithleadingICTcompaniesandresearchinstitutes.EUprojects:www.cern.ch/information-technology/about/projects/eu/eu-funded-projectsListofthecurrentEUprojectshttp://information-technology.web.cern.ch/about/projects/eu/current/projectsUNOSAT:https://unitar.org/unosat/CERN’spowerfulITinfrastructureisusefulforfieldsotherthanfundamentalresearch.For15years,UNOSAThasbeenusingtheLaboratory’scomputingcentreinfrastructureforthepurposesofitshumanitarianwork.
6|P a g e
Toknowmore:http://home.cern/about/updates/2016/10/unosat-15-years-humanitarian-mappinghttp://cds.cern.ch/record/2223516?ln=enTHEPAST:
• Workinprogress,aguideforguidesaboutthehistoryoftheDataCentreisbeingprepared.• Thefirstsectionofthisvideoisgivesabriefoverviewofthehistoryofthedatacentre(but
therestofthevideoisnotanymoreuptodateintermsofnumbers,etc.):CERNITin8minutes(2013)https://cds.cern.ch/record/1604210
THEFUTURE:
• Workinprogress(thissectionwillbefilledatalaterstage)Thankyoutoallthepeoplewhohaveproducedmaterialthathasbeenincluded/copypastedinthisdocument.PleasedonothesitatetomodifyasappropriatethisdocumentwiththetrackchangesfeatureonandtosenditbacktoMélissaGaillardsothatthemodificationsaretakenintoaccountinthenextversion(thisdocumentwillbeupdatedonamonthlybasisandmadeavailableontheITwebsiteandaspartoftheITMMminutesonamonthlybasis).