Upload
others
View
4
Download
0
Embed Size (px)
Citation preview
SAY-Go:TowardsTransparentandSeamlessStorage-As-You-GowithPersistentMemory
Hyeonho Song,SamH.NohUNIST
HotStorage2018
§ PersistentMemory
§ Motivation
§ SAY-Go• Design• Implementation
§ Evaluation
§ Summary
§ Futurework
Contents
1
Persistentmemory
PRAM
3DXPoint STT-MRAM
MemoryStorage
• Non-volatility• Byte-levelrandomaccess• Fastaccesstime(nanoseconds)
PersistentMemoryFeatures
2
Persistentmemory
Intel®Optane SSDDCP4800X
Intel®OptaneMemory
DIMMBasedOptane memory
1H’2017 Comingsoon2H’20181H’2017
Blockdevice Blockdevice Characterdevice
3
Persistentmemory
Intel®Optane SSDDCP4800X
Intel®OptaneMemory
DIMMBasedOptane memory
1H’2017 Comingsoon2H’20181H’2017
Blockdevice Blockdevice Characterdevice
4
Ourtarget
Persistentmemory
CPU
DRAM
PM
CPU
DRAM
traditionalHDD,SSD
PM(cache)
Intel®Optane SSDDCP4800X
Intel®OptaneMemory
DIMMBasedOptane memory
1H’2017 Comingsoon2H’20181H’2017
5
CPU
DRAM
traditionalHDD,SSD
PM
Motivation
SFS
VFS
Devicedrivermemorymanager
Virtualmemorymanager
DRAM storage
6
Applicationsdemandmoreandmorememory
Motivation
SFS
VFS
Devicedrivermemorymanager
Virtualmemorymanager
fullyusedused
free
7
DRAMcapacityneverenough!
storageisrelativelyroomyduetohighdensity
Traditionalsolution
SFS
VFS
Devicedrivermemorymanager
Virtualmemorymanager
fullyusedused
free
8
DRAMcapacityisnotenough!
Swapmechanismwithstorage
§ WhenDRAMrunsout
Swapmechanism
Mainmemory
DRAM
Storagefullyused swap
area freeblocks
Userapplication
9
traditionalHDD
pageallocationisnotpossible
datasave(copy)toswaparea
§ SecurefreespaceinDRAM
Swapmechanism: swap-out
Mainmemory
DRAM
Storagefullyused swap
area freeblocks
Userapplication
10
traditionalHDD
Mainmemory
§ SecurefreespaceinDRAM
Swapmechanism: swap-out
DRAM
Storagenotfull swap
area freeblocks
Userapplication
11
traditionalHDD
pageallocationispossible
§ Whenapplicationwantstoaccessdatainswap area
Swapmechanism: swap-in
Mainmemory
DRAM
Storagenotfull swap
area freeblocks
Userapplication
12
traditionalHDD
§ Whenapplicationwantstoaccessdatainswaparea
Swapmechanism:swap-in
Mainmemory
DRAM
Storagenotfull swap
area freeblocks
Userapplication
13
traditionalHDD
datarestore(copy)tofreespaceinDRAM
blockdeviceisnotdirectlyaccessiblebytheCPU
§ Whenapplicationwantstoaccessdatainswaparea
Swapmechanism:swap-in
Mainmemory
DRAM
Storagenotfull swap
area freeblocks
Userapplication
14
traditionalHDD
accesstodata
§ Largeoverheadwithslowstorage
• Datacopyisessential
• Selectingpagestobeswappedout
isacomplexprocess
§ Swapareacannotbeusedasstoragespace
• Staticpartitionoffilesystem
Weaknessoftraditionalswapmechanism
15
Mainmemory
DRAM
Storagenotfull swap
area freeblocks
Userapplication
traditionalHDD
swap-out
swap-in
Simplesolution:swapwithPMstorage
SFS
VFS
Devicedrivermemorymanager
Virtualmemorymanager
fullyused
16
DIMMBased PM
§ PMperformanceisbetterthantraditionalstorage• Reduceddatacopytime• Overheadoftheswapmechanismisalsoreduced
§ Doesthissuffice?
Thereshouldbeabetterway?
SFS
VFS
Devicedrivermemorymanager
Virtualmemorymanager
fullyused
17
DIMMBased PM
§ PMstoragecanbeaccesseddirectlyfromtheCPU
§ PMasbothmainmemoryandstorage• DynamicallyusePM• Increasecapacityoftheworkingmemorywhenneeded
§ Dynamicadjustmentofthememoryandstorageboundary
OurGoal
User
DRAM traditionalHDD,SSD
18
SFS
VFS
Devicedrivermemorymanager
Virtualmemorymanager
User
DRAM
SFS
VFS
Devicedriver
Virtualmemorymanager
PM
memorymanager
SAY-Gosystem(Storage-As-You-Go)
dynamicusageofPM
§ Dynamicadjustmentofthememoryandstorageboundary
OurGoal
User
DRAM traditionalHDD,SSD
19
SFS
VFS
Devicedrivermemorymanager
Virtualmemorymanager
User
DRAM
SFS
VFS
Devicedriver
Virtualmemorymanager
PM
memorymanager
SAY-Gosystem(Storage-As-You-Go)
dynamicusageofPM
similarattemptswereproposed
§ DynamicusageofPM
Relatedwork
20
User
DRAM
SFS
VFS
Devicedriver
Virtualmemorymanager
PM
memorymanager 1) Jung,J.Y.andCho,S.“Memorage:Emergingpersistentrambasedmalleablemain
memoryandstoragearchitecture.” InProceedingsoftheInternationalACMConference
onInternationalConferenceonSupercomputing(ICS)(2013).
2) Kannan,S.,Ada,G.,andKarsten,S.“pVM:persistentvirtualmemoryforefficientcapacity
scalingandobjectstorage.”InProceedingsoftheEuropeanConferenceonComputer
Systems(EuroSys)(2016).
Memorage1) – ICS2013: Workingmemoryexpansion
pVM2) – EuroSys 2016:DirectallocationofPMpagetoworkingmemory
§ DynamicusageofPM
Relatedwork
21
User
DRAM
SFS
VFS
Devicedriver
Virtualmemorymanager
PM
memorymanager
Memorage1) – ICS2013: Workingmemoryexpansion
pVM2) – EuroSys 2016:DirectallocationofPMpagetoworkingmemory
1) Jung,J.Y.andCho,S.“Memorage:Emergingpersistentrambasedmalleablemain
memoryandstoragearchitecture.” InProceedingsoftheInternationalACMConference
onInternationalConferenceonSupercomputing(ICS)(2013).
2) Kannan,S.,Ada,G.,andKarsten,S.“pVM:persistentvirtualmemoryforefficientcapacity
scalingandobjectstorage.”InProceedingsoftheEuropeanConferenceonComputer
Systems(EuroSys)(2016).
§ WhenDRAMrunsout
Memorage
22
Mainmemory
DRAM
fullyused
Userapplicationpageallocationisnotpossible
Storage
freeblocks
PM
§ Workingmemoryexpansion
Memorage
23
Mainmemory
DRAM
Storagefullyused freeblocks
Userapplication
PM
1.selectfreeblockstobeusedinfilesystem
§ Workingmemoryexpansion
Memorage
24
Mainmemory
DRAM
Storagefullyused freeblocks
Userapplication
PM
2.blocksaretransformedintoallocatablefreepagesbyreorganizingthemasdatastructuresusedintheOSmemorymanagementlayer
1.selectfreeblockstobeusedinfilesystem
expand
§ Workingmemoryexpansion
Memorage
25
Mainmemory
DRAM
Storagefullyused freeblocks
Userapplication
PM
3. setthatregiontoadditionalmemoryusinghot-plugfeatureofOS
2.blocksaretransformedintoallocatablefreepagesbyreorganizingthemasdatastructuresusedintheOSmemorymanagementlayer
1.selectfreeblockstobeusedinfilesystem
expand
§ Workingmemoryexpansion
Memorage
26
Mainmemory
DRAM
Storagefullyused freeblocks
Userapplication
PM
pageallocationbecomespossible
§ Largeruntimeoverhead• Processofchangingfreeblocks
intopagesproceedsatruntime
§ Memoryhot-plugfeatureisrequired
§ Overlookstheissueofconsistency• Consistencyofpageallocatorisnotconsidered• Memoryleakcanoccurwithfaults
WeaknessofMemorage
27
blocks->pages
§ DynamicusageofPM
Relatedwork
28
User
DRAM
SFS
VFS
Devicedriver
Virtualmemorymanager
PM
memorymanager
Memorage1) – ICS2013: Workingmemoryexpansion
pVM2) – EuroSys 2016:DirectallocationofPMpagetoworkingmemory
1) Jung,J.Y.andCho,S.“Memorage:Emergingpersistentrambasedmalleablemain
memoryandstoragearchitecture.” InProceedingsoftheInternationalACMConference
onInternationalConferenceonSupercomputing(ICS)(2013).
2) Kannan,S.,Ada,G.,andKarsten,S.“pVM:persistentvirtualmemoryforefficientcapacity
scalingandobjectstorage.”InProceedingsoftheEuropeanConferenceonComputer
Systems(EuroSys)(2016).
§ WhenDRAMrunsout
pVM
Mainmemory
DRAM
Persistentstorefullyused freePMpages
Userapplication
29
pageallocationisnotpossible
PM
§ WhenDRAMrunsout
pVM
Mainmemory
DRAM
Persistentstorefullyused freePMpages
Userapplication
30
directallocation- npmalloc()- nvmmap(PERSISTorNOPERSIST)
PM
§ Requiresapplicationmodification
• Explicitlibrary functioncallsneedtobemade
− npmalloc(),nvmmap()
§ Similarapproachtopreviouswork• NV-Heap
• Mnemosyne
• NVMDuet
WeaknessofpVM
31
directallocation
Comparisonofrelatedworks
32
Swap Memorage pVM
Goal freeupworkingmemoryspace
memoryexpansion
persistentstore
Memory-Storagedivision fixed fixed fixed
Consistency notconsidered notconsidered yes
Transparency yes yes no
Runtimeoverhead yes yes no
Direct accesstodata no yes yes
strengthweakness
SAY-Godesigngoals
33
PM
User
SFS
VFS
Devicedrivermemorymanager
Virtualmemorymanager
DRAM
Dynamicpartition
• Seamlessintegration- PMroleisdynamicallyadjustedasneedbe
• Transparency- Automaticscalingofworkingmemory- Withoutmodificationoftheapplication
• Consistency- EnsureconsistencyofthePMallocator- Preventmemoryleaks
forworkingmemory forstorage
§ Consistsoftwoparts
SAY-Gocomponents
34
Filesystemwithdynamicpartitioning
Virtualmemorymanager
2. Integratedmemorymanager- Pageallocatorforbothworkingmemoryandstorage- Ensureconsistencyofmemoryallocator
1. Filesystemwithdynamicpartitionsupport- Supportdynamiclayout- Regionmanagedinpage unit
DRAM PM
Integratedmemorymanagerforworkingmemory forstorage
§ Consistsoftwoparts
Layout
35
Filesystemwithdynamicpartitioning
Virtualmemorymanager
DRAM PM
Integratedmemorymanagerforworkingmemory forstorage
DynamicPartitionFileSystem(DPFS), inprogress
1. Filesystemwithdynamicpartitionsupport- Supportdynamiclayout- Regionmanagedinpage unit
2. Integratedmemorymanager- Pageallocatorforbothworkingmemoryandstorage- Ensureconsistencyofmemoryallocator
PersistentMemoryBuddy(PMB), HotStorage ‘18
§ Consistsoftwoparts
Layout
36
Virtualmemorymanager
DRAM PM
Integratedmemorymanagerforworkingmemory forstorage
2. Integratedmemorymanager- Pageallocatorforbothworkingmemoryandstorage- Ensureconsistencyofmemoryallocator
PersistentMemoryBuddy(PMB), HotStorage ‘18
ImplementedinLinux
DynamicPartitionFileSystem(DPFS), inprogress
1. Filesystemwithdynamicpartitionsupport- Supportdynamiclayout- Regionmanagedinpage unit
Filesystemwithdynamicpartitioning
PMBimplementationfeatures
37
PersistentMemoryBuddy(PMB)Filesystemwithdynamicpartitioning
Virtualmemorymanager
DRAM PM
forworkingmemoryPMB
forstorage
• Seamlessintegration- PMroleisdynamicallyadjustedasneedbe
• Transparency- Automaticscalingofworkingmemory- Withoutmodificationoftheapplication
• Consistency- EnsureconsistencyofthePMallocator- Preventmemoryleaks
PMBdesignedwithSAY-Gogoalinmind
PMBLayoutinLinux
38
• DRAMandPMaremanagedinZONEunits- ZONE_NORMAL- plus2newZONEs*ZONE_MIGRATEand ZONE_STRG
• EachZONEhasfreepages- buddymemoryallocator
VMmanager
ProcessesDPFS
Devicedriver
VFS
ZONEMIGRATE
ZONESTRG
ZONENORMAL
PM pagesDRAMpages
PMB
forstorageforworkingmemory
§ CharacteristicsinLinux
• Unsorted
• Circular linkedlist
• Pushandpopoperation
occursonlyattheendofthelist
• Pushandpopoperations
areimplementedbyatomicoperations
Buddymemoryallocator
ZONE
ZONE
ZONE
...
order(10)Free_list
order(1)order(0)
order(2)
...
page
page
page4KB
4MB
...
page
page
4MB
...
page
page
4MB
page4KB
39
PMBbasedonbuddymemoryallocator
PMBLayoutinLinux
40
VMmanager
ProcessesDPFS
Devicedriver
VFS
ZONEMIGRATE
ZONESTRG
ZONENORMAL
PM pagesDRAMpages
PMB
forstorageforworkingmemory
• DRAMandPMaremanagedinZONEunits- ZONE_NORMAL- plus2newZONEs*ZONE_MIGRATEand ZONE_STRG
• EachZONEhasfreepages- buddymemoryallocator
• EachZONEhasdifferentrole
RoleofeachZONE
41
• ZONE_NORMAL- DRAMiscoveredbythisZONE
• ZONE_STRG- Minimumstoragearea- Persistentmetadatasavedtothisarea- Persistentpagedescriptor- Bitmapthatmaintainstheallocationstate- LogtableforPMBconsistency
VMmanager
ProcessesDPFS
Devicedriver
VFS
ZONEMIGRATE
ZONESTRG
ZONENORMAL
PM pagesDRAMpages
PMB
forstorageforworkingmemory
RoleofeachZONE
42
• ZONE_MIGRATE- KeycomponentinPMB- Eitherallocateddependingonitsuse
*aspersistent(storage)page*asnon-persistent(memory)page
VMmanager
ProcessesDPFS
Devicedriver
VFS
ZONEMIGRATE
ZONESTRG
ZONENORMAL
PM pagesDRAMpages
PMB
forstorageforworkingmemory
Pageallocation
43
• ZONE_NORMAL- Allocateworkingmemory
• StorageallocatedfromZONE_STRG- Allocatestorage
• ZONE_MIGRATE- AllocatePMpagestobothzones
- Allocationdoneusingtheexistinginterface
- Allocationdonebykernelthreadinthebackground
VMmanager
ProcessesDPFS
Devicedriver
VFS
PM pagesDRAMpages
PMB
ZONEMIGRATE
ZONESTRG
ZONENORMAL
forstorageforworkingmemory
ZONEcapacity
44
• ZONE_NORMAL- Initially,onlyDRAMpages
- Willgrowwithdemand
• ZONE_STRG- Initially,essentialstoragearea
- Willgrowwithdemand
VMmanager
ProcessesDPFS
Devicedriver
VFS
ZONEMIGRATE
ZONESTRG
ZONENORMAL
PM pagesDRAMpages
forstorageforworkingmemory
PMB
ZONENORMAL
§ ZONENORMALexpansion
Pagemovement
45
VMmanager
ProcessesDPFS
Devicedriver
VFS
ZONEMIGRATE
ZONESTRG
ZONENORMAL
PM pagesDRAMpages
forstorageforworkingmemory
PMB
PMpages
• ZONE_NORMAL- Initially,onlyDRAMpages
- Willgrowwithdemand
- PMpagemigration
§ ZONENORMALexpansion
Pagemovement
46
VMmanager
ProcessesDPFS
Devicedriver
VFS
ZONEMIGRATE
ZONESTRG
ZONENORMAL
PM pagesDRAMpages
forstorageforworkingmemory
PMB
• ZONE_NORMAL- Initially,onlyDRAMpages
- Willgrowwithdemand
- PMpagemigration
- IncreasedZONE_NORMALsize
§ ZONENORMALretraction
Pagemovement
47
VMmanager
ProcessesDPFS
Devicedriver
VFS
ZONENORMAL
ZONEMIGRATE
ZONESTRG
PM pagesDRAMpages
forstorageforworkingmemory
PMB
ZONENORMAL
• ZONE_NORMAL- Initially,onlyDRAMpages
- Willgrowwithdemand
- PMpageretrieval
§ ZONENORMALretraction
Pagemovement
48
VMmanager
ProcessesDPFS
Devicedriver
VFS
ZONEMIGRATE
ZONESTRG
ZONENORMAL
PM pagesDRAMpages
forstorageforworkingmemory
PMB
• ZONE_NORMAL- Initially,onlyDRAMpages
- Willgrowwithdemand
- PMpageretrieval
- ReducedZONE_NORMALsize
§ ZONE_MIGRATE toZONE_NORMAL• BothzonesmanagedwithBuddymemoryallocator• Whennumberoffreepagesfallsbelowwatermark• Transfers4MBcontiguouspages
− simplepointermanipulation
Pagemigration
ZONENORMAL
ZONESTRG
ZONEMIGRATE
...
order(10)Free_list
order(1)order(0)
order(2)
...
page
page
...
order(10)Free_list
order(1)order(0)
order(2) 4MB
2. linkpointer
1. un-linkpointer
49
pagepage
pagepage
2GB
4MB
ZONENORMAL
ZONEMIGRATE
ZONESTRG
ZONENORMAL
PM pagesDRAMpages
forstorageforworkingmemory
PMB
...
page
pagepagepage
2GB
§ ZONE_NORMAL to ZONE_MIGRATE• Nowatermark• Retrievedreleasedbyapplication
− simplepointermanipulation
Pageretrieval
ZONENORMAL
ZONESTRG
ZONEMIGRATE
...
order(10)Free_list
order(1)order(0)
order(2)
...
order(10)Free_list
order(1)order(0)
order(2)
2. linkpointer
1. un-linkpointer
...
page
page
...
page
page
...page
page
50
4MB
4MB
ZONENORMAL
ZONEMIGRATE
ZONESTRG
PM pagesDRAMpages
forstorageforworkingmemory
PMB
ZONENORMAL
Managementunit
ZONENORMAL
ZONESTRG
ZONEMIGRATE
...
order(10)Free_list
order(1)order(0)
order(2)
4KB
...
order(10)Free_list
order(1)order(0)
order(2)
2MB
4MB
2GB
4MB • PMBtakesthe2MBhugepagesize
• Contiguousmemoryspace(20 ~210 pages)- ZONE_NORMAL:4KB~4MB- ZONE_MIGRATE,ZONE_STRG:2MB~2GB
• Pagemovementunitis4MBcontiguouspage- PerformedbetweenZONE_NORMAL(order10)andZONE_MIGRATE(order1)
51
§ Memoryleakmayoccurinpagemovingprocess
Consistency
uponcrash
52
...
page
page 4MB
2. linkpointer
pagepage
pagepage
2GB
4MB
1. un-linkpointer
ZONENORMAL
ZONESTRG
ZONEMIGRATE
...
order(10)Free_list
order(1)order(0)
order(2)
...
order(10)Free_list
order(1)order(0)
order(2)
§ Memoryleakmayoccurinpagemovingprocess
Consistency
53
...
page
page 4MB
pagepage
pagepage
2GB
4MB
1. un-linkpointer
thesepagesareleaked
ZONENORMAL
ZONESTRG
ZONEMIGRATE
...
order(10)Free_list
order(1)order(0)
order(2)
...
order(10)Free_list
order(1)order(0)
order(2)
§ Useloggingtopreventmemoryleak
Consistency
54
...
page
page 4MB
3. linkpointer
pagepage
pagepage
2GB
4MB
1. Writelogaboutpagemovementorallocation
2. un-linkpointer
4. commitlog
ZONENORMAL
ZONESTRG
ZONEMIGRATE
...
order(10)Free_list
order(1)order(0)
order(2)
...
order(10)Free_list
order(1)order(0)
order(2)
§ Useloggingtopreventmemoryleak
Consistency
55
...
page
page 4MB
3. linkpointer
pagepage
pagepage
2GB
4MB
4. commitlog• Logentryfields- Pagestructureaddress- Operationtype- Destinationofpage- Commitbit
• Logentrysize<64bit
• Usemfence andclflush instructionsforlogging
1. Writelogaboutpagemovementorallocation
2. un-linkpointer
ZONENORMAL
ZONESTRG
ZONEMIGRATE
...
order(10)Free_list
order(1)order(0)
order(2)
...
order(10)Free_list
order(1)order(0)
order(2)
PreviousworkandSAY-Go
56
Swap Memorage pVM SAY-Go
Goal freeupworkingmemoryspace
memoryexpansion
persistentstore efficient useofresources
Memory-Storagedivision fixed fixed fixed dynamic
Consistency notconsidered notconsidered yes yes
Transparency yes yes no yes
Runtimeoverhead yes yes no no
Direct accesstodata no yes yes yes
strengthweakness
§ Testplatform• Machinespecification
− IntelXeonE5-2620v32.4GHz(24cores)− 256DRAM
− DividetoDRAMandPseudo-PM
− Linuxv4.11.1
§ Workloadscharacteristics
Evaluation
57
FFT RedisScale Memory intensiveapplication In-memorydatabase
Domain Signal processing Key-valuestore
Benchmarksuite Splash2xinParsec3.0 YCSB
Input Native(largest) 1:1 (read:write)
Max memoryusage 12GB 20GB
§ Comparisonwithswap• Memoryconfiguration• ZONE_NORMALcapacityvariedbasedonworkload
− FFT:16GBforDRAM− Redis:32GBforDRAM
Evaluation
58
DRAM Pseudo-PM
ZONENORMAL
ZONESTRG
ZONEMIGRATE
16or32GB 16GBremainder
SwapPMB
DRAM Pseudo-PM
ZONENORMAL
EXT4withDAX
swaparea
16or32GB remainder(setby/dev/pmem)
tobeuseddynamically
§ Comparisonwithswap
Evaluation
0
20
40
60
80
100
120
140
16 9 6 3 0
PMBSwap
Runtim
e(sec)
DRAM(GB)usedbyapplication(ZONE_NORMALsize:32GB)
RedisFFT
59
DRAM(GB)usedbyapplication(ZONE_NORMALsize:16GB)
0
50
100
150
200
250
300
350
32 15 10 5 0
§ Measurementmethod:FFTexample
Evaluation
0
20
40
60
80
100
120
140
16 9 6 3 0
PMBSwap
Runtim
e(sec)
DRAM(GB)usedbyapplication(ZONE_NORMALsize:16GB)
FFT
60
• FFTmaximumworkingsetsize:12GB
• LimitamountofmainmemoryusedbyFFT- Usestresstool toconsumemainmemory
• x-axisreferstothemaximumDRAMusedbyFFT
FFTstress6GB 6GB10GB
Pseudo-PMDRAM
ZONENORMAL16GB
ZONESTRG16GB
ZONEMIGRATE224GB
§ Measurementmethod:FFTexample
Evaluation
0
20
40
60
80
100
120
140
16 9 6 3 0
PMBSwap
Runtim
e(sec)
DRAM(GB)usedbyapplication(ZONE_NORMALsize:16GB)
FFT
61
FFTstress3 9GB13GB
ZONENORMAL16GB
ZONESTRG16GB
ZONEMIGRATE224GB
Pseudo-PMDRAM
• FFTmaximumworkingsetsize:12GB
• LimitamountofmainmemoryusedbyFFT- Usestresstool toconsumemainmemory
• x-axisreferstothemaximumDRAMusedbyFFT
§ Comparisonwithswap
Evaluation
0
20
40
60
80
100
120
140
16 9 6 3 0
PMBSwap
Runtim
e(sec)
DRAM(GB)usedbyapplication(ZONE_NORMALsize:32GB)
RedisFFT
62
DRAM(GB)usedbyapplication(ZONE_NORMALsize:16GB)
0
50
100
150
200
250
300
350
32 15 10 5 0
0
50
100
150
200
250
300
350
32 15 10 5 0
§ Comparisonwithswap
EvaluationRu
ntim
e(sec)
DRAM(GB)usedbyapplication(ZONE_NORMALsize:32GB)
Redis• Swapperformance
degradesconsiderably
• PMBperformanceremainsstable
FFT
0
20
40
60
80
100
120
140
16 9 6 3 0
PMBSwap
63
DRAM(GB)usedbyapplication(ZONE_NORMALsize:16GB)
Tworeasonsbehindresults
0
50
100
150
200
250
300
350
32 15 10 5 0
§ Comparisonwithswap
EvaluationRu
ntim
e(sec)
DRAM(GB)usedbyapplication(ZONE_NORMALsize:32GB)
RedisFFT
0
20
40
60
80
100
120
140
16 9 6 3 0
PMBSwap
64
DRAM(GB)usedbyapplication(ZONE_NORMALsize:16GB)
• Mechanismoverhead- PMBmigrationandretrieval:~400ns
- Swap-in/out:~6/41us
0
50
100
150
200
250
300
350
32 15 10 5 0
§ Comparisonwithswap
EvaluationRu
ntim
e(sec)
DRAM(GB)usedbyapplication(ZONE_NORMALsize:32GB)
FFT Redis• PMB“unaffected”by
DRAMsize- swappagecopiesincreasewithreducedDRAMsize
0
20
40
60
80
100
120
140
16 9 6 3 0
PMBSwap
65
DRAM(GB)usedbyapplication(ZONE_NORMALsize:16GB)
§ Numberofcopiedpages
Evaluation
0
10
20
30
40
50
60
70
16 9 6 3 0
Swap-outSwap-in
0
10
20
30
40
50
60
70
32 15 10 5 0DRAM(GB)usedbyapplication(ZONE_NORMALsize:16GB)
DRAM(GB)usedbyapplication(ZONE_NORMALsize:32GB)
Num
bero
fswappe
dpages
(unit:million)
0.0M
8.3M
21.8M
60.5M63.8M
0.0M 0.2M 0.4M
7.8M
26.4M
FFT Redis• Reportedby
/proc/vmstat
66
§ PMpagemigrationandretrievalbehavior• Measured per 3s interval
Evaluation
0
200
400
600
800
1000
1200
3 6 9 1215182124273033363942454851540
1000
2000
3000
4000
5000
6000
3 27 51 75 99 123
147
171
195
219
243
267
291
315
339
#ofpages
executiontime(s)executiontime(s)
migrationretrievalpagesinNORMALmovedfromMIGRATE
FFT Redis
67
§ PMpagemigrationandretrievalbehavior• Measured per 3s interval
Evaluation
0
200
400
600
800
1000
1200
3 6 9 1215182124273033363942454851540
1000
2000
3000
4000
5000
6000
3 27 51 75 99 123
147
171
195
219
243
267
291
315
339
#ofpages
executiontime(s)executiontime(s)
migrationretrievalpagesinNORMALmovedfromMIGRATE
FFT Redis
68
§ PMpagemigrationandretrievalbehavior• FFT and Redis executed simultaneously
Evaluation
#ofpages
executiontime(s)
0
2000
4000
6000
8000
10000
120003 21 39 57 75 93 111
129
147
165
183
201
219
237
255
273
291
309
327
• Redis runsonce
• FFTrunningtime:60seconds- Restarteduponterminationaftera20secondlapse
migrationretrievalpagesinNORMALmovedfromMIGRATE
69
§ PMpagemigrationandretrievalbehavior• FFT and Redis executed simultaneously
Evaluation
#ofpages
executiontime(s)
0
2000
4000
6000
8000
10000
120003 21 39 57 75 93 111
129
147
165
183
201
219
237
255
273
291
309
327
• Redis runsonce
• FFTrunningtime:60seconds- Restarteduponterminationaftera20secondlapse
- Pagesarebeingfreedandretrievedinthevicinityof60,140,220,300seconds
60s 60s 60s 60s20s 20s 20s 20s
70
§ ProposedasystemcalledStorage-As-You-Go(SAY-Go)• TransparentlyadjuststheuseofPM• PMcanbeusedasmemoryaswellasstorageasneedbe• Breaktheboundarybetweenmemoryandstorage
§ PersistentMemoryBuddy(PMB)• Memoryallocationservicethatcanfreelygrowandshrinkworkingmemory• ExperimentalresultsshowPMBperformsbetterthantraditionalswap
Summary
71
§ DPFS• Requiresdynamicpartitionresizingsupport• Inthedesignandearlyimplementationphase
§ PMB• Moredetailedpoliciesformanagingpage• Variousoptimizations
Futurework
Virtualmemorymanager
DRAM PM
DPFS
PMB
72
Q& A
73