Upload
others
View
7
Download
0
Embed Size (px)
Citation preview
Persistent Memory forArtificial Intelligence
Bill GervasiPrincipal Systems Architect
[email protected] Clara, CAAugust 2018 1
Demand Outpacing Capacity
Santa Clara, CAAugust 2018 2
In-Memory Computing
Artificial Intelligence
Deep Learning
Machine Learning
Memory Demand
DRAM Capacity
Driving New Capacity Models
Santa Clara, CAAugust 2018 3
Memory Demand
DRAM Capacity
Non-volatile memories
Industry successfully snugglinglarge memories to the processors…
…but we can do oh! so much more
My Three Talks at FMS
Santa Clara, CAAugust 2018 4
NVDIMM AnalysisMemory Class Storage
Artificial Intelligence
History of Architectures
Santa Clara, CAAugust 2018 5
Let’s go back in time…
Santa Clara, CAAugust 2018 6
Historical Trends in Computing
EdgeComputing
Co-Processing
Power FailureData Loss
Santa Clara, CAAugust 2018 7
Some Moments in History
CentralProcessing
DistributedProcessing
Processor per userShared ProcessorDumb terminals
Peer-to-peer networks
Santa Clara, CAAugust 2018 8
Some Moments in History
CentralProcessing
DistributedProcessing
Hercules graphicsSound Blaster audio
Rockwell modemEthernet DSP
“Native Signal Processing”Main CPU driversCheap analog I/O
Tightly-coupledcoprocessing
Santa Clara, CAAugust 2018 9
The Lone Survivor…
Graphics add-in cards
Integrated graphics
…survived the NSP war
Santa Clara, CAAugust 2018 10
Some Moments in History
CentralProcessing
DistributedProcessing
Phone apps providelocal services
Phone providerscontrolled all
data processing
Edge computingreduces latency
Santa Clara, CAAugust 2018 11
When the Playing Field Changes
The speed of networkingdirectly impacts the
pendulum swing fromcentralized to distributed
A faster network favorsdistributed computing
Santa Clara, CAAugust 2018 12
Winners and Losers
Often, the maturity of the softwaredevelopment environment
determined who won and who lost
Santa Clara, CAAugust 2018 13
Maintaining an Edge
Coprocessor
CPU Time
See how greatit is???
Oops!!!
Santa Clara, CAAugust 2018 14
The Tail Wagging the Dog
I won’t say “It’s the Software, Stupid”because I know you’re not stupid
however
To succeed, AI needs GREATsoftware infrastructure
Driving some companies to designhardware to the software
instead of software to the hardware
Santa Clara, CAAugust 2018 15
Wild Array of Programmer Options
Santa Clara, CAAugust 2018 16
AI on Traditional Server
No magic
AI applications are likeany other
Data processing doneon main CPU
CPU
Disk
Downside is main CPU isoverkill in floating point,and weak in parallelism
Santa Clara, CAAugust 2018 17
AI Evolution
CPU
Disk
Addition of AI Accelerator
Offloads main CPU forAI tasks
AIAccelerator
Santa Clara, CAAugust 2018 18
AI Evolution
AI Accelerator Characteristics
Wide array of simple processing elements
Reduced floating point precision
Tuned for matrix operations
GoesoutaGoesinta
Proc
Proc
Proc
Proc
Proc
Proc
Dis
tribu
te
Rec
ombi
ne
Santa Clara, CAAugust 2018 19
In-Memory Computing
CPU
Disk
In-memory computinglets the AI acceleratorcontrol the memory
directly
Also great forencryption
AI Accelerator
Santa Clara, CAAugust 2018 20
Data Processing Paradigms
Traditional database
Data mining
Inferencing
Fuzzy logic
Recognition
DataAccessData
Reliability
etc
Santa Clara, CAAugust 2018 21
The Actualization Gap
Research projects Deployments
Santa Clara, CAAugust 2018 22
The “Research” ProjectsMany interconnects between
storage elements andprocessing elements
Weighted calculationsproduce parallelpossible results
R R R R
R R R R
R R R R
Focus for a number ofstartup companies
Santa Clara, CAAugust 2018 23
What Most People Mostly Building
Dense matrix memoryfor highest storage
capacity
Shared memorycontroller for many
execution units
Pipes fornetworking
Santa Clara, CAAugust 2018 24
Practical I/O Connection LimitsAny-to-any would
be awesomeToroid is a more
practicable solution
Limits how quickly data can flow in and out
Santa Clara, CAAugust 2018 25
Network UtilizationRESET
PowerFail
Fill processorswith code
Fill caches withmodel seed data
Send newInput data
Process input datathrough model Retrieve results
Consumes I/O
Time to Checkpoint
Model?
Retrieve updatedmodel data
xxx
Yes
No
Santa Clara, CAAugust 2018 26
Lossless Versus Lossy
Persistent data: reload needed
Transient data: reload, restart calculations
Time to reload is always an issue
Accumulated data: modified models are expensive to rebuild
Santa Clara, CAAugust 2018 27
Recovering From Power Fail
CPU
Data pulled frommain memory
…or worse…
Backing store
Data requiresmultiple hopsthrough the
interconnects
Not uncommon fordata reload to take3 minutes or more
Before recalculationcan begin!
Disk
Santa Clara, CAAugust 2018 28
Distributed Memory Complications
This may help explain the gapbetween research projects and
actual deployments
Distributed cells complicatedownload time into the arrays
R R R R
R R R R
R R R R
Santa Clara, CAAugust 2018 29
Persistent Main Memory
CPU
NVDIMMs are movingdata persistence to the
main memory bus
and in some casesincreasing memory
capacity
See my other talklater this week
Disk
Santa Clara, CAAugust 2018 30
Cost of Power Failure
Statistics vary but all agree…downtime costs a LOT
Santa Clara, CAAugust 2018 31
Persistent Memory
DRAMLoses data
Must be refreshedCan’t lose power
Persistent MemoryHolds data
forever, evenon power fail
Santa Clara, CAAugust 2018 32
Nantero NRAM™
Nantero NRAM is a persistent memoryusing carbon nanotubes to build
resistive arrays which can bearranged in a DRAM compatible device
……………..……………..……………..
DDR4DDR5
HBM
See my other talkslater this week
Classes of Persistent Memory
Santa Clara, CAAugust 2018 33
Non-volatility
Endurance
Read Time
Write Time
No
No limit
10 ns
10 ns
DRAM
Yes
Limited
X
X
MRAM
Yes
Limited
X
X
ReRAM
Yes
Limited
X
X
PCM /3DXpoint
Yes
No limit
X
X
FeRAM
Yes
103
50M ns
25M ns
Flash
Memory &Memory Class Storage Storage Class Memory Storage
Yes
No limit
10 ns
10 ns
NRAM
See my other talkslater this week
Santa Clara, CAAugust 2018 34
Applying Persistent Memory
Replace DRAM withPersistent Memory
Completely eliminates theneed to reload on
Power fail
Next generationpersistent
memory willtarget SRAM,
too
Persistentshadow registersaren’t such a bad
idea, either
Santa Clara, CAAugust 2018 35
NRAM for Main Memory
CPU
NRAM replaces DDR4,DDR5 for main memory
Santa Clara, CAAugust 2018 36
Enables the New Architectures
NRAM cells in the array
Permanent storagethrough power fail
Programmed once duringmanufacturing, no reload
Santa Clara, CAAugust 2018 37
NRAM Everywhere
Soon we will look back and say
“Remember when data was lostwhen power went out?”
and laugh
Santa Clara, CAAugust 2018 38
Full Disclosure
My first home computerhad an 8” floppy disk
I earned my gray hair
Summary
Santa Clara, CAAugust 2018 39
Centralized versus distributed computing is a long term cycle
Quality of software infrastructure typically determines the winner
Artificial intelligence accelerators are a recent co-processing addition
Data loss on power failure is worsened by AI architectures
Persistent memory in AI device solves major problems
Nantero NRAM addresses many usages of PM in AI systems
If you remember 8” floppies, you probably can’t read this screen