Embed Size (px)
Citation preview
Efficient ML in 2 KB RAM for the Internet of ThingsAshish Kumar (MSR)
Saurabh Goyal (IIT Delhi)Manik Varma (MSR)
Bonsai – A Compact Tree Model
Prediction Accuracy vs Model Size
Sparse Projection (Z) Yes No
Yes No
Bonsai – Key Ideas
Prediction Costs on the Arduino Uno
Comparison to Uncompressed Methods
Prediction Accuracy vs Model Size
0.2 1 2 4 6 8 10 12 14
48
49
50
51
52
Model Size (KB)
nDCG
@1
FastRankBonsai
0.2 0.3 0.4 0.5
52.68
52.7
52.72
Model Size (KB)
nDCG
@1
Bing L3/L4 Ranker Results
Disadvantages of Tree Ensembles
Bonsai Optimization
• To build an efficient tree classifier • Which can be trained on the cloud • But which can make predictions on tiny IoT devices
Arduino Uno
Arduino Uno
Objective
8 bit ATmega328P Processor at 16 MHz with 2 KB RAM & 32 KB read only Flash
• Tree Ensembles might not fit in Kilobytes and might not be accurate
• We design Bonsai to be a single, shallow, sparse tree with powerful nodes for accurate prediction
• We reduce model size by learning Bonsai in a low-dimensional space into which all data is projected
• We jointly learn tree and projection parameters so as to maximize accuracy within the given budget
The Bonsai Objective Function
Accuracy (%)
30
65
100
RTWhale-2 Chars4K-62 Chars4K-2 WARD-2
BonsaiGBDT kNN RBF-SVM Neural Nets
Compressed : Uncompressed :
Model Size (KB)
RTWhale-2 Chars4K-62 Chars4K-2 WARD-2
1 GB
1MB
1KB
Accuracy (%)
Pred Time (ms)
Pred Energy (mJ)
Eye-2
RTWhale-2
01.673.33
5
4
11
18
75
82.5
90
Eye-2
2187 1312
0123
2
6
10
50
60521 313
Bonsai Linear LDKL NeuralNet Cloud-GBDT
Bonsai’s Decision Boundaries
Code for Bonsai code can be downloaded from http://manikvarma.org/
Model Size (KB)0 5 10 15
Ac
cu
racy
(%
)
80
82
84
86
88
90
92
Eye-2
Model Size (KB)0 5 10 15
Acc
urac
y (%
)
55
60
65
70
75
Chars4K-2
BonsaiBonsaiOptGBDTTree PruningLDKLLDKL-L1NeuralNet PruningSNCDecision JungleBudgetRF
Model Size (KB)0 50 100 150
Accu
racy
(%)
10
20
30
40
50
60Chars4K-62
BonsaiBonsaiOptNeuralNet PruningSNCDecision JungleBudgetRF
Model Size (KB)0 5 10 15
Accu
racy
(%)
55
60
65
70
75
Chars4K-2
BonsaiBonsaiOptGBDTTree PruningLDKLLDKL-L1NeuralNet PruningSNCDecision JungleBudgetRF
Model Size (KB)0 5 10 15
Acc
urac
y (%
)
55
60
65
70
75
Chars4K-2BonsaiBonsaiOptGBDTTree PruningLDKLLDKL-L1NeuralNet PruningSNCDecision JungleBudgetRF
Model Size (KB)0 5 10 15
Accu
racy
(%)
66
68
70
72
74
76
CIFAR10-2
Model Size (KB)0 5 10 15
Accu
racy
(%)
80
82
84
86
88
90
92
Eye-2
Model Size (KB)0 5 10 15
Accu
racy
(%)
86
88
90
92
94
96
MNIST-2
Model Size (KB)0 5 10 15
Accu
racy
(%)
50
52
54
56
58
60
62RTWhale-2
Model Size (KB)0 5 10 15
Accu
racy
(%)
90
91
92
93
94
95
96
97USPS-2
Model Size (KB)0 5 10 15
Accu
racy
(%)
90
92
94
96
98WARD-2
