Upload
others
View
3
Download
0
Embed Size (px)
Citation preview
T-CVAE: Transformer-Based Conditioned Variational Autoencoder for Story Completion
Source : IJCAI 2019
Speaker : Ya-Fang, Hsiao
Advisor : Jia-Ling, Koh
Date : 2020/04/24
for Story Completion
My Dad loves chocolate chip cookies. _______________. I decided I would learn how to make them. I made my first batch the other day. My Dad was very surprised and quite happy!
My Mom doesn't like to make cookies because they take too long.
He has been making them all week.
He always ate them.
T-CVAE:
ℒ 𝑥 = −KL 𝑞 𝑧 𝑥 ||𝑝(𝑧) + 𝔼𝑞 𝑧 𝑥 [𝑙𝑜𝑔𝑝(𝑥′|𝑧)]
ℒ 𝑥, 𝑥′ = −KL 𝑞 𝑧 𝑥, 𝑐 ||𝑝(𝑧|𝑥) + 𝔼𝑞 𝑧 𝑥, 𝑐 [𝑙𝑜𝑔𝑝(𝑥′|𝑐, 𝑧)]
VAE:
CVAE:
T-CVAE:
ℒ 𝑥, 𝑥′ = −KL 𝑞 𝑧 𝑥, 𝑐 ||𝑝(𝑧|𝑥) + 𝔼𝑞 𝑧 𝑥, 𝑐 [𝑙𝑜𝑔𝑝(𝑥′|𝑐, 𝑧)]
𝐶𝑡 = tanh( 𝑧, 𝐷𝑜𝑢𝑡,𝑡𝐿 𝑊𝑐)
𝑂𝑡 = 𝐶𝑡𝑊𝑜 + 𝑏𝑜
Combination layer:
𝑃𝑡 = 𝑠𝑜𝑓𝑡𝑚𝑎𝑥(𝑂𝑡)