Brief History of Neural Machine Translation

Jamie Seol

Jamie Seol

Jamie Seol

Machine Translation

Lee, et al. "Naver machine translation system for WAT 2015." Proc. WAT 2015

Jamie Seol

Traditional Statistical Machine Translation

심광섭, and 김영택. "[기술해설] 기계 번역 시스템." 정보과학회지 12.8 (1994): 17-23.

Jamie Seol

Traditional Statistical Machine Translation

Nguyen, et al. "A tree-to-string phrase-based model for statistical machine translation." Proc. CoNLL 2008.

Jamie Seol

Neural Machine Translation

• pros• no need domain knowledge• training multiple features jointly

• cons• training and decoding is quite slow• out-of-vocabulary problem

Lee, et al. "Naver machine translation system for WAT 2015." Proc. WAT 2015

Jamie Seol

Neural Machine Translation

• sequence-to-sequence (Sutskever et al. NIPS 2014)

Sutskever, et al. "Sequence to sequence learning with neural networks." NIPS 2014.

Jamie Seol

Neural Machine Translation

• encoder-decoder (Cho et al. EMNLP 2014)

Cho, et al. "Learning phrase representations using RNN encoder-decoder for statistical machine translation." EMNLP 2014.

Jamie Seol

Neural Machine Translation

• alignment problem!

Tamura, et al. "Recurrent Neural Networks for Word Alignment Model." ACL 2014.

Yang, et al. "Word Alignment Modeling with Context Dependent Deep Neural Network." ACL 2013.

Jamie Seol

Neural Machine Translation

• jointly learning to align (Bahdanau et al., ICLR 2015)

Bahdanau, et al. "Neural machine translation by jointly learning to align and translate." ICLR 2015. (arXiv 2014)

Jamie Seol

Neural Machine Translation

• most of recent works use encoder-decoder model• Lu, Liang, et al. "A study of the recurrent neural network encoder-decoder for large

vocabulary speech recognition." Proc. Interspeech 2015. • Luong, et al. "Effective approaches to attention-based neural machine translation."

EMNLP 2015. • Eriguchi, et al. "Tree-to-Sequence Attentional Neural Machine Translation." ACL

2016. • Cho, et al. "On the properties of neural machine translation: Encoder-decoder

approaches." SSST 2014. • Lee, et al. "Naver machine translation system for WAT 2015." Proc. WAT 2015