LSTM Networks for Sentiment Analysis with Keras

LSTM Networks for Sentiment Analysis

YAN TING LIN

Summary• This tutorial aims to provide an example of how a Recurrent Neural

Network (RNN) using the Long Short Term Memory (LSTM) architecture can be implemented using Theano. In this tutorial, this model is used to perform sentiment analysis on movie reviews from the Large Movie Review Dataset, sometimes known as the IMDB dataset.• In this task, given a movie review, the model attempts to predict

whether it is positive or negative. This is a binary classification task.

• Ref: http://deeplearning.net/tutorial/lstm.html

Data• Ref: https://keras.io/datasets/• Dataset of 25,000 movies reviews from IMDB, labeled by sentiment

(positive/negative). Reviews have been preprocessed, and each review is encoded as a sequence of word indexes (integers). For convenience, words are indexed by overall frequency in the dataset, so that for instance the integer "3" encodes the 3rd most frequent word in the data. This allows for quick filtering operations such as: "only consider the top 10,000 most common words, but eliminate the top 20 most common words".• As a convention, "0" does not stand for a specific word, but instead is

used to encode any unknown word.

Data Label

Train Data : X_trainTrain Data Answer: y_trainTest Data: X_testTest Data Answer: y_test

Understanding LSTM Networks• Ref: http://colah.github.io/posts/2015-08-Understanding-LSTMs/• Recurrent Neural Networks• The Problem of Long-Term Dependencies• LSTM Networks• The Core Idea Behind LSTMs• Step-by-Step LSTM Walk Through• Variants on Long Short Term Memory• Conclusion

Install TensorFlowImportError: No module named tensorflow# creating virtual environment using python 2.7 version• conda create -n tensorflow python=2.7# enter Conda Virtual Environment• source activate tensorflow# Using pip to install # Mac OS X, GPU enabled, Python 2.7:• Export

TF_BINARY_URL=https://storage.googleapis.com/tensorflow/mac/gpu/tensorflow-0.11.0-py2-none-any.whl• sudo pip install --upgrade $TF_BINARY_URL

Install Keras (conda) • conda install -c conda-forge keras• # you may use conda-forge to install Tensorflow• # ref: https://conda-forge.github.io• conda install -c conda-forge tensorflow

Data Preprocessing

Make each comment of imdb data be fixed length (80)

Train Model

• In the neural network terminology:• one epoch = one forward pass and one backward pass of all the training

examples• batch size = the number of training examples in one forward/backward pass.

The higher the batch size, the more memory space you'll need.• number of iterations = number of passes, each pass using [batch size] number

of examples. To be clear, one pass = one forward pass + one backward pass (we do not count the forward pass and backward pass as two different passes).• Example: if you have 1000 training examples, and your batch size is 500, then

it will take 2 iterations to complete 1 epoch.

Result - 1 : It takes much time to download data and train model

Result - 2 : After 1 hour

Time Reduction• Make the training data smaller. 5x smaller and 5x faster.

Visualizing your model

# install pydot and graphvis conda install -c anaconda pydot=1.0.28conda install -c anaconda graphviz=2.38.0# in python code

Dropout Comparison - 1

Dropout Comparison - 2

Why Keras?

LSTM Networks for Sentiment Analysis with Keras

Presentations & Public Speaking

SC-LSTM: Learning Task-Specific Representations in Multi-Task … · 2020-06-30 · 2398 3.1 LSTM Cell An LSTM cell (Hochreiter and Schmidhuber, 1997) is made up of four functional

Understanding LSTM Networks - UNRCdc.exa.unrc.edu.ar/rio2016/sites/default/files/6.Understanding LSTM... · 11/2/2016 Understanding LSTM Networks ... Recurrent Neural Networks

MOGRIFIER LSTM - OpenReview

december 2009 CardMaps · sentiment © pagemaps sentiment © pagemaps mops sentiment pagemaps Q pager-naps © pacem page Irôqps © pacema © pager-naps sentiment sentiment sentiment

Robust Visual-Textual Sentiment Analysis: When Attention ...llcao.net/paper/t-lstm-acmmm16.pdf · textual features and mid-level visual features [2]. More recently, You et al. [42]

MazeExplorer: A Customisable 3D Benchmark for Assessing ...ieee-cog.org/2020/papers2019/paper_210.pdfA2C A2C + LSTM Fig. 2: Learning curves for PPO, A2C, PPO+LSTM and A2C+LSTM, smoothed

Learning Statistical Scripts with LSTM Recurrent Neural ... · PDF fileLearning Statistical Scripts with LSTM Recurrent Neural Networks ... • Language Modeling ... LSTM Script models

Twitter Sentiment Analysis via Bi-sense Emoji Embedding and … · 2018-08-08 · Twitter Sentiment Analysis via Bi-sense Emoji Embedding and Attention-based LSTM Yuxiao Chen∗ Department

GRID-LSTM€¦ · Grid-LSTM: Motivation Stacked LSTM, but LSTM units connections along depth dimension as well as temporal dimension

Proﬁling Resource Bottlenecks in Sentiment Analysis · agogical example for LSTM training, performing senti- ... In monitoring a distributed Apache Spark system the researchers

LSTM-in-LSTM for generating long descriptions of …LSTM-in-LSTM for generating long descriptions of images 381 VggNet [17]). Object detection systems based on a well trained DeepCNN

Lstm shannonlab

LSTM 및 CNN-LSTM 신경망을 활용한 도시부 간선도로 속도 예측

Context-driven Multi-stream LSTM (M-LSTM) for Recognizing ...In this paper, we propose a novel deep neural network called Multi-stream LSTM (M-LSTM) for recognising ne-grained activities

On behalf of the PROSPECT collaboration Andrea Delgado, PhD · Open slide master to edit •Siamese LSTM for Pulse MatchingCast pulse matching to a “similarity” problem. • Keras-based

Retail sales forecasting using LSTM and ARIMA-LSTM: A

COMPARISON OF RNN, LSTM AND GRU ON SPEECH … · 2020. 1. 24. · class of RNN, Long Short-Term Memory [LSTM] networks. LSTM networks have special memory cell structure, which is

LSTM Online Archive BRAIN

lstm neural network

LSTM-based sentiment analysis for stock price forecastStock price forecast In the past few decades, more and more people have conducted researches of stock prices forecast with the