Research ArticleBullet Subtitle Sentiment Classification Based on AffectiveComputing and Ensemble Learning

Lei Yu ,1 Yu Wu ,2 Jie Yang ,3 and Yunkai Zhang 1

1School of Computer Science and Technology, Chongqing University of Posts and Telecommunications, Chongqing 400065, China2School of Cyber Security and Information Law, Chongqing University of Posts and Telecommunications, Chongqing 400065, China3School of Innovation and Entrepreneurship Education, Chongqing University of Posts and Telecommunications,Chongqing 400065, China

Correspondence should be addressed to Yu Wu; wuyu@cqupt.edu.cn

Received 25 January 2021; Accepted 10 June 2021; Published 25 June 2021

Academic Editor: Hasan Khattak

Copyright © 2021 Lei Yu et al. This is an open access article distributed under the Creative Commons Attribution License, whichpermits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

The bullet subtitle reflects a kind of instant feedback from the user to the current video. It is generally short but contains richsentiment. However, the bullet subtitle has its own unique characteristics, and the effect of applying existing sentimentclassification methods to the bullet subtitle sentiment classification problem is not ideal. First, since bullet subtitles usuallycontain a large number of buzzwords, existing sentiment lexicons are not applicable, we propose Chinese Bullet SubtitleSentiment Lexicon on the basis of existing sentiment lexicons. Second, considering that some traditional affective computingmethods only consider the text information and ignore the information of other dimensions, we construct a bullet subtitleaffective computing method by combining the information of other dimensions of the bullet subtitle. Finally, aiming at theproblem that existing classification algorithms ignore the importance of sentiment words in short texts, we propose a sentimentclassification method based on affective computing and ensemble learning. Our experiment results show that the proposedmethod has higher accuracy and better practical application effect.

1. Introduction

In recent years, the bullet subtitle video site has becomemore and more popular among young people, and Bilibiliis one of the typical representatives. The bullet subtitle texthas the characteristics of traditional short text such as shorttext length and sparse features, but it is different from tra-ditional short text at the same time. First, the bullet subtitletext is more simplified, colloquially, and symbolized; sec-ond, it often contains many teasing contents; third, it alsocontains conversational properties. Analyzing the sentimentof the bullet subtitle text, and then analyzing the overallsentiment distribution of the bullet subtitle video, can beused as a reference basis for website operators to make per-sonalized video recommendations to users and can alsoprovide decision support for public opinion governance incyberspace security.

Text sentiment classification is a key technique for min-ing sentiment in the field of natural language processing

and is now widely used in public opinion monitoring, market-ing, and finance [1]. There are two main types of sentimentclassification methods: sentiment lexicon-based methods andmachine learning-based methods. The two methods have dif-ferent advantages in different fields.

In the sentiment lexicon-based classification method, thesentiment lexicon used to calculate the sentiment value of thetext plays a key role. Currently, English sentiment lexiconsmainly use Word-Net-Affect [2] and Senti-WordNet [3],and Chinese sentiment lexicons mainly use HowNet ChineseSentiment Lexicon and the Sentiment Vocabulary OntologyLibrary proposed by the Dalian University of Technology[4]. Based on the English sentiment lexicon, Taboada et al.[5] proposed a semantic-oriented calculation method SO-CAL, which is based on the lexicon to classify the sentimentpolarity of the text. Based on the Chinese sentiment lexicon,Zhang et al. [6] realized the sentiment analysis of Chinesemicroblog text by extending the sentiment lexicon. Yaoet al. [7] added Weibo emoji to the sentiment lexicon, which

HindawiWireless Communications and Mobile ComputingVolume 2021, Article ID 5563104, 9 pageshttps://doi.org/10.1155/2021/5563104

improved the result of sentiment judgment on Weibo con-taining emoji. Li et al. [8] proposed a seven-dimensionalaffective computing method for the bullet subtitle, which isrelatively simple and only uses text information. The senti-ment lexicon-based classification method performs well fortext data containing sentiment words, but cannot handle textdata without sentiment words and has insufficient generali-zation ability.

In machine learning-based classification methods, prela-beled training data is usually needed to train the model andthen make predictions on the data to be classified. Word2Vecmodel proposed by Mikolov et al. [9] effectively solves theproblem of dimensionality disaster caused by traditional fea-ture extraction methods and takes into account the semanticrelationships between words. Zhang et al. [10] combinedWord2Vec and SVMperf to improve the accuracy of senti-ment classification tasks. Ma and Zhang [11] proposed abullet subtitle sentiment polarity analysis model based onTF-IDF and SVM, which can effectively classify the senti-ment of bullet subtitle texts. However, this method ignoresthe semantic information of words and can only analyze sim-ple bullet subtitle texts. Liu and Xing [12] improved the ran-dom forest algorithm by studying its shortcomings in textclassification and optimized the hyperparameter tuning toimprove the performance of the model. In recent years, neu-ral networks and deep learning have been developing moreand more rapidly, Recurrent Neural Network (RNN) [13]and its corresponding variants Long Short-Term Memory(LSTM) [14] and Gated Recurrent Unit (GRU) [15] also per-form well in text classification. Kim [16] proposed a convolu-tional neural network model TextCNN for text sentimentclassification. The model hardly needs to adjust parametersto achieve high classification accuracy, but the model ignoresthe sequential relationship between words in the text. Onan[17] proposed a five-layer neural network based on CNN-LSTM, which performs better in sentiment analysis than tra-ditional neural networks. Yuan et al. [18] proposed apersonality-based Weibo sentiment analysis model, PLSTM,which trains different classifiers through the Weibo of differ-ent personalities, and finally fuses the results. Onan andToçoğlu [19] proposed a three-layer stacked bidirectionallong and short-term memory architecture to identify satir-ical text documents, and the model’s classification accu-racy rate reached 95.3%. Zhuang and Liu [20] extractedhighlight video clips by combining attentional mechanismsand LSTM for sentiment classification. Most of the abovemethods are based on Word2Vec for feature extraction.Although Word2Vec effectively captures the position andlexical meaning information of words, the bullet subtitletext belongs to the short text, and using Word2Vec aloneis not effective.

Ensemble learning is to train multiple weaker classi-fiers and a certain fusion strategy to finally get a powerfulclassifier. Wang and Yue [21] pointed out that in specificproblems and scenarios, combining the advantages of mul-tiple models, the classification results may be better. Xieet al. [22] proposed a high-precision EEG sentiment recog-nition model by fusing XGBoost, LightGBM, and randomforest.

The contributions of this paper mainly include first, byanalyzing the bullet subtitle text data and collecting buzz-words on the Internet in recent years, we obtain the buzz-word lexicon and the emoji lexicon, and finally, they arecombined with the Sentiment Vocabulary Ontology Libraryto construct the Chinese Bullet Subtitle Sentiment Lexicon.Second, using the multiple dimensional information of thebullet subtitle, gain intensity is constructed, and combinedwith the text affective computing method, we propose BulletSubtitle Calculation (BS-CAL), a multidimensional senti-ment value calculation method for bullet subtitle. Finally,based on the analysis of existing algorithms, we propose asentiment classification method for the bullet subtitle text,and comparative experiments are conducted between differ-ent methods and data sets.

2. Related Work

2.1. GRU. GRU [15] is a variant of RNN. Compared withLSTM, GRU has the advantages of fewer parameters and fas-ter convergence speed, which can greatly reduce the trainingtime of the model and achieve results close to or even sur-passing LSTM (see Figure 1).

2.2. Ensemble Learning. For some classification tasks, it maybe more difficult to train a single learner with the best perfor-mance, but it is possible to combine multiple learners withgeneral performance into a combined classifier with betterperformance. Ensemble learning is divided into homoge-neous ensemble learning and heterogeneous ensemble learn-ing. Homogeneous ensemble learning means that eachlearner is of the same type. Such learners are also called baselearners. This method usually cooperates with boosting andbagging learning method, while heterogeneous ensemblelearning means that each learner is of a different type. Suchlearners are also called component learners, and this methodis usually used in conjunction with a fusion strategy.

2.3. Other Methods. Word2Vec model [9] effectively solvesthe problems of dimensional disaster and lexical dividebrought by traditional text data feature extraction methodssuch as bag-of-words model, one-hot, and tf-idf.

The attention mechanism [23] is based on the differentdistribution of human attention when observing objects.When the attention mechanism is applied to a machine

+x

1–

SS

x x

T

Figure 1: GRU structure.

2 Wireless Communications and Mobile Computing

learning model, the model can focus on the current impor-tant information and fully learn and absorb, thereby improv-ing the classification performance of the model.

Naive Bayes [24] is a generative model, which is an opti-mized classifier based on Bayesian networks and Bayesianstatistics. The model assumes that each feature is indepen-dent of each other, can predict the posterior probability ofthe sample belonging to each category based on the prior dis-tribution of sentiment words in the sentiment lexicon, andselect the category with the highest probability as the predic-tion result.

3. Bullet Subtitle SentimentClassification Method

3.1. Sentiment Classification Process. Currently, existing Chi-nese sentiment lexicons do not contain buzzwords in bulletsubtitle texts, which will lead to inaccurate affective comput-ing or even the inability to calculate sentiment. Therefore, wecrawl a large amount of bullet subtitle data from Bilibili andconduct systematic analysis, and we collect common buzz-words on the Internet, and finally fuse them to obtain theChinese Bullet Subtitle Sentiment Lexicon. At the same time,the analysis reveals that the information of other dimensions

of the bullet subtitle has some influence on the bullet subtitlesentiment, which is not taken into account by the traditionaltext affective computing method. Therefore, we propose BS-CAL by fusing the textual information of bullet subtitle textsand other dimensional information. Finally, the classificationproblem of bullet subtitle sentiment can be handled in twocases according to whether the bullet subtitle contains senti-ment words: when it contains sentiment words, a heteroge-neous ensemble learning method is used for prediction;when it does not contain sentiment words, the sentimentlexicon-based method cannot be used, and it is backed offto a single model method for prediction (see Figure 2).

3.2. Chinese Bullet Subtitle Sentiment Lexicon Construction.The sentiment lexicon used in this paper is mainly derivedfrom the Sentiment Vocabulary Ontology Library [4]. Thelexicon contains more than 27,000 sentiment words anddescribes sentiments in terms of seven sentiment dimen-sions, including happiness, like, surprise, sadness, fear, anger,and disgust. Happiness and like belong to the positive senti-ment, while the other five belong to the negative sentiment.The weight of each sentiment is divided into five levels: 1,3, 4, 7, and 9, with 1 representing the lowest weight and 9representing the highest weight.

Sentimentlexicon

Bilibili

Begin

End

Obtaining bulletsubtitle data

Whether it containssentiment words

Preprocessing data

Single classifierBS-CALclassifier Other classifiers

Tuning and evaluation ofclassification performance

Yes No

Figure 2: Sentiment classification flowchart.

3Wireless Communications and Mobile Computing

There are a large number of special words, buzzwords,and internet phrases in the bullet subtitle text, and weobtained the bullet subtitle buzzword lexicon containing2659 words. At the same time, the bullet subtitle text containsa large number of emojis, which contain rich information.Therefore, we collected 431 emojis as the emoji lexicon.

According to the ranking criteria of the SentimentVocabulary Ontology Library, the above buzzwords andemojis are artificially weighted and scored. Finally, the Senti-ment Vocabulary Ontology Library, the bullet subtitle buzz-word lexicon, and the emoji lexicon are fused together toget a relatively complete Chinese Bullet Subtitle SentimentLexicon. The weights and sentiment categories of some emo-jis are shown in Table 1.

3.3. Affective Computing. In the problem of text sentimentanalysis, sometimes it is necessary not only to know thesentiment tendency of the text but also to represent thesentiment quantitatively, while the current text affectivecomputing method does not combine the characteristicsof the bullet subtitle, resulting in poor quantitative accuracyof sentiment. In view of this, this paper proposes BS-CALto describe the sentiment value of each dimension in detail.Some users deliberately choose some prominent bullet sub-title formats such as font size and color in order to expresstheir strong sentiments when sending bullet subtitles.Therefore, BS-CAL combines the information of otherdimensions of the bullet subtitle for quantitative calculationbased on the traditional text affective computing method.The calculation formula of BS-CAL is as follows:

sentiValue d, cð Þ = ξ d, cð Þ + textValue d, cð Þ, ð1Þ

where sentiValueðd, cÞ is the sentiment value of bullet sub-title d under sentiment category c, and ξðd, cÞ is the gainintensity of the bullet subtitle itself. The calculation formulaof ξðd, cÞ is as follows:

ξ d, cð Þ =0, if textValue d, cð Þ = 0,

Wcj jfontSize dð Þ+θ dð Þ+η dð Þ, otherwise,

(

ð2Þ

where Wc is the set of sentiment words belonging to cate-gory c, fontSizeðdÞ is the font size of the bullet subtitle, gen-erally speaking, the bigger the font, the stronger thesentiment. θðdÞ is whether the color of the bullet subtitleis the default color black. The calculation formula of θðdÞis as follows:

θ dð Þ =1, if color ≠ black,0, otherwise:

(ð3Þ

ηðdÞ is whether the bullet subtitle is a special bullet sub-title such as a flashing bullet subtitle or a rotating bulletsubtitle. The calculation formula of ηðdÞ is as follows:

η dð Þ =1, if type ≠ 1,0, otherwise:

(ð4Þ

textValueðd, cÞ is the text sentiment value of bullet sub-title d under sentiment category c. The calculation formulais as follows:

textValue d, cð Þ = 〠∣Wc∣

w=1−1ð Þnegw+φw,cμw 〠

∣Pw∣

p=1αp

Y∣Dw∣

d=1βd , ð5Þ

where negw is the number of negatives preceding the senti-ment word w, μw is the magnitude of the sentiment valueof the word itself, Pw is the set of sentiment punctuationsimmediately following the sentiment word w, αp is the sen-timent value of the sentiment punctuation, Dw is the set ofdegree adverbs preceding the sentiment word w, and βd isthe strength of the degree adverbs. φw,c is the sentimentreversal variable of word w in the sentiment category cwhen calculating the sentiment category c. The calculationformula of φw,c is as follows:

φw,c =1, c ∈ Happiness, Like½ � and?∈Pw,0, otherwise:

(ð6Þ

For the sentiment category c, BS-CAL ssssthm is describedas follows:

Assuming that the number of words of bullet subtitle is n,the time complexity of BS-CAL is linear complexity OðnÞ. Inthe task of this paper, the bullet subtitle with sentiment wordscan be directly calculated according to the above formula,and the final sentiment classification can be obtained by

Table 1: Part of the emoji lexicon.

Emoji Weight Categories Emoji Weight Categories

(∗^-^∗) 9 Like 1 Happiness

(∗≧m≦∗) 3 Happiness 3 Happiness

o(≧▽≦)o 7 Like 5 Sadness

(`へ´) 5 Anger 3 Sadness

￣△￣ 1 Sadness 5 Anger

(┬_┬) 5 Sadness 3 Fear

(￣m￣) 1 Surprise 3 Like

(-_-;) 1 Disgust 3 Disgust

\(≧▽≦)/ 5 Happiness 7 Fear

o(≧口≦)o 3 Fear 3 Disgust

(∗￣︿￣) 3 Anger 1 Happiness

(́ Д`) 3 Sadness 5 Anger

4 Wireless Communications and Mobile Computing

judging the relationship between the sum of positive senti-ment values and the sum of negative sentiment values.

3.4. Classification Algorithm. In the problem of bullet subtitletext classification, there are many sentiment words that canplay a decisive role in the result of text classification, butsome bullet subtitle texts do not contain any sentimentwords, which leads to the classification method based onthe sentiment lexicon is not fully applicable in bullet subtitlesentiment analysis. In view of this, we divided the bullet sub-title into two categories for processing according to whetherthey contain sentiment words: when no sentiment wordsare included, the Gated Recurrent Unit classification modelcombined with the attention mechanism is used for predic-tion (ATT-GRU); when sentiment words are included, theheterogeneous ensemble learning method of BS-CAL, NaiveBayes, and ATT-GRU three models is adopted.

Using the three models as above can fuse the advantagesof each model, ATT-GRU classification model based onWord2Vec makes full use of the semantic and positional rela-tionships between words; the BS-CAL method is good athandling the bullet subtitle containing sentiment words andhas a high performance in classifying the bullet subtitle withstrong sentiment; Naive Bayes method based on sentiment

lexicon fully considers the implicit influence brought by dif-ferent combinations of sentiment words. The model con-struction is shown in Figure 3.

We abbreviate the bullet subtitle sentiment classificationmethod as BSSCM. The specific algorithm is described asfollows:

Assuming that each bullet subtitle contains n words, thetime complexity of the above algorithm isOðm ∗ nÞ for NaiveBayes classifier, Oðm ∗ nÞ for BS-CAL, and Oðn ∗m2Þ forATT-GRU, which has the highest time complexity. Accord-ing to the heterogeneous ensemble learning strategy, theoverall time complexity is determined by the componentlearner with the highest complexity, so the overall time com-plexity is Oðn ∗m2Þ.

4. Experiments and Evaluations

4.1. Data Set. In this paper, we use a crawler program to crawlthe bullet subtitle data in Bilibili and filter out about 120,000bullet subtitle data in the fields of animation, movies, andother major events such as Huawei event and TikTok event,and finally retain about 100,000 bullet subtitle data after datacleaning. Among them, each bullet subtitle data contains 9

ATT-GRU Data d Prediction value y

(a) Without sentiment words

BS-CAL

ATT-GRU

Naive bayes

Data d Prediction value y

Heterogeneousensemble

(b) With sentiment words

Figure 3: The model construction.

Input: bullet subtitle d, the Chinese Bullet Subtitle Sentiment LexiconOutput: the sentiment value of bullet subtitle d under sentiment category cStep 1: extract all sentiment words Wc belonging to sentiment category c in d; initialize i = 0, score = 0, tmp = 0Step 2: find all the degree adverbs Dwi and the number of negatives negwi

between the sentiment word wi and the previous sentiment

wordStep 3: find all the sentiment punctuation Pwi

between the sentiment word wi and the next sentiment wordStep 4: use equation (6) to obtain the sentiment reversal variable φwi ,cStep 5: use equation (5) to calculate the text sentiment value of the current sentiment wordStep 6: when i is greater than or equal to ∣Wc ∣ , execute Step7; otherwise, execute Step2, tmp = tmp + value, i = i + 1Step 7: calculate the gain intensity ξ of d using the corresponding dimensional features of d and equation (2); score = tmp + ξStep 8: output score and the algorithm ends.

Algorithm 1

5Wireless Communications and Mobile Computing

dimensional features, and the information of each feature isshown in Table 2.

About 13,000 data were extracted from all the bullet sub-title data for manual sentiment annotation as the trainingdata set, and finally, about 6,000 positive sentiment bulletsubtitles and about 7,000 negative sentiment bullet subtitleswere obtained, and about 8,500 bullet subtitles with senti-ment words were included in the annotated data.

In order to verify the stability of the classification perfor-mance of the method proposed in this paper, the bullet sub-title under the recent hot event video was crawled. Thedataset contains three events, event one is “Ma becamefamous overnight” (event 1), event two is “nCoV mutation”(event 2), and event three is “innocent young man Ding”(event 3). The specific data is shown in Table 3.

4.2. Experiment Scheme

4.2.1. Sentiment Lexicon Comparison. In order to verify thatthe sentiment lexicon proposed in this paper is more applica-ble to the bullet subtitle domain, we compare it with the Chi-nese Sentiment Vocabulary Ontology Library [4] andHowNet Sentiment Lexicon for experiments, and the experi-mental algorithm uses the bullet subtitle sentiment classifica-tion method proposed in this paper.

4.2.2. Affective Computing Method Comparison. In order toverify that the BS-CAL method proposed in this paper canquantify the bullet subtitle sentiment more effectively, wecompare it with Traditional Text Calculation (TTC) and

the Sentiment Value Calculation of Danmaku (SVCD)method proposed by Li et al. [8].

4.2.3. Algorithm Comparison. In order to verify the effective-ness of the method proposed in this paper, we compare withthe following algorithms: Ma and Zhang [11] proposed a TF-IDF and SVM-based sentiment polarity analysis model of thebullet subtitle text. This method is the first to use machine

Table 2: Feature information of the bullet subtitle.

Feature number Example Meaning

1 3337.11600 The time of video start (second)

2 5 The type of the bullet subtitle

3 25 The font size of the bullet subtitle

4 16777215 The font color of the bullet subtitle (decimal)

5 1554387395 The unix timestamp

6 0 The type of the bullet subtitle pool

7 d826ed2 The sender’s encrypted uid

8 14284342245720068 The rowID of the bullet subtitle in the database

9 Hahahaha really funny The bullet subtitle text

Table 3: Empirical study data.

Event Original dataset Marked dataset

Event 1 18,102 1,800

Event 2 24,752 2,400

Event 3 21,547 2,100

Table 4: Sentiment lexicon comparison results.

Sentiment dictionary Class Accuracy Precision Recall F1

HowNetN

0.8330.825 0.802 0.813

P 0.764 0.963 0.852

DUT-SDN

0.8970.872 0.911 0.891

P 0.914 0.897 0.905

BS-SDN

0.9490.935 0.951 0.943

P 0.965 0.947 0.956

Input: a bullet subtitle setDfd1, d2, d3,⋯, dmg, the Chinese Bullet Subtitle Sentiment LexiconOutput: the sentiment category dic of bullet subtitle diStep 1: split di into words and initializing f lag = 1Step 2: determine whether di contains sentiment words, if it does, execute Step3; otherwise, execute Step5, f lag = 0Step 3: input di into BS-CAL described in Algorithm 1 and get the classification result aStep 4: input di into Naive Bayes classifier to obtain the classification result bStep 5: use one-hot to process the splited data, and feed into ATT-GRU classifier to get the classification result cStep 6: if f lag == 1, then dic is the result of the fusion of a, b, c; otherwise, c is the final result of dicStep 7: output the bullet subtitle sentiment category dic and calculate the accuracy, precision, recall and F1-score.

Algorithm 2

6 Wireless Communications and Mobile Computing

learning methods for the bullet subtitle sentiment classifica-tion problem; Zhuang and Liu [20] proposed the sentimentanalysis based on AT-LSTM, which performed well in thesentiment classification problem of the bullet subtitle; Ada-Boost and Random Forest are typical representatives ofensemble learning methods. Naive Bayes [31] and TextCNN[16] are often used as benchmarks for sentiment classifica-tion problems.

4.2.4. Algorithm Execution Time Comparison. We comparethe training time and test time in different models to analyzemodel performance.

4.2.5. Empirical Analysis. The practicality of the classificationmethod proposed in this paper is verified by conductingexperiments on the latest events (see Table 3).

4.3. Evaluation Metrics. In this paper, we use a common eval-uation metric for text sentiment classification to evaluate theexperimental effect, including accuracy, precision, recall, andF1-score. Precision mainly measures the classifier perfor-mance from prediction results, recall mainly measures theclassifier performance from the sample itself, and F1-scorecan be combined with these two metrics to evaluate the clas-sifier performance as a whole.

5. Results and Analysis

5.1. Sentiment Lexicon Comparison. This paper verifies theeffectiveness of the sentiment lexicon proposed in this paperby conducting comparison experiments with other differentsentiment lexicons, and the classification method used isBSSCM proposed in this paper. The results are shown inTable 4, where DUT-SD represents the Sentiment Vocabu-lary Ontology Library, BS-SD represents the sentimentlexicon proposed in this paper, and N and P denote the neg-ative and positive sentiment categories.

As shown in Table 4, BS-SD has the highest classificationaccuracy compared with the other two traditional sentimentlexicons, and although HowNet sentiment lexicon has thehighest recall in the positive category, it has the worst relativeaccuracy, while precision and F1-score of BS-SD are higherthan the other lexicons in the positive and negative catego-ries. The above results show that BS-SD performs better inthe bullet subtitle sentiment analysis task.

5.2. Affective Computing Method Comparison. In order toverify that BS-CAL proposed in this paper can more accu-rately quantify the bullet subtitle sentiment, we compare itwith different computing methods. The results are shownin Table 5.

As can be seen from Table 5, SVCD performs better onnegative sentiment, with the highest F1-score of 89%, whileBS-CAL achieves 92.5% precision on positive sentimentand has the relatively highest F1-score and classificationaccuracy. In summary, BS-CAL can quantify the bullet subti-tle sentiment more accurately.

5.3. Algorithm Comparison. In order to verify the accuracyand generalization ability of the classification method pro-posed in this article, we compare different algorithms withit. The results are shown in Table 6.

From Table 6, it can be seen that AT-LSTM has the high-est precision of 96.2% in the negative category, and ATT-GRU has the best overall performance in the negative

Table 6: Algorithm comparison results.

Algorithm Class Accuracy Precision Recall F1

Naive BayesN

0.9370.900 0.963 0.930

P 0.952 0.939 0.945

TF-IDF + SVMN

0.7810.878 0.740 0.803

P 0.675 0.871 0.761

AT-LSTMN

0.9180.962 0.913 0.937

P 0.845 0.929 0.885

ATT-GRUN

0.9340.933 0.961 0.947

P 0.936 0.893 0.927

TextCNNN

0.9160.901 0.913 0.907

P 0.944 0.910 0.927

AdaBoostN

0.9370.924 0.945 0.934

P 0.958 0.923 0.940

RFN

0.9380.914 0.944 0.929

P 0.940 0.955 0.947

BSSCMN

0.9460.909 0.963 0.935

P 0.958 0.954 0.956

Table 7: Algorithm execution time results.

TimeModel Training time (ms) Testing time (ms)

BS-CAL 0 101.37

BS-SD+NB 3677.44 558.72

TF-IDF + SVM 12549.54 357.48

AT-LSTM 21333.26 289.88

ATT-GRU 19565.23 260.65

AdaBoost 25323.63 923.40

RF 19687.44 634.28

BSSCM 23242.68 795.36

Table 5: Computing method comparison results.

Affective computingmethod

Class Accuracy Precision Recall F1

TTCN

0.8120.716 0.833 0.770

P 0.794 0.923 0.854

SVCDN

0.8660.893 0.886 0.890

P 0.872 0.814 0.842

BS-CALN

0.9020.879 0.892 0.885

P 0.925 0.911 0.918

7Wireless Communications and Mobile Computing

category. Both AdaBoost and BSSCM have the highest pre-cision of 95.8% in the positive category, and RF has thehighest recall in the positive category. BSSCM has the high-est classification accuracy of 94.6% and best performance inthe positive category. In summary, for the bullet subtitlesentiment classification problem, BSSCM has the best over-all performance.

5.4. Algorithm Execution Time Comparison. This paper hascalculated the training and testing time of each model, andthe final results are shown in Table 7.

The above experiments were conducted on a PC com-puter with 2.6GHz hexa-core Intel Corei7. From Table 7, itcan be seen that AdaBoost takes the longest time in training,while BSSCM is the second. BS-CAL does not require train-ing and can directly perform prediction, so the training timeis 0. The ensemble learning method takes longer in the testphase, AdaBoost takes the longest time in testing, BSSCMtakes the second longest, and BS-CAL does not require morecomplex model parameters, so the test time is also theshortest.

5.5. Empirical Analysis. In order to verify the practicability ofthe bullet subtitle sentiment classification method proposedin this paper, we use three events for experimenting (seeTable 3), and the results are shown in Figure 4.

Figure 4 shows that under different hot topics, the perfor-mance of each model is slightly different. RF has a higheraccuracy than BSSCM in event 1, which may be due to thepresence of some unregistered words in this topic, whichleads to a slight decrease in the performance of BSSCM, whilethe effect of BSSCM on other topics is significantly betterthan other models. Therefore, the method proposed in thispaper is relatively effective in the sentiment classification ofthe bullet subtitle.

6. Conclusion

Based on the traditional sentiment lexicon, this paper pro-poses the Chinese Bullet Subtitle Sentiment Lexicon by com-bining the bullet subtitle buzzword lexicon and the emojilexicon. Then, combined with the information of otherdimensions of the bullet subtitle, BS-CAL is proposed.Finally, considering that the bullet subtitle text does not nec-essarily contain sentiment words, a single classificationmodel ATT-GRU and a heterogeneous ensemble learningmethod based on BK-CAL, Naive Bayes, and ATT-GRU areproposed to deal with different situations, respectively.

However, this article still has some shortcomings. First, astime goes by, there will be more and more new vocabulary,and the timeliness of the lexicon is poor. The lexicon can befurther studied, such as automatic mining of buzzwords inthe bullet screen, so that the lexicon can be adaptivelyupdated. Second, there is no public dataset for the bullet sub-title and the labeled data in this paper is subjective to a cer-tain extent. Therefore, a general dataset for the bulletsubtitle domain can be considered to propose in the future.Finally, this paper divides the sentiment categories into pos-itive and negative. In the future, we can study how to classifysentiment from a more granular perspective.

Data Availability

The data used to support the findings of this study areincluded in the article.

Conflicts of Interest

The authors declare that there are no conflicts of interestregarding the publication of this paper.

Acc

urac

y

Prec

ision

TextCNNATT-GRUAdaBoot

RFBSSCM

0

0.2

0.4

0.6

0.8

1

Event 1 Event 2 Event 20

0.2

0.4

0.6

0.8

1

Event 1 Event 2 Event 3

Reca

ll

0

0.2

0.4

0.6

0.8

1

Event 1 Event 2 Event 30

0.2

0.4

0.6

0.8

1

Event 1 Event 2 Event 2

F1-S

core

Figure 4: Empirical analysis results.

8 Wireless Communications and Mobile Computing

Acknowledgments

This work was supported by the Research on Cyber GroupEvents Management and Pre-warning Mechanism, TheNational Social Science Fund of China (17XFX013).

References

[1] E. Cambria, “Affective computing and sentiment analysis,”IEEE Intelligent Systems, vol. 31, no. 2, pp. 102–107, 2016.

[2] C. Strapparava and A. Valitutti, “W ordnet-affect: an affectiveextension of wordnet,” Proc. Int'l Conf. Language Resourcesand Evaluation, vol. 4, pp. 1083–1086, 2004.

[3] S. Baccianella, A. Esuli, and F. Sebastiani, “SentiWordNet 3.0:an enhanced lexical resource for sentiment analysis and opin-ion mining,” LREC, vol. 10, pp. 2200–2204, 2010.

[4] J. M. Chen, H. F. Liu, and Z. H. Yang, “Automatic acquisitionof emotional vocabulary based on syntax,” CAAI Transactionson Intelligent Systems, vol. 4, no. 2, pp. 100–106, 2009.

[5] M. Taboada, J. Brooke, M. Tofiloski, K. Voll, and M. Stede,“Lexicon-based methods for sentiment analysis,” Computa-tional Linguistics, vol. 37, no. 2, pp. 267–307, 2011.

[6] S. X. Zhang, Z. L. Wei, Y. Wang, and T. Liao, “Sentiment anal-ysis of Chinese micro-blog text based on extended sentimentdictionary,” Future Generation Computer Systems, vol. 81,pp. 395–403, 2018.

[7] H. P. Yao, C. Fang, J. D. Zhang, and P. Y. Zhang, “Microblog-ging sentiment analysis method with the combination ofsemantic rules and emoticon weighting,” Journal of ChongqingUniversity of Posts and Telecommunications (Natural ScienceEdition), vol. 28, no. 4, pp. 503–510, 2016.

[8] Z. Li, R. Li, and G. Jin, “Sentiment analysis of danmaku videosbased on Naïve Bayes and sentiment dictionary,” IEEE Access,vol. 8, pp. 75073–75084, 2020.

[9] T. Mikolov, I. Sutskever, K. Chen, G. Corrado, and J. Dean,“Distributed representations of words and phrases and theircompositionality,” Advances in neural information processingsystems, pp. 3111–3119, 2013.

[10] D. W. Zhang, H. Xu, Z. C. Su, and Y. F. Xu, “Chinese com-ments sentiment classification based on word2vec and SVM,”Expert Systems with Applications, vol. 42, no. 4, pp. 1857–1863, 2015.

[11] M. X. Ma and J. Zhang, “Sentiment polarity analysis model ofdanmaku text based on TF-IDF and SVM,” Modern BusinessTrade Industry, vol. 40, no. 8, pp. 205–207, 2019.

[12] Y. Liu and Y. Y. Xing, “Research and application of text classi-fication based on improved random forest algorithm,” Com-puter Systems & Applications, vol. 28, no. 5, pp. 102–107, 2019.

[13] J. J. Hopfield, “Neural networks and physical systems withemergent collective computational abilities,” PNAS, vol. 79,no. 8, pp. 2554–2558, 1982.

[14] S. Hochreiter, “The vanishing gradient problem during learn-ing recurrent neural nets and problem solutions,” Interna-tional Journal of Uncertainty, Fuzziness and Knowledge-Based Systems, vol. 6, no. 2, pp. 107–116, 1998.

[15] K. Cho, B. V. Merrienboer, C. Gulcehre et al., “Learning phraserepresentations using RNN encoder–decoder for statisticalmachine translation,” in Proceedings of the 2014 Conferenceon Empirical Methods in Natural Language Processing(EMNLP), pp. 1724–1734, 2014.

[16] Y. Kim, “Convolutional neural networks for sentence classifi-cation,” in Proceedings of the 2014 Conference on EmpiricalMethods in Natural Language Processing (EMNLP),pp. 1746–1751, 2014.

[17] A. Onan, “Sentiment analysis on product reviews based onweighted word embeddings and deep neural networks,” Con-currency and Computation: Practice and Experience, articlee5909, 2020.

[18] T. T. Yuan, W. Z. Yang, L. J. Zhong, Z. H. Zhang, and J. Y.Xiang, “Personality-based microblog sentiment analysis modelPLSTM,” Application Research of Computers, vol. 37, no. 2,pp. 342–346, 2020.

[19] A. Onan and M. A. Toçoğlu, “A term weighted neural lan-guage model and stacked bidirectional LSTM based frame-work for sarcasm identification,” IEEE Access, vol. 9,pp. 7701–7722, 2021.

[20] X. Q. Zhuang and F. A. Liu, “Emotional analysis of bullet-screen comments based on AT-LSTM,” Digital Technology &Application, vol. 36, no. 2, pp. 210–212, 2018.

[21] D. Wang and X. S. Yue, “The weighted multiple meta-modelsstacking method for regression problem,” in 2019 ChineseControl Conference (CCC), 2019.

[22] Q. Xie, Z. T. Liu, and X. W. Ding, “Electroencephalogramemotion recognition based on a stacking classification model,”in 2018 37th Chinese Control Conference (CCC), pp. 430–434,2018.

[23] D. Bahdanau, K. Cho, and Y. Bengio,Neural Machine Transla-tion by Jointly Learning to Align and Translate, InternationalConference on Learning Representations, 2015.

[24] D. Lewis, “Naive (Bayes) at forty: the independence assump-tion in information retrieval,” in European Conference onMachine Learning, pp. 4–15, 1998.

9Wireless Communications and Mobile Computing