A Multi-Channel Text Sentiment Analysis Model Integrating Pre-training Mechanism





Sentiment Analysis, Tourism Text, Multi-channel, BERT, BiLSTM, Pre-training mechanism


The number of tourist attractions reviews, travel notes and other texts has grown exponentially in the Internet age. Effectively mining users’ potential opinions and emotions on tourist attractions, and helping to provide users with better recommendation services, which is of great practical significance. This paper proposes a multi-channel neural network model called Pre-BiLSTM combined with a pre-training mechanism. The model uses a combination of coarse and fine- granularity strategies to extract the features of text information such as reviews and travel notes to improve the performance of text sentiment analysis. First, we construct three channels and use the improved BERT and skip-gram methods with negative sampling to vectorize the word-level and vocabulary-level text, respectively, so as to obtain more abundant textual information. Second, we use the pre-training mechanism of BERT to generate deep bidirectional language representation relationships. Third, the vectors of the three channels are input into the BiLSTM network in parallel to extract global and local features. Finally, the model fuses the text features of the three channels and classifies them using SoftMax classifier. Furthermore, numerical experiments are conducted to demonstrate that Pre-BiLSTM outperforms the baselines by 6.27%, 12.83% and 18.12% in average in terms of accuracy, precision and F1-score.

Author Biographies

Shengbin Liang, School of Software, Henan University, Kaifeng, 475004, China; Institute for Data Engineering and Science, University of Saint Joseph, Macao, 999078, China

Shengbin Liang received his M.S. degree form Southwest Jiaotong University in Chengdu, China and he received his Ph.D degree at City University of Macau. He is currently an associate professor work in Henan University, China and now he is also doing his postdoctoral research at the University of Saint Joseph. His research is mainly in the area of recommendation systems and deep learning. He has published several research papers in scholarly journals in the above research areas and has participated in servral conferences.


Jiangyong Jin, School of Software, Henan University, Kaifeng, 475004, China

Jiangyong JIN received his bachelor's degree in Henan University in Kaifeng, China. He currently is a condidate master student at Henan University, his research is mainly in the area of machine learning and recommendation systems and algorithm.


Wencai Du, Institute for Data Engineering and Science, University of Saint Joseph, Macao, 999078, China

Wencai DU received the B.S. from Peking University, China, two Master Degrees from Twente University (ITC), The Netherlands and Hohai University, China, respectively, the Ph.D. degree from South Australia University, Australia, and Post-doct fellow in Israel Institute of Technology, Haifa, Israel.

From 2002 to 2015, He was a professor of Hainan University, China. From 2015-2021 he was a chair professor of City University of Macau, Macau, and now he is a professor of University of Saint Joseph, Macau. His research interests cover in the area of communication engineering, data science and machine learning. He has published more than 200 papers and more than 20 books in his research areas. He holds twenty patents for scientific and technologic inventions.


Shenming Qu, School of Software, Henan University, Kaifeng, 475004, China

Shenming QU received the B.S degree in computer science from Hebei University, Baoding, China, in 2004, the M.S degree in computer application technology from Henan University, Kaifeng, China, in 2010, and Ph.D degree from Wuhan University, Wuhan, China, in 2015.

He is an associate professor at Henan University. His research interests include image processing, computer vision.