Implementation of Deep Learning Based Sentiment Classification and Product Aspect Analysis

With the increase in E-Commerce businesses in the last decade,the sentiment analysis of product reviews has gained a lot of attention in linguistic research. In literature, the survey depicts the majority of the research done emphasizes on mere polarity identification of the reviews. The proposed system emphasized on classifying the sentiment polarity and the product aspect identification from the reviews. Proposed work experimented with traditional machine learning techniques as well as deep neural networks such as Convolutional Neural Network (CNN), Recurrent Neural Network (RNN) and Long Short Term Memory(LSTM) Networks. The proposed system gives a better understanding of these algorithms by comparing the outcomes. The Deep Learning approach in the proposed work successfully provides a mechanism which identifies the review polarity and intensity of the reviews and also analyses the short form words used by people in the reviews. The experimental results in this work, applied on amazon product dataset, shows that the LSTM model works the best for sentiment analysis and intensity of reviews with 93% accuracy. This research work also predicts polarity for short-form word reviews which is the common trend these days while writing the reviews.


Introduction
In recent years, there has been an increase in research activities focused on analyzing sentiment in textual resources. The number of articles published on sentiment analysis has been growing in recent years. Sentiment analysis or opinion mining is one of the subtopics of this research, in which it computationally investigates people's views, assessments, attitudes and emotions regarding entities, individuals, situations, events, texts.  [2] utilize this data to conduct semantic analysis of trends in market and buyers sentiment, which might improve forecasting product quality. Despite this, monitoring online opinion sites, as well as extracting the information hidden in them, is still a difficult undertaking due to the proliferation of different sites. In extensive forum postings and blogs, each site often has a large amount of opinionated content that is not always easy to read. The ordinary reader may face trouble locating appropriate websites and precisely extracting the hidden details and viewpoints included therein [5]. Furthermore, teaching a computer to understand sarcasm is a difficult and time-consuming endeavour, considering that computers cannot yet think like humans. The goal of this work is to identify positive and negative opinions of various products and to develop a model which is a supervised model, having the capability to polarize enormous volumes of data. Customers' reviews and ratings, obtained from Amazon's Consumer Reviews, are used as a dataset for the proposed system. In proposed research, it retrieves the features from the input dataset and uses those features to build various supervised models. Traditional machine learning mechanisms and deep learning techniques like RNN, CNN and LSTM are included in these models. The accuracy for various such models were analyzed and a thorough evaluation of the obtained sentiments regarding the reviews was done.
These models make use of 70% of the data for training and 30% of data serves as testing data.
Yi-Fan and Maria Soledad Elli gathered sentiments by analyzing reviews and evaluated the results to develop a business model in their paper [3]. Authors said that this model gained them a high level of precision. The major classifiers were Support Vector Machine (SVM) and Multinomial Naive Bayesian (MNB). Callen Rain [6] advocated expanding current Natural Language Processing (NLP) research. To determine whether a review was good or not, the Naive Bayes classifier was used by authors. In the field of sentiment analysis, deep-learning neural networks are popular too. For the sentiment labeling challenge, some researchers deployed a convolutional neural network with the purpose of avoiding unnecessary task-specific feature engineering.
In contrast to this, [8] recommended employing recursive neural networks to gain a better grip over sentiment prediction.

Related Work
There have been several research papers published so far on product reviews, sentiment analysis and opinion mining. Wei Zhao et. el. [1] proposed a deep learning approach for product review sentiment classification, which uses existing ratings as input values. The work includes two steps: (1) to make the model understand the representation which evaluates the sentiment values for input sentences through ratings; (2) deploy the classification layer above the embedding layer and make the use of tagged sentences for supervised learning.
Author proposes two types of low level network structures for determining review text, one is Convolutional Feature Extractors and the other is Long Short-Term Memory network. In [2], the author has given a deep survey of reviews and concluded that the internet is the most preferred method of learning, acquiring ideas and obtaining various product ratings.
Millions of evaluations on a commodity, service, individual or location are posted on the internet daily. It's tough to read and evaluate such assessments one by one because of their sheer volume and quantity. As shown in   Many algorithms exist to deal with NLP challenges. The categorization of reviews is presented in the proposed system. In comparison to Naive Bayes' and extreme entropy approaches, this research proves that SVM provides higher accuracy. On Yelp's rating dataset, Xu Yun [8] et al used current supervised learning techniques like supporting vector machine or perceptron algorithm to determine review's rating.

Proposed Work
Product review analysis is basically more concerned about analyzing the text to determine the polarity of the reviews. This is the most tedious task as the reviews are user-written and there is no specific pattern or rules to be mapped. As the data is highly temporal in nature, the use of artificial intelligence and right choice of AI model is 2. The system emphasizes on also integrating the short form words used nowadays in reviews, which is not included in conventional data dictionaries.
3. The proposed system also emphasizes the polarity aspect of the product.

Proposed System Design
The proposed system first uses Meta Dictionary Mapping for short form words in reviews. Nowadays, there is a trend of people writing textual reviews by using short form words e.g. "gud" for "good" or "nyc" for "nice". So the standard dictionary does not have any mappings for such words and hence any review stating "the product is gud" won't be considered to be positive. Hence the proposed system maintains the key value pair of the short form words and actual words, thereby making the system capable of mapping the short form words as well for sentiment prediction.
The proposed system's overall flow is as follows: • The proposed system first loads the dataset and cleans the dataset by removing unnecessary columns from the dataset.
• The cleaned dataset is then processed by VADER sentiment analysis after text pre-processing, for labeling the review's polarity (0,1).
• The Meta Dictionary is maintained for storing the short form words and their corresponding actual words, so that before the model is being trained, words are mapped to actual words.
• After labelling data for its polarity, the data is provided to the proposed LSTM model with some preliminary neural network configurations.
• The model training is then started. If the training output is not optimal, the configurations are varied and the model is retrained till the optimal results are obtained.
• Once the optimal results are obtained, the system then makes use of Part of Speech (POS) tagging on the input reviews. These input reviews are provided as test reviews for testing the model and product review aspects are extracted by segregating nouns from the tagged review tokens. From this, it gives the summary statistics of the listed data such as total number of reviews, total reviews for different ratings values as seen in Figure 4.

Data Preparation
In order to avoid unwanted data, only some part of the entire data is used.  and n_estimators = '100' was set to fit the model to achieve the prediction for review classification.

Applying Deep Learning LSTM
The forget gate (f) If it's found that the forget gate is equal to 0, then the old stored value in the memory state is straight away forgotten.

Fig. 5. LSTM Cell Overview
If it's found that the forget gate is equal to 1 then the old value in the memory state is straight away passed to the cell. So, with the present memory state C t it calculates the new cell state in coordination with the C layer and the input state.
Where, C t = present memory at time-stamp t which gets passed to next time-stamp. Fig 6. shows a flow diagram for C t . The proposed system architecture has the standard Neural Network Blocks for generating the model for analyzing the sentiment from the reviews.

Experimental Setup and Dataset
The information includes Amazon reviews on mobile phones. The product feature names and the associated emotions are marked in the reviews. Clearly, each of these statements mention the entity being reviewed as well as the level of opinion held about it. After conducting a variety of experiments, this research concluded that the third party website reviews are not sufficient for the research purpose. Therefore, more than 5000 reviews were labelled manually. On a scale of -5 to 5, aspect ratings are labelled. This system has additionally pre-processed publicly available data, so as to fit the indicated element into corresponding buckets. consists of polarity intensity identification also, which will provide a more detailed prediction. e.g. "the product is good" -Positive, but "the product is extremely good" -Very positive. The Proposed work does not only determine the polarity of the reviews, but also determines the aspects of the reviews. Which means that if the review indicates some aspect of the product and the reason for its polarity, the proposed system will extract that information too e.g. "the product battery life is awesome", so the system will also determine that the review is positive and the aspect being "battery" of the product. The proposed system is also capable of identifying the polarity of reviews in which short form words are used for expressing the sentiments. If the person writes the review "the product is gud", the word "gud" is not a standard dictionary word, so the CNN model is deployed to predict the actual words from the trained dataset, which comprises of actual words and possible short form words that can be used in now a days. As seen in Fig 7, the input review uses the word "gud" which is a short form, but the final analysis shows the actual word "good" instead of the short form word.

Fig. 7. Product Aspect Analysis
The meta dictionary is maintained for mapping short-form words with actual words which are replaced while the pre-processing phase is computed.

Performance Evaluation
The

Conclusion
The obtained results show that the efficiency of the proposed LSTM model is better than the traditional machine learning models. Also, the proposed system does not restrict its scope over polarity or sentiment Hence it is observed that the proposed model gives more accurate and elaborative details about the reviews which is helpful in terms of analyzing the aspects of the products whose polarity is also identified. So because of this, consumers and service providers will get better clarification on products' market value and help them make important business decisions.