Chatbot for Mental Well-being

. With the world becoming more and more competitive every passing day, the number of people suffering from stress and other mental health issues is exponentially increasing. Even school children and senior citizens are becoming victims of stress and pressure in today's world. The mental health of an individual is as important as physical health is. But unfortunately, there is a lack of awareness and proper mental health facilities in today's society, which demands the individuals to fight against their odds without any support. Therefore, we propose a solution in the form of a chatbot that will act as a medium for the users to communicate with, let out their feelings and thus get relieved of the stress that is clogged up. A chatbot application will be available at all times at the user's expense and will also keep track of the user's mood over a span of time.


Introduction
Just like physical health, mental health also plays a vital role in determining the lifestyle of an individual. A healthy mind is crucial towards leading a healthy lifestyle. But unfortunately, due to lack of proper awareness and social stigma, mental health is not given the due importance it deserves. A major portion of the population succumbs to stress and pressure, due to the perpetual magnification of competition in day-to-day life. A large population, ranging from school kids to senior citizens, suffer from mental health disorders and a major portion of the same are either devoid of proper medical facilities and awareness or are reluctant to make use of the same. A poor mental health hampers the daily functioning of an individual and at times are accompanied by comorbidities like hypertension, heart disease, suicidal intentions, etc. So it is really important to maintain mental health as it is to maintain physical health in the modern era In this project, we propose to build a therapy chatbot with the aim of providing a solution to the discussed dilemma. A therapy chatbot will provide an easily accessible medium that will provide a hassle-free service that is always at one's disposal. While a chatbot cannot replace a human therapist entirely, it can provide a platform for individuals to let out their emotions and get moral support whenever they feel distressed.
The motivation behind this project is that there are several situations wherein a person feels the need to talk to someone and share his/her feelings but cannot do so due to various reasons like lack of time, personal issues, non-availability of a listener, etc. Most of them are reluctant to go through therapy due to social or monetary obligations. We intend to make a chatbot which will be easy to chat with and available 24x7, so that the user can share his/her emotions while he/she feels stressed out or worried, which will help him/her improve their mood.

Existing System and their Limitations
It is difficult for individuals suffering from panic attacks, anxiety attacks or undergoing any stress-related scenario, to find someone to have a conversation with in order to relieve the stress. It is practically unfeasible to find an ally to talk to every time an individual suffers from unpredictable panic attacks. Also, most individuals don't feel comfortable talking about their issues with a real person due to various reasons.
From the chatbot perspective, it would be apt to say that, presently, it is quite a herculean task to develop a chatbot that can mimic human conversation. Chatbot's performance almost always ends up deteriorating when it comes to long conversations. Most of the chatbots available in the market are restricted to a particular use-case and are mainly retrieval-based models.

Literature review
Miriam Romero, Cristina Casadevante y Helena Montoro [1] is a paper in which the fundamental aspects of designing a psychologist chatbot, along with the advantages and disadvantages of chatbots and the scientific guarantees that they need to fulfill. Eileen Bendiga,Benjamin Erbb Lea Schulze-Thuesinga, Harald Baumeistera,Karger publishers [2] is an IEEE paper in which Comparative analysis of most relevant psychological chatbots and their area of interest along with methodology used.
Takeshi Kamita ,Tatsuya Ito,Atsuko Matsumoto, Tsunetsugu Munakata and Tomoo Inou, Hindawi publishers [3] is an IEEE paper in which a different way to approach this with self-guided mental healthcare course and a system using a chatbot based on Digital-SAT method. Cameron Megav Bond O Neil [4] is a paper which includes chatbot for mental health counselling, its basic overview and future development possibilities for the same.
Kien hoa ly Annmaria Ly Gerhard Andersson [5] is an IEEE paper in which A study/analysis of a group of people who used SHIM chatbot which combines different techniques along with its limitations. Kyo-Joong Oh DongKunLee ByungSoo Ko Ho-Jin Choi [6] is a paper which focuses on emotion recognition, monitoring and conversation understanding on chat assistant.

Recurrent Neural Network (RNN) A Recurrent Neural Network (RNN) is a type of Neural
Network that recollects the past information that it has learnt and utilizes the same in its decision making process. Apart from recollecting the information that they have learnt during their training phase, RNNs also allow output of the previous step to be fed to the current step input. Therefore, unlike traditional neural networks wherein inputs and outputs are assumed to be independent, in RNN information from the previous inputs play a significant role in the decision making for the current step. Because of this ability of RNN to recollect and use the previous input, they are generally used when dealing with sequence or temporal data. Now, Vanilla RNN suffers from the 'Vanishing Gradient' problem due to which, the gradients (used to update the weights) shrink over time as they backpropagate and this affects the weight updates for long input sequences, often leading to a loss in the early context of the input. Therefore, variants of RNN like LSTM or GRU are preferred over RNN when dealing with long input sequences.

Long Short-Term Memory (LSTM)
LSTM (Long Short-Term Memory) network was proposed in 1997. It is a type of RNN that tries to overcome the drawbacks faced by Vanilla RNN by introducing the concept of gates and cell states. The information travels between the LSTM cells in the form of cell states. Each cell has three gates, viz. Forget gate, Input gate and Output gate The forget gate is used to decide what information from the previous cell state should be retained. Basically, the previous cell's hidden state and current input are passed through a sigmoid function that returns a vector with values in the range of 0 to 1. This is multiplied with the previous cell state that helps determine what information to forget and what to retain (where value closer to 0 means forget and value closer to 1 means retain). In the input gate, the previous hidden state and current input are passed through the sigmoid function and the output is multiplied with the output obtained by passing previous state and current input through the tanh function. This output determines which information is essential. This is then pointwise added with the updated previous cell state to get the current cell state. In the end, we have the output gate that helps determine what the hidden state of the current cell should be.

Proposed Work
The primary goal of the proposed project is to develop a generative chatbot that will communicate with the users with the aim of helping ease their state of mind and keep track of their mood over a span of time. The chatbot will act as a medium for users to let out their emotions and will be available 24/7 at the user's expense. The motive behind this project is to help people relieve their stress by providing a medium to talk to.
The proposed project will be implemented using the following two major modules: SVM classifier: Will detect the mood parameter based on the emotional alignment of the input Seq2Seq model: It makes use of two RNNs in the form of encoder and decoder to take in the tokenized input and generate an apt response for the same as output.

Dataset
For the mood classification model, we have used the Empathetic Dialogues dataset. The dataset is divided into 2 parts. The training file consists of a novel dataset of 12,424 conversations grounded in emotional situations. It contains 79,190 lines. Along with that, it contains 32 emotion labels mapped to each conversation. The test file consists of 10,956 lines. This file is used in the evaluation of the model. Further while preprocessing, we have considered two main columns which are named as Emotion and Text for training and testing purposes. For the seq2seq model along with the above dataset we have used the Counselchat dataset. The CounselChat corpus has 10 unique columns which includes the question title, question text, answer text from multiple therapists along with the url for the same are available.
There are a total of 1659 lines of question-answers with the therapist.

Data Preprocessing
First, both the training and the test file have been appended together, and then the sample function was used, to shuffle the sorted dataset randomly. Then, each of the 32 Emotions was mapped to a unique number. For preprocessing, regular expressions are used to remove the special symbols, punctuation, links, etc. After that, the strip function was used to remove the blank spaces from the processed text and remove the stop words and then Word level tokenization was used on each sample. Snowball Stemmer is used on the tokenized samples to further improve it. This preprocessed data is then passed to a vectorizer which converts the data into a sparse matrix. In our case, we used a Term Frequency -Inverse Document Frequency vectorizer. Along with vectorization, samples are converted into unigrams+bigrams. Then the vectorized data is passed on to a balancer which balances the data using the random oversampling technique. For this, we used a RandomOverSampler provided by the imblearn library. Then the balance data is then split with a ratio of 80% training and 20% testing data. We have used Regular Expressions to clean the dataset, like removing punctuations, resolved clitics into their base terms, etc. Then, we have appended to the input and target sequence lists, sentences from the dataset with lengths less than or equal to 30. Next, we have appended <SOS> and <EOS> tags to the end and beginning of the target list entry, respectively. Then from these lists, we have created a vocabulary dictionary of words having a frequency greater than 5 in the input and target sequence lists. The vocabulary assigns a unique number to each word. Inverting this vocabulary, we get the inverse vocabulary that the model uses to generate the sentences. We have used 'GloVe (Global Vectors for Word representation) Embedding' to create the embedding layer. Word embedding in simple terms is a representation of text wherein words with similar meanings have similar representation, which means in the vector space, two words having similar vectors will be placed closely. The GloVe is an unsupervised learning algorithm in which the ratio of co-occurrence probability of two words is encoded as vector differences in the vector space.

Architecture
We have used Keras to develop the Seq2Seq model and linear SVM model of the Sklearn library. We have also incorporated the additive Attention mechanism proposed by Bahdanau et al., to improve the efficiency of the Seq2Seq model.

SVM Classifier
Support Vector Machine is a Supervised learning algorithm, which primarily helps with the classification problem. In layman terms, the basic functionality of SVM is to find out from the many possible decision boundaries in an N-dimensional space that classifies the input, the one that has the maximum distance from data points of both the classes (maximum margin). Such a decision boundary is referred to as the Hyperplane. So, in our project, the SVM algorithm will be generating a predetermined numerical output that corresponds to a particular mood based on the input provided by the user, thus determining the mood of the user's chat.

Seq2Seq Model
The Seq2Seq model is an encoder-decoder model developed to work with data with some sequential order, like sentences and time series, and convert the same into another sequence of information required by the user. It makes use of Recurrent Neural Networks internally for converting the input sequence to the required output sequence. But, Vanilla RNNs are not often used, because of the vanishing gradient problem they suffer from, which affects the weight updates of the same for long input sequences, often leading to a loss in the early context of the input. Hence, variants of RNN like LSTM or GRU are used when dealing with long input sequences.
The Seq2Seq model has an encoder-decoder architecture, where both the encoder and decoder comprise stacked RNN cells or its variants (like LSTM or GRU). The embedded version of the input is fed to the encoder, which then operates upon the same and outputs the context vector, a fixed size feature vector encoding the context of the input sequence. This is then fed as input to the decoder, wherein each decoder cell takes in the previous decoder cell's hidden states and output as input, works upon the same to generate the output, that is fed to the softmax layer. Then the probability distribution over the entire vocabulary is generated, and

Attention Model
Now, the encoding of the entire input sequence into a fixed-length context vector leads to the loss of a significant amount of information when dealing with a long input sequence. To deal with this drawback, we make use of a mechanism known as the Attention mechanism. We make use of the Attention mechanism proposed by Bahdanau et al., which is an additive attention mechanism. Instead of creating the context vector from the output of the last LSTM cell of the encoder, for every decoder cell, a context vector is generated as the weighted sum of all hidden states of the encoder and the attention weights calculated for the decoder cell. This helps the decoder cell focus on the apt part of the input sequence rather than working on a fixed-sized encoded version of the input. For every decoder cell, the alignment score tells how well the jᵗʰ position input and output at position "i" match, calculated based on the previous decoder's hidden state (s₍ᵢ₋₁₎) and the hidden state hⱼ of the encoder.
Then, using the softmax function, the attention weights are calculated from the alignment scores as The context vector is calculated as the weighted sum of the hidden states of the encoder and the attention weights as In this way, the decoder can focus on the local context of the input part it needs to pay attention to, rather than working with a fixed-length encoding of the entire input, wherein significant information might get lost for long sequences.

Result
Presently, we have implemented the mood classifier module with the help of SVM classifier. Since we have implemented the mood classifier as a standalone module for our current implementation, the code first does the task of tokenization using TfidfVectorizer to obtain individual words from the provided input and produce a vector of weights where each weight is associated with a word in input. The classifier is then fed with this vector, based on which it outputs the mood of the input. The Seq2Seq model has been trained for 30 epochs and it gives an accuracy of 96.53 and validation accuracy of 90.69.
The overall working of the system is in the following manner: 1. The user first provides the system with the chat input. 2. The system tokenizes the input to get the n-gram array which is then further modified to get the vector of weights linked to each pair 3. The vector is fed as input to the mood classifier, which determines the mood of the input given by the user.
4. The user input is fed as input to the Seq2Seq model, which, based on its training with a dedicated dataset, generates an apt response for the given input. The same will then be provided to the user as the output of the system.

Conclusion
This project serves the primary goal of developing a virtual intelligent environment in the form of a chatbot for the users to communicate with and express their emotions, with the intention of helping the user relieve of the bottled stress and improve their mood to a certain extent. The chatbot will provide a simple interface for the users to put in the input chats, to which the chabot will do mood classification and generate an apt response using the Seq2Seq model that will then be displayed to the user with the aim of helping the user ease his/her mood. Another hallmark of the system is its 24/7 availability at the user's expense.
From the future work perspective, the system can generate a weekly report of how the user's mood varied throughout the week which will give the user an idea of how much he/she is improving over time. Furthermore, a more distinctive dataset with real therapy conversations will further improve the working of the system.