Sentiment Analysis using VADER in Python

Sentiment analysis (also known as opinion mining) is an automated process (of Natural Language Processing) to classify a text (review, feedback, conversation etc.) by polarity (positive, negative, neutral) or emotion (happy, sad etc.).

Sentiment analysis helps businesses to identify customer opinion toward products, brands or services through online review or feedback.

In this tutorial I will explain on below points:
  • What is sentiment analysis?
  • Where you can apply sentiment analysis
  • Types of Sentiment Analysis
  • What is VADER?
  • How VADER works
  • Command to install VADER
  • Hands on VADER sentiment analysis in Python

Where to use Sentiment analysis

There are so many areas where sentiment analysis is helping to take better decision. Like..

Politics:In politics sentiment analysis is using to determine opinion of population towards a particular decision made by some party or people’s interest towards a party. You can also find good or bad thing of a party. Sentiment analysis can also help to predict which party is going to win the election.

Business:In recent time most of the companies use feedback and review to improve their quality of service and quality of product by applying sentiment analysis.

Also Read:

Types of Sentiment Analysis

There are mainly two types of sentiment analysis
1.    Polarity Detection: Predict a review whether it is positive, negative or neutral
2.    Emotion Detection: Predict a reviewer’s emotion while writing the review like sad, happy, angry etc.
There are so many tools available for doing sentiment analysis in Python. In this post I will show you how to do sentiment analysis in Python using a package called VADER. In this post I will only concentrate on Polarity detection using VADER.

How VADER works

VADER (Valence Aware Dictionary and sEntiment Reasoner) is a lexicon and rule-based sentiment analysis tool that is specifically designed to extract sentiments expressed in social media. It is fully open-sourced under the MIT License.
Lexicon is a list of words. For example positive lexicon is a list of all possible positive words (like good, amaze, appreciable etc.). Similarly negative lexicon is a list of all possible negative words (like bad, abolish, ambush etc.)
Rule based approach of sentiment analysis each words of your product review text pass though those two dictionaries or lexicons (positive and negative word list) and count the number of positive and negative words in your review text. Based on that number you can determine whether your review is positive or negative.
VADERuses same technique but with a smart way. It also included Emoji to determine sentiment. In social media Emojis are often used to express emotion. Thus VADER is extremely useful for doing social media sentiment analysis.

Command to install vaderSentiment

$ pip install vaderSentiment

VADER Sentiment Analysis in Python

Below is the code to do sentiment analysis using VADER in python.

from vaderSentiment.vaderSentiment import SentimentIntensityAnalyzer

# function to print sentiments
# of the sentence.
def vader_sentiment_scores(sentence):

# Define SentimentIntensityAnalyzer object of VADER.
SID_obj = SentimentIntensityAnalyzer()

# calculate polarity scores which gives a sentiment dictionary,
# Contains pos, neg, neu, and compound scores.

polarity_dict = SID_obj.polarity_scores(sentence)

print("Raw sentiment dictionary : ", polarity_dict)
print("polarity percentage of sentence ", polarity_dict['neg']*100, "% :: Negative")
print("polarity percentage of sentence ", polarity_dict['neu']*100, "% :: Neutral")
print("polarity percentage of sentence ", polarity_dict['pos']*100, "% :: Positive")

print("Overall polarity percentage of sentence", end = " :: ")

# Calculate overall sentiment by compound score
if polarity_dict['compound'] >= 0.05 :
print("Positive")

elif polarity_dict['compound'] <= - 0.05 :
print("Negative")

else :
print("Neutral")
Now let’s test how VADER sentiment performs

# Simple text
vader_sentiment_scores('I do not like flipkart')
# Emoji test (refer to image)
vader_sentiment_scores(':)')

# Complex text
vader_sentiment_scores('Your service has never been good')

What is compound score in VADER

Compound Score in VADER

The ‘compound’ score is calculated by summing the valen ce scores of each word in the lexicon after that adjusted according to the rules, and then normalized to be between -1 (most extreme negative) and +1 (most extreme positive). This is the most useful metric if you want only one score of sentiment for a given text/ sentence.

Pos, Neg, Neu Score in VADER

These are the most useful metrics if you want to know each (positive, negative and neutral) score for a given sentence. If you sum all (positive, negative and neutral) score will get result 1.

Conclusion

If you don’t have training data for sentiment analysis you must use rule based approach. You can make your own rule or you can simply use tool like VADER. There are multiple tools for sentiment analysis available like: StanfordCore NLP, TextBLOB etc.

Also Read:  Sentence Similarity Matching using Transformer

8 thoughts on “Sentiment Analysis using VADER in Python”

  1. I want to thank you for the efforts you have put in writing this blogpost. I am hoping the same best blog post from you in the future as well. In fact your creative writing abilities has inspired me to start my own blog now. Truly the blogging is spreading its wings rapidly. Your write up is a fine example of it.

    Reply
  2. Yes Vader is detecting emojis also. It is the best ever logic-based sentiment analysis package in python I found. Thanks for your explaination.

    Reply
  3. So I created a secondary blog off my first one, but I now want my secondary blog to be my personal or my main blog. So that as soon and I sign on the newer blog I made would be my default blog. If that makes any sense… Is there a way to do this?.

    Reply

Leave a comment