Sentiment analysis (also known as opinion mining) is an automated process (of Natural Language Processing) to classify a text (review, feedback, conversation etc.) by polarity (positive, negative, neutral) or emotion (happy, sad etc.).
Sentiment analysis helps businesses to identify customer opinion toward products, brands or services through online review or feedback.
In this tutorial I will explain on below points:
- What is sentiment analysis?
- Where you can apply sentiment analysis
- Types of Sentiment Analysis
- What is VADER?
- How VADER works
- Command to install VADER
- Hands on VADER sentiment analysis in Python
Where to use Sentiment analysis
There are so many areas where sentiment analysis is helping to take better decision. Like..
Politics:In politics sentiment analysis is using to determine opinion of population towards a particular decision made by some party or people’s interest towards a party. You can also find good or bad thing of a party. Sentiment analysis can also help to predict which party is going to win the election.
Business:In recent time most of the companies use feedback and review to improve their quality of service and quality of product by applying sentiment analysis.
- Complete Guide for Natural Language Processing in Python
- Advanced Natural Language Processing with Stanford CoreNLP
- Natural Language Processing Using TextBlob
Types of Sentiment Analysis
There are mainly two types of sentiment analysis
1. Polarity Detection: Predict a review whether it is positive, negative or neutral
2. Emotion Detection: Predict a reviewer’s emotion while writing the review like sad, happy, angry etc.
There are so many tools available for doing sentiment analysis in Python. In this post I will show you how to do sentiment analysis in Python using a package called VADER. In this post I will only concentrate on Polarity detection using VADER.
How VADER works
VADER (Valence Aware Dictionary and sEntiment Reasoner) is a lexicon and rule-based sentiment analysis tool that is specifically designed to extract sentiments expressed in social media. It is fully open-sourced under the MIT License.
Lexicon is a list of words. For example positive lexicon is a list of all possible positive words (like good, amaze, appreciable etc.). Similarly negative lexicon is a list of all possible negative words (like bad, abolish, ambush etc.)
Rule based approach of sentiment analysis each words of your product review text pass though those two dictionaries or lexicons (positive and negative word list) and count the number of positive and negative words in your review text. Based on that number you can determine whether your review is positive or negative.
VADERuses same technique but with a smart way. It also included Emoji to determine sentiment. In social media Emojis are often used to express emotion. Thus VADER is extremely useful for doing social media sentiment analysis.
Command to install vaderSentiment
$ pip install vaderSentiment
VADER Sentiment Analysis in Python
Below is the code to do sentiment analysis using VADER in python.
from vaderSentiment.vaderSentiment import SentimentIntensityAnalyzer
# function to print sentiments
# of the sentence.
# Define SentimentIntensityAnalyzer object of VADER.
SID_obj = SentimentIntensityAnalyzer()
# calculate polarity scores which gives a sentiment dictionary,
# Contains pos, neg, neu, and compound scores.
polarity_dict = SID_obj.polarity_scores(sentence)
print("Raw sentiment dictionary : ", polarity_dict)
print("polarity percentage of sentence ", polarity_dict['neg']*100, "% :: Negative")
print("polarity percentage of sentence ", polarity_dict['neu']*100, "% :: Neutral")
print("polarity percentage of sentence ", polarity_dict['pos']*100, "% :: Positive")
print("Overall polarity percentage of sentence", end = " :: ")
# Calculate overall sentiment by compound score
if polarity_dict['compound'] >= 0.05 :
elif polarity_dict['compound'] <= - 0.05 :
Now let’s test how VADER sentiment performs
# Simple text
vader_sentiment_scores('I do not like flipkart')
# Emoji test (refer to image)
# Complex text
vader_sentiment_scores('Your service has never been good')
What is compound score in VADER
Compound Score in VADER
The ‘compound’ score is calculated by summing the valen ce scores of each word in the lexicon after that adjusted according to the rules, and then normalized to be between -1 (most extreme negative) and +1 (most extreme positive). This is the most useful metric if you want only one score of sentiment for a given text/ sentence.
Pos, Neg, Neu Score in VADER
These are the most useful metrics if you want to know each (positive, negative and neutral) score for a given sentence. If you sum all (positive, negative and neutral) score will get result 1.
If you don’t have training data for sentiment analysis you must use rule based approach. You can make your own rule or you can simply use tool like VADER. There are multiple tools for sentiment analysis available like: StanfordCore NLP, TextBLOB etc.
8 thoughts on “Sentiment Analysis using VADER in Python”
I want to thank you for the efforts you have put in writing this blogpost. I am hoping the same best blog post from you in the future as well. In fact your creative writing abilities has inspired me to start my own blog now. Truly the blogging is spreading its wings rapidly. Your write up is a fine example of it.
Good to hear. thanks
You need to get upset! Really its a must to take a look past everything and get upset. Generally this will allow you to take the inititive to make things happen.
Yes Vader is a good library to start learning Sentiment Analysis in NLP
The accuracy of VADER sentiment analysis is very good for my project. Thanks for your well-explained article.
Yes Vader is detecting emojis also. It is the best ever logic-based sentiment analysis package in python I found. Thanks for your explaination.
Good post to start sentiment analysis using python
So I created a secondary blog off my first one, but I now want my secondary blog to be my personal or my main blog. So that as soon and I sign on the newer blog I made would be my default blog. If that makes any sense… Is there a way to do this?.