If you remember in part 2 we discussed what Key Word Analysis is and how this can be implemented to gain deeper insight from textual data. But we can go one step deeper and extract feelings and opinions from the same data. We can do this through Sentiment Analysis and Opinion mining! In this blog I will talk you through what they are and how we can implement them using Microsoft’s Cognitive Services.
What is Sentiment Analysis?
We should probably be clear on what we mean by sentiment; ‘a view or opinion which is held or expressed’. This is what we extract from textual data we are analysing. In particular we find out if a piece of text is ‘positive’, ‘negative’, ‘neutral’ or ‘mixed’. The idea is to encapsulate the feeling of a piece of text, be that a document, a paragraph or a sentence, we can extract the feeling. But that gets more and more inaccurate the larger the text as there will be conflicting themes, contexts and sentiment the more text there is. For a document or a paragraph you will be extracting the overall sentiment of that text, the combined sentiment of its constituent sentences. For this reason it is best to apply sentiment analysis to smaller sections of text to get the most accuracy and thus extract the most accurate indication of sentiment.
What is Opinion Mining?
Opinion mining is somewhat of a derivative of Sentiment Analysis in the context that is Sentiment Analysis performed at a more granular level to show and analyse the individual opinions that shape the overall Sentiment of a piece of textual data. In its most basic form Opinion Mining extracts the words which have prompted a certain sentiment to be reported, as well as showing the specific target for which this opinion is aimed. Its something which is much easier to understand with an example like the following you can get from Microsoft’s Cognitive Services documents:
The above also highlights the power of using both of these techniques in unison, as they can allow you to drill a layer deeper to really understand what is driving a certain sentiment and get a feel for the weight words have in terms of driving this. If we use the example of a business examining its customer reviews; they will most probably what to identify the reviews which are negative (Sentiment Analysis will deliver this), and then drill down into these to understand what exactly is driving this review to be negative (Opinion Mining).
Applications of Sentiment Analysis and Opinion Mining
The main focus of the applications of these techniques are allowing business valuable insights into how people feel about your product brand or service. If used in the context of social media you can identify high points of sentiment and thus find potential advocates of your products such as influencers, or you could identify negative threads as they emerge online so you can therefore be proactive in your response to them. Internally to a corporate environment you could also apply the techniques to your email servers to see the general feeling of emails within your business. You can already do something similar using Microsoft Editor which links into Outlook and give suggestions to adjust the tone of your emails.
One of the most popular applications of this is in customer reviews where it can be extremely useful to cut through the jargon in reviews and really know the emotion a customer is trying to express, as well as the opinions they are expressing. For example a company could get all the Reviews from their Trust Pilot site (or other review sites), and perform Sentiment analysis and Opinion Mining on these reviews. They could then filter to find reviews which have only a negative sentiment and from their see what is making this review have a negative sentiment by reviewing the output of the Opinion Mining to see what opinions are being expressed about specific things. Using this methodology, you can drill through and extract real meaning from the reviews in a fast and efficient manner which is easily palatable in its output; this saves a lot of time and resource and also gives you more info than doing the analysis by hand as some companies do!
How can one perform Sentiment Analysis and Opinion Mining?
As I said in my previous blog post about Key Word Analysis, there are many packages and options from which you can perform these techniques; whether this be programmatically or not. But once again I will be talking specifically about what is available using Microsoft’s Cognitive service; Language Studio. For broader information on the Language Studio then please refer to my previous blog post where I go into detail about the resource as a whole, as I will be talking specifically about its Sentiment Analysis and Opinion Mining options.
When you open the Language Studio and have logged in, you will be presented with the below screen within which you want to make sure you navigate to the ‘Classify Text’ tab (as is selected below).
You then need to click on the Sentiment Analysis and Opinion Mining tile to access the resource to try it out! you will be presented a very similar format to what we saw with Key Word Analysis.
After ensuring you have logged in and selected the azure resource you have set up (if you didn’t have one, it should have asked you to provision one before you got to this page), you are left with the option of providing your own text or using one of the examples Microsoft provides( of which I have selected the first one for simplicity).Ensure Opinion Mining is turned on using the toggle to the right of the screen, click the acknowledgement and press run and you will get the below output(s).
You can see at the top of this print screen you have the sentiment analysis of the textual data as a whole. We see a mixed sentiment overall (of which we can see the percentage breakdown of the sentiments detected), Following this you can also see a sentence by sentence sentiment output as well of an identical output to the overall output. We also get a pretty good graphic displaying the output of the Opinion Mining, linking the assessments and targets. But this is just a pretty frontend output that Microsoft has put together for the ‘demo’ page. It is all derived from the real output you would get from this service which is in a JSON format. You can see this by clicking the ‘JSON’ tab at the top of the results. This will give you results similar to the below:
This is only a small section as the JSON can be quite extensive as it will contain a lot of information. But this is what you would get if you used the service, and this is actually quite easy to interrogate. You can see above the JSON contains all of the information that was displayed so beautifully earlier, but it is now in a structured and easily parsed format which we can take advantage off to feed into other sources such as Python or Power BI for further analysis.
Next Post
The next post will be the last one (and a long one) where I will be tying together everything we have discussed already (Web Scraping, Key Word Analysis, Sentiment Analysis and Opinion Mining). I will give you what will be basically a start to end blueprint to perform a Natural Language Processing project; from data gathering and manipulation via web scraping through Python to leveraging the API’s provided by Language Studio which allow us to perform the Natural Language Processing we want on our textual data. I will then show you how we can present the results of these within a Power BI dashboard.
