Last night, as the general election results poured in and Britain learnt its fate for the next five years, I took to Twitter armed with a sentiment analysis tool to gauge how the country was reacting to the unfolding news of a Conservative win.
I learnt two things. One, Twitter is not a reliable litmus test for the mood of the country (ok, so I did know this beforehand), and two, sentiment analysis tools have a long way to go before they can be considered reliable.
Go deeper with GlobalData
Applying Twitter sentiment analysis to general election reaction
The tool I used was the Tweet Sentiment Visualization App from NC State University’s Department of Computer Science, which is widely considered to be the best non-subscription resource for sentiment analysis on Twitter.
This is designed to estimate whether a tweet is unpleasant or pleasant, and active or subdued, by looking up each word up in a sentiment dictionary, finding the average sentiment and plotting each analysed tweet on a four-quadrant chart.
Between 22:55 and 00:30, I used this to measure the sentiment of tweets at five-minute intervals using the #electionresults2019 hashtag.
This covered the period where the exit poll predicting a Conservative majority had been released but before any seats had been announced, through to when the first three seats had been announced. This included Blyth Valley, a former Labour stronghold that had been won by the Conservatives, indicating the exit poll was correct.
How well do you really know your competitors?
Access the most comprehensive Company Profiles on the market, powered by GlobalData. Save hours of research. Gain competitive edge.
Thank you!
Your download email will arrive shortly
Not ready to buy yet? Download a free sample
We are confident about the unique quality of our Company Profiles. However, we want you to make the most beneficial decision for your business, so we offer a free sample that you can download by submitting the below form
By GlobalDataSentiment analysis gone wrong: When “disgusting” is a pleasant sentiment
Browsing through Twitter, you’d expect any sentiment analysis of the #electionresults2019 hashtag to produce overwhelmingly negative results.
Despite much of the country turning blue and so, presumably, being pleased with a Conservative win, the generally leftwing twitterverse was not happy. Swear-laden tweets abounded, with many users proposing impromptu moves to Canada or decrying the political mindset of the rest of the country. While these weren’t every tweet, negative responses were certainly the overwhelming majority.
However, if you were to rely on the sentiment analysis tool, you’d get a very different picture of the mindsets of UK Twitter users.
While tweets judged to have a negative sentiment did climb throughout the night, spiking around the time that the Blyth Valley result was called, at no point did they outweigh tweets deemed to have a positive sentiment.
In other words, the app was interpreting more than half of tweets using the #electionresults2019 hashtag to be positive reactions to the news, not the pit of furious screeching that they, in reality, were.
Looking into the app’s tag cloud, which shows the most common words associated with each sentiment during each five-minute window, the issue became a bit clearer.
While the pleasant/subdued quadrant was consistently populated by neutral or positive terms, such as “country”, “people”, “tories”, “voted” and “hope”, the pleasant/active quadrant highlighted an issue with the sentiment analysis.
Words such as “disgusting”, “fucked”, “austerity”, “poverty” and “goodbye” appeared regularly.
In fact, out of the 18 tag clouds I captured over the course of the evening, every single one contained at least one variant of the word “fucked” – and usually more than one – as one of the most frequent words.
This seems to reveal two issue with the sentiment analysis tool. In some cases, there seems to be a simple mischaracterisation of the sentiment of words, but in many cases it appears that the issue is more complex.
Why sentiment analysis didn’t get the general election reaction right
In the English language, words can mean many things, both good and bad. The app does attempt to account for this – for example, it can tell the difference between the term ‘happy’ and the phrase ‘not happy’ – but the level of sophistication does not appear to be anywhere near enough to match the reality of language use.
Furthermore, British people in particular will often deploy dry, sarcastic terminology that a sentiment analysis tool – particularly one developed with US English in mind – may not pick up.
I observed a large number of tweets along the lines of ‘to all those who voted for the Tories, I hope you’re happy’ which appear to have been ranked as positive despite the fact that they are clearly intended to be critical.
So with all this in mind, is there any point to sentiment analysis? Well, yes. If we get to the stage where we can accurately gauge sentiment consistently, there are dramatic benefits to be had. Being able to measure live reactions to events at a granular level can provide significant insight into the mood of groups of people – albeit not the entire country – and the ability for governments, brands and organisations to respond to this is significant.
Furthermore, it is highly likely that other enterprise-focused tools that are accessible with an eye-watering fee are likely to be better, due to the increased development time that can be afforded to them.
But for now, we must always take the results of any sentiment analysis tool, particularly those used to measure political events such as a general election, with a handful of salt.
Read more: EU Brexit sentiment: Europeans increasingly positive about EU membership
