by Marcia Hannigan
Sentiment analysis (SA) uses a combination of natural language processing (NLP) methods to analyze a text to estimate the implied sentiment. Sentiment may be classified into categories such as positive, negative or neutral, or be measured by a range of numerical scores. It is used frequently in business to determine consumer attitudes towards products where data such as customer reviews may be too voluminous for traditional analysis, but the use of SA and NLP helps to identify meaningful trends.
SA can be used to analyze a wide range of texts, including short snippets of text such as Twitter feeds to generate meaningful insights. A new study by Jim Samuel (Rutgers University), Gavin C. Rozzi (Rutgers University), Ratnakar Palle (Apple, Inc.) in SSRN (Jan. 2022) reviews known issues with SA as documented by prior research and then compares the application of multiple of-the-shelf lexicon and dictionary methods to stock market and vaccine tweets. The intention of this research is to identify and discuss critical aspects of the “dark side” of SA and develop a conceptual discussion of the characteristics of the dark side.
The study demonstrates flaws with a plug-and-play approach to SA and concludes with notes on conceptual solutions for the dark side of SA. It points to future strategies that could be used to improve the accuracy of SA. This research can help align researcher and practitioner expectations to understand the limits and boundaries of NLP-based solutions for sentiment analysis and estimation.
The study concludes that lexicons and dictionaries help in implementing sentiment analysis. While an in-depth analysis of SA is necessary before drawing conclusions, it is important to know the limits of SA methods and tools. SA modeling may need to be customized for some situations while acknowledging the absence of satisfactory SA solutions for other situations. SA tools are very useful and must continue to be used for research and practice – however, as demonstrated and described in this study, it is vital to understand the conflicts and ways to acknowledge and address them.
It is expected that this study will lead to deeper attention to applied SA and spur new strategies for the improvement of sentiment analysis research and practice.