Is feature selection needed to be performed in lexicon-based sentiment analysis?

March 27, 2018 / Ask Slater

The two are typically at odds. Generally speaking your lexicon defines your features. So you’re performing feature selection as you build the lexicon and once you have your lexicon you stop choosing new features.

If you’re starting with a lexicon and want to add features you could add them to the lexicon you’re working with, or attempt to create some kind of ensemble approach that combines the features in your lexicon with whatever features you create yourself.

However, for sentiment analysis you really shouldn’t be manually creating features, or using lexicons. These approaches generally lead to extremely brittle models with very poor performance. The problem is that if you are engineering your features or changing your lexicon in response to test errors then you’re manually overfitting.

View original question on Quora >

Follow Slater on Quora >>
Slater Victoroff

Don't Miss a Post

Get our best content on Intelligent Process Automation sent to your inbox weekly.