EXECUTIVE SUMMARY

To have a better understanding of each post and its sentiment, a topic was defined for each post and it is evident that some topics are likely to be more positive, while some are not. Since these topics are overlapping on words like "elderly" and "crow", those unique words will explain more for the topics. The average Pos-Neg Ratio of all posts is 1.64, while the ratio of Topic 4 (more about accusation) is only 1.13, compared to 3.94 for Topic 3 (more about improvement). Therefore, the topic is indeed a important feature to determine the sentiment of a post, which is proved in the following analysis.

Topic Modeling - LDA

Different Topics, Different Keywords, Different Sentiments

LDA, one of the most famous topic modeling methods was utilized here to define a topic for each post. It can be recognized that some specific topics tend to be more positive, while some are not. Some keywords appear in more than one topic, so those unique words will explain more for the topics. The average Pos-Neg Ratio of all posts is 1.64, while the ratio of Topic 4 (more about accusation) is only 1.13, compared to 3.94 for Topic 3 (more about improvement)

Normally Distributed on Each Dimension

After defining topics for all the posts, it is necessary to know about the distribution of our topic to see if the model works well. Therefore, t-SNE, a tool to visualize high-dimensional data, was applied onto the results of TF-IDF to reduce the data to 2 dimensions, and these points were colored based on their topics. In agreement with the plot, each topic is normally distributed on these two dimensions, elucidating the successful seperation of LDA model.

Sentiment Prediction
Based on the Features of Post

Complex Model Does Not Make a Big Difference

To find the relationship between the post sentiment and the post other features, three different types of model were constructed, and for each type of model, three different hyperparameter sets were applied.

Before constructing these models, the data were subset in the first half of 2020, since we want to find the influence of the pandemic, and only some important features were kept, including author_flair_text, score, topic, covid, etc. Also, training set, validation set and test set were sampled from the whole data set based on the following proportion (0.8, 0.18, 0.02).

According to the evaluation metrics, those complex models did not make a big difference, but they did require much more time in training, hence, the simpler model would be a better choice here.

Besides, in line with the AUC score, which is nearly close to 0.5, these models actually did not make a really good prediction on the test set, which is acceptable since the sentiment of a post is usually directly related with the text itself, instead of the topic or post score.

Racial Equality, Goverment and Topic

In order to figure out the determinants of the sentiment for a post, the bar plots of feature importance were made for all these three simplest models, revealing that whether the post is racial equality and government related or not and its topic are the most important three determinants of a post's sentiment.

Stock Price Prediction
Based on the Daily Sentiment of Subreddit

LSTM Can Help to Defeat the Average

Stock Market fluctuates all the time because of tons of factors, making it more difficult to predict than other data. In this project, SVR and LSTM were chosen to construct models, trying to predict the Nasdaq Index, based on the daily sentiment of subreddit.

Feature engineering is always the first step to take when constructing predictive models, and daily number of posts, pos-neg ratio, post average score, negative post average score, positive post average score were calculated from the big reddit dataset. Besides, COVID daily new cases and new deaths were also included in the features. Since stock price is timeseries data, it cannot be random splitted into training and testing set. Consequently, the data were splitted by the date 04/01/2021.

In LSTM, a lookback period was predifined so that the model can not only learn about the information on one day, but also learn more patterns about the past trend. In accordance with the evaluation metrics, LSTM performs better than the Moving Average, especially with 60 days' lookback, while SVR actually has a rather poor performance on the test set.

LSTM with 60 Days' Lag

LSTM now can be proved to be able to defeat the moving average, and among all these choices of lookback days, 60 days' lag can provide enough information for the model to learn about the patterns and trends. When it is moving forward and getting farther away from the last training date, the predicted price will also get farther away from the true price

Summary

1. Some specific topics tend to be more positive, while some are not. The average Pos-Neg Ratio of all posts is 1.64, while the ratio of Topic 3 (more about bankrupt and persecution) is only 1.13, compared to 3.94 for Topic 2 (more about integrate and conspiracy).
2. The sentiment of a post is usually directly related with the text itself, instead of the topic or post score.
3. Whether the post is racial equality and government related or not and its topic are the most important three determinants of a post's sentiment.
4. LSTM performs can defeat the Moving Average, especially with 60 days' lookback.

Resources