Three Horror Writers Walk into a Predictive Model

-

How I learned to stop worrying and love the vectorizor

In the world of horror writers, there are few that can top the likes of H.P. Lovecraft, Mary Shelley, and Edgar Allan Poe (maybe Stephen King, but that's mostly due to volume. Not to say he's bad. I've actually never read any of his stuff. This aside has gone on too long.) An avid fan could probably tell the difference between their writings, but how well could we train a computer to do the same. Computers are notorious for their lack if interest in literature, so getting them to absorb the text in a language they understand it tricky. Furthermore, getting those converted text to play nicely with the other features is a bit of a mess, so her is vecorizing and feature union, spooky style. But first, wordclouds!
Inside of our wordy Cthulhu, We see a number of words that don't seem to have to much significance

And the same with Poe's Raven. The vocabulary seems pretty similar with the exception being the word "upon"
Mary Shelly on the other hand has more variety with positive words like "love" and "life" and even names like "Raymond." Nothing specific can be gleamed from this exercise, but its pretty to look at.


If you want to create a model that you can interpret after, there are really two options. CountVectorizor and TF-IDS Vectorizor. CountVectorizor, the simpler of the 2 options gives a count of certain words and ranks them based on their frequency. TF-IDF (Term Frequency - Inverse Document Frequency) adds an added weight to the proceedings by downplaying words that appear often in a document. This helps common words (that are not excluded by the stop words) to not overpower the model. I used TF-IDF to vectorize the sentences.
I also made a number of features based on the text like sentiment factor, parts of speech, and the most syllables in a sentence. Now making a model on the vecorized text alone is easy and with the created features alone is easy as well, but putting them together will require some feature union.
First a class must be created that take in feature names and their vecorizing methods.
After that, separate pipelines are created for the text features and the numeric features. The text pipeline had the vectorizer put into it while the numerical is passed through. They will be fit and transformed in the same manner.
With the two pipelines, we pass them through the FeatureUnion function and create a new object.
Finally, we can pass this through a new pipeline that will run (in this case) and logistic regression to predict the model
With the model run, we get an accuracy of 79.5% and the following confusion matrix
The upshot of doing the feature union like this is that we can easily get the Feature Importance from it.

For a full look at the code and my full analysis, please click here

Comments

Post a Comment

Popular posts from this blog

You Know You Can Be Too Fit

-
Image
A review on the concept of overfitting No, this isn't an incitement on the culture of cross-fit or other fitness related clubs. I've never had much of an issue as long as people are getting in shape and enjoying themselves that's fine. I know people find they talk about their new found hobby too much but so does anyone with a new found hobby. Have you ever talked to someone who just started improv classes?  Anyway, the overfitting issue that I'm referring to has more to do with data and the idea that less is more. We all live with the idea that the more information we have, the better we are equipped at dealing with an issue, but that simply is not the case. Overfitting is when a model performs extremely well on the data it has been trained on, but poorly on data it has never seen. Imagine you are cooking at a friends house. You have trained in your kitchen and have become so used to the set up there that when you more to a kitchen you aren't used to, you b...
Read more

Bring Balance to the Data

-
Image
When the Yays far outweigh the Nays, and vice versa One would probably assume that a 95% accuracy is a damn fine model, but in many cases that is about as bad as it gets. For a classification problem, the more the predicted value counts vary from each other, the more you have issues with imbalanced classes.  To demonstrate this concept, let's look at the issue of West Nile Virus in Chicago. Due to the heavy population density and the species of mosquito that live in the area, Chicago summers are a hot bed of activity. Using weather data and mosquito trap data, it is possible to build a predictive model see when/where the virus would be most prevalent. While West Nile is very prevalent in Chicago, When looking at the traps, we see that the number of observations is, in fact, quite low (about 5%) With that information, spraying can be done to prevent the spread of the virus. However, If you asked a computer to make the most accurate model it can to deal with the issue, i...
Read more