Saturday, August 09, 2014

Dropping Actual Science

Matt Daniels is a data scientist at Undercurrent, a New York-based strategy and management consulting firm. (As an aside, if your kids ask you what they should be when they grow up, you should tell them to be data scientists. Those fuckers are gonna own the world.) Matt Daniels also loves hip hop. He combined his gift for data analysis with his passion for music to create 'The Largest Vocabulary in Hip Hop' project.

Version 2.0 of Daniels' work was just released in June, and compares the lyrical range of 85 different artists (since the analysis requires at least 35,000 words in an artists' recorded body of work, it doesn't include newer acts, or those with limited recordings). As Daniels explains, "
I used a research methodology called token analysis to determine each artist’s vocabulary. Each word is counted once, so pimpspimppimping, and pimpin are four unique words. To avoid issues with apostrophes (e.g., pimpin’ vs. pimpin), they’re removed from the dataset. It still isn’t perfect. Hip hop is full of slang that is hard to transcribe (e.g., shorty vs. shawty), compound words (e.g., king shit), featured vocalists, and repetitive choruses."

The analysis is published on the project website, but you can purchase it in poster form from Pop Chart Lab. Aesop Rock is the runaway lyrical champion of the world, his vocabulary so impressive that Daniels had to change the x-axis to accommodate him. Of note, as Daniels says, "Wu-Tang Clan at #6 is fucking impressive given that ten members, with vastly different styles, are equally contributing lyrics."

On the other end of the scale, DMX sits in dead last, joined by a number of hip hop's biggest names, including Kanye, Lil Wayne, Snoop, and 2Pac. Goes to show, I guess, that lyrical virtuosity may not be the most important factor in moving records. Which is something Sean Carter not only figured out, but let us know on The Black Album cut, "Moment of Clarity":

I dumbed down my audience to double my dollars
They criticized me for it, yet they all yell 'holla'
If skills sold, truth be told, I'd probably be
Lyrically Talib Kweli
Truthfully I wanna rhyme like Common Sense
But I did 5 mil - I ain't been rhyming like Common since

So Jay-Z is already down with data science. What did I tell you about owning the world?

41 comments:

T.J. said...

Gheorghies, if you have any recommendations for Budapest, Vienna, or Prague...now's the time to share.

rob said...

i got nothing

TR said...

Get the linguini and white clam sauce. And a Coke with no ice.

Dave said...

go to kutna hora in the czech republic, short trip from prague., and go to the sedlec ossuary. bone chandeliers and other bone and skull mayhem. a must see.

Clarence said...

Budapest is actually split into Buda and Pest. Teej, Pest would be good for you.

I recommend Wienerschnitzel heartily. Pounded veal is the schnitzel.

T.J. said...

Thank you, Summer Dave and Clarence

zman said...

Didn't Mark already post this? Or at list link to it in a comment?

Regardless ... lookit rob, quoting Hova! Who knew he knew?

Mark said...

I believe I may have posted a link to it. It showed up on my twitter timeline about a month ago.

Mark said...

Air conditioning: still somehow underrated.

zman said...

Not by me. I'm a sweaty bastard. AC is my shit.

Mark said...

Me either. Living most of your life in Florida regularly brings the point home. Some around me have a new appreciation though.

To be fair, it was hot as fuck in this house for about a day. Like so hot I chose to do yard work.

Clarence said...

H I G H E O R G H I E S

zman said...

Oh snap! A caps lock hi Gheorghies!

T.J. said...

Good afternoon Gheorghies from the Munich airport

rob said...

tony stewart's prickliness doesn't seem so cute this morning

Clarence said...

Just got told a story about me from last night. We saw The Killers in a tiny SF venue and they played a cover of Tommy James and the Shondells' "I Think We're Alone Now." Which I like. And I'm pretty sure it was a great cover. Though I don't really remember.

Nor do I remember shouting "Tiffanyyyyyyy!!" At the top of my lungs over and over while they played it. What a douche.

rob said...

next on a very special 'that's so clarence'...

rob said...

got a little bit of a golf tournament here

rob said...

played two hours of hoops this morning, then did yard work for another 90 minutes. showered, ate, sat down on the couch to do a bit of work on the computer. stood up an hour later and my right ankle nearly collapsed. now i can barely put any weight on it. there was no pain after playing hoops at all, nor any particular trauma. ideas, doctors?

Danimal said...

Rob....that's weird. Swelling?

The one time we have a decent tourney it's going to finish on a Monday. Oh wait, we'll watch it at work.

Danimal said...

Danimal smoked the shit out of some fish tonight....if you go to my twitter page and see the most recent saved/favorited tweet, 'tis a recipe you smokers might want to take a gander at. Exceptional. Or I guess I can just tweet it to you. I will do that.

rob said...

no noticeable swelling, danimal. fucking hurts, though. beer and alive barely making a dent.

rob said...

can someone explain the ice bucket challenge to me? i could google it, but that would take more effort than i'm interested in making.

Danimal said...

Rob....my prediction is a stressline fracture.

The ice bucket challenge is likely coming to you very soon.....it's to raise awareness for ALS.

Danimal said...

Hairline

rob said...

i'm pretty aware of als. does that mean i can skip it?

Danimal said...

Sure. But those that do not accept the challenge are supposed to make a donation. I look forward to seeing your dousing on the Facebook.

Shlara said...

How about Rory today?
And I thought it would be Rickies turn

Danimal said...

Rors is the new Tiger. And we will be lucky to get 6 points in the Ryder Cup.

rob said...

feel for rickie, but was pulling for rors. cool reaction after the win.

Danimal said...

Rob, since I told you about the ice bucket challenge, what can you tell me about transvaginal mesh?

Mark said...

I saw you tweet that recipe a couple days ago or whenever it was you originally tweeted it, Dan. I favorited it then as it looked great. I'll be trying it soon.

The wife braised short ribs tonight and threw some bone marrow in the mix too. The ribs were fantastic but were upstaged by the marrow. Bone marrow is so god damn good.

Mark said...

I'm not the golf fan/aficionado that you guys are but has anyone ever had a better year in the majors without winning a major than Fowler?

Danimal said...

I don't know for sure but doubt it....maybe Jack.

Danimal said...

Or Norman...

Marls said...

Jack's 1977 season was the closest I could find. 2nd, 10th, 2nd, 3rd.

mayhugh said...

Rob - whereabout in your foot/ankle is the pain? Is there any redness? What does the pain feel like? Is it throbbing? Did you have any indication (soreness, etc.) prior to the pain coming on fully? Do you drink a lot of beer? Do you eat a lot of red meat and/or seafood?

Mr KQ said...

Have you ever been in a Turkish Prison?

T.J. said...

EuroTeej says good mornin, Gheorghies

Clarence will be happy to know I am sporting my FUBP shirt around Budapest today

Danimal said...

mark - do grocery stores sell what you need and how you need it to do bone barrow? Or do you have to go to a butcher?

Rob - Mayhugh thinks you might have the gout. Are you carrying around a huge container of uric acid everywhere you go, like the guy in the commercial? If so, you have the gout.

Danimal said...

Teej, our roving reporter. Is Teej scouting out possible Hooters International locations?