Twitter Grader Part 2

Social Media

by Kenny Hyder

Over the wee­kend, ever­yone who is an avid follo­wer of Twit­ter Gra­der, noti­ced a sig­ni­fi­cant change in the algo­rithm. In my ori­gi­nal post I tal­ked about what I noti­ced to influence the algo­rithm for twit­ter gra­der most hea­vily. I was quite sur­pri­sed to see a com­ment from Dhar­mesh Shah, the deve­lo­per for, who appa­rently read my post and said “it’ll be inte­res­ting to recon­duct the analy­sis once we go into beta”.

Well, I don’t know if they’re quite in beta yet, but I know that my grade was at a 90 over the wee­kend, and now its at an 84! So there’s got to be a rea­son! Here’s what I think…

You will all remem­ber my friend grou­ping from the last post? Here they are with upda­ted num­bers:








These num­bers are all as of this mor­ning 11–3, bet­ween 11:15 & 11:30 PST.

And just for refe­rence, here is the ori­gi­nal list from the last post:








So to start, we can see that there has been quite a bit of varia­tion on the user accounts and on the sco­ring for almost all users. At the top end, we now have mul­ti­ple users in the 99th per­cen­tile, with seve­ral other close to follow. And at the bot­tom end, sco­res went from being in the high 50’s — low 60’s to now being high seven­ties to low to mid 80’s. This is obviously just a result of the user group being an extre­mely active sam­pling from twit­ter. Sco­res are bound to go up. ;)

But, there are a few things to note that I think are inte­res­ting. For exam­ple, take a look at davesny­der, his follo­wing count went down, follo­wer count went up, and he saw a sig­ni­fi­cant grade change after the algo­rithm update. :) -> Clue #1

Clue #2: Aus­sie­Web­mas­ter, inte­res­tingly enough, saw NO ACTIVITY on his account except for an inc­rease in follo­wers (can you tell me how you did that?), and sees a grade change of almost a full 2 points!

What does this sound like? It sounds like @ons­tar­tups read my post and imple­men­ted some chan­ges! LOL! Which is good, I think that follo­wer to follo­wing ratios should count toward their ran­king sys­tem, which it clearly didn’t before.

BUT, thats not to say that what I had dis­co­ve­red pre­viously with the update ratios isn’t still part of the algo..

If you take a look at mar­tin­bow­ling, with the most upda­tes @ 10,862, he is up on follo­wers & follo­wing less peo­ple, but only saw a grade jump of .9. Mar­tin­bow­ling was also a suf­fe­rer from the algo update, he was pre­viously in the 99 point range until the update. (Sorry Buddy!) Com­pare him to oil­man, who saw a simi­lar rise in upda­tes and follo­wers, but is follo­wing more peo­ple, whe­reas mar­tin­bow­ling is follo­wing less, and oil­man saw more than a 1 point gain in ran­king. (FYI — before the algo update, oil­man and mar­tin­bow­ling were .1 apart in score)

The only evi­dence I can take away from this, is that the update ratio still plays, although less sig­ni­fi­cant than before. And now we also see the incor­po­ra­tion of a follo­wing to follo­wer ratio.

I see this as good pro­gress for twit­ter gra­der. My recom­men­da­tion for the next algo update? Figure out a way to incor­po­rate user inte­rac­tion (@‘s, ret­weets, replys) After all, it’s all about the con­ver­sa­tion ;)

Peo­ple men­tio­ned in this post:

  • Mar­tin Bow­ling aka @martinbowling
  • Todd Frie­sen aka @oilman
  • Dave Sny­der aka @davesnyder
  • Kate Morris aka @katemorris
  • David Brown aka @NeOBlog
  • Chris Win­field aka @chriswinfield
  • Frank Watson aka @AussieWebmaster
  • Robert Pal­mer aka @robertpalmer
  • Kenny Hyder aka @kennyhyder
  • Aus­tin Cur­tis aka @austincurtis

    Austin November 3, 2008 at 1:19 pm

    nice post Kenny. I agree with you about @replies. Another cool consideration would be the pace at which followers were added and tweets were logged.

    Jack Leblond November 3, 2008 at 1:50 pm

    Nice work – I know it’ll be hard for @grader to stay on top of the spammers and other idiots but it’ll be nice to have the algo give a fair and accurate grade to all users.

    When I look at all I can do is shake my head in disbelief.

    admin November 3, 2008 at 2:04 pm

    @austin totally agree about pace of followers being added, i think a consistent stream of new followers should boost your grade.

    @jack exactly, that’s why they need to account for user interaction more.. If lancearmstrong has 2,694 followers, but isn’t talking to any of them, his score shouldn’t be as high cause he isn’t sociable! It’s not right..

    David Niall Wilson November 3, 2008 at 3:10 pm

    I don’t see the follower to following as a good trend because it encourages those of us who enjoy following a variety of people who may, or may not share enough interests to follow back — to drop those following relationships…

    Just my two cents.


    Dharmesh Shah November 3, 2008 at 5:56 pm

    Thanks for the second round of thoughtful analysis.

    The algorithm was indeed updated this weekend (and I think it’s “better”).

    Quick note: We have 200,000+ unique profiles that have been graded. In a way, this is good, but the flip side is that it takes some time to “normalize” the database of grades. We’ve got significant server resources powering the software, but I tend to “trickle-in” the recalculations. That’s a long-winded way of saying: It’s going to take some time for the actual grades to completely reflect the updated algorithm.

    Making progress (I think). The point about tracking retweets and the “quality” of conversations for a given user profile is a really good one. I’ve been thinking about that one and will try to incorporate something like this in a future update.


    Previous post:

    Next post: