Twitter Grader Part 2

by Kenny on November 3, 2008

Over the wee­kend, ever­yone who is an avid follo­wer of Twit­ter Gra­der, noti­ced a sig­ni­fi­cant change in the algo­rithm. In my ori­gi­nal post I tal­ked about what I noti­ced to influence the algo­rithm for twit­ter gra­der most hea­vily. I was quite sur­pri­sed to see a com­ment from Dhar­mesh Shah, the deve­lo­per for twitter.grader.com, who appa­rently read my post and said “it’ll be inte­res­ting to recon­duct the analy­sis once we go into beta”.

Well, I don’t know if they’re quite in beta yet, but I know that my grade was at a 90 over the wee­kend, and now its at an 84! So there’s got to be a rea­son! Here’s what I think…

You will all remem­ber my friend grou­ping from the last post? Here they are with upda­ted num­bers:

 

 

 

 

 

 

 

These num­bers are all as of this mor­ning 11–3, bet­ween 11:15 & 11:30 PST.

And just for refe­rence, here is the ori­gi­nal list from the last post:

 

 

 

 

 

 

 

So to start, we can see that there has been quite a bit of varia­tion on the user accounts and on the sco­ring for almost all users. At the top end, we now have mul­ti­ple users in the 99th per­cen­tile, with seve­ral other close to follow. And at the bot­tom end, sco­res went from being in the high 50’s — low 60’s to now being high seven­ties to low to mid 80’s. This is obviously just a result of the user group being an extre­mely active sam­pling from twit­ter. Sco­res are bound to go up. ;)

But, there are a few things to note that I think are inte­res­ting. For exam­ple, take a look at davesny­der, his follo­wing count went down, follo­wer count went up, and he saw a sig­ni­fi­cant grade change after the algo­rithm update. :) -> Clue #1

Clue #2: Aus­sie­Web­mas­ter, inte­res­tingly enough, saw NO ACTIVITY on his account except for an inc­rease in follo­wers (can you tell me how you did that?), and sees a grade change of almost a full 2 points!

What does this sound like? It sounds like @ons­tar­tups read my post and imple­men­ted some chan­ges! LOL! Which is good, I think that follo­wer to follo­wing ratios should count toward their ran­king sys­tem, which it clearly didn’t before.

BUT, thats not to say that what I had dis­co­ve­red pre­viously with the update ratios isn’t still part of the algo..

If you take a look at mar­tin­bow­ling, with the most upda­tes @ 10,862, he is up on follo­wers & follo­wing less peo­ple, but only saw a grade jump of .9. Mar­tin­bow­ling was also a suf­fe­rer from the algo update, he was pre­viously in the 99 point range until the update. (Sorry Buddy!) Com­pare him to oil­man, who saw a simi­lar rise in upda­tes and follo­wers, but is follo­wing more peo­ple, whe­reas mar­tin­bow­ling is follo­wing less, and oil­man saw more than a 1 point gain in ran­king. (FYI — before the algo update, oil­man and mar­tin­bow­ling were .1 apart in score)

The only evi­dence I can take away from this, is that the update ratio still plays, although less sig­ni­fi­cant than before. And now we also see the incor­po­ra­tion of a follo­wing to follo­wer ratio.

I see this as good pro­gress for twit­ter gra­der. My recom­men­da­tion for the next algo update? Figure out a way to incor­po­rate user inte­rac­tion (@‘s, ret­weets, replys) After all, it’s all about the con­ver­sa­tion ;)

Peo­ple men­tio­ned in this post:

  • Mar­tin Bow­ling aka @martinbowling
  • Todd Frie­sen aka @oilman
  • Dave Sny­der aka @davesnyder
  • Kate Morris aka @katemorris
  • David Brown aka @NeOBlog
  • Chris Win­field aka @chriswinfield
  • Frank Watson aka @AussieWebmaster
  • Robert Pal­mer aka @robertpalmer
  • Kenny Hyder aka @kennyhyder
  • Aus­tin Cur­tis aka @austincurtis
  • Popu­la­rity: 1% [?]

    { 2 trackbacks }

    Twitter Grader Updates Algo to Incorporate @ Replies - Kenny Hyder
    November 5, 2008 at 1:12 pm
    14 Tools of Highly Effective Twitter Users - Kenny Hyder
    August 18, 2009 at 12:05 am

    { 5 comments }

    Austin November 3, 2008 at 1:19 pm

    nice post Kenny. I agree with you about @replies. Another cool con­si­de­ra­tion would be the pace at which follo­wers were added and tweets were logged.

    Jack Leblond November 3, 2008 at 1:50 pm

    Nice work — I know it’ll be hard for @grader to stay on top of the spam­mers and other idiots but it’ll be nice to have the algo give a fair and accu­rate grade to all users.

    When I look at http://twitter.grader.com/lancearmstrong all I can do is shake my head in disbelief.

    admin November 3, 2008 at 2:04 pm

    @austin totally agree about pace of follo­wers being added, i think a con­sis­tent stream of new follo­wers should boost your grade.

    @jack exactly, that’s why they need to account for user inte­rac­tion more.. If lan­cearms­trong has 2,694 follo­wers, but isn’t tal­king to any of them, his score shouldn’t be as high cause he isn’t socia­ble! It’s not right..

    David Niall Wilson November 3, 2008 at 3:10 pm

    I don’t see the follo­wer to follo­wing as a good trend because it encou­ra­ges those of us who enjoy follo­wing a variety of peo­ple who may, or may not share enough inte­rests to follow back — to drop those follo­wing relationships…

    Just my two cents.

    David

    Dharmesh Shah November 3, 2008 at 5:56 pm

    Thanks for the second round of thought­ful analysis.

    The algo­rithm was indeed upda­ted this wee­kend (and I think it’s “better”).

    Quick note: We have 200,000+ uni­que pro­fi­les that have been gra­ded. In a way, this is good, but the flip side is that it takes some time to “nor­ma­lize” the data­base of gra­des. We’ve got sig­ni­fi­cant ser­ver resour­ces powe­ring the soft­ware, but I tend to “trickle-in” the recal­cu­la­tions. That’s a long-winded way of saying: It’s going to take some time for the actual gra­des to com­ple­tely reflect the upda­ted algorithm.

    Making pro­gress (I think). The point about trac­king ret­weets and the “qua­lity” of con­ver­sa­tions for a given user pro­file is a really good one. I’ve been thin­king about that one and will try to incor­po­rate something like this in a future update.

    Cheers,
    Dharmesh

    Comments on this entry are closed.

    Previous post:

    Next post: