Twitter API Restrictions and AJAX to the Rescue

We’re working on a facebook app that pulls in Twitter information into a facebook page tab. Unfortunately, Twitter limits its API accesses to 150/hour identified by IP address. So if you are on a shared hosting plan, or just need a lot of data rapidly, this is quite limiting. While researching this problem, I found several people caching their data, to limit Twitter API accesses. However, even if you cache, that’s still pretty limiting. Here is how we worked around it.

If you load Twitter data from the client, you suddenly get a large pool of IP addresses to collect data. Its quite unlikely that your clients will access Twitter APIs at a rate greater than 150 hits/hour. So mix in a little jQuery and you get:

$(document).ready(function() {
$.getJSON('http://api.twitter.com/1/statuses/user_timeline.json?screen_name=voltampmedia&callback=?', function(data) {
var items = [];

var text = '';
var tweet_date;
var first = false;
$.each(data, function(key, val) {
text = val.text;
tweet_date = new Date(val.created_at);
pub_date = twitter_dates(val.created_at);
if(first == false){
items.push(first_html(pub_date,text.parseUsername().parseHashtag().parseURL()));
first = true;
} else {
items.push(remaining_html(pub_date,text.parseUsername().parseHashtag().parseURL()));
}
});

$('div#twitter').append(items.join(''));
});
});

Make your API call, process the data in javascript, and voila, Twitter is none the wiser… for now…
if Twitter is wiser, we could have the clients transmit back Twitter data for caching, do caching ourselves at a frequency that meets the rate limit, or a whole host of things.

On a side note, I borrowed some of those parsing functions. I can’t recall where I found them, so here there are again.

String.prototype.parseURL = function() {
var exp = /(\b(https?|ftp|file):\/\/[-A-Z0-9+&@#\/%?=~_|!:,.;]*[-A-Z0-9+&@#\/%=~_|])/ig;
return this.replace(exp,"<a href='$1'>$1</a>");
};

String.prototype.parseUsername = function() {
return this.replace(/[@]+[A-Za-z0-9-_]+/g, function(u) {
var username = u.replace("@","")
return u.link("http://twitter.com/"+username);
});
};

String.prototype.parseHashtag = function() {
return this.replace(/[#]+[A-Za-z0-9-_]+/g, function(t) {
var tag = t.replace("#","%23")
return t.link("http://search.twitter.com/search?q="+tag);
});
};