Latest and Greatest?

queues background rails ruby redis

11|04|2009

I have been very interested and locked to background processing ever since I've employed it. I have tried Starling, Delayed::Job, beanstalk, ActiveMessaging, ZeroMQ, and RabbitMQ. Here's a quick rundown:

ActiveMessaging, ZeroMQ, RabbitMQ

ActiveMessaging is basically the library that handles communication using AMQP. ZeroMQ and RabbitMQ are both AMQP based.

In my opinion, both are not that easy to get up and running (RabbitMQ especially), and both are too heavyweight for a typical site which basically just needs to background a few stuff. And I have this sense that AMQP is not really ideal for your typical backgrounding needs.

Delayed::Job

I guess the feature set of Delayed Job is very good. I'm probably just biased against using a Database to do stuff like this, which I feel is not something it was built for. With around 30K jobs, pushing and popping jobs go up to about 2 seconds.

beanstalk

What can I say, I feel that it's pretty stagnant in terms of development. Although the library is fast, I'm missing persistence.

Enter Resque

You can read about the story here : http://github.com/blog/542-introducing-resque

Basically it's an in-house background queue built and used by github. Now off to hacking and trying it out.

blog comments powered by Disqus