Peons: A simple fail-safe queue

redis errorsafe queue worker peons

10|04|2010

Peons:

n. Hard working dependable slaves, with a keen attention to mining your gold.

If you have been doing a lot of web apps lately, chances are that you have noticed the massive attention towards background processing queues. Peons is just a simple, yet robust attempt, to solve the same problem, albeit with less code.

Why reinvent the wheel?

I'm sure you also know that there exists tons of solutions out there for solving this same problem. Here's a couple of reasons that will favor Peons over the other solutions:

  1. If you're already using Redis in your project, then this won't require an added dependency, like RabbitMQ, ZeroMQ, etc (the list goes on).

  2. If you like dead simple solutions (lib/peons.rb is only 53 lines of code).

  3. If you want error-safe processing (e.g. the power of your server runs out during the act of extracting a job from your queue).

Getting started

Get it

  1. cyx/peons in github, or
  2. gem install peons

Install it

Play with it

What's next?

There's still some work to be done with peons. Here are just a few off the top of my mind:

  1. Build a great CLI interface (heroku comes to mind)
  2. Build a simple sinatra (or maybe [rum][rum] or [cuba][cuba]) app that allows you to see the queues and optionally empty the queues, re-insert backup items, etc.
  3. More documentation, examples, guidelines.
blog comments powered by Disqus