April, 2010
For one of our projects, we’re using beanstalkd in combination with the async_observer plugin for Rails to run slow tasks in the background.
This is a nice setup. Although we’re not using all of its more advanced features, beanstalkd is a very powerful and efficient queueing system, and async_observer provides a clean integration with Rails. (On other projects, I’ve happily used delayed_job for similar purposes, but it was not suitable for this project due to its heavy dependency on a ActiveRecord and a SQL database. By contrast, async_observer’s support for ActiveRecord was quite easy to rip out.)
However, we would like to be able to queue async_observer jobs from cron or the command line. Why launch a new process with a copy of the Rails stack when there’s already a long-lived one running? So, I wrote a ruby script with minimal dependencies which can stick tasks in beanstalkd for our async_observer worker to run.