We started using the redis-backed Resque for background jobs. So far it’s been pretty great to work with. The syntax is clear and the whole thing just works. There are very good articles describing how to actually code Resque. Here’s one. I won’t bother with yet-another-walk-through.
However, I do want to talk about one gotcha and one thing you should think about.
The Gotcha: Possible Message Failure
Messages in Resque are not necessarily going to run. There is an, admittedly edge, case where a worker can pop a message off of a queue and then die. The message never gets put in a fail queue and it never actually runs. The message is gone. I have a fork that fixes this (makes it so messages can’t be lost unless Redis loses ‘em) https://github.com/tobowers/resque. However, that’s awaiting a merge.
Until my branch (or some other fix) gets merged, make sure you think about what’s going to happen if your job doesn’t run. For many apps this can be “ok” (since it won’t happen very often). Just make sure it’s OK for yours.
One Thing You Should Think About: Tracking Message State
Since you can’t reach into a Resque queue and say “talk to me about this job,” tracking message state is up to your application. We happen to do this in our database.
We put a “job” record into the database with body, result, state columns. The body is there so we can always re-create a message from the database if something goes wrong. The result keeps track of what happened and the state lets us track it along its merry way.
If you have to make absolutely sure that your messages get processed, you’ll have to think about how you’re using Resque.
The New York Times published the article “Once a Dynamo, the Tech Sector Is Slow to Hire.”
The piece comes across a little “woe is me.” Which I find strange. The tech industry that I know and love isn’t hurting at all. However, as the article states, the mediocre development jobs and the grunt work is being shipped overseas. That’s good for America and American developers.
This particular paragraph in the article kind of sums it up (I think):
Ms. Mann said, that an employer has sent her job abroad since she received her master’s in computer science more than two decades ago; the last time was in 2001. This week she starts a yearlong program to upgrade her programming skills, paid for by a federal program that assists workers who have been displaced by international trade.
The tech industry is so wide open for positions right now. Companies are scrambling to find the best talent and it’s incredibly difficult to find excellent developers. Software development also has vastly greater resources available to the self starter than many other industries.
All it takes to become an excellent developer is to decide on something to learn, and go learn it. You probably don’t even have to buy a book. In general, the communities are very supportive of “newbs” and there are tons of open source projects that you could work on to provide you with “something real.”
In any industry, if you “show up for your job, work a little, and then leave” you will probably be out of a job. There are hungry, passionate people all over the world that want your job. The good news is that if you care about what you do, constantly learn, and adapt then it will be impossible to ship your job overseas.
All it takes is to love your job. Do that – and I bet you keep it.
Also: read Linchpin