cfis : Rails - Simple Asynchronous Processing

Comment on Rails - Simple Asynchronous Processing by jim Cropcho

Tue, 17 Jun 2008 07:11:30 -0600

thanks for getting this up; I’m def checking out the worker plugin.

Comment on Rails - Simple Asynchronous Processing by aleco

Tue, 17 Jun 2008 09:03:38 -0600

this sounds like a very interesting plugin. are you planning to add scheduled tasks that are not triggered by a controller action?

ps: personally I’d strongly prefer github over rubyforge.

Comment on Rails - Simple Asynchronous Processing by Jim Cropcho

Tue, 17 Jun 2008 10:29:21 -0600

I noticed that the .zip doesn’t include any tests. Has any testing been done on the plugin yet? If not, I suppose writing them would be a first step, should a formal project be created.

Comment on Rails - Simple Asynchronous Processing by Charlie

Tue, 17 Jun 2008 11:48:17 -0600

Aleco - Github is also fine if that’s what people want.

Comment on Rails - Simple Asynchronous Processing by Charlie

Tue, 17 Jun 2008 11:50:32 -0600

Jim - Sorry, there aren’t any direct tests with the plugin. Having said that, we’ve been running the code in production for the last couple of months. In addition, it is tested indirectly through our unit and functional test suites (the tests verify that certain background requests get completed correctly).

So I so feel fairly confident about the code - although I’m sure there are some lurking bugs. Adding some direct tests would be a good thing.

Trackback from roScripts - Webmaster resources and websites: cfis : Rails - Super Simple Asynchronous Processing on Rails - Simple Asynchronous Processing

Tue, 17 Jun 2008 20:15:04 -0600

cfis : Rails - Super Simple Asynchronous Processing

Comment on Rails - Simple Asynchronous Processing by Noel

Wed, 18 Jun 2008 14:24:10 -0600

The instructions for the test and development environment file modifications are the same – is this correct?

Comment on Rails - Simple Asynchronous Processing by Thomas Kadauke

Wed, 18 Jun 2008 15:18:57 -0600

Hey there,

Here is a completely different approach to background processing, that can be used with any of the above mentioned choices:

http://devblog.imedo.de/2008/6/18/running-ruby-blocks-in-the-background

Instead of implementing a complete background task communication protocol, this solution builds on top of any communication protocol to execute code in a background process. Plus, it is the only solution that I know of, that has DRY error recovery support.

Comment on Rails - Simple Asynchronous Processing by Charlie

Wed, 18 Jun 2008 17:33:08 -0600

Good catch Noel - the test and development modes use different workers. Test mode uses an in-process worker (similar to render_component), while development mode uses an out of process worker (well sort of, its the same Rails instance but its done via a 2nd http request).

Comment on Rails - Simple Asynchronous Processing by Thomas Kadauke

Wed, 18 Jun 2008 17:56:07 -0600

Hey there,

I’m not sure if my last comment made it all the way through. Anyway, here is it:

Here is a completely different approach to background processing, that can be used with any of the above mentioned choices:

http://devblog.imedo.de/2008/6/18/running-ruby-blocks-in-the-background

Comment on Rails - Simple Asynchronous Processing by Charlie

Thu, 19 Jun 2008 14:15:35 -0600

Hi Thomas,

Thanks for the link. Looks like youv’e done some great work - the Imedo solution of passing blocks is quite interesting, I’ve never seen anything like that before.

By passing blocks, its a ruby-centric take on distributed objects. As you could probably guess from the article, I’m not a fan of such solutions, since I think they bind processes to tightly together and are tough to debug. Having said that, they can can work well in a local network.

As for working with any communication protocol, I think that’s over engineering. I know it seems like a good idea, but every time I’ve tried it the extra complexity has never been worth the flexibility. Is someone really going to switch communication protocol? And of course, you still need at least one communication protocol anyway, so now a user needs to install two things.

Error recovery is definitely a good point since its non-obvious what the best way is for a background process to report an error. So curious what you mean that it has DRY error support?

And as for DRY generally - one of the thing that I like about the solution we’re using is that everything is a Rails instance. Thus there is no need to implement a bunch of supporting code in background instances that duplicate what you already have in your foreground instances.

Comment on Rails - Simple Asynchronous Processing by niko

Fri, 20 Jun 2008 01:24:22 -0600

I really like the approach and I got i working… basically. But I had to set

session :cookie_only => false

in the controller. Otherwise I’d get a SessionFixationAttempt error on the process method. I’m working with Rails 2.1.

Now the bad thing is, with :cookie_only set to false, the :process action can’t add flash-notices. I’d really like the user to be noticed about the progress…

What am I doing wrong?

Niko.

PS: +1 for github :D

Comment on Rails - Simple Asynchronous Processing by niko

Fri, 20 Jun 2008 03:18:54 -0600

pls forget about the last comment… of course the flash notice isn’t added until the process action finishes.

Comment on Rails - Simple Asynchronous Processing by Charlie

Fri, 20 Jun 2008 13:25:45 -0600

Hey Niko - Good point I forgot to mention. You have to tell rails that you’re passing the session id as URL parameter.

As far as notifying a user about progress - you can’t really since this is a background asynchronous process. What we do instead is send a message to a user’s “Inbox” telling them when the action has completed. You could also send an email, sms, etc.

Comment on Rails - Simple Asynchronous Processing by Thomas Kadauke

Mon, 23 Jun 2008 03:46:45 -0600

@Charlie: Our background processing plugin does not implement any communication protocol (like e.g. ActiveMessaging does), but it relies on the user to use an existing one, or implement a new one. In the case of ActiveMessaging, that is a single line of code. It’s really not that complicated, just a matter of how the code block is sent to the background process.

The advantage of being able to switch communication protocols on the fly is that you get error recovery for free. Shipped with the plugin are fallback handlers that execute the code block in-process, write it to disk, or discard it altogether.

The DRY part about this is that for using fallback handlers, you still only need to write

background do
  # your code
end

if it fails to connect to your background process, the code block is automatically sent to a fallback handler (which can be configured in environment.rb).

Comment on Rails - Simple Asynchronous Processing by Charlie

Mon, 23 Jun 2008 10:00:25 -0600

Thomas - Thanks for the comments. I understand that your background process can use various communication protocols to send blocks to background processes. What’s involved on the receiving end though to support the various protocols?

As far error handling, is it really the case that if using ActiveMessaging to send a message fails then rolling over to some other protocol is going to work? That seems unlikely to me, but what are you seeing in practice.

Trackback from Remmrit Bookmarking: Super Bookmarks on Rails - Simple Asynchronous Processing

Wed, 13 Aug 2008 13:00:08 -0600

Remmrit.com user has just tagged your post as Super!

Comment on Rails - Simple Asynchronous Processing by aleco

Sat, 04 Oct 2008 13:43:29 -0600

The link http://cfis.savagexi.com/files/worker.zip returns a 404 error. Has the plugin been moved to github/rubyforge? I can’t seem to find it anywhere…