Restful Rails Revisited

Posted by Charlie Mon, 03 Apr 2006 05:51:00 GMT

Anyone interested in making Rails a bit more restful should check out the current thread on the Microformats Rest mailing archive.

A good place to start is message. You can also find more information on Peter’s blog.

Posted in ,  | no comments | no trackbacks

Firebug

Posted by Charlie Fri, 31 Mar 2006 20:12:00 GMT

Just noticed that Joe Hewitt has posted an updated version of Firebug. He's taken a great extension and turned it into the best Firefox extension by far. Its an absolutely invaluable development tool. If you develop web apps, you should go download it now.

Posted in ,  | no comments | no trackbacks

Rails and Content Negotiation Revisited

Posted by Charlie Fri, 31 Mar 2006 07:41:00 GMT

Let's revisit the subject of content negotiation in Rails. Now that I've upgraded to Rails 1.1 I've had a chance to play with the new responds_to functionality. To get some feedback, I also posted a comment to Jamis' blog and a message to the REST microformat list.

My conclusion is that adding content negotiation to Rails would be valuable. The new responds_to functionality is useful when you want to override the default behavior. Having content negotiation would be useful for making the default behavior more intelligent.

Templates already work in a similar manner. For example, say you write this code:

class MyController < ApplicationController
  def list
  end
end
Rails will assume that you want to render a template called list.rhtml. However, you can override this behavior as needed like this:
class MyController < ApplicationController
  def list
    render(:template => 'not_the_default_template')
  end
end

Content negotiation would work in a similar manner. It would load the appropriate template based on a client's request as specified in the HTTP accept header.You can then override that behavior when needed using responds_to.

The way I propose this works is that Content Types are mapped to templates via their file extensions. For instance:

Content Type Template Name
text/html list.rhtml
application/xhtml+xml list.rxhtml
text/javascript, application/javascript list.rjs
application/atom+xml list.ratom
application/atom+rss list.rss
application/x-yaml list.ryaml

 

One key thing to understand - the template extension does not dictate the template type. Thus, list.rxhtml could be an ERB template or a Builder template or any other sort of template.

Right now Rails doesn't support such a scheme because template extensions are hard-coded to template types. This is done in ActionView::Base::create_template_source:

def create_template_source(extension, template, render_symbol, locals)
  if template_requires_setup?(extension)
    body = case extension.to_sym
    when :rxml
      "xml = Builder::XmlMarkup.new(:indent => 2)\n" +
      "@controller.headers['Content-Type'] ||= 'application/xml'\n" +
      template
    when :rjs
      "@controller.headers['Content-Type'] ||= 'text/javascript'\n" +
      "update_page do |page|\n#{template}\nend"
   end
  else
    body = ERB.new(template, nil, @@erb_trim_mode).src
  end
  
  @@template_args[render_symbol] ||= {}
  locals_keys = @@template_args[render_symbol].keys | locals
  @@template_args[render_symbol] = locals_keys.inject(
                                    {}) { |h, k| h[k] = true; h }
  locals_code= ""
  locals_keys.each do |key|
    locals_code << "#{key} = local_assigns[:#{key}]
                         if local_assigns.has_key?(:#{key})\n"
  end
  
  "def #{render_symbol}(local_assigns)\n#{locals_code}#{body}\nend"
end

It would be easy to work around this code since the template parameter includes the full source of the template. With a little regular expression magic, you could figure out what type of template has been loaded and then run the appropriate code.

However - what if we did something more clever? What if every template started with an comment that specified its type - similar to a Unix shebang or XML processing instruction. Maybe something like this:

For ERB:
<!-- template_type :erb -->

For RJS:
# template_type :rjs

For Builder:
<!-- template_type :erb -->

It would then be easy to extract out the template type when it was loaded. If the template comment did not exist (i.e., for all existing templates) then you could revert to regular expression magic.

This solution has one very nice property - it would make it really easy to add additional template types to Rails (not that its that hard now).

To recap, the idea is:

  • Implement default content negotiation by mapping content types to template names via file extensions (.rhtml, .rxthml, .rjs., .ratom, etc.)
  • Add code to Rails so that it determine a template's type when the template is loaded
  • Use responds_to to override the default behavior as needed.

 

What do people think of this idea? If there is interest, I'll update my content negotiation plugin and add this functionality to Rails 1.1.

Posted in ,  | 8 comments | no trackbacks

Need Help on HTTP Status Code

Posted by Charlie Fri, 31 Mar 2006 06:21:00 GMT

Imagine the following situation:

  1. You have an HTML sign up form for a user.
  2. One of the fields is for the user's email.
  3. The user fills in the data and POSTs the response to the server.
  4. The server rejects the request, perhaps the email address is already taken.

Is there an appropriate HTTP status code for this? It seems to me this falls into the 4xx range since the client provided invalid data. Its not a 400 though since the server understood the request. And none of the other 4xx codes seem to fit.

I ran into a similar situation today, except that I'm doing it via an XmlHttpRequest. When posting some data to the server, the request always returns HTTP status code 200 whether or not data was actually inserted into the backend database. Thus, the server has to return back a response that tells the client what happened - probably via a little bit of JavaScript. But it seems like using an HTTP status code would be a much cleaner solution.

Any ideas? Or am I trying to misuse HTTP status codes?

Update - I've happily found my answer. A similar problem was posted by Jan Algermissen on the Yahoo Rest discussion list. Bill de hora answered use a new HTTP status code, 418. And in response to that, Roy Fielding, one of the authors of HTTP specification, asked if he should standardize it. Yes please. Now, let's see what happens when I try this trick with the browsers.

5 comments | no trackbacks

Typo Feed Content Types

Posted by Charlie Thu, 30 Mar 2006 07:14:00 GMT

Being a good web netizen, every few days I check that my blog and Yue's blog validate using the W3C feed validator (well someone has to serve valid content - of course having said that I'd better go check again!).

I noticed that Typo, my blogging software, was serving RSS and ATOM as text/html - which really isn't correct. So I submitted a bug, and a really simple patch, and didn't think much about it.

Today I received an email that the patch has been applied by Piers. Thanks Piers - and I have to say I'm impressed by the fast turnaround!

After using Typo for the last week, I'm pretty happy with it. Its quite feature rich, particularly considering its young age. To be fair, its not as polished as WordPress, which is what Yue uses. The biggest difference is in editing. WordPress 2.0+ has a very nice WSYWIG rich text editor that's simple to use. In contrast, with Typo you need to know Markdown, Textile or good ole (X)HTML. For technically inclined users that's no problem, probably even better. But for users who want to just post content without knowing the gory technical details, its a showstopper. It would also be nice to have a spell checker.

Besides those two things, I can highly recommend Typo. Of course, you have to be willing to roll up your sleeves and install your own blogging software (the same is also true for WordPress).

2 comments | no trackbacks

I've Been Found

Posted by Charlie Wed, 29 Mar 2006 18:39:00 GMT

Well it took 10 days but both Google and MSN have added me to their indices. Sadly, Yahoo still doesn’t acknowledge my existence :)

Its interesting watching the search engines work from the outside. I watch the spiders come by every day at about the same frequency. I see that MSN and Google both added me to their indicess on the same day. And I see more or less the same search results from the engines.

Now if only that other Charlie Savage didn’t exist I’d be all set with the search engines. Hopefully, he won’t decide to start a blog.

no comments | no trackbacks

Data and Reality

Posted by Charlie Wed, 29 Mar 2006 17:27:00 GMT

Data and Reality

Every artistic endeavor strives to capture some slice of the world around us. You might be trying to create as realistic a portrayal as possible, or maybe you're after a highly abstract portrayal that tries to evoke some emotion.

In the world of software (an artistic endeavor if I've ever seen one), we create highly-tailored portrayals that are suitable only for the purpose they were constructed.1 Take two or more programs that are in the same problem domain - human resource software, databases, blogging software, etc. None of them will be compatible because they represent a single designer's (or maybe a couple) view of a problem domain. And every one has a different view of the problem to be solved and how to solve it.

Take a simple term - say a street. What it is it? Have you ever been on a street that ends, and then starts up in a few blocks? Is that the same street? How about a street whose name changes - is it still the same street? Does a street include boulevards, highways, freeways? Does a street cross city, county, state boundaries? Do you and I live on the same street, although I live in the United States and you live in Canada?

Most of us just want to bury our heads in the sand when faced with such questions - thinking about them is painful because there is no right answer and you're immediately led to deep philosophical questions about how we perceive the world around us. And most books and papers on the subject are highly technical, and of very little use to people who need to create software by yesterday.

Fortunately, there is a fantastic book on the subject, called Data and Reality, that is deeply grounded in how these issues affect software design. Written by William Kent, it addresses issues that come up every day in design. What is the difference between an identifier and a name? What is a type or category? What is the difference between an attribute and a relationship? For that matter, what's the difference between types, attributes and relationships? What are the strengths and weaknesses of the hierarchical, network and relational models? Don't expect any easy answers - they aren't any.

Don't let the publication date, 1978, fool you. The questions the book raises are as valid today as then. In fact, I'd bet they'll be just as valid one thousand years from now. Luckily the book was reprinted in 2000, so go pick up your copy now before it goes out of print again.

Let me leave you with one of my favorite passages from the book (page 10, 2000 paperback edition):

We seem to have little difficulty with the concept of "one person" despite changes in appearance, personality, capabilities, and, above all, chemical composition...When we speak of the same person over a period of time, we certainly are not referring to the same ensemble of atoms and molecules. What the is the "same person"? We can only appeal to some vague intuition about the "continuity" of - something - through gradual change. The concept of the "same person" is so familiar and obvious that it is absolutely irritating not to be able to define it. Definitions in terms of "soul" and "spirit" may be the only true and humanistic concepts, but significantly, we don't know how to deal with them in a computer-based information system. It is only when the notion of "person" is pushed to some limit do we realize how imprecise the notion is.

1I can't help but think of software products as Mount Everests on a fitness landscape that are infinitely thin. Veer just slightly off their intended purpose and everything comes crashing down. Read more...

no comments | no trackbacks

REST Controller for RAILS

Posted by Charlie Sun, 26 Mar 2006 09:01:00 GMT

Time to continue the REST and Rails theme.

Rails' most egregious violation of REST is that it exposes object methods in URIs:

http://myrailsapp.com/users/show
http://myrailsapp.com/users/list
http://myrailsapp.com/users/update

Remember, one of the fundamental ideas of REST is to manipulate resources (represented by URIs) via the use of a limited set of verbs. For our purposes, the verbs that matter are HTTP's GET, POST, PUT and DELETE.

Rails does not do this, instead it exposes public object methods on the Internet. This is no different than what DCOM, CORBA, RMI, XML-RPC, SOAP and your own home-grown RPC (remote procedure call) system do. And yes, every good developer has written an RPC system or two. In fact, I think that's the main problem. Every developer I've ever met who writes a distributed computing system always implements some sort of RPC mechanism. I know I did it the first time. Its seems perfectly logical - let's hide away the nasty details of the network so the developer doesn't have to worry about it. Of course it doesn't work, but that doesn't mean that 9 out of 10 developers will tell you it does.

The good news of course is that RAILS is built on top of the Web, which is one of the most successful distributed computing system created and is not an RPC system. In addition, RAIL's RPC implementation is much simpler to use than the ones I mentioned above and it strongly encourages you to create nice looking URIs which is a significant benefit.

So at this point, unless you're a RESTafarian, you're probably asking yourself "so what?" since RAILS is a very productive framework. We'll let's go through what would have to change to see if it has any interest to you.

Let's start with the fundamental problem - Rails equates each URI with a single action as opposed to a single resource that can be manipulated via HTTP verbs. For example, let's say we have a products table with an id field and a name field. Then let's run the rails scaffold generator. If you look at the generated code, you'll see its creates a number of public methods that manipulate several resources. More concretely, the following table shows how the generated methods map to resources and HTTP verbs.

Mapping RAILS to REST
Resource GET POST PUT DELETE
products list create    
each product show   update destroy
creator new      
editor edit      

 

Thus, I've identified four types of resources:

  • The resource that represents the collection of all products
  • Any number of resources that identify individual products
  • A creator resource
  • An editor resource

 

The creator and editor resources might be confusing. An analogy I've heard before that helps me is to imagine your resource is a Microsoft Word document. Does that mean that Microsoft Word itself is part of your resource? No, that doesn't make sense. Instead, Microsoft Word is a resource that you can use to manipulate your document resource. Thus, the creator and editor resources provide a representation (i.e., an HTML document) that allows you to create new product resources or edit existing product resources.

So how would these map to URIs. Well, that's pretty easy:

http://myrailsapp/product
http://myrailsapp/product/member/:id
http://myrailsapp/product/creator
http://myrailsapp/product/editor/:id

Instead of the standard RAILS route of :controller/:action/:id we are now doing :controller/:resource/:id. In addition, sInce http://myrailsapp/product/member/:id is so common, I tend to shorten it using routes to:

http://myrailsapp/product/:id

Still with me? If so, then is it possible to fit these ideas into RAILS which did not start with REST in mind? Well, I know of at least two previous attempts, both which provided inspiration but in the end aren't satisfactory. I think this third attempt falls into the same camp, but hopefully will keep the discussion going and throw another idea or two into the pot.

So here's my take - feel free to download this plugin if you'd like experiment with live code:

class ProductController < ApplicationController
  include RestController

  def get
    @product_pages, @products = paginate :products, :per_page => 10
  end

  def post
    @product = Product.new(params[:product])
    begin
      @product.save!
      flash[:notice] = 'Product was successfully created.'
      redirect_to :resource => :collection
    rescue => e
      flash[:product] = @product
      redirect_to :resource => :creator
    end
  end
  
  resource :Member do
    def get
      @product = Product.find(params[:id])
    end

    def put
      @product = Product.find(params[:id])
      begin
        @product.update_attributes(params[:product])
        flash[:notice] = 'Product was successfully updated.'
        redirect_to :id => @product
      rescue => e
        # Send the current invalid values to the editor via the flash
        flash[:product] = @product
        redirect_to :resource => :editor, :id => @product
      end
    end

    def delete
      Product.find(params[:id]).destroy
      redirect_to :id => nil, :resource => nil
    end
  end

  resource :Creator do
    def get
      @product = flash[:product] ||  Product.new
    end
  end
  
  resource :Editor do
    def get
      @product = flash[:product] || Product.find(params[:id])
    end
  end
end

Some things to notice about the code.

First, the REST support is added in via a mixin.

Second, we assume the controller itself represents the product collection resource and therefore define the get and post methods directly on it.

Third, other resources are demarcated by the resource keyword. Each resource can expose up to four methods - get, post, put and delete. However, in general only a subset of these methods makes sense for a given resource.

So how does this work under the hood? First, the resource keyword creates a Ruby module on the fly and includes it in the ProductController at compile time. As part of the merging process, the resource methods are renamed to include the resource name. Thus, the get method for the member resource becomes get_member. This is not ideal, but is needed to avoid name conflicts when multiple resources are included in a controller.

I also played with picking the correct resource at runtime and then merging the resource into an instance of the productsController using Ruby's extend mechanism. That turned out to be fairly difficult to accomplish since it interfered with the way that RAILS combines multiple modules into the Base class in ActiveController (in particular I had problems with the flash). I also wondered about the performance implications, but didn't do any testing to see if there really were any.

Now let me tell you what I don't like about the solution. The main issue is the method renaming. You have to know about it since you need to create templates called get.rhtml, get_memeber.rhtml, etc. It also comes into play if you want to turn on or off filters.

The second issue, umm....cough...., is that a pure REST solution does not work with HTML forms since browsers don't support PUT and DELETE. You can get around this by using XmlHttpRequest which doe support PUT and DELETE. If you need to use PUT and DELETE in forms, then sadly you'll have to add an action parameter to the URL like this - "action=put." For those of you are annoyed about this like I am, first write your favorite browser maker a letter and then let the W3C have a piece of your mind since amazingly enough the HTML specification does not even mention the use of PUT and DELETE on forms. Hopefully, the WHATWG forum will solve this issue in the future.

The last obvious issue you may have caught is look closely at the top level post method (or put). If the post fails we have to store the ill-formed product into the flash and redirect back to the editor since its at a different URL.

If you'd like to play with this you can download the plugin. Even better, also download the content negotiation plugin and you'll have a RESTified version of RAILS.

3 comments | no trackbacks

Stats

Posted by Charlie Sun, 26 Mar 2006 06:48:00 GMT

Perhaps this is a bit vain, but after my first week of blogging I was curious how many people had wandered through.

I’ve heard people say good things about awstats so I gave it try (although it is written in Perl of all things :) I now know the answer - 324. Thanks to all of you, that’s about 300 more than I thought! In case you’d like to check it out, I’ve added a link in the sidebar to the stats. They’ll update themselves every night around 3am.

Things that struck me:

  • The large gif at the top made up 50% of the download bandwidth. Sure enough, the image was a 66k gif. I’ve now slimmed it down to a 28k jpeg.
  • There are lots of bots out there - Google, Yahoo, MSN, Blogline, Feedster, etc.
  • There have been exactly 2 referrals from search engines. Although they’ve all crawled the site, it hasn’t made it into the indexes yet. I remember it took about 10 days for Yue’s blog to show up in the index.
  • The vast majority of referrals have come through Artima. But 18 have come through Peter’s blog - thanks Peter!

Last, thanks to Pat and his ruby script that automatically runs awstats.

no comments | no trackbacks

Content Negotiation And Rails

Posted by Charlie Fri, 24 Mar 2006 06:34:00 GMT

Let's get back to REST and Rails. One of the things Rails doesn't support is HTTP Content Negotiation. Why would you want this? Because different clients understand different content types. For example, you can serve XHTML to Firefox but only HTML to Internet Explorer. Or perhaps you are using AJAX, and want to send JavaScript back to the browser. Or maybe another computer system is talking to yours and it would like to have machine parseable XML thank you.

To accomplish this in Rails 1.0 means manually setting the content type. One place to do this is in a controller. But that doesn't seem right. Controllers help guide a client request from its inception to its fulfillment. A controller should have no knowledge about how the results are rendered. That's the realm of ActiveView. Now you are talking! But wait...by the time a view is invoked by Rails the content type has already been decided (that's not quite true, but you sure don't want to be in an RHTML template generating YAML).

So let's take a step back. We see that Rails has already started a custom - HTML files are in .rhtml templates, XML files in .rxml, JavaScript in .rjs. So let's stick with that. Thus, our goal is to implement code that picks the appropriate template based on a client's stated desires as indicated by the HTTP Accept header.

How do we such a thing? Well, first go read Joe Gregorios' excellent introduction to the subject and implementation in Python. Then download this Rails plugin that I wrote a few month ago that implements support for Mime Media Types in Ruby and integrates them into Rails.

The key bit of code is the negotiate_content method. It iterates, in order of importance as defined by the client, the different Mime Types supported by the client. For each Mime Type, it sees if there is a corresponding template with the right extension. Once it finds one, the template is loaded and the request is fulfilled:

# Copyright (c) 2006 MapBuzz
# Released under the MIT License
# Author: Charlie Savage

module ActionView
  class Base
    def pick_template_extension(template_path)#:nodoc:
      if match = delegate_template_exists?(template_path)
        match.first
      else
        negotiate_content(template_path)
      end
    end

    def negotiate_content(template_path)
      result = 'rhtml'
      
      if !controller.respond_to?(:request)
        return result 
      end
      
      negotiator = ContentNegotiator.new
          (controller.request.env['HTTP_ACCEPT'])

      # In order of preference, as specified by the client via
      # HTTP_ACCEPT, check to see if we can produce the media type 
      negotiator.media_types.each do |media_type|
        extension = media_type.extension
        if media_type.extension and
          template_exists?(template_path, extension)
          result = extension
          # Break out of this block
          break
        end
      end

      # If all else fails return rhtml
      result
    end
  end
end
    

One thing that is not immediately obvious is that this code will run every time a template or partial is run. We can use this our advantage. For example, let's say you want to serve HTML to Internet Explorer and XHTML to Firefox.

First, in your layout file do this:

<%= render(:partial => 'layouts/doctype') %>

Next, create one partial for Internet Explorer, called _doctype.rhtml:

<!DOCTYPE HTML PUBLIC 
 "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/html4/strict.dtd">
<html>
<% controller.response.headers["Content-Type"] = MediaType::HTML %>

And now one for Firefox called _doctype.rxhtml (make sure to get the extension right!):

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE html PUBLIC 
 "-//W3C//DTD XHTML 1.0 Strict//EN"
  "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en">
<% controller.response.headers['Content-Type'] = MediaType::XHTML %>

Notice that we are neatly able to reuse 99% of our templates for both browsers, yet at the same time include the <?xml ...?> header for Firefox but not Internet Explorer.

Last, its good to see that the Rails team has recently discovered the HTTP Accept header and is adding support in Rails 1.1. I haven't had a chance to look at their implementation yet since I'm not running Edge Rails. So take the following comment with a grain of salt. Its certainly a good thing for Rails to become more aware of MimeTypes, but as mentioned above, I don't think controllers are the right place to do this.

There is one caveat though. Mime Types are a blunt instrument (xml/text doesn't tell you much about a document) and therefore do not provide sufficient information in all cases. The most obvious example is trying to serve the same content type either via a standard browser request or an Ajax request. In the former case you generally want to render a layout and in the later case you do not. Of course, if you are able to make Ajax requests use a different content type (i.e., JavaScript) then there aren't any issues. Alternatively, you can always implement two different external apis to take care of this case.

Once I get my hands on Rails 1.1 I'll make sure to update this plugin if it is still useful.

Posted in , ,  | 1 comment | no trackbacks

Older posts: 1 ... 10 11 12 13