Cool Links Don't Change

Posted by Charlie Fri, 13 Jul 2007 01:58:00 GMT

Peter and Tim have been blogging about one of the least undestood parts of REST - how state is handled. Tim puts it nicely:

The essence of REST is to make the states of the protocol explicit and addressible by URIs.

Unfortunately, the official REST terminology obscures this fact by using the indecipherable phrase "Hypermedia as the Engine of Application State."

Whatever it is called, having states addressable by URIs is one of the best parts of the Web. When you do a search on Google, the state of your interaction with Google is returned in the web page of results you get back. That page contains many other links, each of which moves you to a new state. Thus its easy to jump from state-to-state, directly leading to the magic of the Web.

The same thing is true for automated systems as Peter explains. However, I disagree with part of what Peter writes:

With a truly REST based architecture you are free to change just about anything about the disposition and naming of resources in the system, except for a few well known start point URIs. You can change the URI shapes (for example, if you decide that you really hate the currently scheme). You can relocate resources to different servers (for example, if you need to partition you data). Best of all, you can do those things without asking any ones permission, because clients will not even notice the difference.

If I understand him correctly, Peter is arguing that as long as you provide and unchaning "start" link then you are free to change the embedded links in the result pages. I understand what Peter is getting at, as long as the top URI doesn't change, then clients can pick out the next set of URIs and proceed on their way.

However, I still think this is a bad idea. As Tim Berners-Lee famously wrote, Cool URIs don't change. A URI is a contract between you and the world - people expect to be able to bookmark it, write it down a napkin, tatoo it on their bodies, and come back many months later and have the URI work. In a scheme like Peter's, that breaks.

Now, for the work Peter is doing that might be ok - perhaps this system is only for internal consumption. And of course a world of never-changing URIs is nothing but an ideal. In the real world companies go out of business, processes are time-limited (like reserving an airline ticket), etc.

But I still think you should try your best to keep your URIs. If they have to change, then at least redirect the old ones to the new ones, or put a page up saying whatever you were doing is no longer valid.

Posted in ,  | no comments | no trackbacks

A Community for REST

Posted by Charlie Thu, 27 Jul 2006 20:51:00 GMT

One of the great ironies of the Web is that a huge number of technical people refuse to believe what is right in front of their eyes. The Web is the world's most diverse and powerful information system – one that has massively scaled from a single user at its inception to hundreds of millions today and billions tomorrow. REST is deceptively simple - its based on three fundamental design decisions:

  • Agreement on a global addressing scheme, URIs, to identify resources.
  • Agreement on a limited, generic application interface to manipulate resources. HTTP provides this generic interface via its “verbs," which include GET, POST, PUT, DELETE, etc.
  • Agreement on common formats, particularly HTML, to represent resources. Over time additional formats have been added, including JPEG, GIF, PNG, SVG, etc.

The Siren Call

Yet these lessons have been ignored time and time again. People just can't get past the siren call of building distributed systems using distributed objects. Let's count the failures:

  • DCOM
  • CORBA
  • XML RPC
  • SOAP
  • Web Services
  • Java RMI
  • JNI

And hundreds of home grown solutions that will never make it outside the Enterprise walls. Like I wrote last spring, every developer I've ever met who has written a distributed computing system always write their own RPC mechanism. I know, I've done it.

REST proponents have been struggling to get their message across for years. A the height of the Web Services myopia, REST was portrayed as an overly simplistic solution that most certainly was not adequate for building Enterprise applications.

Five years later the tide has turned - the evangelization of REST believers is starting to pay off. And they are about to be joined by a new champion - the Rails community.

A New Champion

It looks to me that the Rails community is about to become the first main-stream web development community to fully embrace REST.

Over the last year, the Rails community has been learning Web architecture best practices at light speed. Watching this has been quite refreshing - there is surprisingly little "not invented here syndrome." Some of the fundamental changes have included:

  • RPC style controllers to REST style controllers
  • Closely related, the move to CRUD interfaces
  • Changing deployment from Apache/cgi to Lighttpd/fastcgi to http load balancing with Apache/Mongrel
  • Some support for content negotiation to allow the reuse of controllers for creating multiple types of representations

As discussed above, its no surprise that Rails started down the wrong path on these architectural issues. What is a surprise is how quickly the Rails community has discovered its mistakes and starting fixing them. The ability to rapidly adopt is partially due to the community's youth, which frees it from the shackles of a backwards compatibility required by an installed base. But its also a characteristic of the community.

At the same time, Rails has been quickly grabbing mind share and market share. Adoption hasn't been driven by technical superiority - if David Heinemeier Hansson used Python I think we'd be talking about Python on Rails today. Instead, adoption has been driven by one part marketing, one part a unified community (Rails is the way to develop Web applications if you use Ruby) and one part luck. Web Services have almost run their course, J2EE is creaking under its own weight and .NET has not taken over the world. Now that the Web is hip again, there is a vacuum that is ripe to be filled by a new platform. PHP has clearly filled most of this void, but Rails has grabbed itself a niche that is growing quickly.

The combination of the Rails marketing juggernaut, its growing market share and the community's move toward REST all point to a new champion for the REST architectural style. It will be interesting to see if this final nudge pushes the REST style architecture over the tipping point so its finally appreciated by the majority of the technical community as opposed to a few enlightened souls.

Posted in ,  | 4 comments | no trackbacks

Updated Rails Plugins

Posted by Charlie Thu, 13 Apr 2006 19:18:00 GMT

Tom Fakes was kind enough to point out that the Rails Rest controller and content negotiation plugins I posted a couple of weeks ago can no longer be downloaded. They unfortunately were deleted when my server crashed last week.

So I've uploaded new versions of the plugins that work with Rails 1.1.1.

If you're interesetd in the Rest controller, I'd recommend looking at this one on RubyForge. It was written by Dan Kubb and then modified based on feedback from Peter and myself. This post to the microformats rest mailing list provides a bit of the technical background.

Posted in , , ,  | no comments | no trackbacks

Updated Rails Plugins

Posted by Charlie Thu, 13 Apr 2006 19:18:00 GMT

Tom Fakes was kind enough to point out that the Rails Rest controller and content negotiation plugins I posted a couple of weeks ago can no longer be downloaded. They unfortunately were deleted when my server crashed last week.

So I've uploaded new versions of the plugins that work with Rails 1.1.1.

If you're interesetd in the Rest controller, I'd recommend looking at this one on RubyForge. It was written by Dan Kubb and then modified based on feedback from Peter and myself. This post to the microformats rest mailing list provides a bit of the technical background.

Posted in , , ,  | no comments | no trackbacks

Rails and HTML and XHTML

Posted by Charlie Wed, 12 Apr 2006 08:47:00 GMT

Rails support for HTML and XHTML is less than exemplary. For some reason, the 1.1.1 release considers the application/xhtml+xml mime type to be a synonym for text/html. Umm....why?

And it gets stranger still. If you've ever looked at the HTML that Rails generates you'll see markup that looks like this:

<head>
  <link ... />
</head>
<input type="text" value="foo" name="bar" /><br />

Notice the extra "/>" characters? This usage comes from none other than the W3C which recommends it if you want older browsers (at this point only Internet Explorer) to be able to render your XHTML as if it was HTML. Whether this is a good idea or not is long-standing religious debate that's not worth rehashing.

What is worth asking though is why is Rails doing this? Right now it provides the worst of all worlds. First, it serves quasi XHTML to browsers as HTML. This is wrong for Firefox, Safari and Opera which all understand XHTML. And it serves XHTML to Internet Explorer which doesn't understand it. In addition, this quasi XHTML masquerading as HTML doesn't validate against the HTML 4.01 strict standard or the XHTML standard.

Wouldn't it be a whole lot easier to generate HTML 4.01 and be done with it? The alternative solution is to implement content negotiation so that XHTML is served to modern browsers and HTLM to Internet Explorer. However, before you can even take the first step in this direction you have to teach Rails that XHTML is in fact not HTML. Here is the code:

module ActionController
  module MimeResponds
    class Responder
      DEFAULT_BLOCKS = {
        :xhtml    => 'Proc.new { render }'
      }

      for mime_type in %w( xhtml )
        eval <<-EOT
          def #{mime_type}(&block)
            custom(Mime::#{mime_type.upcase}, &block)
          end
        EOT
      end
    end
  end
end

module Mime
  # Fix HTML
  #HTML  = Type.new "text/html", :html, %w( application/xhtml+xml )
  HTML  = Type.new "text/html", :html
  
  # Add XHTML
  XHTML  = Type.new "application/xhtml+xml", :xhtml
  
  # Fix xhtml lookup
  LOOKUP["application/xhtml+xml"] = XHTML
end

Once you've done that, you need to change the ActionView::TagHelper::so that it doesn't add in the "/>" characters. However, at this point you run into a problem. To figure out whether to close the tag or not requires knowing the response content type, which is on the response object. Unfortunately, the tag method is mixed into ActionView::Base as well as ActionView::Helpers::InstanceTag (and maybe others classes also). The problem is that getting the the response object is different in each case. It would be nice if there was a global place to get it from. More about this in a future post.

Posted in , , , , ,  | no comments | no trackbacks

Rails and HTML and XHTML

Posted by Charlie Wed, 12 Apr 2006 08:47:00 GMT

Rails support for HTML and XHTML is less than exemplary. For some reason, the 1.1.1 release considers the application/xhtml+xml mime type to be a synonym for text/html. Umm....why?

And it gets stranger still. If you've ever looked at the HTML that Rails generates you'll see markup that looks like this:

<head>
  <link ... />
</head>
<input type="text" value="foo" name="bar" /><br />

Notice the extra "/>" characters? This usage comes from none other than the W3C which recommends it if you want older browsers (at this point only Internet Explorer) to be able to render your XHTML as if it was HTML. Whether this is a good idea or not is long-standing religious debate that's not worth rehashing.

What is worth asking though is why is Rails doing this? Right now it provides the worst of all worlds. First, it serves quasi XHTML to browsers as HTML. This is wrong for Firefox, Safari and Opera which all understand XHTML. And it serves XHTML to Internet Explorer which doesn't understand it. In addition, this quasi XHTML masquerading as HTML doesn't validate against the HTML 4.01 strict standard or the XHTML standard.

Wouldn't it be a whole lot easier to generate HTML 4.01 and be done with it? The alternative solution is to implement content negotiation so that XHTML is served to modern browsers and HTLM to Internet Explorer. However, before you can even take the first step in this direction you have to teach Rails that XHTML is in fact not HTML. Here is the code:

module ActionController
  module MimeResponds
    class Responder
      DEFAULT_BLOCKS = {
        :xhtml    => 'Proc.new { render }'
      }

      for mime_type in %w( xhtml )
        eval <<-EOT
          def #{mime_type}(&block)
            custom(Mime::#{mime_type.upcase}, &block)
          end
        EOT
      end
    end
  end
end

module Mime
  # Fix HTML
  #HTML  = Type.new "text/html", :html, %w( application/xhtml+xml )
  HTML  = Type.new "text/html", :html
  
  # Add XHTML
  XHTML  = Type.new "application/xhtml+xml", :xhtml
  
  # Fix xhtml lookup
  LOOKUP["application/xhtml+xml"] = XHTML
end

Once you've done that, you need to change the ActionView::TagHelper::so that it doesn't add in the "/>" characters. However, at this point you run into a problem. To figure out whether to close the tag or not requires knowing the response content type, which is on the response object. Unfortunately, the tag method is mixed into ActionView::Base as well as ActionView::Helpers::InstanceTag (and maybe others classes also). The problem is that getting the the response object is different in each case. It would be nice if there was a global place to get it from. More about this in a future post.

Posted in , , , , ,  | no comments | no trackbacks

Restful Rails Revisited

Posted by Charlie Mon, 03 Apr 2006 05:51:00 GMT

Anyone interested in making Rails a bit more restful should check out the current thread on the Microformats Rest mailing archive.

A good place to start is message. You can also find more information on Peter’s blog.

Posted in , , ,  | no comments | no trackbacks

Restful Rails Revisited

Posted by Charlie Mon, 03 Apr 2006 05:51:00 GMT

Anyone interested in making Rails a bit more restful should check out the current thread on the Microformats Rest mailing archive.

A good place to start is message. You can also find more information on Peter’s blog.

Posted in , , ,  | no comments | no trackbacks

Content Negotiation And Rails

Posted by Charlie Fri, 24 Mar 2006 06:34:00 GMT

Let's get back to REST and Rails. One of the things Rails doesn't support is HTTP Content Negotiation. Why would you want this? Because different clients understand different content types. For example, you can serve XHTML to Firefox but only HTML to Internet Explorer. Or perhaps you are using AJAX, and want to send JavaScript back to the browser. Or maybe another computer system is talking to yours and it would like to have machine parseable XML thank you.

To accomplish this in Rails 1.0 means manually setting the content type. One place to do this is in a controller. But that doesn't seem right. Controllers help guide a client request from its inception to its fulfillment. A controller should have no knowledge about how the results are rendered. That's the realm of ActiveView. Now you are talking! But wait...by the time a view is invoked by Rails the content type has already been decided (that's not quite true, but you sure don't want to be in an RHTML template generating YAML).

So let's take a step back. We see that Rails has already started a custom - HTML files are in .rhtml templates, XML files in .rxml, JavaScript in .rjs. So let's stick with that. Thus, our goal is to implement code that picks the appropriate template based on a client's stated desires as indicated by the HTTP Accept header.

How do we such a thing? Well, first go read Joe Gregorios' excellent introduction to the subject and implementation in Python. Then download this Rails plugin that I wrote a few month ago that implements support for Mime Media Types in Ruby and integrates them into Rails.

The key bit of code is the negotiate_content method. It iterates, in order of importance as defined by the client, the different Mime Types supported by the client. For each Mime Type, it sees if there is a corresponding template with the right extension. Once it finds one, the template is loaded and the request is fulfilled:

# Copyright (c) 2006 MapBuzz
# Released under the MIT License
# Author: Charlie Savage

module ActionView
  class Base
    def pick_template_extension(template_path)#:nodoc:
      if match = delegate_template_exists?(template_path)
        match.first
      else
        negotiate_content(template_path)
      end
    end

    def negotiate_content(template_path)
      result = 'rhtml'
      
      if !controller.respond_to?(:request)
        return result 
      end
      
      negotiator = ContentNegotiator.new
          (controller.request.env['HTTP_ACCEPT'])

      # In order of preference, as specified by the client via
      # HTTP_ACCEPT, check to see if we can produce the media type 
      negotiator.media_types.each do |media_type|
        extension = media_type.extension
        if media_type.extension and
          template_exists?(template_path, extension)
          result = extension
          # Break out of this block
          break
        end
      end

      # If all else fails return rhtml
      result
    end
  end
end
    

One thing that is not immediately obvious is that this code will run every time a template or partial is run. We can use this our advantage. For example, let's say you want to serve HTML to Internet Explorer and XHTML to Firefox.

First, in your layout file do this:

<%= render(:partial => 'layouts/doctype') %>

Next, create one partial for Internet Explorer, called _doctype.rhtml:

<!DOCTYPE HTML PUBLIC 
 "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/html4/strict.dtd">
<html>
<% controller.response.headers["Content-Type"] = MediaType::HTML %>

And now one for Firefox called _doctype.rxhtml (make sure to get the extension right!):

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE html PUBLIC 
 "-//W3C//DTD XHTML 1.0 Strict//EN"
  "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en">
<% controller.response.headers['Content-Type'] = MediaType::XHTML %>

Notice that we are neatly able to reuse 99% of our templates for both browsers, yet at the same time include the <?xml ...?> header for Firefox but not Internet Explorer.

Last, its good to see that the Rails team has recently discovered the HTTP Accept header and is adding support in Rails 1.1. I haven't had a chance to look at their implementation yet since I'm not running Edge Rails. So take the following comment with a grain of salt. Its certainly a good thing for Rails to become more aware of MimeTypes, but as mentioned above, I don't think controllers are the right place to do this.

There is one caveat though. Mime Types are a blunt instrument (xml/text doesn't tell you much about a document) and therefore do not provide sufficient information in all cases. The most obvious example is trying to serve the same content type either via a standard browser request or an Ajax request. In the former case you generally want to render a layout and in the later case you do not. Of course, if you are able to make Ajax requests use a different content type (i.e., JavaScript) then there aren't any issues. Alternatively, you can always implement two different external apis to take care of this case.

Once I get my hands on Rails 1.1 I'll make sure to update this plugin if it is still useful.

Posted in , , , , ,  | 1 comment | no trackbacks

Content Negotiation And Rails

Posted by Charlie Fri, 24 Mar 2006 06:34:00 GMT

Let's get back to REST and Rails. One of the things Rails doesn't support is HTTP Content Negotiation. Why would you want this? Because different clients understand different content types. For example, you can serve XHTML to Firefox but only HTML to Internet Explorer. Or perhaps you are using AJAX, and want to send JavaScript back to the browser. Or maybe another computer system is talking to yours and it would like to have machine parseable XML thank you.

To accomplish this in Rails 1.0 means manually setting the content type. One place to do this is in a controller. But that doesn't seem right. Controllers help guide a client request from its inception to its fulfillment. A controller should have no knowledge about how the results are rendered. That's the realm of ActiveView. Now you are talking! But wait...by the time a view is invoked by Rails the content type has already been decided (that's not quite true, but you sure don't want to be in an RHTML template generating YAML).

So let's take a step back. We see that Rails has already started a custom - HTML files are in .rhtml templates, XML files in .rxml, JavaScript in .rjs. So let's stick with that. Thus, our goal is to implement code that picks the appropriate template based on a client's stated desires as indicated by the HTTP Accept header.

How do we such a thing? Well, first go read Joe Gregorios' excellent introduction to the subject and implementation in Python. Then download this Rails plugin that I wrote a few month ago that implements support for Mime Media Types in Ruby and integrates them into Rails.

The key bit of code is the negotiate_content method. It iterates, in order of importance as defined by the client, the different Mime Types supported by the client. For each Mime Type, it sees if there is a corresponding template with the right extension. Once it finds one, the template is loaded and the request is fulfilled:

# Copyright (c) 2006 MapBuzz
# Released under the MIT License
# Author: Charlie Savage

module ActionView
  class Base
    def pick_template_extension(template_path)#:nodoc:
      if match = delegate_template_exists?(template_path)
        match.first
      else
        negotiate_content(template_path)
      end
    end

    def negotiate_content(template_path)
      result = 'rhtml'
      
      if !controller.respond_to?(:request)
        return result 
      end
      
      negotiator = ContentNegotiator.new
          (controller.request.env['HTTP_ACCEPT'])

      # In order of preference, as specified by the client via
      # HTTP_ACCEPT, check to see if we can produce the media type 
      negotiator.media_types.each do |media_type|
        extension = media_type.extension
        if media_type.extension and
          template_exists?(template_path, extension)
          result = extension
          # Break out of this block
          break
        end
      end

      # If all else fails return rhtml
      result
    end
  end
end
    

One thing that is not immediately obvious is that this code will run every time a template or partial is run. We can use this our advantage. For example, let's say you want to serve HTML to Internet Explorer and XHTML to Firefox.

First, in your layout file do this:

<%= render(:partial => 'layouts/doctype') %>

Next, create one partial for Internet Explorer, called _doctype.rhtml:

<!DOCTYPE HTML PUBLIC 
 "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/html4/strict.dtd">
<html>
<% controller.response.headers["Content-Type"] = MediaType::HTML %>

And now one for Firefox called _doctype.rxhtml (make sure to get the extension right!):

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE html PUBLIC 
 "-//W3C//DTD XHTML 1.0 Strict//EN"
  "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en">
<% controller.response.headers['Content-Type'] = MediaType::XHTML %>

Notice that we are neatly able to reuse 99% of our templates for both browsers, yet at the same time include the <?xml ...?> header for Firefox but not Internet Explorer.

Last, its good to see that the Rails team has recently discovered the HTTP Accept header and is adding support in Rails 1.1. I haven't had a chance to look at their implementation yet since I'm not running Edge Rails. So take the following comment with a grain of salt. Its certainly a good thing for Rails to become more aware of MimeTypes, but as mentioned above, I don't think controllers are the right place to do this.

There is one caveat though. Mime Types are a blunt instrument (xml/text doesn't tell you much about a document) and therefore do not provide sufficient information in all cases. The most obvious example is trying to serve the same content type either via a standard browser request or an Ajax request. In the former case you generally want to render a layout and in the later case you do not. Of course, if you are able to make Ajax requests use a different content type (i.e., JavaScript) then there aren't any issues. Alternatively, you can always implement two different external apis to take care of this case.

Once I get my hands on Rails 1.1 I'll make sure to update this plugin if it is still useful.

Posted in , , , , ,  | 1 comment | no trackbacks