Posted by Charlie
Mon, 03 Apr 2006 05:51:00 GMT
Anyone interested in making Rails a bit more restful should check out the current thread on the Microformats Rest mailing archive.
A good place to start is message. You can also find more information on Peter’s blog.
Posted in Rails, REST | no comments | no trackbacks
Posted by Charlie
Fri, 31 Mar 2006 20:12:00 GMT
Just noticed that
Joe Hewitt has posted an updated version of
Firebug. He's taken a great extension and turned it into the best Firefox extension by far. Its an absolutely invaluable development tool. If you develop web apps, you should go
download it now.
Posted in Firefox, Tools | no comments | no trackbacks
Posted by Charlie
Fri, 31 Mar 2006 07:41:00 GMT
Let's revisit the
subject of content negotiation in Rails. Now that I've upgraded to Rails
1.1 I've had a chance to play with the new responds_to functionality. To
get some feedback, I also posted a comment to Jamis' blog and a message to
the REST microformat list.
My conclusion is that adding content negotiation to Rails would
be valuable. The new responds_to functionality is useful when
you want to override the default behavior. Having content negotiation
would be useful for making the default behavior more intelligent.
Templates already work in a similar manner. For example, say you write
this code:
class MyController < ApplicationController
def list
end
end
Rails will assume that you want to render a template called list.rhtml. However,
you can override this behavior as needed like this:
class MyController < ApplicationController
def list
render(:template => 'not_the_default_template')
end
end
Content negotiation would work in a similar manner. It would load the appropriate
template based on a client's request as specified in the HTTP accept header.You
can then override that behavior when needed using responds_to.
The way I propose this works is that
Content Types are mapped to templates via their file extensions. For instance:
| Content Type |
Template Name |
| text/html |
list.rhtml |
| application/xhtml+xml |
list.rxhtml |
| text/javascript, application/javascript |
list.rjs |
| application/atom+xml |
list.ratom |
| application/atom+rss |
list.rss |
| application/x-yaml |
list.ryaml |
One key thing to understand - the template extension does not dictate the
template type. Thus, list.rxhtml could be an ERB template or a Builder template
or any other sort of template.
Right now Rails doesn't support such a scheme because template extensions
are hard-coded to template types. This is done in ActionView::Base::create_template_source:
def create_template_source(extension, template, render_symbol, locals)
if template_requires_setup?(extension)
body = case extension.to_sym
when :rxml
"xml = Builder::XmlMarkup.new(:indent => 2)\n" +
"@controller.headers['Content-Type'] ||= 'application/xml'\n" +
template
when :rjs
"@controller.headers['Content-Type'] ||= 'text/javascript'\n" +
"update_page do |page|\n#{template}\nend"
end
else
body = ERB.new(template, nil, @@erb_trim_mode).src
end
@@template_args[render_symbol] ||= {}
locals_keys = @@template_args[render_symbol].keys | locals
@@template_args[render_symbol] = locals_keys.inject(
{}) { |h, k| h[k] = true; h }
locals_code= ""
locals_keys.each do |key|
locals_code << "#{key} = local_assigns[:#{key}]
if local_assigns.has_key?(:#{key})\n"
end
"def #{render_symbol}(local_assigns)\n#{locals_code}#{body}\nend"
end
It would be easy to work around this code since the template parameter includes
the full source of the template. With a little regular expression magic,
you could figure out what type of template has been loaded and then run the
appropriate code.
However - what if we did something more clever? What if every template started
with an comment that specified its type - similar to a Unix shebang or
XML processing instruction.
Maybe something like this:
For ERB:
<!-- template_type :erb -->
For RJS:
# template_type :rjs
For Builder:
<!-- template_type :erb -->
It would then be easy to extract out the template type when it was loaded. If
the template comment did not exist (i.e., for all existing templates) then you
could revert to regular expression magic.
This solution has one very nice property - it would make it really easy to add
additional template types to Rails (not that its that hard now).
To recap, the idea is:
- Implement default content negotiation by mapping content types to template names
via file extensions (.rhtml, .rxthml, .rjs., .ratom, etc.)
- Add code to Rails so that it determine a template's type when the template is
loaded
- Use responds_to to override the default behavior as needed.
What do people think of this idea? If there is interest, I'll update my content
negotiation plugin and add this functionality to Rails 1.1.
Posted in HTTP, Rails | 8 comments | no trackbacks
Posted by Charlie
Fri, 31 Mar 2006 06:21:00 GMT
Imagine the following situation:
- You have an HTML sign up form for a user.
- One of the fields is for the user's email.
- The user fills in the data and POSTs the response to the server.
- The server rejects the request, perhaps the email address is already taken.
Is there an appropriate HTTP status code for this? It seems to me this falls into the 4xx range since the client provided invalid data. Its not a 400 though since the server understood the request. And none of the other 4xx codes seem to fit.
I ran into a similar situation today, except that I'm doing it via an XmlHttpRequest. When posting some data to the server, the request always returns HTTP status code 200 whether or not data was actually inserted into the backend database. Thus, the server has to return back a response that tells the client what happened - probably via a little bit of JavaScript. But it seems like using an HTTP status code would be a much cleaner solution.
Any ideas? Or am I trying to misuse HTTP status codes?
Update - I've happily found my answer. A similar problem was
posted by Jan Algermissen on the Yahoo Rest discussion list. Bill de hora
answered use a new HTTP status code, 418. And in
response to that, Roy Fielding, one of the authors of HTTP specification, asked if he should standardize it. Yes please.
Now, let's see what happens when I try this trick with the browsers.
5 comments | no trackbacks
Posted by Charlie
Thu, 30 Mar 2006 07:14:00 GMT
Being a good web netizen, every few days I check that my blog and Yue's
blog validate using the W3C feed validator (well someone has to serve valid
content - of course having said that I'd better go check again!).
I noticed that Typo, my blogging
software, was serving RSS and ATOM as text/html - which really isn't correct.
So I submitted a bug, and a really simple patch, and didn't think much
about it.
Today I received an email that the patch has been applied by Piers. Thanks
Piers - and I have to say I'm impressed by the fast turnaround!
After using Typo for the last week, I'm pretty happy with it. Its quite
feature rich, particularly considering its young age. To be fair, its not
as polished as WordPress,
which is what Yue uses. The biggest difference is in editing. WordPress
2.0+ has a very nice WSYWIG rich text editor that's simple to use. In contrast,
with Typo you need to know Markdown, Textile or good ole (X)HTML. For technically
inclined users that's no problem, probably even better. But for users who
want to just post content without knowing the gory technical details, its a showstopper. It would also be nice to have a spell checker.
Besides those two things, I can
highly recommend Typo. Of course, you have to be willing to roll up your sleeves and install
your own blogging software (the same is also true for WordPress).
2 comments | no trackbacks
Posted by Charlie
Wed, 29 Mar 2006 18:39:00 GMT
Well it took 10 days but both Google and MSN have added me to their indices. Sadly, Yahoo still doesn’t acknowledge my existence :)
Its interesting watching the search engines work from the outside. I watch the spiders come by every day at about the same frequency. I see that MSN and Google both added me to their indicess on the same day. And I see more or less the same search results from the engines.
Now if only that other Charlie Savage didn’t exist I’d be all set with the search engines. Hopefully, he won’t decide to start a blog.
no comments | no trackbacks
Posted by Charlie
Wed, 29 Mar 2006 17:27:00 GMT
Every artistic endeavor strives to capture some slice of the world around
us. You might be trying to create as realistic a portrayal as possible,
or maybe you're after a highly abstract portrayal that tries to evoke some
emotion.
In the world of software (an artistic endeavor if I've ever seen one), we
create highly-tailored portrayals that are suitable only for the purpose
they were constructed.1 Take
two or more programs that are in the same problem domain - human resource
software, databases, blogging software, etc. None of them will be compatible
because they represent a single designer's (or maybe a couple)
view of a problem domain. And every one has a different view of the problem
to be solved and how to solve it.
Take a simple term - say a street. What
it is it? Have you ever been on a street that ends, and then starts up
in a few blocks? Is that the same street? How about a street whose name
changes - is it still the same street? Does a street include boulevards,
highways, freeways? Does a street cross city, county, state boundaries?
Do you and I live on the same street, although I live in the United States
and you live in Canada?
Most of us just want to bury our heads in the sand when faced with such
questions - thinking about them is painful because there is no right answer
and you're immediately led to deep philosophical questions about how we
perceive the world around us. And most books and papers on the subject
are highly technical, and of very little use to people who need to create
software by yesterday.
Fortunately, there is a fantastic book on the subject, called Data
and Reality, that is deeply grounded in how these issues affect
software design. Written by William Kent, it addresses issues
that come up every day in design. What is the difference between an identifier
and a name? What is a type or category? What is the difference between
an attribute and a relationship? For that matter, what's the difference
between types, attributes and relationships? What are the strengths and
weaknesses of the hierarchical, network and relational models? Don't
expect any easy answers - they aren't any.
Don't let the publication date, 1978, fool you. The questions the book
raises are as valid today as then. In fact, I'd bet they'll be just
as valid one thousand years from now. Luckily the book was reprinted in
2000, so go pick up your copy now before it goes out of print again.
Let me leave you with one of my favorite passages from the book (page
10, 2000 paperback edition):
We seem to have little difficulty with the concept of "one person"
despite changes in appearance, personality, capabilities, and,
above all, chemical composition...When we speak of the same person over
a period of time, we certainly are not referring to the same ensemble
of atoms and molecules. What the is the "same person"? We can only appeal
to some vague intuition about the "continuity" of - something -
through gradual change. The concept of the "same person" is so familiar
and obvious that it is absolutely irritating not to be able to define
it. Definitions in terms of "soul" and "spirit" may be the only true
and humanistic concepts, but significantly, we don't know how to deal
with them in a computer-based information system. It is only when the
notion of "person" is pushed to some limit do we realize how imprecise
the notion is.
I can't help but think of software
products as Mount Everests on a
fitness
landscape that are infinitely thin. Veer just slightly off their intended
purpose and everything comes crashing down.
Read more...
no comments | no trackbacks
Posted by Charlie
Sun, 26 Mar 2006 09:01:00 GMT
Time to continue the REST
and Rails theme.
Rails' most egregious violation of REST is that it exposes object methods
in URIs:
http://myrailsapp.com/users/show
http://myrailsapp.com/users/list
http://myrailsapp.com/users/update
Remember, one of the fundamental ideas of REST is to manipulate resources
(represented by URIs) via the use of a limited set of verbs. For our purposes,
the verbs that matter are HTTP's GET, POST, PUT and DELETE.
Rails does not do this, instead it exposes public object methods on the
Internet. This is no different than what DCOM, CORBA, RMI, XML-RPC, SOAP
and your own home-grown RPC (remote procedure call) system do. And yes,
every good developer has written an RPC system or two. In fact, I think
that's the main problem. Every developer I've ever met who
writes a distributed computing system always implements some sort
of RPC mechanism. I know I did it the first time.
Its seems perfectly logical - let's hide away the nasty details of the
network so the developer doesn't have to worry about it. Of course it doesn't work,
but that doesn't mean that 9 out of 10 developers will tell you it does.
The good news of course is that RAILS is built on top of the Web, which
is one of the most successful distributed computing system created and
is not an
RPC system. In addition, RAIL's RPC implementation is
much simpler to use than the ones I mentioned above and it
strongly encourages you to create nice looking
URIs which is a significant benefit.
So at this point, unless you're a RESTafarian, you're probably asking
yourself "so what?" since RAILS is a very productive framework. We'll let's
go through what would have to change to see if it has any interest to you.
Let's start with the fundamental problem - Rails
equates each URI with a single action as opposed to a single resource that
can be manipulated via HTTP verbs. For example, let's say we have a products
table with an id field and a name field. Then let's run the rails
scaffold generator. If you look at the generated code, you'll see its creates
a number of public methods that manipulate several resources. More concretely,
the following table shows how the generated methods map to resources
and HTTP verbs.
Mapping RAILS to REST
| Resource |
GET |
POST |
PUT |
DELETE |
| products |
list |
create |
|
|
| each product |
show |
|
update |
destroy |
| creator |
new |
|
|
|
| editor |
edit |
|
|
|
Thus, I've identified four types of resources:
- The resource that represents the collection
of all products
- Any number of resources that identify individual products
- A creator resource
- An editor resource
The creator and editor resources might be confusing. An analogy I've heard
before that helps me is to imagine your resource is a Microsoft Word document.
Does that mean that Microsoft Word itself is part of your resource? No,
that doesn't make sense. Instead, Microsoft Word is a resource that you
can use to manipulate your document resource. Thus, the creator
and editor resources provide a representation (i.e., an HTML document)
that allows you to create new product resources or edit existing product
resources.
So how would these map to URIs. Well, that's pretty easy:
http://myrailsapp/product
http://myrailsapp/product/member/:id
http://myrailsapp/product/creator
http://myrailsapp/product/editor/:id
Instead of the standard RAILS route of :controller/:action/:id we are
now doing :controller/:resource/:id. In addition, sInce
http://myrailsapp/product/member/:id is
so common, I tend to shorten it using routes to:
http://myrailsapp/product/:id
Still with me? If so, then is it possible to fit these ideas into RAILS
which did not start with REST in mind? Well, I know of at least two previous attempts,
both which provided inspiration but in the end aren't satisfactory. I think
this third attempt falls into the same camp, but hopefully will keep
the discussion going and throw another idea or two into the pot.
So here's my take - feel free to download this plugin if
you'd like experiment with live code:
class ProductController < ApplicationController
include RestController
def get
@product_pages, @products = paginate :products, :per_page => 10
end
def post
@product = Product.new(params[:product])
begin
@product.save!
flash[:notice] = 'Product was successfully created.'
redirect_to :resource => :collection
rescue => e
flash[:product] = @product
redirect_to :resource => :creator
end
end
resource :Member do
def get
@product = Product.find(params[:id])
end
def put
@product = Product.find(params[:id])
begin
@product.update_attributes(params[:product])
flash[:notice] = 'Product was successfully updated.'
redirect_to :id => @product
rescue => e
flash[:product] = @product
redirect_to :resource => :editor, :id => @product
end
end
def delete
Product.find(params[:id]).destroy
redirect_to :id => nil, :resource => nil
end
end
resource :Creator do
def get
@product = flash[:product] || Product.new
end
end
resource :Editor do
def get
@product = flash[:product] || Product.find(params[:id])
end
end
end
Some things to notice about the code.
First, the REST support is added in via a mixin.
Second, we assume the
controller itself represents the product collection resource and therefore
define the get and post methods directly on it.
Third, other resources are demarcated by the resource keyword.
Each resource can expose up to four methods - get, post, put and delete.
However, in general only a subset of these methods makes sense for a given
resource.
So how does this work under the hood? First, the resource keyword creates
a Ruby module on the fly and includes it in the ProductController
at compile time. As part of the merging process, the resource methods are
renamed to include the resource name. Thus, the get method for
the member resource becomes get_member. This is not ideal, but is needed
to avoid name conflicts when multiple resources are included in a controller.
I also played with picking the correct resource at runtime and then merging
the resource into an instance of the productsController using Ruby's extend
mechanism. That turned out to be fairly difficult to accomplish since
it interfered with the way that RAILS combines multiple modules into the
Base class in ActiveController (in particular I had problems with the flash).
I also wondered about the performance implications, but didn't do any testing
to see if there really were any.
Now let me tell you what I don't like about the solution. The
main issue is the method renaming. You have to know about it since you
need to create templates called get.rhtml, get_memeber.rhtml, etc. It also
comes into play if you want to turn on or off filters.
The second issue, umm....cough...., is that a pure REST solution does
not work with HTML forms since browsers don't support PUT and DELETE.
You can get around this by using XmlHttpRequest which doe support PUT
and DELETE. If you need to use PUT and DELETE in forms, then sadly you'll
have to add an action parameter to the URL like this - "action=put." For
those of you are annoyed about this like I am, first write your favorite
browser maker a letter and then let the W3C have
a piece of your mind since amazingly enough the HTML specification does
not even mention the use of PUT and DELETE on forms. Hopefully, the WHATWG forum
will solve this
issue in the future.
The last obvious issue you may have caught is look
closely at the top level post method (or put). If the post fails we have
to store the ill-formed product into the flash and redirect back to the
editor since its at a different URL.
If you'd like to play with this you can download the plugin. Even better,
also download the content negotiation plugin and you'll have a RESTified
version of RAILS.
3 comments | no trackbacks
Posted by Charlie
Sun, 26 Mar 2006 06:48:00 GMT
Perhaps this is a bit vain, but after my first week of blogging I was curious how many people had wandered through.
I’ve heard people say good things about awstats so I gave it try (although it is written in Perl of all things :) I now know the answer - 324. Thanks to all of you, that’s about 300 more than I thought! In case you’d like to check it out, I’ve added a link in the sidebar to the stats. They’ll update themselves every night around 3am.
Things that struck me:
- The large gif at the top made up 50% of the download bandwidth. Sure enough, the image was a 66k gif. I’ve now slimmed it down to a 28k jpeg.
- There are lots of bots out there - Google, Yahoo, MSN, Blogline, Feedster, etc.
- There have been exactly 2 referrals from search engines. Although they’ve all crawled the site, it hasn’t made it into the indexes yet. I remember it took about 10 days for Yue’s blog to show up in the index.
- The vast majority of referrals have come through Artima. But 18 have come through Peter’s blog - thanks Peter!
Last, thanks to Pat and his ruby script that automatically runs awstats.
no comments | no trackbacks
Posted by Charlie
Fri, 24 Mar 2006 06:34:00 GMT
Let's get back to REST
and Rails. One of the things Rails doesn't support is HTTP
Content Negotiation. Why would you want this? Because different clients understand
different content types. For example, you can serve XHTML to Firefox but
only HTML to Internet Explorer. Or perhaps you are using AJAX, and want
to send JavaScript back to the browser. Or maybe another computer system
is talking to yours and it would like to have machine parseable XML thank
you.
To accomplish this in Rails 1.0 means manually setting the content type.
One place to do this is in a controller. But that doesn't seem right. Controllers
help guide a client request from its inception to its fulfillment. A controller
should have no knowledge about how the results are rendered. That's the
realm of ActiveView. Now you are talking! But wait...by the time a view
is invoked by Rails the content type has already been decided (that's not
quite true, but you sure don't want to be in an RHTML template generating
YAML).
So let's take a step back. We see that Rails has already started a custom
- HTML files are in .rhtml templates, XML files in .rxml, JavaScript in
.rjs. So let's stick with that. Thus, our goal is to implement code that
picks the appropriate template based on a client's stated desires as indicated
by the HTTP Accept header.
How do we such a thing? Well, first go read Joe Gregorios' excellent introduction to
the subject and implementation in Python. Then download this Rails plugin
that I wrote a few month ago that implements support for Mime Media
Types in Ruby and integrates them into Rails.
The key bit of code is the negotiate_content method. It iterates, in order
of importance as defined by the client, the different Mime Types supported
by the client. For each Mime Type, it sees if there is a corresponding
template with the right extension. Once it finds one, the template is loaded
and the request is fulfilled:
module ActionView
class Base
def pick_template_extension(template_path)
if match = delegate_template_exists?(template_path)
match.first
else
negotiate_content(template_path)
end
end
def negotiate_content(template_path)
result = 'rhtml'
if !controller.respond_to?(:request)
return result
end
negotiator = ContentNegotiator.new
(controller.request.env['HTTP_ACCEPT'])
negotiator.media_types.each do |media_type|
extension = media_type.extension
if media_type.extension and
template_exists?(template_path, extension)
result = extension
break
end
end
result
end
end
end
One thing that is not immediately obvious is that this code will run every
time a template or partial is run. We can use this our advantage.
For example, let's say you want to serve HTML to Internet Explorer and
XHTML to Firefox.
First, in your layout file do this:
<%= render(:partial => 'layouts/doctype') %>
Next, create one partial for Internet Explorer, called _doctype.rhtml:
<!DOCTYPE HTML PUBLIC
"-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/html4/strict.dtd">
<html>
<% controller.response.headers["Content-Type"] = MediaType::HTML %>
And now one for Firefox called _doctype.rxhtml (make sure to get the extension right!):
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE html PUBLIC
"-//W3C//DTD XHTML 1.0 Strict//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en">
<% controller.response.headers['Content-Type'] = MediaType::XHTML %>
Notice that we are neatly able to reuse 99% of our templates for both browsers,
yet at the same time include the <?xml
...?> header
for Firefox but not Internet Explorer.
Last, its good to see that the Rails team has recently discovered the
HTTP Accept header and is adding support in Rails 1.1. I haven't had a chance to
look at their implementation yet since I'm not running
Edge Rails. So take the following comment with a grain of salt. Its certainly
a good thing for Rails to become more aware of MimeTypes, but as mentioned above,
I don't think controllers are the right place to do this.
There is one caveat though. Mime
Types are a blunt instrument (xml/text doesn't tell you much about a document)
and therefore do not provide sufficient information in all cases. The most obvious
example is trying to serve the same content type either via a standard browser
request or an Ajax request. In the former case you generally want to render
a layout and in the later case you do not. Of course, if you are able to make Ajax
requests use a different content type (i.e., JavaScript) then there aren't
any issues. Alternatively, you can always implement two different external apis
to take care of this case.
Once I get my hands on Rails 1.1 I'll make sure to update this plugin if it is
still useful.
Posted in Rails, REST, Ruby | 1 comment | no trackbacks