Posted by Charlie Fri, 28 Sep 2007 21:46:00 GMT
Foss4g 2007 witnessed the birth of a new Google Group - GeoRuby. Like I talked about last week, when we started MapBuzz the Ruby and Mapping community could be counted on one hand. Now we're up to at least two! And undoubtedly there are many more of you hiding in the woodwork. So come join the group and let's see if we can create a new Ruby mapping community.
Posted by Charlie Fri, 21 Sep 2007 16:29:00 GMT
Wilhelm Chung had a nice followup to my post last week about Rails Unusual Architecture. He provides more details about how alias_method_chain works and even includes a few pictures (it would be nice to see a few lines connecting the methods though).
Posted by Charlie Wed, 19 Sep 2007 20:12:00 GMT
Back when I started MapBuzz, there was so little work being done in the open source geospatial world with Ruby and/or Rails that Sean even commented on it. Geospatial was the realm of Python and C/C++ on one side, and a completely duplicated Java software stack on the other (an interesting dichotomy that's worth a blog about at some point).
Although Python/Java still remain dominant in the geospatial world, there has been an uptick of interest in the Ruby and Rails communities. So let's take a look what's new.
We'll start with Shoaib Burq's RailsConf Europe 2007 talk Rails GIS Hacks. Although I wasn't at the conference, Shoaib asked me to review his materials a couple months ago. Unfortunately I didn't have much time to provide feedback - but he really didn't need it. His materials are absolutely excellent and are now online - so check them out.
Earlier this year, Apress released a book about using Google Maps with Rails. I haven't read the book, so can't comment on it. But one of the authors, Andre, has an active blog Earthcode that frequently talks about Rails and mapping. Other blogs of interest are The Pochi Superstar Mega Show! by Guilhem Vellut, High Earth Orbit by Andrew Turner and GeoLabs by Nomad Labs.
On the software side, various Rails plugins have appeared over the last years that make it easy to embed a google map (such as ym4r, cartographer) in a Rails application. There are also a slew of geocoding gems, and Ruby bindings for Gdal and Geos (provided by yours truly).
And as far as sites, there is of course my favorite MapBuzz. In addition, OpenStreetMap has moved to Rails and I would guess Andrew's site Mapufacture is also using Rails.
So although community is still quite small, it sure is a lot bigger than last year. And if I missed your totally excellent Ruby Mapping library/blog/site let me know and I'll add you to the list.
Posted by Charlie Mon, 17 Sep 2007 05:14:00 GMT
Let's continue our on-again, off-again series about making Rails better. We've talked about fixing ActiveRecord's awkward handling of attributes, implementing RESTful error handling and using Content Negotiation with Rails. Today let's dive back into ActionView.
This won't make me many friends, but there isn't a nice way to say this - the internals of ActionView are ugly. The fundamental problem is that ActionView lacks the right abstraction - in fact, it has no abstraction at all. Without a guiding design, ActionView descends into a mass of tangled, hard-to-pick apart code that lacks cohesion and has high coupling.
What's bizarre about this state of affairs is that the abstraction is easy to see.
What does ActionView do? It renders templates. Do templates changes - you bet! Rails has three built-in template types (erb, builder, rjs) and there are a myriad of 3rd party templates.
So quick - what's the abstraction? Doh - an AbstractTemplate class of course. Its API would be something simple like this (note there might be one or two methods missing, I'd have to whip up a prototype to be sure):
class AbstractTemplate def render(response, options = {}, assigns = {}) end def template_exists(template_path, extension = nil) end end
Implementers could then inherit from the AbstractTemplate class when creating their own custom templates types and override the base functionality as needed.
Sadly, Rails doesn't do this. Instead we are faced with this ugliness in ActionController::Base:
module ActionController class Base def render_text(text = nil, status = nil, append_response = false) end def render_javascript(javascript, status = nil, append_response = true) end def render_xml(xml, status = nil) end def render_json(json, callback = nil, status = nil) end def render_nothing(status = nil) end end end
Followed by this ugliness in ActionView::Base:
module ActionView class Base def erb_template_exists?(template_path) end def builder_template_exists?(template_path) end def javascript_template_exists?(template_path) end end end
Just looking at the method names should set off alarm bells.
First, the division of labor between ActionController and ActionView is murky - resulting in tight coupling between them and lousy cohesion internally in both classes.
Second, its clear that Rails wasn't designed to support multiple template types and they have been bolted on as ugly warts.
As I mentioned above, there are plethora of 3rd party template types. So how does ActionView supports support them? By punting - you register your template type and define a render method - then you are on your own.
Thus there are two separate code paths in ActionView for built-in templates and custom templates. But what's the difference between a built-in template and a custom template? Nothing of course.
So why are there two code paths? Why can't a custom template type reuse the applicable parts of ActionView (such as checking to see if a template exists on disk, caching a template, etc.).
Introducing an AbstractTemplate class to ActionView would work wonders. It would make ActionView more extensible, simplify its interface, improve cohesion, reduce coupling and make the code a whole lot easier to follow and understand. So here is hoping the Rails core reworks ActionView for Rails 2.0 and expunges the ugliness that exists in it today.
Posted by Charlie Mon, 10 Sep 2007 20:03:00 GMT
Its been over a year since I blogged about GEOS, an open source project that provides rich functionality for analyzing and manipulating geometries and is a key part of PostGIS.
Sandros and Mateusz spent a ton of time refactoring and improving GEOS last year, and it looks like version 3.0 will finally be released this fall. As part of the 3.0 release, I've completely rebuilt the Ruby and Python bindings to use GEOS's C API. In theory this should decouple the bindings somewhat from GEOS releases - we'll see if that works out in practice.
Over the last couple of weeks, Mark Cave Ayland and I have spent time polishing off some rough edges. We managed to:
In addition, I added VC++ project files for the Python and Ruby bindings and updated the Ruby bindings to be more "Ruby" like.
Long story short - building GEOS and the Ruby/Python bindings should now be easier and more fool proof. If you're in the GNU world, its as simple as:
./configure --enable-python --enable-ruby
If you're in the Microsoft world, just open the VC++ solution in the source tree (thanks Mateusz), tweak the locations of your Ruby and Python installations, and hit the compile button.
So if you are using, or planning to use, the bindings I'd recommend grabbing the latest from SVN. Once you've built everything, then take a look at the test scripts I've written in the swig/ruby/test and swig/python/test directories. They should provide enough examples to get you started.
Feedback is of course welcome, particularly from Python users. Since I use Ruby day in and day out, I know the right Rubyisms to put into the bindings. But I don't know Python well enough to know the right Pythonisms.
Posted by Charlie Fri, 07 Sep 2007 16:00:00 GMT
Ealier this week, Yue was interviewed by Christine Lu on The China Business Show. Christine does a great job of interviewing people who are experts on doing business in China, and thus the show is an invaluable source of insider perspectives, secrets and tips.
Yue talked about her company, Skyetek, and the challenges of selling technology in Asia, particurarly China. If you'd like to listen in, the interview is now online.
Posted by Charlie Thu, 06 Sep 2007 20:33:00 GMT
As I've mentioned a number of times, I'm a big fan of Atom and the Atom publishing protocol and believe they will become the lingua franca of Web Services, including spatial web services.
I've found fellow believers in the Geo Web Rest group and it seemed like high time to put theory into practice. So yesterday we had our first interoperability day. Sean, Andrew, Chris, Pat and I (did I miss anyone?) spent most of the day making sure our various clients and servers could talk to each other. At the end of the day we had MapBuzz talking to Mapufacture talking to Hammock talking to OpenLayers. Nice.
We'll be publicly showing our work at Foss4G where Sean, Chris and I are sharing a Tuesday afternoon track. I'd like to have another interoperability day before then, so check out the Geo-Web-Rest page and come join in on the fun.
Posted by Charlie Wed, 05 Sep 2007 06:05:00 GMT
There has been surprisingly little written about how Rails is implemented. If you Google for "architecture Rails" you'll find plenty of articles that describe Rails as a web framework based on the classic Model-View-Controller design pattern. And you'll find plenty of links about Rail's constituent parts - ActiveRecord, ActionController, ActionView, ActiveSupport, etc. But you'll find precious little about how Rails is put together.
Which is shame, since Rails is constructed in a very unusual manner. So let's dig into Rails and see what makes it unusual.
Let's start by considering what a web framework has to do. If you were to build your own framework, the absolute minimum requirements are:
But such a bare bones framework would leave much to be desired. To be truly useful, you would also have to address a number of additional issues, including:
So how can you meld all this functionality into a cohesive system that is easy to use and easy to extend?
The standard answer is use the Model-View-Controller pattern. But that still leaves many questions to be answered - particularly on the controller side. How exactly do you implement the controller piece?
A common approach is to use the Decorator pattern made famous by the Gang of Four. The basic idea is to create a chain of decorators as shown in the diagram below.
Each decorator deals with one issue. So you'd have one decorator for unmarshalling, one for authentication, one for session management, etc.
The Smallworld Internet Application Server uses this architecture. Apache Axis and Apache Cocoon use this architecture. And although I've never used Django, its middleware classes require a common API which is a telltale sign that they are really decorators in disguise.
So why is the Decorator pattern so popular? Because it let's you modularize your framework, making it possible to add or remove functionality as needed based on a particular application's requirements.
Another approach, albeit much rarer, is to use Aspect oriented programming, or AOP, to combine multiple behaviors. Like many computing technologies in use today, aspect oriented programming was invented at Xerox Parc. But it found its home in the Java world via the AspectJ project, which is now part of Eclipse.
There are plenty of articles that explain aspect oriented programming in-depth so I won't rehash them. But the basic idea is to use a domain specific language (DSL) to combine different pieces of functionality without touching the original code.
The classic example is logging. Adding logging statements to each method is tedious and error prone. Even worse, if your logging API changes you have to go back and touch every part of your system. Thus logging is an example of a cross-cutting concern, something that affects every part of a system.
With a little thought, its easy to see why aspect oriented programming could make it easier to write a web framework. And its appears there is such a thing - the Spring Framework. Although I've never used it, the documentation makes it look like it relies heavily on aspect oriented programming.
So how about Rails? Rails is unusual because it uses Ruby's limited aspect oriented support to mimic the Decorator pattern.
Ruby supports aspect oriented programming? Well not really. But it does offer a very crude approximation via its alias method and support of modules. Rails leverages this basic support for all its worth - it forms the very heart and soul of Rails.
To see what I mean let's look at an example. Here is the basic skeleton of Rails session handing functionality, found in actionpack\lib\action_controller:
module ActionController #:nodoc: module SessionManagement #:nodoc: def self.included(base) base.extend(ClassMethods) base.send :alias_method_chain, :process, :session_management_support base.send :alias_method_chain, :process_cleanup, :session_management_support end module ClassMethods # Class methods go here end # Instance methods go here end end
The important things to notice are:
Rails then loads this code in actionpack-1.13.3\lib\action_controller.rb like this:
require 'action_controller/session_management' ActionController::Base.class_eval do include ActionController::SessionManagement end
The end result is that ActionController::Base now has the following method chain:
process -> process_with_session_management_support -> process_without_session_management_support
Voila. Rails has created a Decorator using Ruby's limited aspect oriented programming support.
If you look through the Rails source code you'll see this pattern repeated over and over.
Rails implements the Decorator in such an unusual way that its worth asking if it is a good idea or not. In my opinion it isn't. I don't see any advantages in Rails' implementation - do you?
But I do see several disadvantages. The biggest one is that its removes an obvious extension point from Rails. Once two methods are linked together using Ruby's alias method you can't break them apart. For example, if A aliases B then you can't insert C between them. This might not seem like a big deal, but for some extensions it quite important. For example, when I wrote our REST controller implementation last year I struggled mightily to fit it into Rails aliases method calls and finally gave up and took a different approach. Its worth noting that its often possible to work around this problem since Ruby is such a flexible language. Instead of adding a new decorator, in many cases you can replace the original method to achieve what you want. Its not nearly as elegant, but it usually does the trick.
A second disadvantage is that it complicates Rails architecture. What's the difference between a decorator and a filter? Nothing. By using the Decorator pattern you could merge Rail's filter functionality with its method chaining functionality, thereby reducing its overall code base.
A third disadvantage, albeit a minor one, is that the method chains Rails creates are hidden. You can't look at a single piece of code, or configuration file, and see the method chains. The closest is action_controller.rb and action_view.rb, but I can't get a handle on a chain object at runtime and say "please print yourself on STDOUT so I can see what is happening."
The fourth, and last, disadvantage is much more subjective and personal. I think Rails is trying to be too cute for its own good. When faced with a standard problem use a standard solution - that is the whole point of design patterns. By trying to be too cute Rails provides a solution that is less flexible and less obvious than the standard approach.
Update - Peter rightly pointed out in the comments that I mistakenly used the Chain of Responsibility Pattern instead of the Decorator pattern. There is a subtle, yet extremely important, difference between the two. In the Chain of Responsibility pattern the first handler that can process a request will do so and the rest of the chain will not be executed. In the Decorator pattern every decorator in the chain will process a request.
I've decided to update this entry since I think accuracy is more important the maintaining the original version. Thanks for your feedback Peter!
Posted by Charlie Sun, 26 Aug 2007 06:13:00 GMT
How neat, Matz linked to my article about fixing architecture flaws in ActiveRecord.
Of course I was curious to see what he said, so I ran his comments through online translators from Babelfish, Google and Dictionary.com. Based on past experiences, I wasn't expecting much.
Updated on Sept. 30, 2007. Sam was kind enough to send me his real translation - so take a look at this first:
It says that ActiveRecord's faults are with the way it handles text-based column types and the fact that serialized objects get mixed with text from the database in a hash table.
I'm not in a position to make decisions but pointing out such flaws and making suggestions for improvement is very constructive.
Personally, I think the biggest flaw with Rails is its "self-centeredness". Rails is changing Ruby itself at a rapid pace so when you try to do similar things with other libraries and frameworks you quickly run into a brick wall. When everyone tries to do the same thing you usually get tragic results so I am understanding of how all the "black magic" of redefining core classes is delegated to ActiveSupport.
Well, flaws in Rails are also related to flaws within Ruby itself so I'd like to address that in some way with Ruby 2.0.
And now the automated translations, which were awful. And remarkably similar - they all must be using the same translation engine. Take a look:
Babelfish:
As for the fault of ActiveRecord the text and the object which the Syrian rise from the database is done blending, is in treatment and attribute hash of text based column type when. It cannot judge in me, but as for indication of fault and proposition of improvement you think that it is very healthy. Privately as for the largest fault of Rails, you think that there is a "self central characteristic". In other words, because Rails changes the Ruby very thing with rather enormous spirit, it tries to do kind of that the other library either the framework or are similar with directly it collides. Well, when everyone does similar thing, because it means mostly miserable thing, it seems that rewrites fundamental class as for the "black magic" leaving to ActiveSupport, the bucket, with is the attitude which can understand also the fact that you say, but. As for fault of ま and this Rails being the case that that way it is consequence of the weak point of Ruby, with Ruby 2.0, somehow we would like to cope with here.
Google:
As for the fault of ActiveRecord the text and the object which the Syrian rise from the database is done blending, is in treatment and attribute hash of text based column type when.
It cannot judge in me, but as for indication of fault and proposition of improvement you think that it is very healthy.
Privately as for the largest fault of Rails, you think that there is a “self central characteristic”. In other words, because Rails changes the Ruby very thing with rather enormous spirit, it tries to do kind of that the other library either the framework or are similar with directly it collides. Well, when everyone does similar thing, because it means mostly miserable thing, it seems that rewrites fundamental class as for the “black magic” leaving to ActiveSupport, the bucket, with is the attitude which can understand also the fact that you say, but.
As for fault of [ma] and this Rails being the case that that way it is consequence of the weak point of Ruby, with Ruby 2.0, somehow we would like to cope with here.
Dictionary.com:
As for the fault of ActiveRecord the text and the object which the Syrian rise from the database is done blending, is in treatment and attribute hash of text based column type when.
It cannot judge in me, but as for indication of fault and proposition of improvement you think that it is very healthy.
Privately as for the largest fault of Rails, you think that there is a “self central characteristic”. In other words, because Rails changes the Ruby very thing with rather enormous spirit, it tries to do kind of that the other library either the framework or are similar with directly it collides. Well, when everyone does similar thing, because it means mostly miserable thing, it seems that rewrites fundamental class as for the “black magic” leaving to ActiveSupport, the bucket, with is the attitude which can understand also the fact that you say, but.
As for fault of [ma] and this Rails being the case that way it is consequence of the weak point of Ruby, with Ruby 2.0, somehow we would like to cope with here.
So much for enlightenment.
Posted by Charlie Thu, 16 Aug 2007 19:59:00 GMT
Error handling in web applications is devilishly complex - we ended up rewriting SIAS's error handling code in almost every release.
There are two main sources of complexity. First, how do you handle errors that occur after you've already started returning a response to the client? The easiest solution, and the one Rails adopts, is to cache the generated result until its complete. If an exception occurs you dump the result and then render an error message - making damn sure your error message generating code is beyond flawless. This works if your results tend to be small, but breaks down if you need to return a lot of data to the client. For this article, let's stick with Rails assumption (we couldn't use it for SIAS).
The second issue is how do you return errors to the client - what format and HTTP status codes should you return? Unfortunately, most standards are mum on this point. For example, nowhere in the Atom Publishing Format does it tell you what to do when an error occurs. On the other extreme, the Web Map Server (WMS) standard, which we supported in SIAS, goes overboard. Since a Web Map Server is supposed to return a map, usually in a raster format such as PNG or JPEG, the standard specifies that errors should be returned as bitmaps that contain the error message (bet you never thought of that!). It also requires that you support returning errors as "blank" images (so as not to block out any other maps) and then, for a fun twist, as XML documents.
Its on this second point that Rails is lacking - error messages are always returned in HTML and are mapped to two HTTP status codes - 404 (not found) and 500 (internal server error).
We can do better. Building off the content negotiation plugin I talked about last week, we have created an error handling plugin for MapBuzz called render_exceptions.
The plugin does four things:
rescue => e render(:exception => e) end
module ActiveRecord
class RecordNotFound
def http_status_code
404
end
end
end
Note that ActionController::RecordInvalid is mapped to 418, as described earlier on my blog and rest-discuss. When an exception is raised, the plugin checks to see if the exception object responds to http_status_code. If it does, and if the value is not 500, then the plugin picks one of the error templates using the content negotiation plugin I blogged about last week. Otherwise, the plugin reverts back to the default Rails error handling.
The end result is that errors are returned using the appropriate mime type and http status code. So if a client asks for an Atom feed of a non-existent record, the server returns 404 and an error message as an Atom entry. Similiarly, if the client asks for an HTML document for a non-existent record, the server returns 404 and an HTML error message.
To install the plugin:
And that should do the trick.
