cfis : A Sane Way of Sanitizing HTML

Trackback from cfis: Ruby, libxml and Windows on A Sane Way of Sanitizing HTML

Fri, 09 Feb 2007 01:06:18 -0700

Let's say my last post tempted you to check out libxml's validation functionality. However, you work in shop that develops on multiple platforms, including Windows, OS X and Linux, so you code has to run on all three platforms. You happily note th...

Comment on A Sane Way of Sanitizing HTML by Ambush Commander

Fri, 09 Feb 2007 21:06:01 -0700

Thanks for linking to my comparison article at HTML Purifier. Being the author of that library, I’d like to throw in my two cents on the issue.

You noted some of the problems of attempting to validate a document using the XHTML Basic DTD (no attribute parsing) and the ability of RELAX NG schema to peek in those attributes, but there are still some problems:

The schemas validate entire documents, not HTML fragments that users usually submit. You’ll need to wrap html in at least an html and body tag. In a similar vein, XHTML Basic allows things like html, meta and body tags, which aren’t appropriate for XHTML fragments
The schemas are, to put it lightly, not very forgiving. Even with a WYSIWYG editor, things can get past, and even the slightest element in the wrong place will result in failed validation and a cryptic error message. XML currently does not have facilities for graceful error handling
XHTML Basic allows script tags. And object tags. And events. Sorry, but if you’re going to piggy-back of XML validation functionality, construct your own schema! XHTML Basic is not an XSS-defender.

(Plug Warning): HTML Purifier validates according to hand vetted XHTML 1.0 DTDs, with bad stuff taken out, as well as comprehensive attribute validation schemes. Recent developmental versions validate according XHTML 1.1’s modularization. I strongly recommend you take a second look at the library! Thanks.

Comment on A Sane Way of Sanitizing HTML by cfis

Sat, 10 Feb 2007 12:02:23 -0700

Hi Ambush Commander,

Thanks for the great comment. I was very impressed by your HTML Purifier library. I think your approach of using a full-fledged parser and validating against a white list is on the only workable solution to this problem. And in fact, the approach I discussed, using libxml to validate against a DTD or schema, is the same. The reason why I prefer that route is that you can reuse the xml infrastructure that has been built over the last five years.

Caveats do apply though:

You have to use XHTML.
As you mention, you have to wrap the text to parse with an <html> element. This seems like a good idea anyways though, since otherwise someone could embed a processiong instruction at the top of the submitted text that points to a different schema to validate against (I totally forget to mention that in the article).
Agreed that XHTML Basic still leaves in dangerous elements like script and object, and those need to be removed. That is what I meant by the sentence “with a tweak here and tweak there.” I should have been more explicit. So yes, you have to create your own schema. XHTML Basic is a good starting point since its a lot closer to what you want then full XHTML.

Thanks again for the comment. It sounds to me we are in general agreement, but let me know if that’s not the case! I am curious as to why you took the approach you did though, instead of using libxml or another validating parser?

Comment on A Sane Way of Sanitizing HTML by Edward Z. Yang

Sun, 11 Feb 2007 13:37:40 -0700

You have to use XHTML

Surprisingly, this is not always the case. PHP’s DOM extension has a method called DOMDocument->loadHTML() which accepts poorly formed HTML documents. I believe DOM is based off of libxml.

It sounds to me we are in general agreement, but let me know if that’s not the case!

For the most part. In my opinion, using an XML parser/validator is a quick and easy way to get some HTML filtering capabilities, although it isn’t necessarily the best. But I do agree that the only way to filter HTML is to understand it fully.

I am curious as to why you took the approach you did though, instead of using libxml or another validating parser?

Besides the limitations I enumerated in my first comment as well as the fact that even Relax NG’s treatment of attribute validation isn’t satisfactory either (it will allow javascript:// URIs, for instance), this library, quite simply put, wasn’t available.

HTML Purifier has support for PHP 4 and PHP 5, but libxml is only bundled with PHP 5.1.0 or higher. The PHP 4 library, expat, does not have validating capabilities. When libxml is available, I use it to parse the document (yes, wrapping the code in html tags), but then I use my own validator system.

P.S. It’s Ambush Commander not Ambush Defender. You can use my real name (Edward Z. Yang) though. ;-)

Comment on A Sane Way of Sanitizing HTML by cfis

Sun, 11 Feb 2007 20:45:33 -0700

Hi Edward,

My apologies on your name - got it right in the comment and wrong on the blog. Its fixed now.

As far as HTML and using PHP to correctly parse it, that sounds good. Another alternative that I’m sure you know about (mentioning more for others to see) is HTML Tidy - not sure if that is better or worse.

And thanks for the explanation about when you can and can’t use libxml, definitely makes sense. I do most of my work with Ruby. Ruby doesn’t have libxml support built-in, but the luckily there is an open source extension that provides Ruby bindings.

Thanks again for the great comments.

Comment on A Sane Way of Sanitizing HTML by greenday

Thu, 07 Feb 2008 22:00:44 -0700

check out the htmLawed library

Trackback from cfis: Resurecting libxml-ruby on A Sane Way of Sanitizing HTML

Wed, 16 Jul 2008 10:41:02 -0600

There is general discontent with the state of XML processing in Ruby - see for example here or here. An obvious solution is to use libxml. However that has been a non-starter since the libxml Ruby bindings have historically caused numerous segemen...