Posted by Charlie
Mon, 28 Jan 2008 09:06:00 GMT
In preparation for getting back to this blogging thing, a couple of weeks ago I toyed with changing my blogging software from Typo to WordPress. I originally chose Typo for purely geek reasons - it ran on Rails and supported with Postgresql. But it didn't work particularly well, so soon thereafter I set up Yue with WordPress which was much, much better.
Anyway, before switching I figured I'd install the latest and greatest of Typo and give it one last chance. And a funny thing happened - Typo was so much better than it was before that the pain of a migrating suddenly became much sharper. So I put a couple hours of work into upgrading, patted myself on the back, and moved onto other things. Bad move.
Soon thereafter, Michael Gurski was kind enough to send me an email saying that the feeds on my blog were broken. Bascially link A redirected to link B which redirected back to link A, causing an infinite loop and unhappy clients. Unfortunately, I was so busy last week I didn't have time to investigate. But its now fixed. So thanks Mike, and apologies for everyone subscribed to my feeds.
no comments | no trackbacks
Posted by Charlie
Wed, 05 Dec 2007 08:25:00 GMT
Well, it wasn't quite in the same league as 62-36, but a victory over Nebraska is always cause for celebration. Here's Mike, my brother, and I in the south stands of Folsom field. If it looks like we are cold - well - we were! It was freezing that day.
And of course a home victory over Nebraska requires an obligatory run onto the field:
I suppose at some point one gets to be too old to rush onto a football field with a bunch of college students, but its seems Mike and I haven't hit that age yet 
.
Posted in Home | 3 comments | no trackbacks
Posted by Charlie
Fri, 09 Nov 2007 21:15:00 GMT
Peter wrote a good post last month about the power of links. It strange to be evangelizing links over ten years into the Web revolution, but in an ironic twist, what the average person immediately groks seems to have mostly escaped the notice of geeks.
Anyone who uses the Web immediately grasps the power of links - they are the way in which you navigate through the Web. In fact, I think the value of the Web is almost entirely due to links because they enable Reed's Law.
But for whatever reason, geeks have traditionally equated links with HTML and therefore have not understood their power. On the grand scale, this failure has led to the Soap/WS* detour and the painfully slow adoption of REST. On a smaller scale, it results in developers creating custom xml formats like this (borrowing Peter's example):
<person>
<accounts>
<account><id>3242</id></account>
<account><id>5523</id></account>
</accounts>
</person>
Instead of like this:
<person>
<accounts>
<account href="http://bank.example/accounts/3242"/>
<account href="http://bank.example/accounts/5523"/></accounts>
</person>
The problem with the first example is that is assumes a closed system - thereby throwing away all the benefits of the web including discoverability, interoperability, aggregation, etc.
So while links drove the phenomenal adoption of the human Web, the lack of links has severely stunted the growth of the machine web (computers talking to other computers).
The Rise of Atom
Fortunately, Atom (and its predecessor RSS) has changed the game. Originally considered just a blogging format, Atom instead will become the foundation of the machine web and will eventually become as an important format as XHTML.
Why? The links of course! (you didn't expect another answer, did you?). At its heart, an Atom document is simply a list of links with a bit of standardized metadata. As a result, it finally enables the machine web to leverage Reed's law thereby leading to the explosion of mashups and Web 2.0 sites we see today.
And this is just the beginning of course. Now that the requisite standards are in place, over the coming decades the machine web will follow the human web and grow exponentially as all the devices around us (cars, houses, appliances, sensors, etc.) start to come online and join the Web.
1 comment | no trackbacks
Posted by Charlie
Tue, 06 Nov 2007 06:33:00 GMT
As many of you know, I'm a big fan of Atom, including both the syndication format and publishing protocol. At Foss4g, I gave a talk that argued that Atom offers a better solution for exchanging geographic information than OGC's Web Feature Service (WFS) standard.
If you're interested in learning more, I've finally put my slides online in a Web friendly format. If you're interested in just the quick summary, it goes like this.
WFS is not designed for the web:
- Its lack of URIs means no linking, page rank, etc.
- Its proprietary interface requires custom clients that are tightly coupled to servers
- Its incorrect use of HTTP as a transport protocol throws away the benefits of the web infrastructure (caching, load balancing)
- Its lack of a common GML schema causes information to be segregated by organization, industry type, problem domain
The result is there is no serendipity (pdf file) with WFS - just think of how many WFS mashups you see.
In contrast, Atom is designed for the Web:
- Everything has a URI
- It uses a standard interface based on HTTP
- It uses standard formats
- It has full support for linking and therefore session state
- Atom fully utilizes the Web's infrastructure
The result is that Atom is designed for serendipty - allowing clever developers to build new, and unexpected, applications. And as I said earlier this year:
Atom offers a great foundation for building web services. Its a no-brainer to see that Atom will blow away the WS-DeathStar (which is a mocking tribute to the mountains of specifications that IBM, Microsoft and others have generated for defining web services).
But I think Atom will go farther - it will become the lingua franca of Web Services just as HTML is the lingua franca of the Web.
In my opinion OGC should extract the spatial specific parts of WFS (a suprisingly small part of the specification) and rewrite it as an Atom extension.
6 comments | no trackbacks
Posted by Charlie
Wed, 31 Oct 2007 03:08:00 GMT
Before Sunday's World Series game, Lauren and I did a bit of pumpkin carving to get ready for Halloween. To keep with the spirit of the day, I went with a Rockies logo. And since there was no way I was letting Lauren put a pumpkin out front of our house with a Red Sox logo, he went with a more traditional scary pumpkin look.
Here's our handiwork after we'd finished carving the pumpkins out back:
And here they are lit up at night in front of the house (not that you can see the house, but its there):
Not bad for two guys who totally lack any artistic ability!
no comments | no trackbacks
Posted by Charlie
Tue, 30 Oct 2007 05:39:00 GMT
Boston Red Sox versus Colorado Rockies.
Time for Lauren, one of my best friends and lifelong Sox fan, to make it to Denver for a World Series weekend. We scored two tickets off CraigsList to see Game 3, the first World Series game ever played at Coors Field. Of course we were in the very, very, very top row of Coors Field, even above the line of purple row of seats that demarks one mile high, but no matter.
Here's Lauren more than a half hour before the start of the game - Coors Field is almost already full!
And then fifteen minutes later the national anthem sung by Carrie Underwood (you can't see her on the field because she's standing behind home plate, but you can see her face televised on the left field scoreboard).
And then Lauren looking happy after the Red Sox scored six runs in the third and me looking sad and goofy.
The Rockies actually made a game out of it, but in the end lost. That bit sucked, but it sure was a night to remember.
Posted in Home | 2 comments | no trackbacks
Posted by Charlie
Tue, 30 Oct 2007 05:11:00 GMT
Like most American male teenagers, I loved baseball. When I discovered Bill James' incomparable Baseball Abstract in 1983 it was like finding a slice of heaven. The Baseball Abstracts were baseball for geeks - full of hypotheses that were either confirmed or refuted by masses of statistics and equations. Before long, I had programmed Lotus 1-2-3 on my parent's IBM PC (yes - one of those) to calculate all sorts of statistics about the players on my favorite team back then - the New York Mets (who were truly dreadful at the time).
But year by year my love got whittled away. Most of it was destroyed by watching the insufferable fights between multimillionaire players and owners. And then more was whittled away seeing the huge competitive disadvantages between small and large market teams - where's the fun if you know that 2/3rds of the teams have no chance of winning the World Series? And when a small market team actually did make the playoffs, the Yankees, Red Sox or one of the other privileged few inevitably swooped in and took away their best players.
By the mid-90's I'd pretty much given up on baseball. Sure I'd go to a game or two a year, either to hang out at Camden Yards when I lived in DC or at Coors Field after moving back to Denver. But that's because both fields are great ballparks, and it was always a fun evening hanging out with friends or going out on a date.
It didn't help that my new favorite team, the Rockies, were lousy. And perhaps even worse then being lousy, their roster was the definition of a revolving door. Their general manager (Dan O'Dowd) made 39 trades involving 130 players in his first three years on the job. Its hard to get excited about a team when you haven't got a clue who plays for them.
Jumping on the Bandwagon
And then the Rockies made baseball fun again this year. They're an easy team to root for - young, talented, hard-working and team-oriented - the perfect underdog. And it was clear they were getting better - anything less than a winning record this year (2007) would have been a disappointment.
By the end of August, they had met expectations. But then they put together the best September in baseball history and capped it off with one of the most extraordinary baseball games I've ever seen - a one game playoff victory against the Padres.
I was lucky enough to be at Coors Field that night - my dad had bought two tickets. And it was quite a night - you could just "feel" the Rockies were going to win until they blew a lead in the eight inning. And then a funny thing happened. The game stopped being fun and became downright scary. The capacity crowd stood from the ninth inning on - fidgeting nervously before every pitch, cheering wildly for strikes and outs and groaning with each ball or Padre hit.
And I've never heard a crowd so mercilessly boo a player off the field after Jorge Julio crushed our dreams by giving up a two run home run in the top of the thirteenth inning. And I've never heard a crowd so joyful then after Matt Holliday was called safe to win the game in the bottom of the the 13th.
If you're not into the sports then none of this makes sense. But if you are, then you understand that its greatest moments are pure joy or pure devastation. Its every bit of life all wrapped up into one moment, and its a high that makes people root for their favorite teams year after year after year until the one ultimate moment arrives.
And obviously I was hooked on baseball again.
Posted in Home | no comments | no trackbacks
Posted by Charlie
Wed, 10 Oct 2007 00:34:00 GMT
I had no intention of getting involved in the proprietary versus open source versus free software war. But Tom asked an excellent question in response to my article exploring if open source can dethrone Google maps. He asked if I believe the Jimmy Wales quote that I included?
Let's take a look at the quote again. When talking about the current state of Internet Search, Jimmy Wales said:
It is broken for the same reason that proprietary software is always broken: lack of freedom, lack of community, lack of accountability, lack of transparency.
In my post I didn't agree or disagree with him. Instead, I used his quote to summarize the free software point of view.
So what do I think? I think his assumptions are correct, but his conclusion is wrong.
Transparency, Freedom, Accountability and Community
Let's start with the assumptions. Having worked on proprietary software most of my life, I agree that it lacks transparency, freedom and accountability. I disagree however on community.
Customers have very little visibility into proprietary software. They are almost never granted access to the bug database, they are kept in the dark about upcoming features until the very last moment, and they rarely have control over development work unless they've paid for it. And even in that case, they often end up with one-off solutions that don't get integrated back into the main product line.
Customers also lack freedom, in two fundamental ways. First, they can't change the software because they don't have the source code or because they've signed a license that doesn't let them modify the source code. Second, any software vendor worth its salt will try its hardest (its good business practice) to lock in its customers. There are lots of ways of doing that, with a favorite one being proprietary data formats.
Next, software vendors have been amazingly successful at avoiding accountability, particularly in light of our litigious society. When was the last time you sued your vendor because their software did not perform as advertised? Right, that's what I thought.
But I do disagree with the community assumption. There are large and thriving communities around popular proprietary software products that are extremely valuable. If you want to knit-pick, you could point out these communities tend to be more negative then comparable open source or free software communities. You'll see more complaining and griping, I assume because the participants are at the mercy of their software vendor. However, that's a minor point and doesn't take away from the value the communities do provide.
It's a Question of Value
Now that I've agreed with three out of four of Jimmy Wales assumptions do I agree with his conclusion that proprietary software is broken? No, I don't.
I find it laughable to consider Google's, Microsoft's or ESRI's software broken. In fact, I think the opposite is true. Proprietary software has generated such profound value that it has raised our standard of living by making us richer and our lives better. It plays a huge role in our day-to-day lives. Some of it is obvious, like making it easier to book travel on line using Expedia or socializing with your friends on Facebook. Much of it is hidden, such as the anti-lock brake system in your car, the digital thermostat in your home that saves energy or the software that runs the telephone network so you can call your friends.
But think about the last example for a moment, the proprietary software that runs the telephone network. As wonderful as it is, the open source software that runs the Internet is better. Not only does it provide all the functionality of the older telephone networks, it also enables email, the Web, radio, tv, etc. It simply is more valuable.
And you see the pattern over and over - when there is direct competition between proprietary software and open source software, I believe open source software usually provides greater value. Obviously that's a controversial statement, and should be backed up by concrete proof as opposed to just my observations. Unfortunately, I don't know of any thorough, non-biased studies that explore the economic value generated in each model. I assume they must exist, so if you know of one then drop me a comment.
So based on my observations, I'd modify the original quote to read:
It is less valuable for the same reason that proprietary software is always less valuable: lack of freedom, lack of community, lack of accountability, lack of transparency.
Sadly, it doesn't have quite the same zing as the original, does it?
no comments | no trackbacks
Posted by Charlie
Thu, 04 Oct 2007 02:21:00 GMT
Today I discovered a really interesting blog, Spatial Law, written by Kevin Pomfret. Kevin commented on my last post about open source and Google maps, and pointed out that the legal issues surrounding spatial data are even more murky than those around software.
More importantly, Kevin also talks about privacy issues. As usual, our technology is outpacing our culture, seriously jeopardizing our expectations of privacy. And worse, our fear of terrorism is scaring us into compromising on rights that we should not be compromising. As Benjamin Franklin said:
They that can give up essential liberty to obtain a little temporary safety deserve neither safety nor liberty.
Many of these privacy issues come to a head in the geospatial world. Its now easy to track people. We are constantly filmed on security cameras, most of us have location enabled cell phones, we conduct commerce electronically (ATMs, credit cards), and some of us even show up in Google Street View! Not only that, but many of us freely publish our locations because of the useful applications it enables - where are my friends, what bar am I at right now, etc. And last, if Congress has its way (and other countries are heading down the same path), we'll all soon have national identity cards that makes it even easier to track us.
Of course many of these issues are from clear. What is acceptable to you might not be aceptable to me. But its much too easy for those of us building these applications to get caught up in the technology and "geez-whiz" factor and completely ignore the privacy issues at hand.
How many talks were there at FOSS4G about privacy? Not a single one. How about Where 2.0? One. Exactly 15 minutes was spent this year on privacy issues at two of the biggest geospatial conferences.
We must do better. And it starts with each of us. If you're building a new whiz-bang spatial app, the bare minimum you can do is make sure your users have complete control over what information they share and don't share with others. And make sure your default settings are "private."
And then go educate yourself on the issues, if you haven't already. Many of us already have strong political feelings in the open source versus free software versus proprietary software debate. Yet the importance of that debate doesn't hold a candle to the debate about basic privacy rights.
4 comments | no trackbacks
Posted by Charlie
Sun, 30 Sep 2007 04:14:00 GMT
Since returning from FOSS4G, I've been nagged by a question in the back of my mind - can open source succeed in the geospatial world?
Now you might think I've lost my mind. Didn't hundreds of open source geospatial developers just attend FOSS4G? And haven't I checked MapServer's market share? And haven't I noticed how many people use PostGIS? And don't I remember that Autodesk open sourced MapGuide last year?
More than Just Software
Sure - I noticed all that. But its only part of the story. The geospatial world is different because it requires more than software - it also requires massive amounts of data and lots of hardware to serve that data. That simply is not true in other industries that open source plays - it doesn't apply to operating systems, web servers, databases, drawing programs, web browsers, email clients, sales force management, etc.
Similarly to these other problem domains, open source has created some wonderful geospatial software products that stack up well against their commercial brethen. But can open source also solve the data and distribution pieces? Can it recreate Google Maps - which is the penultimate combination of software, data and hardware?
Depending on your political viewpoint you may or may not think this is important. If you're an open source advocate then developing the software is probably good enough. You'd just leave the data to Navteq and the distribution to Google, Yahoo or Microsoft. But if you're a free software advocate, then clearly it isn't. You'll always be subject to data licensing fees and data distribution charges. And more fundamentally, quoting Jimmy Wales, the founder of Wikipedia :
It is broken for the same reason that proprietary software is always broken: lack of freedom, lack of community, lack of accountability, lack of transparency.
Data
So assuming that the open source world can create the needed software to replicate Google Maps (and I think it already has), then the next challenge is the data. Traditionally, gathering and maintaining geospatial data has been time consuming and hugely expensive. As a result, freely available, high-quality data is only available in countries with enlightened governments - which sadly are few and far between.
But just as the Internet enabled the birth of the open source movement, mass-market GPS devices have enabled the birth of the open source mapping movement. These citizen mappers have congealed around the OpenStreetMap project, which can be thought of as a Wikipedia for maps. Thousands of users upload their GPS data, together creating highly detailed and accurate maps of where they live.
I consider OpenStreetMap wildly successful - it's proven that citizens can create their own maps. Yet OpenStreetMap provides data for a vanishingly small part of the world. Will it continue to grow? Will it be possible to cover the world? What happen in countries with free data, like the United States? Will OpenStreetMap be able to gather and manage important non-geographic data, such as which roads are one way, where are addresses located, road numbers, etc?
My guess is yes - it feels to me that OpenStreetMap is nearing a tipping point. As more data comes online, the more useful it becomes, and the more people want to get involved. Thus starts a virtuous cycle which will propel OpenStreetMap to cover to the globe, just as Wikipedia has become the world's encyclopedia.
And its seems likely this virtuous cycle will lead to a new competitor for Navteq. Navteq has built a $600 million a year company based on selling mapping data. Of that $600 million, last year $275 million went to creating and distributing their massive database.
A startup that leverages Open Street map data could compete directly against Navteq. Its main advantage would be cost - its main disadvantages would be data availability and data quality. But as the startup grew, it would pour resources into OpenStreetMap by paying top volunteers to add more data, ensuring data quality and funding the mapping of previously unmapped areas.
And of course more money means more data which means more utility which means more people. And the virtuous cycle would continue just as it does in the Linux world where RedHat, IBM and Novell bank role much of Linux's development.
When it Costs Real Money
But now we run into trouble. Why do open source projects work? Because participants love what they do and love the recognition they get for doing it. But above all, open source projects are about community and working together with your colleagues and friends.
However, open source projects are predicated on the fact that participation does not entail significant out of pocket expenses. If I want to join an open source software project I need a computer and Internet connection - two things I'm likely to have. And if I want to join OpenStreetMap then I need a GPS, which I probably also have and if I don't then I can buy one for a couple hundred of dollars.
And thus we get to the hard part in duplicating Google Maps. Two parts of it cost money - lots of money. The first part is the satellite and aerial imagery. And the second is the infrastructure to store, process and distribute all this data.
Imagery
One of the most compelling features of Google maps is the imagery data. Who hasn't looked at your house or apartment above? And who hasn't seen pictures of B-2 bombers, Navy buildings that look like swastikas and advertisements big-box stores are putting on their roofs?
Can open source duplicate this data? I don't see how. Very few citizens have the expertise or money to get an airplane, strap on a camera, and start taking high resolution aerial photography. It seems the best that can be done is using freely available government data, just as NASA's World Wind program does. Is that good enough?
And in this case, I don't see why a commercial enterprise would come to the rescue. Why would Digital Globe provide its imagery data to OpenStreetMap? First, they don't get the benefit of user contributed data. And second, they'd seriously tick-off their commercial customers such as Google and Microsoft. Thus there is no virtuous cycle here.
Update - 80n in the comments pointed out that last year Yahoo announced that OpenStreetMap could use its aerial imagery. Very interesting, I had totally missed that. Of course, I think the same licensing worries as mentioned above still apply, but kudos to Yahoo.
Infrastructure
Which leads us to the last point - storing, processing and distributing spatial data takes a whole lot of hardware. Last year I calculated that at zoom level 20 Google maps requires 1,099,511,627,776 (that would be a cool one trillion) tiles to cover the world. And since then, they've added additional zoom levels.
In addition to storing the tiles, you'd need machines to store the original vector data, render the tiles and serving them on the Web. And we haven't even talked about imagery yet, which takes even more space. Even then, having the hardware isn't enough. You'd also have to have the expertise to run it all, which perhaps is Google's most treasured secret.
So its seems to me there is absolutely no chance that open source can duplicate this infrastructure. However, there could be two ways around it.
First, use peer-to-peer software to create a vast network of home computers that can render the tiles and serve the data around the world. SETI, BitTorrent and Skype have proven the basic concept. Grub is trying to prove it can work for crawling the web. OpenStreetMap is following with its tiles@home project, which is an attempt to distribute tile rendering across hundreds (maybe someday thousands) of machines across the Internet. But can peer-to-peer software also work for serving data on the web (I'm assuming no one is going to download a GoogleEarth clone to view maps)?
Second, let a commercial company build the infrastructure. The obvious business model is hosting Open Street map data for free and generating revenue through local advertising. Thus such a company would be competing directly against Google, Yahoo and Microsoft - hardly an easy task. And in the end, would it be any different than Google, Yahoo or Microsoft today?
The Conclusion
So can open source replicate Google Maps? On the software side - yes. On the map data side - yes. On the imagery data side - no. And on the infrastructure side - its a stretch.
So I think the answer is no - open source cannot replicate Google Maps - at least for now. But it can get close. And if WikiSearch succeeds, and proves that even mighty Google is susceptible to open source, then at least there will be a foundation on which to stand and try.
19 comments | 2 trackbacks
