One of the great mysteries of sofware development is what is the best way to do it. Its just like diets – every year or two the latest fad comes roaring through and everyone jumps on board. And just like diets, there is no one right way – it depends on what you are trying to accomplish and what type of team you have.

When it comes to writing software, on one extreme is the belief that every possible piece of functionality should be designed ahead of time in exruciating detail – resulting in thousands of pages of specifications that almost no one reads. On the other extreme is the belief that the best thing to do is jump right in and see how things work out. Surprisingly, sometimes either of the extremes is the right answer. However, most of the time an approach somewhere in the middle is better.

Yet no development process can account for human creativity. Just like painting, or writing music, or desiging buildings, software development is a creative process. A couple of months ago we implemented one of the more complex parts of [MapBuzz](http://www.mapbuzz.com) – who can view or edit features that are on a map. This turns out to be quite tricky – so over a month we thought long and hard to figure out the best design. Once we thought we had it, we spent a couple of weeks implementing it and thoroughly testing it.

And yet we were wrong. In a flash of insight last week, we discovered a much better solution – one that greatly simplifies the code and makes it reusable for permissions on any type of object (map, feature, comment, etc.). It took another week of work to implement the new solution – obviously begging the question why we didn’t see it at first.

And this question goes right to the heart of design. Good design is based on experience – without it you don’t have the requisite knowledge to solve difficult problems. But often times experience is not enough – difficult problems require that you immerse yourself in them trying any number of solutions until one clicks. Even more maddeningly, you usually then get so engrossed in a problem that you lose your ability to see the solution. At that point you have to walk away from the problem and give your subconscious a chance to put all the pieces together. If you’re lucky, it will present you with a solution in the form of a Eureka moment when you least expect it. And that is exactly what happened to us.

Which leads to an obvious conclusion. The more you work, the less creative you are because you don’t give your subsconscious enough time to find the best solution. This is hardly a novel thought – a quick searh on Google turns up a number of studies of this phenomenan. But its worth keeping in mind – if you have a really tough problem, and you’ve put in the necessary hours to fully understand it, then your best bet for solving it is a day at the beach or skiing!