Understanding Problems

There is no way to prove you have understood a problem.

It is a fuzzy, intuitive endeavor, more art than science, more craft than engineering. It requires unending patience, infinite care, and obsession with the tiniest details.

You cannot truly understand problems without an insatiable need to understand the whole, no matter how complex it is. Only a deep-seated hunger to know will compel you into every last corner, driven more by obsessive curiousity than the need to solve.

Alas, the world is not perfect, and humans have very small brains. There will always be unseen flaws in your grasp on a problem.

The deep magic lies in knowing when you have understood a problem deeply enough to solve it well enough for the current context. No team has enough resources to solve all its problems well.

There is no escape from implementation - actually trying an idea is the only way to see if it conforms with reality. Thus, the best test for deep understanding of a problem is implementation.

If your solution surprises and delights its users, and experts describe it as 'simple' or 'beautiful', you have almost certainly understood the problem deeply. The first is insufficient on its own, because people are so used to awful software [1].

Incarnating ideas is difficult, time-consuming, and expensive. Thus, it is a big win to catch flaws in your design as soon as possible.

This is why jumping directly to coding, while sometimes productive, is often a net loss. You can go days, weeks, or even months before discovering a fatal flaw in your solution, when you could have caught the issue by thinking for another hour or two, before a single line of code was written.

Time spent understanding the problem reduces the probability of a fatal flaw in the solution. How much time you should take depends on the value of having a solution, how much the solution's value hinges on level of quality, and the consequences of an unforeseen fatal flaw.

One way to think through a problem is to describe it and the proposed solution in written form. A written description of a proposed solution is often called a 'design document' or 'specification' (the latter is usually much more detailed than the former). Such a document lets you explain your solution to people without talking to them, and thus lets people see how well you understand the problem out-of-band. Of course, that does not guarantee correct understanding, because humans make mistakes.

That's why staying focused on implementation is critical. Writing proof-of-concept solutions for the hard parts of a problem is a great way to validate your understanding at lower cost than a full implementation. You pay only for writing the hard parts, not the copious amounts of glue that holds a system together, and you pay only enough for each proof-of-concept to verify that you have a workable solution, since it does not need to be production-ready.

Of course, even for proof-of-concepts, you should have a decent idea how you plan for it to work before building. As mentioned, finding fatal flaws the hard way is not ideal.

Whether production software or a proof-of-concept, what are some indicators you should think more before building something?

  • You cannot explain the problem clearly to a newcomer.
  • You cannot propose a solution to the problem.
  • You can find holes in your proposed solution.
  • Someone asks you a question about the problem you have a hard time answering.
  • Someone dislikes your proposed solution and can explain why.
  • Someone points out a hole in your proposed solution.

What are some indicators it's time to start building?

  • When you explain the problem to an expert who is not familiar with it, they immediately propose reasonable solutions.
  • You can explain the known solutions to a layperson, as well as their pros and cons.
  • You can explain the problem, and your solution, to a layperson clearly. "If you cannot explain it to your grandmother, you do not really understand it."
  • You can design a new solution to the problem with little effort.

Remember - these are only guidelines, and not a complete set. Understanding problems is an art, not a science.

[1]For instance, if you have only used Harvest for version control, anything that doesn't occasionally eat your files would be amazing. However, that doesn't make CVS a good version control system.

Other Articles In 'fundamentals'


  1. Fundamentals

    "Of the making of many books there is no end..."

  2. Caring For Users

    If a program runs in a forest but no one sees the output...