Monday, November 5, 2007

How could we ever beat Numenta?

Let me start with an...

anecdote used by Jerry Jordan, president of the Federal Reserve Bank of Cleveland, in an article in the Cato Journal last summer. Jordan described a U.S. businessman visiting China a few years ago. The American came upon a team of 100 workers building a dam with shovels. Shovels.

He commented to a local official that, with an earth-moving machine, a single worker could build the dam in an afternoon. The official replied,

"Yes, but think of all the unemployment that would create."
"Oh," said the businessman, "I thought you were building a dam. If it's jobs you want to create, then take away their shovels and give them spoons."

End-of-quote. Numenta is doing great work; but perhaps they have made some choices to be in quite difficult territory. I think that their strict reliance on biological plausibility and the low level in which they work are deeply problematic.

Not that they are wrong.

Of course a sound theory of the mind can be reduced to the level of neurons. Nothing wrong with that idea. But here are two arguments: why stop there?, and how will you deal with the bandwidth problem?

Why stop at the neuron level? Why not go all the way to atoms? Or to quantum physics, perhaps? It seems aesthetically pleasing to model the mind using neurons; the final encounter between biology, psychology, and computation. But there is a more important question here; is this the most productive way to work?

I don`t think so. I think it is very unproductive.

As economists will tell you, there is another way to say that "productivity is low": You are limited in what you can achieve. I think Hawkins and Numenta are limited in what they can achieve, as they have made some design choices which might paint them in a corner for long stretches of time.

In building something so complex, your first questions should be:
  • what are the project`s self-correcting mechanisms?
  • how long does it take to self-correct?

Without facing these issues head-on, right from the start, and finding the most productive way to build this thing, there is no money or intelligence that will pull it off alone.