Thursday, November 29, 2007

Pleasure, maybe. Avoid pain, thanks!

Today I could'nt go to the class. I was giving a seminar of text mining and artificial intelligence on Fluminense Federal University (UFF). Here then goes my today's contribution.

Doing cognitive science is hard job because we just can't look inside us clearly. We can't ask objective questions to our brain. We have only access to a very high level instructions. Thus, last class I refactored my intuition about the hedonic guide of our lives, e.g., one being tries to maximizes pleasure and minimizes pain as well. I realized it might have a possibility that this is not a simetric distribution. Ingrate, but I think one may have much more pain then pleasure. Pain is a strong word, the left extreme. A discomfort or distress may be a better word. Thus, one may have much discomfort in daily life. It might be necessary to make us move on, to act or todo something.

Imagine one exposed to a situation that causes him this feeling. The situation sets hedonic function to a little bit under neutral point. A short-circuit alerts to bring back balance. One only stabilish peace if the short-circuit was released. Ok, my point is that this situation is kind of a bit similar with pleasure, but it is not the same. A pleasure is a move with neutral point as origin. I suppose we have a stick that slip a bit down every time. And this movement is what makes life.

Monday, November 19, 2007

On the verge of a breakthrough

I think we're on the verge of a breakthrough. We are about to solve the shape sorting problem, the most important scientific problem that nobody cares about.

I would like to take this opportunity to thank the Swedish Academy.

Images from qotile.

And here's a live specimen in its habitat:

Saturday, November 10, 2007

NUMBO To-do list

Classes dealing with

  • Activations
  • Nodes & links
  • Slipnet
  • (Bare) Chunks, withdraw and insert chunks and objects
  • Numbo-specific chunks
  • Codelets
  • Temperature
and... a project wrap-up and it's done. Getting there.

Life on the easy street

Things to finish STILL in 2007!

  • NUMBO implementation
  • NUMBO documentation (framework)
  • NUMBO documentation (NUMBO domain-specific)
  • The PhD Seminar
  • Paper on the Capyblanca Project
  • Jarbas' PhD Project
  • Carla's PhD Project
  • Anna's PhD Project
  • Jarbas' paper
  • Brum's psychological experiment
  • Simone's MSc Thesis
  • Anne's MSc Thesis
  • Taisa's MSc Thesis
  • Nicholas Thesis is for january, way way into the far future!

This list does not include any Club of Rome activity whatsoever, as you might have noticed.

What a piece of cake!

I hope I haven't forgot anything.

Numbo's slipnet: nodes

I'll be posting here the NUMBO's real slipnet, directly from Daniel Defays's code. Please disregard my previous guesses!

Node Numbers


Sum nodes

Multiplication nodes

Other nodes

I guess I'll be delving deeper into the lisp code, to see what's the best way to design an OO framework to handle it.

Thank god for Daniel Defays.

Wednesday, November 7, 2007

Compile and run. Nothing has changed? GOOD!

We are, today, taking small, but significant steps towards our framework.

The MOST crucial thing now is to proceed in a manner that gives us total confidence in what has been done. If there is any bad design decision, it will be rapidly found and corrected.

This is the most important thing in building such complex mechanisms. We have a long way to go, but let's take each single step in rock solid ground. This is a framework, not a program. It will be used for a long time period. It has to be rock solid.

We can only move on to new functionality when we feel confident to say, as members of the Cosa Nostra are fond of saying: "forget 'booout it!"

Finally, here's an important tip: Compile and run. Nothing has changed? GOOD!


Tuesday, November 6, 2007

Is Microsoft still the power?

The first war was, at the beginning, Microsoft vs Apple for a graphical user interface (GUI), where in 1984 Apple strikes first. In 1985, Microsoft shows the world Microsoft Windows, a GUI for any personal computer, no only Macs, overtaking Apple.

The second war was against Netscape to the Internet. Netscape strikes first in 1994 with 85% of market share. Microsoft, in 1995 uses its Windows share ands "gives" Internet Explorer to all Internet users, flattening Netscape to 1%.

The third war was to instant communication, ICQ strikes first in 1996. In 1999, Microsoft launches MSN Messenger, using its OS monopoly again.

The fourth war was and is to the search market. Google strikes first, in 1998, with PageRank technology. Internet was bigger then anyone could imagine. Google has 53,6% of it, a virtual monopoly. This time Google vs Microsoft seems like David vs Golias. Microsoft has no quick jabs. Time goes so fast that, today, 10 years later, nothing has stated yet. Google expanded to Gmail, YouTube, Orkut, Earth etc. A new tech-empire. Everybody shouts David, David, David...

Is Microsoft still the power?

Today I saw MSN Video embedded on MSN Messenger. I saw, at a glance, Microsoft again. How subtle and weak an empire might be. Incredible, when things sounds stable I see Facebook overtaking the consumed social networks phenomena. Look at behind of it, Microsoft on the fire. The brand-new Windows Vista comes powered by Microsoft Search as native or default for everything.

Where are Google fast moves? Now I prefer to think about Adobe Systems. While nobody used to give attention, it has 99,1% of computers around the world using its engine. More then Windows, Google and Firefox. Browser style goes down and it is coming Adobe AIR.

Monday, November 5, 2007

Refactoring a method to the strategy pattern

An example from O`Reilly:

Refactoring to Strategy

The getRecommended() method presents several problems. First, it's long—long enough that comments have to explain its various parts. Short methods are easy to understand, seldom need explanation, and are usually preferable to long methods. In addition, the getRecommended() method chooses a strategy and then executes it; these are two different and separable functions. You can clean up this code by applying STRATEGY. To do so, you need to

  • Create an interface that defines the strategic operation

  • Implement the interface with classes that represent each strategy

  • Refactor the code to select and to use an instance of the right strategic class

Three steps involved.

Why is this important? Because, in turning this Rubik`s cube, we will make mistakes. We do not know exactly many parameters and many design decisions will have to be taken without them. Educated guesses are informing all cognitive models in the market. So if we know that we will make mistakes, we should be able to undo those mistakes easily. So here is a design principle for our architecture:

Whenever a design decision is based on an educated guess, the implementation should not be done through class methods, but with the strategy pattern.
I don`t know to what extent we will be able to follow this. That is precisely why we need to start getting used to the refactoring involved and master these three small steps.

We want to create a strategy pattern to deal with potentially different activation curves and potentially different decay curves. We are not sure of which is the best (i.e., psychologically plausible) curve, hence we want this to be easy to change. The easier to change, the higher the productivity. The higher the productivity, the most mistakes we can make without major backtracks. And mistakes we will make, lots of mistakes. On that you can be sure.

But the strategy pattern also gives us something enormously valuable. It gives us the ability to change code on the fly. It gives us the ability, for example, to change behavior should a global signal be received; we know from experience that our memory of intense moments is registered; while expected, boring, moments do not register as well. I used to believe that in the mind there was no space for global variables, or global events. I was wrong. A spike in adrenaline makes you alert, opens your eyes, raises your heart rate, raises your blood pressure, feeds your muscles with blood, preparing them for action, and halts numerous "background" processes, such as digestion. There are global signals, and behavior changes on the fly. (BTW, where in NUPIC does this happen?)

The strategy pattern gives us this ability. And this is something we do want. So we want to be masters at the craft of transforming refactoring a class method into a strategy pattern. So here we go. Here's our current class definition for activations:

Tactivation = class
current_state, level, increment: real;
Signals_Received: real;
procedure Recompute_Level;

Constructor Create;
procedure increase (step: real);
Function Get_Level:real;
function Get_CurrentState:real;
function Get_Increment: real;
Procedure Reset_Increment;
Procedure DeployChange;
procedure decay;

Step 1. Creating an interface that defines the operation

(The syntax java it is easier on the eyes than in Delphi, so translating this code should be a piece of cake.)

Here we go:

IRecompute_Activation = Interface
function Recompute_Activation(Current_state: Real):Real;

Compile and run. Nothing should have changed yet in functionality. No errors should appear.

Step 2. Implement the interface with classes that represent each strategy.

Let's have, for starters, a sigmoid curve, and a linear one (i.e., level=current_state).

TRecompute_Activation_Sigmoid = Class (TInterfacedObject, IRecompute_Activation)
function Recompute_Activation(Current_state: Real):Real;

TRecompute_Activation_equals_state = Class (TInterfacedObject, IRecompute_Activation)
function Recompute_Activation(Current_state: Real):Real;

Copy/Paste the previous method code (in the sigmoid case), without deleting the original.

function TRecompute_Activation_Sigmoid.Recompute_Activation(Current_State:Real):real;
var pyramid, sum, t:real; counter: integer;
for counter:= 0 to floor (Current_State*max_Steps) do
t:= counter/max_steps;
If(t<=0.5) then Pyramid:=t else pyramid :=1-t;
Sum:=(4*(1/max_steps)* Pyramid) + Sum;
Result:= Sum;

function TRecompute_Activation_equals_State. Recompute_Activation (Current_State:Real) :real;
Result:= Current_State;

Now compile and run. Nothing changed in functionality. Good.

Step 3. Refactor the code to select and to use an instance of the right strategic class.

NOW we're changing the original class. Five quick steps are involved here.

(3.1) First we need to include the strategy pattern object, named activation_strategy, then compile and run. No change. Good.

(3.2) Now on to include methods to set_activation_sigmoid, or set_activation_linear; hence we include in the TActivation Class:

Activation_Strategy: IActivation_Strategy;
Procedure set_activation_sigmoid;
Procedure set_activation_linear;

...and the respective methods on the Activation class which call the constructor of the desired strategy:

Procedure Tactivation.set_activation_sigmoid;
Activation_Strategy:= TRecompute_Activation_Sigmoid.create;

Procedure Tactivation.set_activation_linear;
Activation_Strategy:= TRecompute_Activation_Linear.create;

Compile and run. Nothing changed in functionality. Good.

(3.3) Kill the previous code, by commenting out the method and its declaration. Compile... and it doesn't run anymore! Great, because the compiler will point out to you all the previous calls made to the method, so just substitute them for your strategy. In our example, we substitute calls to Recompute_Level to the function Level:=Activation_Strategy. Recompute_Activation (Current_State).

Compile and run, and functionality should be restored!

(3.4) Now, test whether or not the whole strategy is working by changing the pattern at runtime. In my case this means including the following piece of code in the end of the DeployChange method: if Level>0.5 then set_activation_linear;

Compile and run. Now marvel at the runtime behavioral change.

(3.5) Finally, clean the code. Delete the (commented out) method calls and method implementation (and declaration).

Compile and run. Works like a charm. Congratulate yourself now.

Here's to the strategy pattern. This one is truly important. Everybody should master this technique. This MUST be trivial to do, anytime. Unless you are so overconfident to believe that you'll never get paralyzed because of a bad design decision taken ages ago.

Don't be.

How could we ever beat Numenta?

Let me start with an...

anecdote used by Jerry Jordan, president of the Federal Reserve Bank of Cleveland, in an article in the Cato Journal last summer. Jordan described a U.S. businessman visiting China a few years ago. The American came upon a team of 100 workers building a dam with shovels. Shovels.

He commented to a local official that, with an earth-moving machine, a single worker could build the dam in an afternoon. The official replied,

"Yes, but think of all the unemployment that would create."
"Oh," said the businessman, "I thought you were building a dam. If it's jobs you want to create, then take away their shovels and give them spoons."

End-of-quote. Numenta is doing great work; but perhaps they have made some choices to be in quite difficult territory. I think that their strict reliance on biological plausibility and the low level in which they work are deeply problematic.

Not that they are wrong.

Of course a sound theory of the mind can be reduced to the level of neurons. Nothing wrong with that idea. But here are two arguments: why stop there?, and how will you deal with the bandwidth problem?

Why stop at the neuron level? Why not go all the way to atoms? Or to quantum physics, perhaps? It seems aesthetically pleasing to model the mind using neurons; the final encounter between biology, psychology, and computation. But there is a more important question here; is this the most productive way to work?

I don`t think so. I think it is very unproductive.

As economists will tell you, there is another way to say that "productivity is low": You are limited in what you can achieve. I think Hawkins and Numenta are limited in what they can achieve, as they have made some design choices which might paint them in a corner for long stretches of time.

In building something so complex, your first questions should be:
  • what are the project`s self-correcting mechanisms?
  • how long does it take to self-correct?

Without facing these issues head-on, right from the start, and finding the most productive way to build this thing, there is no money or intelligence that will pull it off alone.