Monday, January 28, 2008

Some thoughts on Temperature

Intrinsic to FARG architectures is the notion of temperature. From John Rehling's thesis:

"All of the three modules and the Letter Spirit loop incorporate, each in its own way, temperature – a means of modulating the randomness of behavior – or something very much like it. The Letter Spirit loop, for instance, decreases the amount that style influences the Drafter's attempts at rendering a particular letter category when it has already undergone several attempts at drafting without the Examiner having recognized them as members of the intended letter category. It also turns up the amount of randomness in drafting as a run goes on, under the premise that novel letterforms are more useful at that point than good ones that had been drafted previously. In all of its guises, the use of temperature employs the principle encapsulated by the phrase "desperate times call for desperate measures"—a principle seen at work in many kinds of human behavior." (Rehling p.346-347)

While I do love the notion, I am not satisfied with current implementations.

We know the mind is massively distributed. Imagine, in the future, Fluid Concept architectures running in thousands of processors. It is with this in mind that your blogger asks you to follow his wild speculations.

Today, temperature is implemented as a central variable that the whole system depends upon. As an appetizer, ask yourself: Is this sustainable for a massively parallel future? Do our brains have a central floating point number to access the state of entropy in any given situation? That's highly unlikely, I think.

Now for the main course: how can we change this? How can we turn a single, central, variable, into massively distributed stuff? I'm going to speculate here, but I think that this can be implemented fairly easily, and perhaps by the end of 2008 we can have some running examples.

How does temperature work? It both measures the state of coherence (or, to flip it, the state of disorder), and feeds back that information, so that the system may "behave" itself or go "balooney".

By behaving itself, I mean, of course, that the system should remain concentrated in the most promising ideas. It should ignore most possibilities. If temperature is low, that means that a coherent global view is coming into shape, and we don't want to mess it up by destroying work done--which inevitably happens if a "new" idea comes up.

By going balooney, I mean that the system has not yet figured out what to do. It has not yet built in STM structures that represent, in any cogent way, what the situation is like. Any node in the slipnet is welcome to become as agitated as it wants.

Now, how can that be massively distributed?

Here's a hunch.

What if, during a run, the system is able to track codelets that are destroying previously built structure? If the nodes that are creating and destroying stuff can be tracked by STM chunks, then what about creating, temporarily, "inhibitor" links in the slipnet nodes responsible for the creation and destruction of the structure in question?

With these mechanisms in place, I think they can be managed to gradually increase inhibition between competing worldviews.

After some time, with high enough inhibition, we would have "clusters" of self-reinforcing nodes, and these clusters should inhibit other competing clusters. The emergent behavior should in principle be just like the impact of a lowered temperature. "New", unwanted ideas should rarely creep into the party.

If the system gets stuck on a snag, then it could try moving to another cluster. But how would it find out that it is stuck, without having temperature (a centralized variable), to tell it that "this is probably not the best idea concerning our current situation"?

The structures built, since they would be tracking (some) activity in the slipnet, might become more and more dissatisfied with the top-down "orders" they are receiving. At one point, they might send a message to "shut-up", and bring activation of such node to zero. Maybe also this zero activation might spread to (the cluster's) associated nodes, or maybe even it could flow directly to the inhibited nodes. Perhaps the inhibited nodes (who were dying out) could be instantly activated by such triggering. This could make the system rapidly jump between competing worldviews, rather like our perception of Necker cubes or the Vases/Faces illusion.

Of course, as Christian says, language compiles everything. It may be very tricky to get this working. I do, however, think that it would (coupled with other new ideas) enable massive parallelism (on the tech side), and, more than that, reflect the distributed nature of our own information-processing (on the scientific side). Perhaps this may be one of the right moves in turning that Rubik's cube.

(more than ever, comments very welcome)