Memory makes you smarter

Another sidebar working up to talking about the hide-and-seek demo.

Few words express more exasperation than "I just told you that!", and -- fairly or not -- there are few things that can lower someone's opinion of another person's cognitive function faster than not remembering simple things.

Ironically for systems that can remember much more data much more permanently and accurately than we ever could, computers often seem to remember very little.  For example, I just tried a couple of online AI chatbots, including one that claimed to have passed a Turing test.  The conversations went something like this:

Me: How are you?
Bot: I'm good.
Me: That's great to hear.  My name is Fred.  My cousin went to the store the other day and bought some soup.
<a bit of typical AI bot chat, pattern-matching what I said and parroting it back, trying stock phrases etc.>
Me: By the way, I just forgot my own name.  What was it?
<some dodge, though one did note that it was a bit silly to forget one's own name>
Me: Do you remember what my cousin bought the other day?
<some other dodge with nothing to do with what I said>

The bots are not even trying to remember the conversation, even in the rudimentary sense of scanning back over the previous text.  They appear to have little to no memory of anything before the last thing the human typed.

Conversely, web pages suddenly got a lot smarter when sites started using cookies to remember state between visits and again when browsers started to be able to remember things you'd typed in previously.  There's absolutely nothing anyone would call AI going on, but it still makes the difference between "dumb computer" and "not so bad".

When I say "memory" here, I mean the memory of things that happen while the program is running.  Chess engines often incorporate "opening books" of positions that have occurred in previous games, so they can play the first few moves of a typical game without doing any calculation.  Neural networks go through a training phase (whether guided by humans or not).  One way or another, that training data is incorporated into the weightings that determine the networks behavior.

In some sense, those are both a form of memory -- they certainly consume storage on the underlying hardware -- but they're both baked in beforehand.  A chess engine in a tournament is not updating its opening book.  As I understand it, neural network-based chess engines don't update their weights while playing in a tournament, but can do so between rounds (but if you're winning handily, how much do you really want to learn from your opponents' play?).

Likewise, a face recognizer will have been trained on some particular set of faces and non-faces before being set loose on your photo collection.  For better or worse, its choices are not going to change until the next release (unless there's randomization involved).

Chess engines do use memory to their advantage in one way: they tend to remember a "cache" of positions they've already evaluated in determining previous moves.  If you play a response that the engine has already evaluated in detail, it will have a head start in calculating its next move.  This is standard in AB engines, at least (though it may be turned off during tournaments).  I'm not sure how much it applies for NN engines.   To the extent it does apply, I'd say this absolutely counts as "memory makes you smarter".

Overall, though, examples of what we would typically call memory seem to be fairly rare in AI applications.  Most current applications can be framed as processing a particular state of the world without reference to what happened before.  Recognizing a face is just recognizing a face.

Getting a robot moving on a slippery surface is similar, as I understand it.  You take a number of inputs regarding the position and velocity of the various members and whatever visual input you have, and from that calculate what signals to send to the actuators.  There's (probably?) a buffer remembering a small number of seconds worth of inputs, but beyond that, what's past is past (for that matter, there's some evidence that what we perceive as "the present" is basically a buffer of what happened in the past few seconds).

Translating speech to text works well enough a word or phrase at a time, even if remembering more context might (or might not) help with sorting out homonyms and such.   In any case, translators that I'm familiar with clearly aren't gathering context from previous sentences.  It's not even clear they can remember all of the current sentence.

One of the most interesting things about the hide-and-seek demo is that its agents are capable of some sort of more sophisticated memory.  In particular, they can be taught some notion of object permanence, usually defined as the ability to remember that objects exist even when you can't see them directly, as when something is moved behind an opaque barrier.  In purely behavioral terms, you might analyze it as the ability to change behavior in response to objects that aren't directly visible, and the hide-and-seek agents can definitely do that.  Exactly how they do that and what that might imply is what I'm really trying to get to here ...

Intelligence and intelligence

I've been meaning for quite a while to come back to the hide-and-seek AI demo, but while mulling that over I realized something about a distinction I'd made in the first post.  I wanted to mention that brief(-ish-)ly in its own post, since it's not directly related to what I wanted to say about the demo itself.

In the original post, I distinguished between internal notions of intelligence, concerning what processes are behind the observed behavior, and external notions which focus on the behavior itself (note to self: find out what terms actual cogsci/AI researchers use -- or maybe structural and functional would be better?).

Internal definitions on the order of "Something is intelligent if it's capable of learning and dealing with abstract concepts" seem satisfying, even self-evident, until you try to pin down exactly what is meant by "learning" or "abstract concept".  External definitions are, by construction, more objective and measurable, but often seem to call things "intelligent" that we would prefer not to call intelligent at all, or call intelligent in a very limited sense.

The classic example would be chess (transcribing speech and recognizing faces would be others).  For quite a while humans could beat computers at chess, even though even early computers could calculate many more positions than a human, and the assumption was that humans had something -- abstract reasoning, planning, pattern recognition, whatever -- that computers did not have and might never have.  Therefore, humans would always win until computers could reason abstractly, plan, recognize patterns or whatever else it was that only humans could do. In other words, chess clearly required "real intelligence".

Then Deep Blue beat Kasparov through sheer calculation, playing a "positional" style that only humans were supposed to be able to play.  Clearly a machine could beat even the best human players at chess without having anything one could remotely call "learning" or "abstract concepts".  As a corollary, top-notch chess-playing is not a behavior that can be used to define the kind of intelligence we're really interested in.

This is true even with the advent of Alpha Zero and similar neural-network driven engines*. Even if we say, for the sake of the argument, that neural networks are intelligent like we are, the original point still holds.  Things that are clearly unintelligent can play top-notch chess, so "plays top-notch chess" does not imply "intelligent like we are".  If neural networks are intelligent like we are, it won't be because they can play chess, but for other reasons.

The hide-and-seek demo is exciting because on the one hand, it's entirely behavior based.  The agents are trained on the very simple criterion of whether any hiders are visible to the seekers.  On the other hand, though, the agents can develop capabilities, particularly object permanence, that have been recognized as hallmarks of intelligence since before there were computers (there's a longer discussion behind this, which is exactly what I want to get to in the next post on the topic).

In other words, we have a nice, objective external definition that matches up well with internal definitions.  Something that can

• Start with only basic knowledge and capabilities (in this case some simple rules about movement and objects in the simulated environment)
• Develop new behaviors in a competition against agents with the same capabilities

is pretty clearly intelligent in some meaningful sense, even if it doesn't seem as intelligent as us.

If we want to be more precise about "develop new behaviors", we could either single out particular behaviors, like fort building or ramp jumping, or just require that any new agent we're trying to test starts out by losing heavily to the best agents from this demo but learns to beat them, or at least play competitively.

This says nothing about what mechanisms such an agent is using, or how it learns.  This means we might some day run into a situation like chess where something beats the game without actually appearing intelligent in any other sense, maybe some future quantum computer that can simultaneously try out all a huge variety of possible strategies.  Even then, we would learn something interesting.

For now, though, the hide-and-seek demo seems like a significant step forward, both in defining what intelligence might be and in producing it artificially.

* I've discussed Alpha Zero and chess engines in general at length elsewhere in this blog.  My current take is that the ability of neural networks to play moves that appear "creative" to us and to beat purely calculation based (AB) engines doesn't imply intelligence, and that the ability to learn the game from nothing, while impressive, doesn't imply anything like what we think of as human intelligence, even though it's been applied to a number of different abstract games.  That isn't a statement about neural networks in general, just about these particular networks being applied to the specific problem of chess and chess-like games.  There's a lot of interesting work yet to be done with neural networks in general.