Embedding and meaning

I a previous post entitled Experiences, mechanisms, behaviors and LLMs, I discussed a couple of strawman objections to the idea that an LLM isn't doing anything particularly intelligent: that it's "just manipulating text" and it's "just doing calculations".

The main argument was that "just" is doing an awful lot of work there. Yes, an LLM is "just" calculating and manipulating text, but it's not "just" doing so in the same way as an early system like ELIZA, which just turned one sentence template into another, or even a 90s-era Markov chain, which just generates text based on how often which words appeared directly after which others in a sample text.

In both of those cases, we can point at particular pieces of code or data and say "those are the templates it's using", or "there's the table of probabilities" and explain directly what's going on. Since we can point at the exact calculations going on, and the data driving them, and we understand how those work, it's easy to say that the earlier systems aren't understanding text the way we do.

We can't do that with an LLM, even if an LLM generating text is doing the same general thing as a simple Markov chain. We can say "here's the code that's smashing tensors to produce output text from input text", and we understand the overall strategy, but the data feeding that strategy is far beyond our understanding. Unlike the earlier systems, there's way, way too much of it. It's structured, but that structure is much too complex to fit in a human brain, at least as a matter of conscious thought. Nonetheless, the actual of behavior shows some sort of understanding of the text without having to stretch the meaning of the word "understanding".

In the earlier post, I also said that even if an LLM encodes a lot about how words are used and in which contexts -- which it clearly does -- the LLM doesn't know the referents of those words -- it doesn't know what it means for water to be wet or what it feels like to be thirsty -- and so it doesn't understand text in the same sense we do.

This feels similar to appeals like "but a machine can't have feelings", which I generally find fairly weak, but that wasn't quite the argument I was trying to make. While cleaning up a different old post (I no longer remember which one), I ran across a reference that sharpens the picture by looking more closely at the calculations/manipulations an LLM is actually doing.

I think the first post I mentioned, on experiences etc. puts a pretty solid floor under what sort of understanding an LLM has of text, namely that it encodes some sort of understanding of how sentences are structured and how words (and somewhat larger units) associate with each other. Here, I hope to put a ceiling over that understanding by showing more precisely in what way LLMs don't understand the meaning of text in the way that we do.

Taking these together, we can roughly say that LLMs understand the structure of text but not the meaning, but the understanding of structure is deep enough that an LLM can extract information from a large body of text that's meaningful to us.

In much of what follows, I'm making use of an article in Quanta Magazine that discusses how LLMs do embeddings, that is, how they turn a text (or other input) into a list of vectors to feed into the tensor-smashing machine. It matches up well with papers I've read and a course I've taken, and I found it well-written, so I'd recommend it even if you don't read any further here.

Despite the name, a Large Language Model doesn't process language directly. The core of an LLM drives the processing of a list of tokens. A token is a vector -- an ordered list of numbers of a given length -- that represents a piece of the actual input.

To totally make up an example, suppose vectors are three numbers long. If the word a maps to (1.2, 3.0, -7.5), list maps to (6.4, -3.2, 1.6), of maps to (27.5, 9.8, 2.0),  and vectors maps to (0.7, 0.3, 6.8), then a list of vectors maps to [(1.2, 3.0, -7.5), (6.4, -3.2, 1.6), (27.5, 9.8, 2.0), (0.7, 0.3, 6.8)].

Here I'm using parentheses for vectors, which in this case always have three numbers, and square brackets for lists, which can have any length (including zero for the empty list, []). In practice, the vectors will have many more than three components. Thousands is typical. The list of vectors encoding a text will be however long the text is.

The particular mapping from input to tokens is called the embedding^*.   The overall idea is to encode similarities along various dimensions. There are (practically) infinitely many ways to do this mapping. Over time this has evolved from a mostly-manual process, to an automated process using hand-written code, to the current state of the art, which uses machine learning techniques on large bodies of text. The first two approaches are pretty easy to understand.

An ML-produced embedding (that is, the procedure for turning an actual list of words into tokens), on the other hand, relies on a mass of numbers created during a training phase. This mass of numbers drives a generic algorithm that turns words into large vectors. While the numbers themselves don't really lend themselves to easy analysis, people have noticed interesting patterns in the results of applying embedding.

Because the model-building phase is looking at streams of text, it's not surprising that the embedding itself captures information about what words appear in what contexts in that text. For example in typical training corpora, dog and cat appear much more often in contexts like my pet ___ than, say, chair does. They are also likely to occur in conjunction with terms like paw and fur, while other words won't, and so forth.

While we don't really understand exactly how the embedding-building stage of training an LLM extracts relations like this, the article in Quanta gives the example that in one particular embedding the vector for king minus the one for man plus the one for woman is approximately equal to the one for queen (you add or subtract vectors component by component, so (1.2, 3.0, -7.5) + (6.4, -3.2, 1.6) = (7.6, -0.2, -5.9) and so on).

It's long been known that use in similar contexts correlates with similarity in meaning. But we're talking about implied similarities in meaning here, not actual meanings.  You can know an analogy like cat : fur :: person : hair without knowing anything about what a cat is, or a person, or fur or hair.

That may seem odd from our perspective. A person would solve a problem like cat : fur :: person : ? by thinking about cats and people, and what about a person is similar to fur for a cat, because we're embodied in the world and we have experience of hair, cats, fur and so forth. Odd as it might seem to know that cat : fur :: person : hair without knowing what any of those things is, that's essentially what's going on with an LLM. It understands relations between words, based on how they appear in a mass of training text, but that's all it understands^**.

But what, exactly, is the difference between understanding how a word relates to other words and understanding what it means? There are schools of thought that claim there is no difference. The meaning of a word is how it relates to other words. If you believe that, then there's a strong argument that an LLM understands words the same way we do, and about as well as we do.

Personally, I don't think that's all there is to it. The words we use to express our reality are not our reality. For one thing, we can also use the same words to express completely different realities. We can use words in new ways, and the meaning of words can and does shift over time. There are experiences in our own reality that defy expression in words.

Words are something we use to convey meaning, but they aren't that meaning. Meaning ultimately comes from actual experiences in the real world. The way words relate to each other clearly captures something about what they actually mean -- quite a bit of it, by the looks of things -- but just as clearly it doesn't capture everything.

I have no trouble saying that the embeddings that current LLMs use encode something significant about how words relate to each other, and that the combination of the embedding and the LLM itself has a human-level understanding of how language works.

That's not nothing. It's something that sets current LLMs apart from anything before them, and it's an interesting result. For one thing, it goes a long way toward clarifying what's understanding of the world and what's just understanding of how language works and what combinations of words people actually use.

If an LLM is good at it, then it's something about how language works. If an LLM isn't good at it, then it's probably something about the world itself. I'll have a bit more to say about that in the next (shorter) post.

Because LLMs know about language, but not what it represents in the real world, we shouldn't be surprised that LLMs hallucinate, and we shouldn't expect them to stop hallucinating just because they're trained on larger and larger corpora of text.

The earlier post distinguished among behavior, mechanism and experience. An LLM is capable of linguistic behavior very similar to a person's.

The mechanism of an LLM may, or may not, be similar as far as language processing. We may well learn rules like the way that we use the in relation to nouns in a way that's similar to training an LLM. Whether that's the case or not, an LLM, by design, lacks a mechanism for tying words to anything in the real world. This probably accounts for much of the difference between what we would say and what an LLM would say.

All of this is separate from subjective experience.  One could imagine a robot that builds up a store of interactions with the world, processes them into some more abstract representation and associates words with them. But even if that is more similar to what we do in terms of mechanism, it says nothing about what the robot might or might not be experiencing subjectively, even if it becomes harder to rule out the possibility that the robot is experiencing the world as we do.

^* Wikipedia seems to think it's only an embedding if it's done using feature learning, but that seems overly strict. Mathematically, an embedding is any map from one domain into another, no matter how it's produced.

^** Technically, it might matter what the actual numbers are, for example, an embedding that doubled every numeric value or added (1.0, 2.0, 3.0) to every token might produce different results. I'm quietly assuming that models are insensitive to this kind of change of coordinates. If you buy that, then it's relations like king - man + woman ~= queen that matter, and not the actual numeric values that king, man, woman and queen map to. Even if that's not the case, I don't think that changes the overall argument that nothing in an embedding or a model is even trying to capture anything about referents in the real world.

What part of consciousness is social?

I think a lot of questions about consciousness fall into one of two categories:

• What is it, that is, what features does it have, what states of consciousness are there, what are reasonable tests of whether something is conscious or not (given that we can't directly experience any consciousness but our own)?
• How does it happen, that is, what causes things (like us, for example) to have conscious experiences?

Reading that over, I'm not sure it really captures the distinction I want to make.  The first item deals in experiments people know how to do right now, and there has been quite a lot of exciting work on the first type of question, falling under rubrics like "cognitive science" and "neural correlates of consciousness".

I mean for the second item to represent "the hard problem of consciousness", the "Why does anyone experience anything at all?" kind of question.  It's not clear whether one can conduct experiments about questions like this at all and, as far as I know, no one has an answer to that isn't ultimately circular.

For example, "We have consciousness because we have a soul" by itself doesn't answer "What is a soul?" and "How does it give us consciousness?" or clearly suggest an experiment that could confirm or refute it.  Instead, it states a defining property (typically among others): A soul is something which gives us consciousness.  The discussion doesn't necessarily end there, but if there's an answer to How does consciousness happen in it, it's not in the mere assertion that souls give us consciousness.

Similarly, if we substitute more mechanistic terms like "quantum indeterminacy" or "chaos of non-linear systems" or whatever else for "soul" in "We have consciousness because ...", we haven't explained why that leads to the subjective experience of consciousness or provided a way to test the assertion.  We may well be able to demonstrate that some aspect or another of consciousness is associated with some structure -- some collection of neurons, one might expect -- where quantum indeterminacy or chaos plays a significant role, but that doesn't explain why that structure correlates with consciousness rather than being just another structure along with the gall bladder, earlobe or whatever else.

If we were able to pinpoint some complex of neural circuits that fire exactly when a person is conscious, or perhaps more realistically, in a particular state of waking consciousness, or consciousness of a particular experience, it would be tempting, then, to say "Aha! We've found the neural circuits that cause consciousness," but that's not really accurate, for a couple of reasons.

First, correlation doesn't imply cause, which is why we speak of neural correlates of consciousness, not causes.  Second, even if there's a good case that the neural pattern we locate really is a cause -- for example, maybe it can be demonstrated that if the pattern is disrupted the person loses consciousness, as opposed to the other way around -- we still don't know what is causing a person to have the subjective experience of consciousness.  We can talk with some confidence about patterns of neurons firing, or even of subjects reporting particular experiences, but we can't speak with confidence about people actually experiencing things.

If we didn't already know that subjective experiences existed (or, at least, I know my subjective experiences exist), there's nothing about the experiment that would tell us that they did, much less why.  All we know is that if neurons are firing in such-and-such a state, the subject reports conscious experiences.

Since we do experience consciousness, it's blindingly obvious to us that the subject must be as well, but again that just shifts the problem back a level: We're convinced that we have found something that causes the subject to experience what we experience, but that doesn't explain why we experience anything to begin with.  If we were all "philosophical zombies" that exhibited all the outward signs of consciousness without actually experiencing it, the experiment would run exactly the same -- except that no one would actually experience it happening.

That's more than I meant to say about the second bullet point.  I actually meant to explore the first one, so let's try that.

Suppose you're hanging out in your hammock on a pleasant afternoon (note to self: how did I let the summer go by without that?).  You hear the wind in the trees, maybe birds chirping or dogs barking or kids playing, or cars going by, or whatever.  You are alone with your own thoughts, but for a while even those die down and you're just ... being.  Are you conscious?  Unless you've actually drifted off to sleep, I think most people would answer yes.  If someone taps you on your shoulder or shouts your name, you'll probably respond, though you might be a bit slow to come back up to speed.  If it starts to rain, you'll feel it.  If something makes a loud noise and you manage to regain your meditative state, you're still liable to remember the noise.

On the other hand, it's something of a different state of consciousness than much of our usual existence.  There's nothing verbal going on.  There's no interaction with other people, none of the constant evaluation  (much of which we're generally not aware of) concerning what people might be thinking, or whether they heard or understood you, or whether you're understanding them, or what their motives might be, or their opinions of you or others around, or what they might be aware of or unaware of.  You're not having an inner conversation with yourself or that jerk who cut you off at the intersection, and there's little to no self consciousness, if you're only focusing on the sensory experience of the moment (indeed, this is a major reason people actively seek such a meditative state).

I've become more and more convinced over time that we often underestimate how conscious other beings are.  I don't subscribe to the sort of literal panpsychism that holds that a brick has a consciousness, that "It is something (to a brick) to be a brick".  I doubt this is a particularly widely held position anyway, so much as the anchor at one end of a spectrum between it and "nothing is actually conscious at all".  However, I am open to the idea that anything with a certain minimum complement of capabilities which can be measured fairly objectively, including particularly senses and memory, has some sort of consciousness, and, as a corollary, that there are many different kinds or components of consciousness that different things have at different times.

For example, a hawk circling over a field waiting for a mouse to pop out of its burrow likely has some sort of experience of doing this, and if it spots a mouse, it has some sort of awareness of there now being prey to pursue with the goal of eating it or, if there are no mice, an awareness of being hungry.  This wouldn't be awareness on a verbal, reflective level we experience when we notice we are hungry and tell someone about it, but something more akin to that "I'm relaxing in a hammock and things are just happening" kind of awareness.  I also wouldn't claim that this awareness is serving any particular purpose.  Rather, it's a side effect of having the sort of mental circuitry a hawk has and being embodied in a universe where time exists -- another mystery that may well be deeply connected to the hard problem of consciousness.

I think this is in some sense the simplest hypothesis, given that we have the same general kind of neural machinery as hawks and that we can experience things happening.  It still presupposes that there's some sort of structural difference between things with at least some subjective experiences and things with no such experiences at all, but that "something" becomes a fairly general and widely-shared capacity for sensing the world and retaining some memory of it rather than a specialized facility unique to us.  The difference between us and a hawk is not that we're conscious and hawks aren't, but that we have a different set of experiences from hawks.  For the most part this would be a larger set of experiences, but, if you buy the premise of hawks having experiences at all, there are almost certainly some that they have but we don't.

Which leads me back to the title of this post.

I suspect that if you polled a bunch of people about consciousness in other animals, you'd see more "yes" answers to "is a chimpanzee conscious" or "is a dog conscious" than to "is a hawk conscious" or "is a salmon conscious".  Some of this is probably due to our concept of intelligence in other animals.  Most people probably think that chimps and dogs are "smart animals", while hawks and salmon are "just regular animals".

However, I think our judgment of that is strongly colored by chimps and dogs being more social animals than hawks or fish (even fish that school are probably not social in the same way we are -- I'd go into why I think that, but this post is already running a bit long).  It doesn't take much observation of chimps and dogs interacting with their own species and with humans to conclude that they have some awareness of individual identities and social structure, the ability to persuade others to do what they want (or at least try), and other aspects of behavior that are geared specifically toward interaction with those around them.  Other animals do interact with each other, but social animals like chimps, dogs and humans normally do so on a daily basis as a central part of life.

This social orientation produces its own set of experiences beyond "things are happening in the physical world" experiences like hunger and an awareness that some potential food just popped out of a burrow.  I think it's this particular kind of experience that we tend to gravitate toward when we think of conscious experience.  More specifically, self-awareness is often held out as the hallmark of "true consciousness", and I think there's a good case that self-awareness is closely connected to the sort of "what is that one over there thinking and what do they want" calculation that comes of living as a social animal.

To some extent this is a matter of definition.  If you define consciousness as self-awareness, then it's probably relatively rare, even if several species are able to pass tests like the mirror test (Can the subject tell that the animal in the mirror is itself?).  However, if you define consciousness as the ability to have subjective experiences, then I think it's hard to argue that it's not widespread.  In that formulation, self-awareness is a particular kind of subjective experience limited to relatively few kinds of being, but only one kind of experience among many.

The stuff of dreams

... and then our hero woke up and it was all a dream ...

... has to rank among the most notorious pulled-out-of-thin-air deus ex machina twist endings in the book, along with "it was actually twins" and "it was actually the same person with multiple personalities".  As with all such tropes, there's nothing wrong with these plot twists per se.  The problem is generally the setup.

In a well-written "it was actually twins" twist, you have clues all along that there were actually two people -- maybe subtle shifts in behavior, or a detail of clothing that comes and goes, or the character showing up in an unexpected place that it seemed unlikely they'd be able to get to.  With a good setup, you're reaction is "Oh, so that's why ..." and not "Wait ... what?  Seriously?"

The same goes for "it was all a dream".  In a good setup, there are clues that it was all a dream.  Maybe things start out ok, then something happens that doesn't quite make sense, then towards the end things get seriously weird, but more in a "wait, what's going on here, why did they do that?" kind of way, as opposed to a "wait, was that a flying elephant I just saw with a unicyclist on its back?" kind of way, though that can be made to work as well.

There's a skill to making things dreamlike, particularly if you're trying not to give the game away completely.  Dream logic doesn't just mean randomly bizarre things happening.  Dreams are bizarre in particular ways which are not particularly well understood, even though people have been talking about and interpreting dreams probably for as long as there have been talking and dreams.

A while ago I ran across a survey by Jennifer Windt and Thomas Metzinger that has quite a bit to say about dreams and the dream state, both ordinary dreams and "lucid" dreams where the rules are somewhat different.  They compare the three states of ordinary dreaming, lucid dreaming and waking consciousness to try to tease out what makes each one what it is with, I found, fair success.  I'm not going to go into a detailed analysis of that paper here, but I did want to acknowledge it, if only as a starting point.


First, though, some more mundane observations about dreams.  We tend to dream several times a night, in cycles lasting around 90 minutes.  We don't typically remember this, but a subject who is awakened while exhibiting signs of a dream state can generally recall dreaming while a subject awakened under other conditions doesn't.  The dream state is marked by particular patterns of electrical activity in the brain, near-complete relaxation of the skeletal muscles and, probably best-known, Rapid Eye Movement, or REM.  REM is not a foolproof marker, but the correlation is high.

Dreams in early sleep cycles tend to be closely related to things that happened during the waking day.  Subjects who studied a particular skill prior to going to sleep, for example, tended to have dreams about that skill.  I've personally had dreams after coding intensely that were a sort of rehash of how I'd been thinking about the code in question, not so much in the concrete sense of writing or reading particular pieces as more abstractly navigating data and control structures.

Later dreams -- those closer to when you wake up -- tend to be more emotional and less closely associated with recent memories.  Since these are more likely to be the ones you remember unless someone is waking you up as part of a sleep experiment, these are the kind of dreams we tend to think of as "dreamlike".  These are the "I was in this restaurant having dinner with such-and-such celebrity, except it didn't look like them, and I could hear my third-grade teacher yelling something, but everyone just ignored it and then a huge ocean wave came crashing in and we all had to swim for it, even though the restaurant was in the Swiss Alps" kind of dreams.

In my experience this kind of dream can often be linked back to relevant events, but in a sort of mashed-up, piecemeal, indirect way.  Maybe you heard a news story about a tidal wave yesterday and a couple of days ago some relative or old friend had mentioned something that happened to you in grade school.  Celebrities, by definition, are frequently in the news, and it was the Swiss Alps just because.  That doesn't really explain what the dream might mean, if indeed it meant anything, but it does shed some light on why those particular elements might have been present.

But why that particular assemblage of elements?  Why wasn't the third grade teacher your dinner companion?  Why did all the other diners ignore the teacher?  Why wasn't the restaurant on the beach? And so on.

My personal theory on such things is pretty unsatisfying: it just is.  Whatever part of the mind is throwing dream elements together is distinct from the parts of the mind concerned with cause and effect and pulling together coherent narratives.

To draw a very crude analogy, imagine memory as a warehouse.  From time to time things have to be shuffled around in a warehouse in for various logistical reasons.  For example, if something that's been stored in the back for months now needs to be brought out, you may have to move other items around to get at it.  Those items were put there for their own reasons that may not have anything to do with the item that's being brought out.

Now suppose someone from management in a different part of the company -- say media relations -- comes in and starts observing what's going on.  A pallet of widgets gets moved from section 12D, next to the gadgets, to section 4B, next to the thingamajigs.  This goes on for a while and our curious manager may even start to notice patterns and make tentative notes on them.

Suppose upper-level management demands, for its own inscrutable reasons, a press release on the warehouse activity.  The media relations person writing the release is not able to contact the warehouse people to find out what's really going on and just has to go by the media relations manager's notes about widgets moving from next to the gadgets to next to the thingamajigs.  The resulting press release is going to try to tell a coherent story, but it's not going to make much sense.  It's almost certainly not going to say "We had to get the frobulator out of long-term storage for an upcoming project so we moved a bunch of stuff to get at it."

My guess is that something similar is going on in the brain with dreams.  In normal waking consciousness, the brain is receiving a stream of inputs from the outside world and putting them together into a coherent picture of what's going on.  There are glitches all the time for various reasons. The input we get is generally incomplete and ambiguous.  We can only pay attention to so much at a time.

In order to cope with this we constantly make unconscious assumptions based on expectations, and these vary from person to person since we all have different experiences.  The whole concept of consciousness is slippery and by no means completely understood, but for the purpose of this post consciousness (as opposed to any particular state of consciousness) means whatever weaves perception into a coherent picture of what's going on.

Despite all the difficulties in turning perception into a coherent reality, we still do pretty well.  Different people perceiving the same events can generally agree on at least the gist of what happened, so in turn we agree that there is such a thing as "objective reality" independent of the particular person observing it.  Things fall down.  The sun rises in the morning.  It rains sometimes.  People talk to each other, and so on.  Certainly there's a lot we disagree on, sometimes passionately with each person firmly believing the other just doesn't know the simple facts, but this doesn't mean there's no such thing as objective reality at all.



In the dream state, at least some of the apparatus that builds conscious experience is active, but it's almost completely isolated from the outside world (occasionally people will incorporate outside sounds or other sensory input into a dream, but this is the exception).  Instead it is being fed images from memories which, as in the warehouse analogy, are being processed according to however memory works, without regard to the outside world.  Presented with this, consciousness tries to build a narrative anyway, because that's what it does, but it's not going to make the same kind of sense as waking consciousness because it's not anchored to the objective, physical world.

If the early memory-processing is more concerned with organizing memories of recent events, early-cycle dreams will reflect this.  If later memory processing deals in larger-scale rearrangement and less recent, less clearly correlated memories, later-cycle dreams will reflect this.


As I understand it, Windt and Metzinger's analysis is broadly compatible with this description, but they bring in two other key concepts that are important to understanding the various states of consciousness: agency and phenomenal transparency.

Agency is just the power to act.  In waking consciousness we have a significant degree of agency.  In normal circumstances we can control what we do directly -- I choose to type words on a keyboard.  We can influence the actions of others to some extent, whether by force or persuasion.  We can move physical objects around, directly or indirectly.  If I push over the first domino in a chain, the others will fall.

In a normal dream the dreamer has no agency.  Things just happen.  Even things that the dreamer experiences as doing just happen.  You can recall "I was running through a field", but generally that's just a fact.  Even if your dream self decides to do something, as in "The water was rushing in so I started swimming", it's not the same as "I wanted to buy new curtains so I looked at a few online and then I picked these out".  Your dream self is either just doing things, or sometimes just doing things in a natural reaction to something that happened.

Even that much is a bit suspect.  It wouldn't be a surprise to hear "... a huge ocean wave came crashing in and then I was walking through this city, even though it was underwater".  In some fundamental way, in a dream you're not making things happen.  They just happen.

Likewise, one of the most basic forms of agency is directing one's attention, but in a dream you don't have any choice in that, either.  Instead, attention is purely salience based, meaning, more or less, that in a dream your attention is directed where it needs to be -- if that ocean wave bursts in you're paying attention to the water -- rather than where you want it to be.

Phenomenal transparency concerns knowing what state of consciousness you're in.  Saying that dreaming is phenomenally transparent is just a technical way of saying "when you're in a dream you don't know you're dreaming" (So why coin such a technical term for such a simple thing?  For the usual reasons.  On the one hand, repeating that whole phrase every time you want to refer to the concept -- which will be a lot if you're writing a paper on dreaming -- is cumbersome at best.  It's really convenient to have a short two-word phrase for "the-quality-of-not-knowing-you're-dreaming-when-you're dreaming".  On the other hand, defining a phrase and using it consistently makes it easier for different people to agree they're talking about the same thing.  But I digress.)

If someone is recalling a dream, they don't recall it as something that they dreamed.  The recall it as something that happened, and happened in a dream.  It "happened" just the same as something in waking consciousness "happened".  During the dream itself, it's completely real.  Only later, as we try to process the memory of a dream, do we understand it as a dream.  I've personally had a few fairly unsettling experiences of waking up still in a dreamlike state and feeling some holdover from the dream as absolutely real, before waking up completely and realizing ... it was all a dream (more on this below).  I expect this is something most people have had happen and this is why the "it was all a dream trope" can work at all.

In some sense this seems related to agency.  When you say "I dreamed that ..." it doesn't mean that you consciously decided to have thus-and-such happen in your dream.  It means that you had a dream, and thus-and-such happened in it.

Except when  it doesn't ...

Windt and Metzinger devote quite a bit of attention to lucid dreams. While the term lucid might suggest vividness and clarity, and this can happen, lucidity generally refers to being aware that one is dreaming (phenomenal transparency breaks down).  Often, but not always, the dreamer has a degree of control (agency) over the action of the dream.  In a famous experiment, lucid dreamers were asked to make a particular motion with their eyes, something like "when you realize you're in a dream, look slowly left and then right, then up, then left and right again", something that would be clearly different from normal REM.  Since the eyes can still move during a dream, even if the rest of the body is completely relaxed, experimenters were able to observe this and confirm that the dreamers were indeed aware and able to act.

Not everybody has lucid dreams, or at least not everyone is aware of having had them.  I'm not sure I've had any lucid dreams in the "extraordinarily clear and vivid" sense, but I've definitely had experiences drifting off to sleep and working through some problem or puzzle to solve, quite consciously, but blissfully unaware that I'm actually asleep and snoring.  I've also had experiences waking up where I was able to consciously replay what had just been happening in a dream and at least to some extent explore what might happen next.  I'm generally at least somewhat aware of my surroundings in such cases, at least intermittently, so it's not clear what to call dreaming and what to call remembering a dream.

In any case, I think this all fits in reasonably well with the idea of multiple parts of the brain doing different things, or not, none of them in complete control of the others.  Memory is doing whatever memory sorting it needs to do during sleep (it's clear that there's at least something essential going on during sleep, because going without sleep for extended periods is generally very bad for one's mental health).  Some level of consciousness's narrative building is active as well, doing its best to make sense of the memories being fed to it.  Some level of self awareness that "I'm here and I can do things" may or may not be active as well, depending on the dreamer and the particular circumstances.

This is nowhere near a formal theory of dreams.  Working those out is a full-time job.  I do think it's interesting, though, to try to categorize what does and doesn't happen in dream states and compare that to normal waking consciousness.  In particular, if you can have A happen without B happening and vice versa, then in some meaningful sense A and B are produced by different mechanisms.

If we draw up a little table of what can happen with or without what else...

Can there be ... without ...	Consciousness	Agency	Phenomenal transparency
Consciousness	—	yes1	yes2
(Conscious) Agency	no	—	?
Phenomenal transparency	no	yes3	—

¹ In ordinary dreams, but also, e.g., if paralyzed by fear

² In ordinary dreams

³ In a lucid dream, if you're aware that you're dreaming but can't influence the dream

... it looks like things are pretty wide open.  I didn't mention it in the table, but agency doesn't require consciousness.  We do things all the time without knowing why, or even that, we're doing them.  However, conscious agency requires consciousness by definition.  So does phenomenal transparency -- it's consciousness of one's own state.

Other than that, everything's wide open except for one question mark: Can you have conscious agency without phenomenal transparency?  That is, can you consciously take an action without knowing whether you're awake or dreaming (or in some other mental state).  This isn't clear from lucid dreaming, since lucid dreaming means you know you're dreaming.  It isn't clear from ordinary dreaming.  Ordinary dreams seem passive in nature.

In a related phenomenon, though, namely false awakening, the dreamer can, while actually remaining asleep, awaken and start to do ordinary things.  In some cases, the dreamer becomes aware of the dream state, but in other cases the illusion of being awake lasts until the dreamer awakens for real.

All of this is just a long way of saying that our various faculties like consciousness, agency and awareness of one's state of consciousness seem to be mix and match.  The normal states are waking consciousness and ordinary dreaming, but anything between seems possible.  In other words, while these faculties generally seem to operate either together (waking consciousness), or with only consciousness (ordinary dreaming) they're actually independent.  It's also worth noting that nothing in the table above distinguishes waking from dreaming.  The difference there would seem to be in whether we're processing the real world or memories of it.

This is an interesting piece of information, one which would have been considerably harder to come by if we didn't have the alternate window into consciousness provided by dreams.

Can we prove a dog is happy?

The previous post talked about qualia, or subjective experiences, but why should we care?  This being a matter of philosophy, there are a variety of answers to that, starting with "Why care about anything?" but nonetheless, there seems to be something significant about the question.  At least from my own subjective point of view.

For one thing, it seems like one of those fundamental questions.  How can we come to a complete understanding of the universe without understanding how we experience it?  Perhaps more than that, there are ethical concerns.  If we wish to increase happiness or we do not wish to cause unnecessary suffering in the world, we should understand what happiness and suffering are.  Outward appearances will only tell us so much.  It would be good to have more reliable indicators, or at least to know how reliable the ones we have are.

The problem with subjective experiences, though, is that they are subjective.  I can be well convinced that my own subjective experience is real.  Sentio ergo sum -- I feel, therefore I am.  There are several reasons for me to believe that someone else's feelings are real: I can see their reactions, they can tell me, and we know that humans have, for the most part, essentially the same neural apparatus.

Nonetheless I cannot know for sure what another person's feelings are in the same way that you and I could both put the same object on a balance scale and agree on its mass.  Each of the common-sense indications I just gave can fail.  Someone may not react visibly to a feeling or experience, or I may not catch the reaction.  They may not be able to tell me for any number of reasons.  Different people can have different ranges of feeling -- what seems intense to me might seem like nothing special to you, or vice-versa.

From a purely philosophical point of view we don't know for sure that having the same kind of neural pathway means having the same kinds of experiences.  Perhaps the ability to experience requires both a certain type of pathway and something else intangible that not everyone has.   Even if there is no such intangible, we're still far from knowing what physical pieces are associated with experience, though we do have some clues.  Without knowing just what pathways gives rise to subjective experience we have no way to be sure everyone has it.

When we go beyond human experience to other species, which react differently, can't verbalize their experiences (or at least not in ways we can presently understand), and have clearly different neural circuitry, we have even less to go on.  We can presume that a dog wagging its tail and barking when its human returns is happy, but it's always possible that dogs have simply co-evolved with us for long enough that they are able to act happy when that would be to their advantage (most people with dogs would dispute this, I expect).

Artificial constructs are even more problematic.  If I build a robot that avoids walls even if you push it toward one, it's easy to say "it doesn't like walls" because it's acting like a sentient being that disliked walls would, but it seems a much bigger step to say "it avoids walls because it experiences negative emotions when it's near one", particularly when we can point to the exact code that causes it to avoid walls.

Even if the code for the control system is extremely complex or has gone through some sort of machine learning process to develop an avoidance of walls, so that we couldn't point to exactly what was making it avoid walls, it still seems hard to argue that the robot is feeling emotions.  If incomprehensible code were the basis of sentience, there would be a lot of sentient software around.

When it comes to what we generally refer to as inanimate objects, the best we can say is that we have no reason to believe that a rock feels pain if we smash it with a hammer.  Nothing in our understanding of how we feel pain seems to apply to something like a rock.  Even so, how can we really know?


But how do we know anything?  We have no way of knowing whether we really live in a universe where the laws of physics hold.  It's possible that tomorrow things dropped will fall up instead of down.  Some theories of cosmology assign a non-zero (but still exceedingly small) chance that we live in such a universe.

In the absence of certain knowledge all we can do is try to build a coherent framework and constantly test and adjust the assumptions it rests on, a process we call "science".  From a scientific point of view we can figure out what sort of neural structures correspond with the subjective experiences that people report.  We can assess whether other organisms have such structures and even whether a particular combination of hardware and software has something functionally equivalent.

We can tell whether something's reactions to various stimuli are consistent with it having such capabilities, based on what people have reported.  We can conclude from that that it's likely or unlikely that the organism or construct we're examining is experiencing feelings, but we can never know for sure, no matter what philosophical machinery we develop for understanding qualia.

But this is nothing new.  Recently it was announced that gravitational waves had finally been detected, stemming from the collision of two black holes over a billion years ago.  The chain of inferences that rests on is mind-boggling.  A more accurate statement would have been "In two separate places, specially constructed instruments registered a signal that indicated that test masses had moved, over a distance much less than the size of an atom, in a way that indicated that space-time had been distorted in a way consistent with the collision of two black holes over a billion light-years away.  We feel confident about this because we believe that science works in general and we're convinced by a large web of observations and theoretical conclusions that the observable universe is billions of years old and billions of light-years in extent, black holes exist and, consistent with a distinct but overlapping web of observations and theoretical conclusions, in certain cases they should produce detectable gravitational waves.  We have also done extensive measurements to convince ourselves that the detectors are in fact detecting gravitational waves and not just trucks driving by ..."

And that would be the short version.  The full version fills textbooks and takes entire careers to grasp even a small portion of.

If science can accept that, can it come to accept that a dog is happy?

Not exactly.  The sticking point here is not whether we can accept a long chain of inference like "People report feeling happy when certain neurons are firing in certain ways, they behave in certain ways when this is happening, dogs have analogous neural pathways, and these tend to fire when dogs are engaged in behavior analogous to that of happy people, and/or people report that the dogs seem happy."  That's not a problem, particularly not compared to the detection of gravitational waves.

The problem is that science depends fundamentally on objective, repeatable measurements of numbers.  Happiness is subjective, and happiness is not a number.  Science can get quite close to measuring happiness, but it's up to us to decide where to go from there -- just like with any other scientific result.

Qualia, or why do we experience anything at all?

Today I'd like to discuss a topic which has baffled (at least some) philosophers for quite some time and which I am even more ill-qualified to address than usual.  Since I'm giving general impressions from general ignorance I'll be citing a few well-known examples without attribution.  You can find a good summary here, or it least it seemed like a good one to me.  Rest assured I'm not claiming to be doing any original work here, just ... conjecturing.

The term qualia has come to encompass experiences, and in particular subjective experiences.  For example, what is it like to see the color red, or what is it to be a bat.  Such experiences seem to be subjective, in that the experience depends, at least in principle, on who's experiencing it.  To take a very old example, cliche but no less valid for being cliche, I have no obvious way of knowing whether you experience the color red in the way I do.  Perhaps you experience it the way I experience the color blue, and vice versa, or perhaps you experience it some completely different way.

For that matter, how do I know that you experience anything?  If you and I are at an intersection, stopped at a red light, I can see you react to the light turning green, but that doesn't mean that you had the same experience I did of seeing a red light and then a green light.  I assume that you experienced the sensation of something red and then something green, and that the color red seemed essentially the same way to you as it did to me, but how would I know?

Suppose you were actually in a self-driving car browsing the news on your phone.  You didn't see the light at all.  Rather, the car's cameras recorded the light changing and the car's control system caused the car to go when the light turned green.  I'm perfectly comfortable saying "The car saw the light change and drove through the intersection when it turned green", anthropomorphizing the car, but that doesn't mean I think the car experienced the colors red and green in anything like the way you or I would (or at least, I think you would).

Trying to account for distinctions like this in some objective way has been referred to as "the hard problem of consciousness", as opposed to easier, more empirical problems like "How does the brain record memories?" or "To what extent are we conscious of our own decisions?"

In some sense it's quite likely that all experiences are distinct.  If I see a red paint chip today and then again tomorrow, I will almost certainly have different associations each time.  The first time might put me in mind of a stop sign, or blood, or a red apple.  The second time I might be more focused on whether it's the same paint chip I saw yesterday.  Likewise, you will almost certainly have different associations than I will even if we're looking at the same chip.

And yet, we would probably all agree that we are experiencing seeing something red, and that it feels like something to have that experience.   Even if there's no emotional response, you're still having some sort of experience.  How do we account for that?

Suppose we could account for every firing of every neuron in the nervous system (including the optic nerve, which is actually doing quite a bit of processing before the signal even gets to the brain).  Have we accounted for the experience?  Suppose that after decades of research we compile an exhaustive list of experiences and how they correlate to brain activity.  We bring in a new subject and scan their neural activity.  Pointing at a display, we say "That pattern of firing always occurs in response to seeing the color red".  We can say "that person is experiencing the color red", but how, exactly, do we know that for sure?

It's not hard to imagine what kind of data would back this up.  We hook hundreds of subjects from all over the world and all walks of life up to our highly-advanced brain scanner, flash colors at them and note the results.  We may even ask them to describe what they're experiencing.  When we see the same patterns for our new subject it's a reasonable inference that their brain is processing the color red, and it's reasonable to expect that if we ask them what they're experiencing, their answer will involve the color red.

That's probably good enough for a cognitive scientist, but not a philosopher.  The philosopher may well insist that you don't know what the subject experienced, but only how they would answer a question.  They -- and for that matter any of your other subjects -- might just as well be philosophical zombies who exhibit all the expected behaviors and responses without actually experiencing anything.  We may know intuitively, but we can't prove that the test subjects aren't just like the self-driving car, only on a more elaborate level.


There are a couple of ways out of this.  One is to deny that qualia exist in any well-defined way.  From a logical point of view, this seems quite plausible.  We can talk about the abstract concept of redness, but in real life we don't experience redness in the abstract.  We experience a particular something red at a particular place and time.  That feels a particular way at that place and time, and quite possibly nothing has ever felt quite the same before or ever will.  Maybe we should just stick to our knitting and figure out what happens in real brains in response to real stimuli.  We can still generalize and define abstractions, but if we want an objective description of the world we have to start with objective data.

And yet, we still experience things, subjectively, each of us (or at least I'm pretty sure about me).

So how do we distinguish between a person at a stop light and a self-driving car?  Maybe we don't need to make a strong distinction.  Maybe we're ... not so different.

There's no particular reason, beyond our innate sense of specialness, to assume that only human beings can have experiences.  If we see a hungry dog, our intuition tells us the dog is experiencing hunger.  Our intuition is probably right.  The dog may not be having exactly the same kind of experience we do, but there's no reason to assume it's a philosophical zombie that only looks like it's experiencing hunger.

One way of handling this is to assert that along with the physical properties of the world -- mass, position, velocity and so forth -- there is an experiential component that's completely distinct but which we might still be able to reason about.  Perhaps we will even discover laws that govern it and develop a comprehensive theory of experience.

One objection to this approach is that it seems to imply panpsychism, the idea that everything has consciousness.  There are already schools of thought that believe exactly that, but the concept doesn't sit particularly well in materialist circles (materialist in the philosophical sense).

However, this seems misguided.  If consciousness in the sense of being able to experience qualia is a property in a way similar to mass being a property of things, that doesn't mean that everything has to have that property.  Just as photons are massless, there's no contradiction in saying a rock is unconscious.

Rather than stating that everything has consciousness, we are asserting that objects can have consciousness, and we are trying to investigate under what circumstances that happens.  However, we are explicitly punting on the question of how it has consciousness.  We are saying that when the conditions are right "it just does", just as when a particle interacts with the Higgs field it has mass* (I believe physics has a more detailed account of this than "it just does", but at some point even physics has to make some base assumptions).

From that point of view it's still reasonable to say that a rock has no feelings or consciousness, but a human does, a dog does and just possibly a self-driving car has some limited degree of consciousness as well.  Moreover we may be able to prove that in the scientific sense of having a coherent theory and data to support it.  If so, it seems this theory will look a lot like a purely material explanation of memory, attention and other aspects of consciousness, together with an assertion that when certain of these are present, the thing in which they are present experiences qualia.

What is it to be a self-driving car?  Probably not much, but perhaps something.

* [That's not a really rigorous way to phrase that, but I don't know well enough to give a better one --D.H.]