The Exchanges

Q So is it fair to say that space and movement have changed how we describe things? Is that fair to say?

A Yes, and in fact, it's, I think it's fair to say that all our spatial representations depend on, on interactions. They're a coupling of what we do and how we perceive it, and those studies go way back, um, a 150 years when, when Stratton and later others put lenses on people's eyes that moved the visuals System or the visual perception, left words or right words, you can do it upside down. And people adapted by moving, by grasping things. And if they didn't grasp things and recalibrate how space was represented in the brain, then they didn't learn. To behave in a new situation. And it's undoubtedly something we're doing continuously all our lives is recalibrating, um, our many visual spatial systems to the way they allow us to behave in the world.

AI assessment note: “Yes, and in fact, it's, I think it's fair to say”

Q shapes language, you're in for a real treat with this episode. We cover her nine laws of cognition, why movement, not language, is the foundation to thought, how language changes how we think, gestures, and the relationship to language, why she dove into the work of Leonardo da Vinci, and the importance of perspective taking. It's time to listen and learn. What get you interested in how the mind works?

A I think everybody is puzzled by the mind. Whether to pursue it or not is a choice and how to pursue it is a choice. When I got into newly birthed cognitive psychology, I had been interested anyway in the visual-spatial world. Um, we are constantly Experiencing it, and so it's a large part of our lives, and most of the theories at that point were based on what philosophers called propositions on language-like representations, and attempting to reduce everything in the visual-spatial world to these propositional statements, which are minimal statements About truth conditions, and that struck me as odd, because the visual spatial world precedes language evolutionarily. It takes up far more of the cortex than language, and it has to have its own logic, not logic based, not a logic or a structural Based on language. If anything, I thought maybe language is based on the way we perceive and interact, perceive space and interact with things in space. So I began looking at various aspects of the visual-spatial world, looking for its own, own structure and own logic, and that brought me back to language. Because we use language to describe and talk about the world, and spatial language seems both primitive and primitive in the sense of its origins, both ontologically and phylogenetically. We talk about where things are, where to look, what to do. Those are all about our interactions with…

AI assessment note: “When I got into newly birthed cognitive psychology, I had been interested anyway in the visual-spatial”

Q Do gestures override language too? Like, I'm thinking of a scenario where somebody says yes, but they're shaking their head no.

A Right. And, and My husband, who was a paratrooper and had an uncanny sense of direction, um, right and left, um, in a, in any language eluded him, as it does many people. Many people confuse right and left for good reason, and he would .1, he'd say go right and point left, and I knew follow the gesture. Um, it's not, um, the language is not, it comes later, Gestures precede language, um, and they're more, more in tune because they're more direct. Susan Goldenmeadow and some of her colleagues, she's done gorgeous work on gesture, especially in children, looked at children on the cusp of, of, um, conceptual thinking. They're going from this sensory motor, so they're, they're, you get Two cups, one is thin, and one is fat, and, and pour water from one to another, and children who are small think that pouring the water changes the amount in the glass, and you ask them to explain, and children on the cusp of getting that idea will gesture something like pouring before they can articulate it. So they've got the concept of conservation, That pouring water doesn't change the quantity. They've got the concept. They can't say it in words, but they can show it in gesture. So their understanding is ahead of their ability to articulate it, which makes sense even for us, um, not just for children. The articulation is an additional layer that's really quite difficult.

AI assessment note: “he'd say go right and point left, and I knew follow the gesture.”

Q I want to get into some of the implications of that in a second. The question that came to mind is that people tend to think of themselves as a visual or verbal or mathematical thinker. Is there any evidence behind that? Or can you be good at more than one of them? Or how does that work?

A Yeah, I mean, the, the disappointing is that you can be good or bad at any of them more or less independently, so that's discouraging in, in some way. You think if you're a visual person, if you're not very good at visual, maybe you're better at verbal, and maybe you are, because we do tend to compensate and use Use media that, um, and channels that were, that we feel more at home and we're more adept in, but they do seem to be pretty independent. And I should say that a lot of our spatial thinking is really not visual. Um, the taxi drivers in London that get you to any of these obscure streets aren't imagining a map as they're navigating. They simply have a spatial sense of Of where things are relative to each other that may not, you might not necessarily always have a conscious experience of them. Blind people navigate quite well, many of them, and it's spatial, not visual, and they'll navigate by other cues, acoustic cues, smells, wind, the texture of the ground. These are all kind of landmarks And cues that keep people without sight oriented, and I know many people who've been blind from birth who, who are blind later, who have amazing spatial abilities. They can be excellent mathematicians and get, get around space better than I can. So space is not just visual, and in fact, the areas of the brain that capture space are multimodal. They're, they get input from many differe…

AI assessment note: “you can be good or bad at any of them more or less independently”

Q so you have to adapt and incorporate learned knowledge. And yeah, I think you, I think you're onto something there. I want to switch gears to part of your book. You, you, Talk about the nine laws of cognition. I think they're more like nine generalizations, but let, let's start with the first one. There are no benefits without cost, meaning creativity versus learning. How do they offset each other?

A So the, the big example there I think is, or one big example is categorizing. So if we awoke anew as a newborn each day, and didn't know how to categorize things into animate, inanimate, parent, friend, um, chair, table, um, these sorts of things that we do, that our brain does quite quickly. If we didn't know how to make categories very quickly, And had to depend on new learning every day. We'd never advance beyond, um, that new learning. And people wonder when they have a baby, what are they looking at? What are they thinking about? Well, they're doing a huge amount of learning about what the world looks like and acts like and sounds like and smells like and what happens in time. These are huge things that Uh, a child needs, a baby needs to learn. Some of it, the proclivities are built in, and even maybe some of the kinds of knowledge, but a great deal of it, words, um, have to be learned, and if we had to go through that learning every day anew, we'd, we'd remain helpless infants. So, we need to categorize, but any kind of categorization Has, um, error in it. We can miscategorize. You know, I've, I've mistaken twins for their sibling and said hello and wondered why they greeted me so coldly. I think that's an experience many people have had. The extreme leads to biases that, you know, I'm, I'm a small woman, so alone at night, I'm very alert. To any kind of danger, and becau…

AI assessment note: “we need to categorize, but any kind of categorization Has, um, error in it.”

Q So does that have implications for how we present? Like, should we be presenting in, uh, more graphically in terms of bullet points and maps and diagrams and relationships, explaining them in, in a spatial sort of way to people?

A I think it's a good idea, and there's more and more of it. Now that pixels don't care if they're representing words or diagrams, and we do have access to many, many, many pixels, and So that making diagrams, it's certainly easier digitally, and then it's easier in printing books, um, before books were typeset, and that gave an advantage to language, whereas diagrammatic things had to be done differently and integrated in a different way, in a, a far more work-intensive way. So there's still a bias, I think, in publishing towards language, Rather than toward diagrams. The bias isn't simply the ease of printing. There are all kinds of reasons. But, I mean, language is great, so I, I use it quite frequently. So, but I do think that diagrams go a long way to conveying conceptual information. There is reams of experiments supporting that. Now, diagrams in science, even in history, for showing time and, and dynasties and so forth, they're, they can be well-crafted, just as texts can be well-crafted or poorly crafted, so you have to take that into consideration, but well-crafted diagrams are superior to language for explaining Many, many, many, um, kinds of information more directly, more succinctly. Diagrams also allow huge numbers of inferences. So, I mean, Venn diagrams for logic, you have intersecting circles, and you can make a great deal of, of a huge number of inferences just f…

AI assessment note: “I think it's a good idea, and there's more and more of it.”

Q change this? So what are the positive and negative aspects of communication over something like zoom versus face to face? And how does communicating over video, which sort of changes our, our spatial perceptions, um, Affect what's communicated and what's understood. Like face-to-face communication uses our whole body, and even the language we use, right? We say this or that, or we look at something to draw attention to it.

A Yeah, no, I'm with you completely, and the research needs to be done, and probably is being done, and obviously it depends on what's being communicated. A big thing for me is that we're only seeing face and shoulder, so we don't have a shared workspace. And if you look at designers or planners or any kind of committee, you often have a whiteboard or something that's central on the table that everybody can point at and add to and subtract from, and that a great deal of the communication and the, the, the joint understanding is externalized in that way. And we've done a small amount of work, and others have too, on collaborative design, and found that when people can work over the same diagram, the communication is much more successful. People are looking at each other's hands on the diagram, not at each other's faces. The words are annotating that diagram. Um, they aren't independent. So people are more, they're, the communication is better, the result is better, um, the product is better. People are happier with the collaboration when they can do that when, then when it's just over the phone and we have, um, separate diagrams. So I think for any kind of collaborative work, and teaching is often collaborative, uh, conversations certainly are, um, Um, but in many cases, we need that external representation of our shared understanding, and there's a huge amount of research and pra…

AI assessment note: “we're only seeing face and shoulder, so we don't have a shared workspace.”

Q So how about the, the inverse of that? How does the physical environment influence how we think? And if the physical environment does influence how we think, how can we design that to improve our thinking?

A You know, so diagrams are one way to improve our thinking in, in, In, in the small, um, there's certainly one, and even written text is a way of improving our thinking, making lists. We have so many, all those ways of externalizing mind can help bring it into the mind and improve it, especially if it's well designed. When you think about the world, you know, I can go into a supermarket in any country, buy food, and put it on the counter. And that's a communication to the, to the person at the cash register that I want to purchase those items. I don't have to talk. It's, it's, um, the way the world is designed and it's universal. Now I have to go to the automatic cash registers and find the barcode and scan it, and I can do that now in any country except China because I can't pay. Because I don't have the right app on my, um, on my phone. It does change our actions in many ways. It makes it more efficient. I can't help but think that babies growing up in a world where, where their homes are organized by themes, and where books are on a shelf in an order, where toys are kept in different categories and so forth, that, that can't help but influence their minds. And I wish someone would study that. Um, you know, I remember my own children, and if they're in the living room, everyone's talking and playing, and I'd pick up a child and walk toward the bedroom. They'd scream because th…

AI assessment note: “all those ways of externalizing mind can help bring it into the mind and improve it”

Q Has language changed how we think too? The language that we use to communicate is also, I would assume, somewhat the language that we use to think in.

A Right, and, and that, I think, goes deep, um, on how, so there's beautiful work on, on rodents and other primates, on how space is represented, the larger space that we move in, not space of hand interaction, but the larger space we knew. And this won the Nobel Prize for, uh, the Mosher's and John O'Keefe, the combinations of their work. So O'Keeffe with Nadel and other people had found in hippocampus of rats single cells that respond when the rat is in particular places. And what's beautiful about these cells is the single cell brings together information about vision, smell, touch, um, the movement of paws on the, on the ground, And so forth. All this multi-sensory information into a single cell, and that encodes a place. They thought, in fact, they wrote a book called The Hippocampus is a Cognitive Map. However, those place cells weren't arrayed spatially. So the problem was still open. How do you get all these place cells related one to another? And the Mosher is working in O'Keeffe's lab, Found grid cells one synapse away in entorhinal cortex that arrayed the place cells relative to one another, essentially a spatial array. Um, and later work, work that came quite a bit later, beautiful work, showed that place, the single cells in hippocampus, also in cold people, Events in time, conceptual relations, and that grid cells map them in temporal space, conceptual space, social…

AI assessment note: “spatial thinking is the foundation of all thought. And I qualify that”

Q Let's talk about some of those modalities, I guess, in the sense of, like, how does this change how we write or how we present material? And maybe we'll keep Zoom separately and some of the implications for that, but when we're writing an email or we're writing a document at work or we're giving a presentation, what are some of the impacts of this?

A I don't have direct information. I can describe some studies that we've done where you're learning, and how you're learning, and there we put people alone in a room. They're not talking to anyone, and they're alone in a room, and we've given them complicated material to learn, and they're going to be tested, and these are students, so they're used to learning and being tested, And they're good at it. And the material we gave them is varied. We started with descriptions of environments, so describing where four or eight landmarks were in a gym or a small town or a museum, and describing incidental information about the environments to make them more vivid. And we told them that they would be tested, and they could Read these descriptions four times, and we filmed them as they were reading, and as they were reading, they were making gestures. So they're not talking. The gestures are for themselves. And for the, the environments, they were drawing lines for the paths in the environments and, and points, dots for the various landmarks. And they were doing it over and over the way you might rehearse a phone number. So they were rehearsing that, checking it, and doing it in different ways. But for the environments, Just about, about 70% of the people did this spontaneously. We never suggested, ah, that, that gestures would be helpful. We never gave them a hint that we were interested…

AI assessment note: “I don't have direct information. I can describe some studies that we've done”

Q how we interpret things. And I know you've done some work where you got people to think creatively, and one of the ways that you got them to think creatively was to ask them to think about the different roles of people. So like, how would a doctor use this? How would a gardener use this? Uh, I wonder if you can talk to me a little bit about that.

A Yeah, sure. We've done a lot of work on perspective taking, starting with With the map situation, are we taking an overview perspective or embedded in the environment? And that is really hard. We work with Google Maps, and we're seeing something. How do we reconcile them? There's been beautiful work on the brain, on that, and how the brain might do it by Russ Epstein and Penn, um, and undoubtedly others. So there's perspective taking there. There's perspective taking in Should I take your perspective or my perspective when I'm describing things? And if you're in front of me, I generally take your perspective, because I understand, and your cognitive load is higher. But, um, so, changing perspective is key in many, many places in, in our lives. We found that even in situations that are non-communicative, Or where you and I have the same perspective, and we're looking at somebody else acting, we take the perspective of the actor instead of our own perspective, and I think it's a way of understanding their action, um, is taking their perspective, and we do that more quickly and automatically than taking our own perspective. So that was surprising to many people who assumed that an egocentric Perspective was primary, and overcoming it was difficult, but by now that experiment has been replicated by many people in many cases. We talk about social perspective taking and taking the pe…

AI assessment note: “We talk about social perspective taking... and there we found it useful in creativity.”

Q In your writing, you've used the term cognitive collage. What, what did you mean by that?

A So, people have talked about cognitive maps, And a term that came from Tolman, who, who showed that rats learning roots could put those roots together into a map of where things were relative to each other, and of course the beautiful work on place cells and grid cells has shown that, um, conclusively that rats have ways of organizing places in a Space that's viewpoint independent, and, and then navigating in it, and, or using that to navigate. So we studied, along with many other people, various errors that people make systematically in, in making judgments about spatial relations, distance and direction in particular. So people think that Philadelphia is, is north of Rome. When it's actually south of Rome, and it seems to be because people line up Europe and North America, and Europe is actually farther north than North America, and anyone who goes to Europe at this time of year knows the sun sets very late in Paris and Rome compared to, to Northern American cities. So people line them up, and then they see that Philadelphia is Northern part of America, and Rome is sort of south in Europe, and things use that to infer the direction between Philadelphia and, and, um, and Rome. People think that an ordinary building, say Jacques' house, is closer to the, to a landmark Eiffel Tower than Eiffel Tower to Jacques' house, and that violates any metric assumption. And it seems to me b…

AI assessment note: “people have talked about cognitive maps... So we studied... various errors that people make systematically”