Nina Kraus on Hearing, Noise, and Of Sound Mind
Intro. [Recording date: October 27, 2021.]
Russ Roberts: Today is October 27th, 2021, and my guest is author and neuroscientist Nina Kraus of Northwestern University. Her book, and the subject of today’s conversation is Of Sound Mind: How Our Brain Constructs a Meaningful Sonic World. Nina, welcome to EconTalk.
Nina Kraus: Well, thank you, Russ. I think you can hear from the sound of my voice how happy I am to be talking to you.
Russ Roberts: This is an utterly fascinating book on a huge part of our daily life that we rarely think about, and often don’t fully perceive, which is the role of sound in our lives–music, language, everything else that makes noise–how we hear those sounds, how the brain is shaped by them, and in turn how the brain shapes how we hear.
I want to start with the importance of sound because, as you point out in the book, a lot of people would list it a little bit down the list of their favorite senses. They’d say, ‘Eh, seeing, that’s crucial. I’d hate to be blind. But, deaf–I could live with it if I had–it’s not as important as seeing.’ And, I think we hardly ever think about it until we lose our sense of hearing, in which case we do think about it, or if we’re deaf. But, I think that one of the themes of your book is that we greatly undervalue the importance of sound and we only think of it as, ‘Oh, that’s hearing stuff. I get that. I know what I hear. I like to hear things.’
But, it’s so much more profound than that, and I think one of the great insights of the book for a non-auditory scientist as you are is to appreciate it. So, why do we undervalue the importance of sound?
Nina Kraus: Yeah. Well, I think one of the reasons is because sound is invisible. And, like many of the powerful forces in our lives, like gravity, you can’t see it. We live in an increasingly visually biased world. So, I mean, people just don’t recognize and appreciate what an important part sound has in our lives, in our world, and in what makes us, us. And, the book is my love letter to sound.
If you look at the homepage of our website, Brainvolts, you’ll see that we study music and bilingualism, aging, language, reading, concussion, hearing a noise, autism. You might look at that and think, ‘What are they doing at Brainvolts?’ But really, the overarching umbrella is sound and the brain.
Just by looking at that list, all those different topics, you can see how many aspects of our lives sound is a powerful, powerful part of, and indeed we don’t recognize it.
Russ Roberts: One of the things you talk about in the beginning of the book is just the mechanism of sound. I know I have an ear drum and I know there’s a couple of other things in there–been in a doctor’s office where there’s a model of the ear and you go, ‘Oh yeah, there’s a lot going on there.’ It’s kind of extraordinary. We’re not going to go into it in depth. But, the part that I thought was particularly interesting is that the brain moves very quickly and it has to convert moving molecules of air pressure into electric signals. Kind of extraordinary.
Nina Kraus: Yeah. It’s marvelous. The process fills me with awe. So, the first part of the book–and it’s just the first third–and it’s called “How Sound Works.” And, I feel like a little kid sometimes. You know how little kids like to read the same book again, and again, and again? They want to hear the same story. I have heard, I have told the story of sound, how sound works from a biological standpoint many, many times, and I want to hear it again. I want to hear it again and again and again.
But, basically, sound is the movement of air; and we have sound waves and we have brain waves; and the sound waves are air and the brain waves are electricity. And electricity is the currency of the nervous system. So, this transformation needs to happen. And, so much of what happens, not only it enters the ear, but it’s really the brain that makes a lot of sense of the sound.
But, now, if we just think about: so what makes up sound? Again, visually nobody has difficulty. I’ve got this visual object and it’s got a shape, a size, a weight, a color, texture. It has ingredients.
Well, sound, if you even pay attention to sound at all, sound also is rich with ingredients. It has pitch, timbre, harmonics, phase, how loud it is. There are many, many–how fast. There’s so many timing, rhythmic cues. And, the brain, our sonic brain, our sound-mind, our hearing brain needs to make sense of all of these ingredients.
And, one of the metaphors that I use in the book is a mixing board. So, there are all of these different ingredients in sound. And, if you think of the faders of a mixing board going up and down, and each fader reflecting the brain’s processing of one of these ingredients, we can see how good a job our brain is doing at processing these different ingredients in sound. And, not only does this processing occur in a very interactive manner, but it matters what our life in sound has been. So, these faders really reflect who we are in terms of our life in sound.
Russ Roberts: We tend not to think that, right? We tend to think like, ‘Well, we all heard the same sound, so it’s the same.’ But, of course, we also think we see the same thing. And, of course we don’t. Our brain is constantly interpreting what we see and literally editing things out. And, that’s true in sound as well, as I learned from this book. We know that very directly because people say, ‘Aren’t you listening?’ And, you go, ‘What do you mean?’ ‘What?’ Of course, your ears were taking in the sound, but your brain decided to just kind of not take it in.
Russ Roberts: One of the examples of this that I thought was so extraordinary–it’s a very small thing, but I loved it–is the difference between ‘bill’ with a ‘B’ as in boy and ‘pill’ with a ‘P’. Your mouth is doing the same thing. What’s the difference between bill and pill that allows you to hear that difference?
Nina Kraus: It’s timing. So, timing is one of my favorite and also important ingredients. The difference between bill and pill is the amount of time that it takes between the onset of the movement of the mouth and then the vocalization. So, bill, pill: it’s a 30-millisecond timing difference that even though the motions that you’re making are the same, there is a very, very clear acoustic difference. If you’re a signals man like me, and you measure these brain responses, you can see as clear as day that there’s a 30-millisecond timing difference between the onset of the voice in bill and in pill.
Russ Roberts: So, B and P–I’m trying to say P carefully not to pop my mic, my microphone–B and P are not really a vocalization. We have to get to the I, the ‘-ill’, which are the B and the P there. After the P, I make a tiny unconscious hesitation when I want to say the letter P. Correct?
Nina Kraus: Yeah. And, then there are languages like in Hindi, for example, and Armenian–there is something called pre-voiced. So, you voice ‘mmp’, ‘mmb’. And, that’s something that our English-speaking ear or brain does not pick up. And, we can be trained to hear those differences–but again, a beautiful example, I think of how our sound mind is a product of our experiences. When we’re born, babies–they’re citizens of all the world’s languages. They can hear any of the sounds, but based on what is meaningful to us.
So, if we grow up in speaking a language where pre-voicing is meaningful, or in Mandarin, where these tone changes within a syllable are meaningful, well, then our brain is going to change fundamentally, to the point that, when we are asleep and you measure the brain’s response to these vocalizations, to these particular sounds, just depending on who you are, if you have made repeated sound-to-meaning connections with certain sounds, in your sleep, your brain will respond to that. And, it won’t, if you are presented with sounds that you haven’t learned to make any kind of a meaningful association with.
Russ Roberts: So, I’m living in Israel right now. I moved here four months ago. I came here with a rudimentary Hebrew vocabulary and I can read the Hebrew letters. And, Hebrew is a very impossible language. But, of course I’m 67 and many languages would be hard for me. Now, French would be a little easier or Spanish. I know them each a little bit. But Hebrew, of course, is not from the same root as English. But, of course, in my mind–well, English iseasy, because I know it; and Hebrew is hard because it’s not like English.
I was fascinated by this data point you gave us that there are 40 or so sounds in English. Forty. I assume a sound is ‘ee’, ‘ih’, ‘eh’, ‘ah’. My name in Hebrew is Russ. There’s no ‘uh’ in Hebrew. So, people have trouble with my name. I happen to have a Hebrew name, as a Jew, which is Reuven, which they’re very good at, but they’re not so good at ‘Russ’. So, there’s 40 sounds in English and those are represented by a mere 1,120 different letter combinations, according to your book. As opposed to Italian, which has 25 sounds and 33 different letters.
So, English has 40-ish, roughly, times more combinations of letters to read[?] all those sounds. It’s unimaginable that people come to America from a foreign country and learn English. It’s such a hard language.
Nina Kraus: Yeah. But in fact, our sound mind–our hearing brain–is so miraculous in that we learn how to make sense of these sounds. And, one of the stories in my book is the owl story that is relevant, I think, to what you’re saying about being an older person and starting to learn a language later in life. One of the things that you can do in an animal model is go directly and measure electrical activity from individual neurons. And, you can see–okay, so owls are nocturnal predators and so they really depend on sound. They found that you can change the owl’s auditory and visual fields and you can actually see it reflected in the mapping of the neural activity in the brain. And, they saw this initially in younger animals and they were wondering, ‘Well, is this something that an older animal can do?’
And, the first experiment that they did, they found, No: in fact, the older animal couldn’t learn a new auditory visual space. What they do with these owls is they put psychedelic prisms on them that shift the visual field to the right or to the left. And so, you have to learn to make new associations.
But, then the scientists had an idea and they said, ‘Well, maybe the older animals just are learning in a different way.’ So, they used a different strategy. And what they did is they made the changes in the visual fields and in the auditory visual connections smaller.
And, over the same amount of time it took the younger animals to learn this new task and for their brain to get rewired in a way that you could physically measure it and see it, the older animals could do that, too.
So, that’s telling us [inaudible 00:14:37] the older brain–and we know this just from so much converging evidence–the older brain, our brain is malleable until we die. And, of course we are different now than when we were four or five years old, or two or three. So, maybe we need a different strategy for learning. And, that’s fascinating to me.
Russ Roberts: I guess that’s a little bit of a comfort. My wife is studying daily her Hebrew. I don’t have time unfortunately, but she goes to what’s called an ulpan, where you study Hebrew–you start learning three hours a day and then she has homework and she studies grammar and everything. My view is that’s not going to help me. I have to listen and speak. I’m not going to study anything. I’m not going to read charts or read rules.
I don’t know if that’s going to be true or not. What I do know is that for 10 years, I took French–third grade through 12th grade–and learned almost nothing. So, I’m hopeful. I’m hoping that I can do with a little bit better here. But you can teach an old dog new tricks, I’m hoping.
Nina Kraus: But, also you and your wife are different; and you hear the brain differently. One of the things that we discovered at Brainvolts is you can take a sound wave and we can listen to it. And, then using scalp electrodes, we can measure the electricity that happens in response to sound. So, as I’m talking to you now, the neurons in your brain are producing electricity. And we can pick that up. And, we can determine then that your wife, and you, and me hearing the same sound is going to–so the sound is the same–but the brain response is going to be different and you can see it in the electrical wave form.
We can even hear it because you can sonify the electrical signal and you can hear that your brain and your wife’s brain are hearing the same sound differently. So, you’re also going to learn–your way of learning through sound is going to be very different.
Russ Roberts: I can’t help but add that George Steiner, the essayist and writer, has written an extraordinary book called After Babel–a reference to the Tower of Babel and different languages–where he argues that all language is translation. You and I process the sound differently, of course, as you point out, but the words mean different things to us, too. And we think we understand–we understand that we don’t understand Shakespeare very well, but we think we understand Jane Austin because she uses modern English. But, Steiner shows that actually the words have evolved since then. And, what you hear versus a contemporary verse[?], not the same thing.
Russ Roberts: The thing I really liked about the owls in your book is that an owl–tell the example about the football field. That just blew me away.
Nina Kraus: Oh. Well, just an owl–and I’m not remembering exactly what the–you know, owls are just remarkable. Again, this gets to a philosophical issue of: we often think philosophically in terms of them and us. So, us being humans and them being animals and plants. And, plants and animals have extraordinary, extraordinary abilities that we are just beginning to understand a little bit. But, to put that in context with the owl: I can be at one end of a football field and you can be at the other and can snap my finger on one hand and the other hand, and the owl can resolve that difference, that spatial difference–it’s a phase difference–very, very well. We certainly couldn’t.
Russ Roberts: So, the owl is 100 yards away and I snap my fingers on my right hand, and the owl knows it’s my right hand, not my left hand–not by seeing it. Even wearing the goggles–the owl is not going to be able to cheat. But it’s going to know from–its eye is going to move toward the hand thats snapped, as opposed–
Russ Roberts: Which is–a few weeks ago, we had an episode about dogs’ ability to smell. I love this one. I just want to mention: there’s a Robert Penn Warren poem called the “Heart of the Backlog,” which has a very powerful and dark reference to owls’ hearing; and we’ll put a link up to that.
Let’s talk about songbirds, because we’re on birds. I just want to be clear: it’s only a small part of the book on owls, about a page and a half. There is a chapter on songbirds toward the end, but it was one of my favorite chapters. Why does a person, an auditory neuroscientist, care about songbirds?
Nina Kraus: Well, first of all, they’re beautiful. They’re beautiful to listen to; and historically people have listened for birds to determine where to settle, because it turns out that places that are friendly to birds and to the survival of birds are often also friendly to humans. So, this has been a force that has been driving us.
Birds are fascinating from a number of standpoints. And, again, they help us understand human hearing, because we’re able, again, to do various experiments with a bird that you can’t do with a human.
For example, I mean, what we do know is that birds learn their song. They learn their songs from a tutor–from their father. And, you also need to know that with birds, it’s the males that sing, and it’s the females that choose. So, the male bird learns his song from his father. And, there are all kinds of things that you can get into in terms of dialects and what–the lady bird is going to prefer the bird with the closest dialect. She’s also going to prefer the song that is most intricate, that has some nice improvisations, but that there’s strength there because it’s really a measure of fitness.
And, I also want to say that part of our research has been looking at sex differences in hearing. And, there are some; and it’s kind of interesting to see this in the animal kingdom, because the male bird has a different job than the female bird.
So, the way that a bird learns from his daddy is that he imitates. He imitates the song. And, there are not very many species that imitate; and humans are one of them. Chimpanzees don’t. People have tried to get them to vocally imitate.
Your dog can understand if you want to go outside. He’ll understand certain words. But, if you’re talking to your kid and you’re saying, ‘I’m going outside,’ the kid will say, ‘outside’; and your dog is never going to do that. And, this has to do with the fact that birds are vocal learners. We–humans–are vocal learners. And, it seems to be also attached to a very fundamental thing that humans do, which is–and birds–is that we’re very sensitive to rhythms in sound. A human baby will move in sync with the beat of a song.
And, certain birds’ song–certain birds–will also move in the same way. But, again, your dog is not going to be wagging his tail in time with the beat.
So, the fact is, and this brings me this issue of reciprocity and between this as McGilchrist talks about it–you know, this idea of back and forth of what you’re hearing and what you’re perceiving, and then you’re adjusting what you’re saying and your movements and what you’re thinking–in the present–to interact beautifully with your environment.
So, birds give us a glimpse of this because they are one of few species that are vocal learners. They know how to imitate, and they can move rhythmically.
Russ Roberts: That’s really beautiful, and I really love of the idea of the–I mean, I find birdsong, especially in the morning, very emotionally powerful. And I don’t know–I love the idea that it could be part of my evolutionary past. And, I think that it’s–it’s ‘comforting,’ is the word I would use. Right? The idea that it’s tied to a friendly, fertile environment that we evolve to appreciate that is a fascinating idea.
It also reminds me, though, that I think it’s part of my childhood. So, it doesn’t just evoke maybe my evolutionary past: It maybe evokes a time when I was a young boy. And, I had a certain feeling about waking up and being excited. And the birdsong taps into that, somewhat in the way that the music from my childhood has a certain comforting and belovedness to it that–I think, often, that we love the music we grew up with, not because it was particularly good: it just happened to be the soundtrack of our lives at that point. And, it’s not obvious to me that the music that Baby Boomers grew up with, like me, is going to still be popular in 50 years because people then will have grown up with different things.
But, that birdsong idea, and the rise and fall of it, the melodic part of it, the rhythmical part–it’s a musical element to life that I think we often don’t notice, and I really appreciated noticing in your book.
Nina Kraus: Yeah. I think that one of the things that we may have noticed when we were in lockdown is that we were noticing the birds not only because there was less noise, but actually people who measured these responses from birds found that the birds were singing more quietly. The birds were singing more quietly because birds have to scream to be heard over the noise. So, they were not only singing more quietly, but they were singing more intricate songs–you know, intricate in terms of pitch and timing and timbre. The things that excite the lady birds. They had the energy to do that because weren’t shouting above the din.
Russ Roberts: Well, we understand that because of what you identify as the cocktail-party effect. So, if I’m at a cocktail party and we’re yelling because it’s a noisy room–the acoustics of the room, or a restaurant, which happens–it totally changes the range of things you can communicate. Because, you’ve got to be loud, period. You can’t be quiet. Talk about that challenge for our hearing system and the listening part of it.
Nina Kraus: Yeah. And, you can’t be nuanced, either, because there is so much nuance in the sound. I started the episode–you know, you welcomed me to EconTalk and I know you could hear from the sound of my voice that I was happy to be there. And so, there is this nuance that you would completely lose in a noisy room where you’re just trying to get the message across.
And again, I don’t think that I don’t think that people notice and realize how important this is. And, again, part of my book is really trying to get people to realize that this is important: that communication and connection is important. And so that we should honor it. But, we learn throughout our lives.
So, for example, you talked about memory and memories for songs. Well, in a noisy place, you will be able to hear your wife better than a stranger because you know something about the rhythm of her voice and you can pick up cues that–they do have–you’ve just learned. And you know, when you’re in a room, depending on the music that’s playing, it will affect you or not.
But, we, you know, we learn because we–you know, the sound enters our ear into our brain, and there we have this confluence, this engagement of what we call the efferent system, which is more massive than the pathways that are going upstairs from the ear to the brain. And, with evolution, these are more and more evolved. And, with our life and sound, our hearing, our actual hearing is sculpted not only by the sounds, but what is happening at the same time.
So, how we are moving. What we are thinking about. What we are feeling. And, the information from our other senses.
Again, this philosophically has to do with a binding problem. And, when you look at the sonic mind, when you look at the sound mind, the hearing brain, you see that these operations are–this is a reverberating circuit. And, you know, scientists like to compartmentalize. But this is a distributed, but very interactive process.
Russ Roberts: It’s a broad–it’s an incredible network which I started to appreciate from reading your book. But, the challenge of hearing multiple voices at the same time and only listening to one of them is quite an achievement, right? Isn’t that what–with the background noise going, too–
Nina Kraus: Yes. It is an achievement because–but again, we need to learn what to pay attention to, and we need to learn what to ignore.
So, again, if you are a speaker of multiple languages, it turns out you’re particularly good at, they call it ‘inhibitory control’–that you’re pretty good at ignoring information and sounds that are irrelevant so that that particular part of your sound mind is going to affect how you hear a noise. [More to come, 30:39]
This article appeared firshere