The human brain is amazingly complex, and full of fun little oddities when it comes to how it interacts with music!
To explore some of these quirks, Bey and Kirsten sit down with one of the founders of the field of music cognition, Dr. Robert Zatorre from McGill University, to talk everything from perfect pitch to how we can hear songs in our head. Then, they’re joined by Dr. Jayatri Das, chief bioscientist at the Franklin Institute, for another Body of Knowledge segment, this time exploring the science behind auditory illusions and misheard lyrics! And to round out the episode, Bey and Kirsten chat with Philadelphia-based composer Andrea Clearfield to learn about her experience with synesthesia, and how seeing colors when she hears music (and vice versa) affects her creative process.
Links for today’s episode:
- Learn more about Dr. Rober Zatorre
- Check out some more examples of auditory illusion from The Franklin Institute!
- The infamous dress that divided the internet, what colors do you see?
- Learn more about Andrea Clearfield, and the two specific compositions she shared; Cal-3, based on the music she heard when she saw Heiner Riepl’s paining, and her opera, MILA, Great Sorcerer
- Have your own experiences with synesthesia, or anything else you want to share? Email us at socurious@fi.edu
Transcript
Yo, yo, yo. What's up, everyone?
Speaker:I'm Kirsten Michelle Cills. And I'm The Bul Bey!
Speaker:We are your host of the So Curious
Speaker:podcast, presented by the Franklin Institute.
Speaker:And this season is all about the science of music.
Speaker:And today we are looking at all of the
Speaker:fascinating little things that our brains do when we are listening to music.
Speaker:Yeah.
Speaker:First, we'll be speaking with cognitive neuroscientist Dr.
Speaker:Robert Zatorre to go in depth about how our brains understand and interpret music.
Speaker:And then we are going to be joined by the Franklin Institute's very own Dr.
Speaker:Jayatri Das.
Speaker:You know her, you love her.
Speaker:She is a staple on this show, friend of the pod Dr.
Speaker:Jayatri Das!
Speaker:For her Body of Knowledge segment, teaching us about psychoacoustics.
Speaker:And to close out the episode, we'll be
Speaker:speaking with Philadelphia-based composer Andrea Clearfield about her experience
Speaker:with synesthesia and how that affects the music she creates.
Speaker:All right, as you can hear, we're pretty
Speaker:excited about this one, so we might as well just dive in.
Speaker:What do you say? Let's do it.
Speaker:Dr.
Speaker:Robert Zatorre, welcome so very much to the So Curious podcast.
Speaker:Can you introduce yourself?
Speaker:Tell us a little bit about what you do?
Speaker:Feel free to brag about yourself.
Speaker:Yeah, big brags.
Speaker:Hi, Bey and Kirsten.
Speaker:It's nice to be on your show.
Speaker:I am a neuroscientist at the Montreal
Speaker:Neurological Institute, which is part of McGill University, and my area of
Speaker:specialization is the study of the brain and music and why music even exists, and
Speaker:how it is that it gives us so much pleasure, and all of that.
Speaker:Big topics, wow! Yeah.
Speaker:Honestly, we're really excited to have you, because you're considered the OG
Speaker:pioneer in the world of music cognition research.
Speaker:What led you to start this research on this topic?
Speaker:Yeah, that's a great question.
Speaker:So when I was an undergrad, I sort of fell
Speaker:in love with music, like many people in their adolescence.
Speaker:And so I decided I wanted to study music, but I had also always had a very strong
Speaker:interest in science, and so I thought I would do both.
Speaker:And so I actually studied some physiology and psychology and also music.
Speaker:During the course of my training, I realized that my friends in the music
Speaker:program were actually far better than I was in terms of being musicians, but I was
Speaker:pretty good at the science part, so I thought, well, maybe I should pursue the
Speaker:science, but at the same time , keep a hand in the music world.
Speaker:I managed to convince my PhD supervisor, who was very open minded, that maybe we
Speaker:should try to study how music is perceived.
Speaker:When I started my PhD, there was very,
Speaker:very little work in that area, but there were a few crazy souls out there like me.
Speaker:And, yeah, it sort of, it blossomed!
Speaker:And now it's a topic that there are many,
Speaker:many very good laboratories throughout the world.
Speaker:It's been very rewarding to see how it's sort of grown.
Speaker:Yeah, and that's incredible. And we're really excited to talk to you.
Speaker:You were at the forefront of it all. That's awesome.
Speaker:Thank you.
Speaker:So you have done so much work exploring music and brain function.
Speaker:Right.
Speaker:So we wanted to ask you, can musical training physically change your brain?
Speaker:That's definitely true, and our lab and a
Speaker:lot of other labs have shown this, and we think of it in terms of adaptation.
Speaker:In other words, your brain is capable of
Speaker:many, many different things, some of which are very complicated, right?
Speaker:You could decide to be an astronaut, you
Speaker:could decide to be a farmer, you could decide to be a musician.
Speaker:And each of those specialties requires some kind of change in your brain.
Speaker:And so, in the case of music, we
Speaker:definitely see certain adaptations in musicians who, for example, you won't be
Speaker:surprised, not only have a better ear in terms of their ability to hear. But also,
Speaker:when we look at the way that the auditory part of the brain responds to sound, we
Speaker:see that sound is represented more accurately in the brain of a musician.
Speaker:Wow. We call that functional specialization,
Speaker:because it's the way that your brain is functioning.
Speaker:But then there are also structural
Speaker:specializations, meaning that the actual fibers that connect different parts of the
Speaker:brain are strengthened in musicians, compared to people who do other things.
Speaker:And in particular, the fibers that connect the motor parts of the brain, that control
Speaker:your fingers, for example, and also your vocal musculature, they're more strongly
Speaker:connected to the auditory parts of the brain.
Speaker:And that makes perfect sense, because when
Speaker:you're playing an instrument or if you're singing, you need to have a lot of back
Speaker:and forth between the actions that you're producing and the sounds that come from
Speaker:those actions, and you need to be able to adjust very often on the fly, right?
Speaker:If you're playing an instrument, especially with, like, a band or with a
Speaker:group of people, you have to be very attentive to what the others are doing,
Speaker:and you have to adjust your movements in milliseconds, right?
Speaker:And then as you produce the sound, the ear
Speaker:has to immediately pick up on what you're doing to see if it's correct or not.
Speaker:So we call that a feedback loop.
Speaker:The action generates a sound.
Speaker:The sound, in turn, influences the action, which generates a sound, and so forth.
Speaker:Okay. Dr.
Speaker:Zatorre, I got to ask you a bit of a strange question, it's just on my mind.
Speaker:Have you ever held a musician's brain?
Speaker:I have held brains, actual physical
Speaker:brains, especially when I was training as a student.
Speaker:In those days, we didn't have MRI scans.
Speaker:That's how ancient I am. So the only way
Speaker:to learn about anatomy was to actually dissect a real human brain.
Speaker:Wow. And usually, we would just not know
Speaker:anything about the person who had donated their brain.
Speaker:It's sort of a matter of
Speaker:confidentiality. And so, in a sense, very selfless, if you think about it.
Speaker:But usually we just didn't know who those folks were.
Speaker:So maybe, in response to your question, maybe I did a musician's brain!
Speaker:Can you walk us through what it feels like?
Speaker:What did it feel like?
Speaker:What was the texture?
Speaker:Well, when you hold a brain, it's typically after it's been fixed.
Speaker:So, it's been sitting in a bucket with formalin.
Speaker:I've seen live human brains in the middle of a surgical procedure.
Speaker:There are certain procedures when they have to stimulate parts of the brain to
Speaker:see, like where seizures are coming from, for instance.
Speaker:And so on a few occasions, I sort of sat in to the operating theater.
Speaker:And so you see the actual brain that's open because they've removed a chunk of
Speaker:skull, and you see the surgeon kind of poking at it with an electrode.
Speaker:And then you can see that it's very, very soft.
Speaker:It's kind of like jello. Jello...
Speaker:It's got that consistency. Ooooooo!
Speaker:So imagine poking jello with your spoon.
Speaker:That's sort of what it feels like. Oh, man!
Speaker:This is going to be great the next time we have jello!
Speaker:But it's pulsating, right? It's pulsating
Speaker:because blood is flowing through it. So, it's pulsating with your heartbeat.
Speaker:Going back to the musical stuff for a
Speaker:minute, because I know Bey has a question about when it comes to musical talent,
Speaker:because this is, I'm sure, a hot debated topic.
Speaker:So, Bey, go ahead. Yeah.
Speaker:Like, you know, are you born with musical
Speaker:talents or is this something that you learned?
Speaker:I guess it's like a nature and nature kind of a question.
Speaker:It's the classic, classic question, and the classic answer is that it's, of
Speaker:course, the interaction between genetics and environment.
Speaker:There's a big variation in people in what their native talents are.
Speaker:And sometimes that's thought of, like, in negative terms, but I always think of it
Speaker:in very positive terms, like in terms of diversity, right?
Speaker:Because imagine if everyone had the identical talents.
Speaker:The world would be incredibly boring, right?
Speaker:Number one.
Speaker:And also, we wouldn't be able to solve certain problems, because everyone would
Speaker:approach a certain problem in exactly the same way.
Speaker:So the fact that we all think in different ways and we all have different abilities
Speaker:and strengths and weaknesses, I think is a plus.
Speaker:Yeah.
Speaker:Not only like, for biological survival, but for society.
Speaker:Right? Well said, I love that.
Speaker:That's why we learn stuff from each other, right?
Speaker:Yeah...
Speaker:But then, obviously, your brain is very strongly influenced by your environment,
Speaker:and that starts even literally before you're born.
Speaker:And then, during infancy and early
Speaker:childhood, the interactions the child has with the world are very, very important.
Speaker:And we've actually done research showing that if you want to become a musician,
Speaker:there's a time window during which the training is particularly important.
Speaker:And that time window is somewhere between like five and seven, roughly.
Speaker:And so, I mentioned earlier that I was
Speaker:kind of a mediocre musician, and I attribute that to the fact that I only
Speaker:started musical training when I was like, eleven or twelve.
Speaker:And so I always wonder, I wish I had
Speaker:started earlier, because maybe I would be more skilled than I am.
Speaker:But that's a good example of a gene-by-environment interaction.
Speaker:If you hope to be a very good musician, it
Speaker:helps, a lot, to have started early, but it also helps if you have some genetic
Speaker:propensity, and we can actually measure that, also.
Speaker:We can look at the genetic link, for example, in twins.
Speaker:If one twin is very musical, it's much more likely that the other twin will be
Speaker:also very musica,l than two siblings who share some genes, but not all the genes.
Speaker:Wow!
Speaker:And so I have a question about perfect pitch, because, I was saying I had a very
Speaker:good friend in college who had actual perfect pitch.
Speaker:There's a lot of people who think they do,
Speaker:but they just have relative, and can you explain what that is?
Speaker:And is it something that you can see on a brain scan?
Speaker:Yes, actually.
Speaker:So it's this ability that some, small-ish number of people have to identify a
Speaker:particular tone without any reference to any other tone.
Speaker:Now, it's another case of gene-environment
Speaker:interaction, because going back to this idea of earlier exposure, most people who
Speaker:have perfect pitch started some kind of training before age ten or so.
Speaker:And if you started training after that, it's very rare that it will develop.
Speaker:Not impossible, but quite rare.
Speaker:However, not everyone who begins the early training will develop perfect pitch.
Speaker:And again, there's probably a genetic component.
Speaker:When we look in their brains, we see that there are kind of more connections from
Speaker:the auditory regions of the brain to other areas.
Speaker:So in these experiments that I'm talking
Speaker:about, we do MRI scans, magnetic resonance imaging.
Speaker:Got you. And so we go out and we search for people
Speaker:who do or do not have absolute pitch and we measure it, see how accurate they are,
Speaker:and then we analyze the brain scans using different kinds of mathematical techniques
Speaker:Oh that's cool! And that's where we see these differences.
Speaker:So let me ask, why is it that we can hear
Speaker:songs in our head, even when we aren't physically listening to it?
Speaker:Like, why is it that songs kind of stick around in our psyche?
Speaker:Great question.
Speaker:So we've actually done a lot of research on this.
Speaker:We call it musical imagery.
Speaker:And it's something that if you're a
Speaker:musically trained person, you have a more developed version of it.
Speaker:Like, you can call to mind music that you
Speaker:are familiar with, that you've learned to play, for example.
Speaker:But even if you are completely untrained,
Speaker:it's something that almost all people have.
Speaker:An exception for, there are a small number
Speaker:of people who actually can't imagine sounds, and they're very interesting.
Speaker:But it's not just music.
Speaker:They can't imagine the voice of their mom, or they can't imagine a dog barking.
Speaker:But that's really quite rare.
Speaker:The vast majority of people, even without
Speaker:musical training, easily can call to mind a song.
Speaker:It's not effortful at all.
Speaker:And we've actually done brain scans when people are doing this, which is very
Speaker:tricky, because you have to control what they're doing.
Speaker:And we don't just take their word for it.
Speaker:We have ways of measuring what's going on.
Speaker:And what we see is that, when people are imagining sound, we see many of the same
Speaker:brain areas that are involved when they're perceiving sound, except that it's driven
Speaker:entirely by internal cognition, not by external sounds coming into the ear.
Speaker:So in complete silence, we can scan them,
Speaker:and we ask them, imagine song X or imagine song Y.
Speaker:And then we see activity in those areas of
Speaker:the brain, especially the auditory areas, also the motor areas.
Speaker:And the activity is subtly different for song A or for song B.
Speaker:So we can even tell, we can use a computer
Speaker:algorithm, and figure out, which song are you imagining?
Speaker:Is it Stairway to Heaven?
Speaker:Or is it Beethoven's Fifth?
Speaker:And we can actually determine that. Wow.
Speaker:And so, is there a way to train your musical imagery?
Speaker:Like if you were to work on that ability in your head?
Speaker:Yes. When you go to music courses, it's usually
Speaker:called audiation, it's a funny word that they use in the music training world.
Speaker:But it's the same thing that we're talking about.
Speaker:They will play you certain chords and
Speaker:they'll say, "okay, now imagine that chord" just without playing anything.
Speaker:"Now I'm going to play you a random chord.
Speaker:You tell me if it matches what you were imagining," things like that.
Speaker:So musicians will actively train to do that.
Speaker:And it's very, very helpful because, for example, when you're rehearsing, I know of
Speaker:people who rehearse just by imagery, especially if they play an instrument that
Speaker:you can't carry around easily, like you're a tuba player.
Speaker:It's a big, heavy thing.
Speaker:You may not have it right in front of you, but you can look at the music, the printed
Speaker:music, and imagine, okay, what do I have to do?
Speaker:What is it going to sound like?
Speaker:And that actually does help you.
Speaker:I've heard of professional musicians who
Speaker:didn't have time to rehearse and they're going to a concert.
Speaker:And so they're flying to LA, and they say, okay, I've got 3 hours, I'm going to study
Speaker:the score and when I get there, I'm going to play.
Speaker:And if their imagery is good enough, they can actually do that.
Speaker:That is wild!
Speaker:I wouldn't recommend it, but, you know.
Speaker:It's for the pros. Wow.
Speaker:That is incredible. Dr.
Speaker:Robert Zatorre, thank you so much for
Speaker:being on the So Curious podcast, this has been very enlightening!
Speaker:Very cool, very cool. This is wild.
Speaker:Thank you so much. Great.
Speaker:Thank you very much. Thank you!
Speaker:Take care. Have fun.
Speaker:Bye bye.
Speaker:Thank you so much, Dr.
Speaker:Zatorre, for sharing some of your
Speaker:knowledge with us, and specifically, the fact that you actually have held a brain.
Speaker:That is incredible.
Speaker:We really appreciate it! You did it, Bey.
Speaker:Yes!
Speaker:You finally got what you were looking for, wow!
Speaker:That was awsome! We could just end the whole show here.
Speaker:Yeah, I'm good! Cause you got to talk to a brain holder.
Speaker:See you, guys! Yeah, right!
Speaker:And now, I am so freaking excited to
Speaker:welcome back on the podcast for her Body of Knowledge segment, our only recurring
Speaker:guest, Chief Bioscientist at the Franklin Institute, Dr.
Speaker:Jayatri Das!
Speaker:Hey!
Speaker:Hey Kirsten, hey Bey.
Speaker:Great to be back! Great to have you!
Speaker:Tell us what we have today. Yeah.
Speaker:All right, well, today we're going to talk about psychoacoustics.
Speaker:Oh, man. Very, very simple.
Speaker:What do you think that means?
Speaker:Right off the bat?
Speaker:I would think it's the way that your brain hears and makes sense of it?
Speaker:You nailed it! Well done.
Speaker:Well done. Right?
Speaker:So it's how we experience sound, because
Speaker:sound is a physical phenomenon, but again, it's interacting with how we perceive it,
Speaker:and that can be different for different people.
Speaker:So, at the very basic level, there's
Speaker:always the variation in just hearing perception.
Speaker:As we age, we start to lose the ability to hear higher and higher frequencies.
Speaker:Part of that is just because of the damage to our hearing system.
Speaker:All those years as teenagers.
Speaker:I know!
Speaker:Being in the emo space, blasting music...
Speaker:Be like old people wear earplugs when you go to concerts, right?
Speaker:Yeah yeah yeah, right.
Speaker:But we don't perceive all frequencies equally, either.
Speaker:And then on top of that, the fact that we
Speaker:have two ears that tell us information about spatial location.
Speaker:Oh, wow. So sound is like, an experience.
Speaker:You hear it, and that's a very finite
Speaker:thing that people hear, but also your brain interprets it.
Speaker:Exactly.
Speaker:So let me give you a non music related example.
Speaker:Okay.
Speaker:Do you guys remember the picture of that dress that broke the Internet?
Speaker:Oh, yeah.
Speaker:Like, if you see it blue or if you see it gold.
Speaker:Blue and black. Blue and black or white gold,
Speaker:something like that, right? Yeah.
Speaker:Yes. Go ahead.
Speaker:So, the dress itself, in reality was blue
Speaker:and black, but this particular picture was -
Speaker:It was also an ugly dress, sorry!
Speaker:No comment, no comment, withhold judgment!
Speaker:But the photograph that was shared on the Internet was somewhat ambiguous in its
Speaker:lighting, and that ambiguity left the door open for each of our brains to interpret
Speaker:the visual experience of that image in a different way.
Speaker:And that's true for lots of things.
Speaker:Your brain is always drawing on past experience to interpret new information.
Speaker:Yeah, because I've always been told, like, the brain is essentially like, it
Speaker:recognizes patterns, and I guess it remembers patterns, too.
Speaker:So it goes back to the referred patterns
Speaker:to try to figure out what new patterns it might be trying to figure out!
Speaker:Exactly! Cause right, if you had to process every moment from scratch, like
Speaker:collecting, okay, what is all these sensory information I have about this?
Speaker:What does that mean?
Speaker:That kind of bottom up.
Speaker:Information processing can be more accurate, but it takes a lot more time.
Speaker:So our brains have these shortcuts of what we call top-down processing, that we're
Speaker:kind of going back to patterns or experiences or environments that we know
Speaker:from previous experience, and using that to interpret new information coming in.
Speaker:What would be, I guess, the equivalent in terms of audio?
Speaker:Well, let me play you an example! Please!
Speaker:Okay.
Speaker:All right, so I am going to play you a sound clip of a spoken sentence, but it's
Speaker:been transformed using a computer algorithm.
Speaker:All right? So let me play you that first clip.
Speaker:(unintelligable, robotic voice)
Speaker:I'll play for you one more time.
Speaker:(unintelligable, robotic voice)
Speaker:Okay, sounds like sad R2D2.
Speaker:Yeah, it sounds like a robot voice speaking maybe a language we don't know.
Speaker:Can I hear it one more time?
Speaker:(unintelligable, robotic voice) Is it saying something?
Speaker:It is saying something, but like I said,
Speaker:this particular sound file has been transformed.
Speaker:Okay, alright.
Speaker:So let me play you what the actual sound clip is now.
Speaker:Okay. The man read the newspaper at lunchtime.
Speaker:So let me go back and play you the
Speaker:original clip, which, I'll remind you, you both thought it sounded like gibberish.
Speaker:The man read the newspaper at lunchtime.
Speaker:Whoa!
Speaker:Wow, yeah yeah, that came through a little bit more clear.
Speaker:So once you know the information that it's supposed to be, your brain can now apply
Speaker:that to what sounded like meaningless information in the first place.
Speaker:Oh, my God. Yeah.
Speaker:I'm mind blown right now. Right?
Speaker:As soon as you know what you're listening
Speaker:for, it's like an informational bias, basically.
Speaker:Yeah.
Speaker:We have this exhibit at the Franklin Institute in the Brain exhibit.
Speaker:So, Bey, to come back to your original
Speaker:question is, what's the analog to the dress when it comes to sound and music?
Speaker:Think About misheard lyrics.
Speaker:What's your misheard lyric, Jayatri, that
Speaker:blew your mind when you learned the real words?
Speaker:I don't know what the one that blew my
Speaker:mind, but the one that stuck with me the most was Michael Jackson.
Speaker:It was years before I knew it was not The Man in the Middle.
Speaker:Oh, my God!
Speaker:Oh, Interesting!
Speaker:Wow!
Speaker:Because I'm looking at the man in the middle.
Speaker:And once you've decided those are the words, you hear them confidently, right?
Speaker:Yeah.
Speaker:Because I think as a kid, I heard that song.
Speaker:We always were playing, Monkey in the Middle.
Speaker:That was a phrase that my brain was familiar with.
Speaker:And Dr.
Speaker:Das followed up with, I'm asking him to change his ways.
Speaker:Yeah!
Speaker:That's so interesting.
Speaker:That's hilarious. Wow, okay.
Speaker:Yeah.
Speaker:So misheard lyrics are the equivalent of the dress.
Speaker:Yeah.
Speaker:So to all of our listeners out there now, you know what the science behind that is.
Speaker:Wow! Well, this has been sufficiently mind
Speaker:blowing, and it wouldn't be a Body of Knowledge segment without that.
Speaker:Oh, my gosh. I love it, love it, I love it!
Speaker:Well, Dr.
Speaker:Das, thank you so much for being here, wow!
Speaker:Love it.
Speaker:Thank you again to Jayatri for coming on the show.
Speaker:It's always such a treat to get her.
Speaker:And we have a jam packed show today, so
Speaker:we're moving right along to the last interview of the episode.
Speaker:Andrea Clearfield, welcome to the So Curious podcast!
Speaker:So happy to be here! Thank you so much for inviting me.
Speaker:Absolutely. This is exciting.
Speaker:We are pumped, can you introduce yourself and tell us what it is you do?
Speaker:Yeah! My name is Andrea Clearfield.
Speaker:I'm a Philadelphia-based composer, writing music for opera, chorus, orchestra,
Speaker:chamber ensembles, and multidisciplinary works.
Speaker:And I also host a salon concert series that's now celebrating its 36th year.
Speaker:Ooo, Congratulations! Wow.
Speaker:Thank you.
Speaker:And so, we have a lot to talk with you about today, in our short time, because we
Speaker:are learning about your work with synesthesia.
Speaker:Can you talk to us a little bit about that?
Speaker:What is synesthesia?
Speaker:What is your relationship with it?
Speaker:Yeah, it's something that I didn't even
Speaker:realize that I had, that maybe not everybody had.
Speaker:It was just the way my brain worked.
Speaker:So synesthesia is essentially when one sense activates or triggers another sense.
Speaker:The word synesthesia comes from a Greek
Speaker:word which means to perceive together, to perceive all together.
Speaker:So this one sense will then activate this
Speaker:other sense, and you're perceiving them together.
Speaker:So, in my case, I see colors to music, and
Speaker:I hear specific musical pitches to specific colors that I see.
Speaker:And I've heard that there are people who smell numbers or taste words.
Speaker:So there's all different kinds of synesthesia!
Speaker:Can you talk about your own personal experience?
Speaker:You know, it's different for everyone, but
Speaker:what is it like for you, and just kind of expound on that a little bit more?
Speaker:I'll do my best.
Speaker:I mean, it's kind of hard to describe
Speaker:since it happens involuntarily, but if I look at something red, any shade of red,
Speaker:I'll hear the note A or various related pitches to A.
Speaker:So could be A, a flat, A sharp.
Speaker:And even if I think about that color, I'll hear that note.
Speaker:And if I listen to music, I'll see the related color to that pitch.
Speaker:And even if I think about the music in my
Speaker:head, I'm not listening to it on the outside.
Speaker:I'll hear that note.
Speaker:So the color comes with it.
Speaker:So, for instance, A is red, b is kind of a
Speaker:grayish, and that one also comes with a feeling of velvet.
Speaker:C is yellow, and it's something that's bright.
Speaker:D is blue.
Speaker:And of course, there are all different
Speaker:shades of blue, depending on how the D is being heard, the musical D.
Speaker:E and F, they range between orange and brown, and G is green.
Speaker:I'm looking out my window right now, I'm composer-in-residence in Vermont.
Speaker:Even though I'm based in Philadelphia, I'm here at an artist residency program, and I
Speaker:have this beautiful studio. And I'm looking outside, I'm seeing all the trees
Speaker:on the mountain, and they're kind of swaying in the wind.
Speaker:And so I'm hearing these various shades of
Speaker:the note g that go along with the various shades of green.
Speaker:And can you walk us through how you might be perceiving these colors?
Speaker:How do they appear in your field of vision?
Speaker:Is it like an overlay?
Speaker:It depends.
Speaker:I'll talk about a work sample in a little bit where I was actually looking at a
Speaker:painting and seeing the colors in the painting.
Speaker:So then, yes, they were overlaid.
Speaker:And so then the pitches formed chords.
Speaker:There was almost like a vertical stacking of pitches that went along with the
Speaker:overlay of colors in the painting, but it can also be very pure.
Speaker:Wow.
Speaker:And so when there are things that have colors that people maybe put together, and
Speaker:let's say that in the way that you hear those, those are dissonant or clashing,
Speaker:does that kind of stuff, like, give you a headache or bother you when you're looking
Speaker:at - you're hearing a chord that is, like, very clashed and dissonant?
Speaker:No, that doesn't bother me. Okay.
Speaker:I listen to all kinds of music and write music that's very tonal or very disonant.
Speaker:So nothing really surprises me in that way or bothers me.
Speaker:But I did experience a kind of a
Speaker:limitation at one point in my own composing process,
Speaker:in that I found that whenever I was writing something that was energetic or
Speaker:compelling and fast, I was seeing the color red.
Speaker:And so that movement would be centered around the pitch A, right?
Speaker:Because A is red for me.
Speaker:And then I started to look at my oeuvre
Speaker:and I thought, "wow, all my fast movements are around A." Like, not necessarily an A
Speaker:major or minor, but centered around that pitch.
Speaker:And so then I went through a whole period
Speaker:of time where I was trying to push against the synaesthesia.
Speaker:It took some energy, but I was able to do that, and challenge myself to write fast
Speaker:movements that had a different color scheme.
Speaker:However, when I was commissioned to write
Speaker:my first opera, which took seven years and was produced in 2019,
Speaker:I couldn't really do that, because in the libretto, the writers had written a
Speaker:libretto - or a play - with characters that had colors associated with them.
Speaker:So, this was a Tibetan theme, and there
Speaker:were these fantastical bird -like deities that had colors.
Speaker:So there was the red deity and the green deity and the yellow deity.
Speaker:And I just thought, "okay, I'm going right with whatever I see and whatever I hear",
Speaker:because it just didn't seem right for me and my creative process to try to take the
Speaker:energy to push against what came naturally.
Speaker:Wow. So I went back to allowing the sensations
Speaker:to shape the notes that I chose in the opera.
Speaker:(sample from the opera)
Speaker:Is there any specific works you could you
Speaker:could tell us about and explain how your synesthesia guided that creation?
Speaker:Yeah, I was thinking about this.
Speaker:I wanted to tell you the story of the
Speaker:first time that I wrote a piece directly related to the synesthesia.
Speaker:That was my intention with the piece.
Speaker:And I also really love collaboration, so I love collaborating with visual artists.
Speaker:I always -Me too.
Speaker:Like to spout off ideas.
Speaker:And it's so interesting when you get together in a room with other artists and
Speaker:it's like, you can create these things you never would have done alone.
Speaker:So I love that with visual artists, filmmakers, dancers.
Speaker:This was in 2009.
Speaker:I was invited to an artist residency program near Santa Cruz in California
Speaker:that's called Djerassi, and I was the only composer.
Speaker:There were also writers, and painters, and dancers.
Speaker:And although I was working on my own projects - it was a vocal chamber piece to
Speaker:poetry - every day, I would walk by a painter studio on the way to our meals.
Speaker:And he had his large paintings hung up on
Speaker:the walls, and they were vibrant and colorful and big shapes and blocks, and
Speaker:they would grab my attention as I walked by every day.
Speaker:And I kept hearing music every time I passed the studio.
Speaker:The painter's name is Heiner Riepl, he's a German expressionist painter.
Speaker:And so I asked him one day, rather hesitantly, if he would allow me to put
Speaker:one of his paintings in my studio so I could write down what I heard.
Speaker:And he generously agreed.
Speaker:And he hung the painting on the wall in front of the piano.
Speaker:So I followed all the blocks of color, which created these chords and patterns.
Speaker:And it was the first time I actually tried
Speaker:to write down the music directly from what I saw.
Speaker:And there were dancers who were also in residence, and they were staying in the
Speaker:same barn where we all had our studios, and they heard me composing, and they
Speaker:liked music, and they wanted to dance to it.
Speaker:So, they ended up choreographing to the music that I wrote to the painting.
Speaker:And we brought all three disciplines
Speaker:together on the final day of our residency for a performance.
Speaker:I played piano, the dancers danced to it.
Speaker:They also dressed in the colors of the painting.
Speaker:And the piece is called Cal-3, which is the name of Heiner's painting.
Speaker:It was part of his California series.
Speaker:The dancers were Debbie Kagyama and Jose Navarrette.
Speaker:And I have an audio file that's from a
Speaker:rehearsal of me playing on a little upright piano.
Speaker:And then there's also a video.
Speaker:(music sample from Cal-3)
Speaker:That's awesome.
Speaker:Wow, that's like, genuinely, the definition of collaboration, right?
Speaker:It's the ultimate. Wow.
Speaker:I love it.
Speaker:You taking a visual art, turning it into what you hear musically -
Speaker:Someone else hearing it and saying, "let me dance to it."
Speaker:Wow.
Speaker:Thank you for joining us.
Speaker:It was a pleasure to chat with you.
Speaker:I have just learned an infinite amount
Speaker:that's going to take me a while to process!
Speaker:Yeah, andrea Clairfield, thank you so much
Speaker:for coming on So Curious, we appreciate it.
Speaker:Thank you! Bye!
Speaker:Thank you so much, Andrea, for taking the time to come and hang out with us today.
Speaker:Coming into this interview, I didn't know
Speaker:what synesthesia was, and I kind of just played along.
Speaker:I was like, "yeah, I know what that is." I did not know what that was!
Speaker:I mean, now that you know what it is, that's pretty dope, right?
Speaker:That was pretty mind blowing to me.
Speaker:Now I'm going to be, like, specific about what I wear. Because it's like, what if
Speaker:when I wear a lot of colors, I'm walking down the street and someone passing me has
Speaker:synesthesia and they're hearing like, a whole song.
Speaker:Is it a good song? Oh, my gosh.
Speaker:I love it, love it!
Speaker:So be sure to join us next week where we
Speaker:explore the future of music technology, and what it looks like.
Speaker:And so we arrived at this idea of doing a
Speaker:musical instrument competition, where the competitors don't play a musical
Speaker:instrument, but they invent musical instruments.
Speaker:All right, this is going to be a fun one.
Speaker:So please make sure that you subscribe to
Speaker:So Curious wherever you listen and don't miss out.
Speaker:And you know what I'm going to say, please leave us a zero star review.
Speaker:Just kidding! Please leave us a five star review.
Speaker:It goes such a long way.
Speaker:Go into the reviews and write what it is
Speaker:that you learned most from this podcast so far.
Speaker:And specifically, do you have any colors that you associate with any sound?
Speaker:Because I'm curious. Oh, I'm curious!
Speaker:Ding ding ding! She said it.
Speaker:Hit the button.
Speaker:Also, if you have synesthesia, oh, my gosh!
Speaker:Email us. I really want to know!
Speaker:You can email us at socurious@ fi.edu. It'll be in the show notes.
Speaker:Please go ahead and subscribe, leave us a
Speaker:review, and be sure to tune in next Tuesday!
Speaker:Wherever you listen to podcasts.
Speaker:This podcast is made in partnership with
Speaker:RADIO KISMET, Philadelphia's premier podcast production studio.
Speaker:This podcast is produced by Amy Carson.
Speaker:The Franklin Institute's director of digital editorial is Joy Motefusco.
Speaker:Dr.
Speaker:Jayatri Das is the Franklin Institute's chief bioscientist, and Erin Armstrong
Speaker:runs marketing, communications and digital media.
Speaker:Head of operations is Christopher Plant.
Speaker:Our mixing engineer is Justin Berger, and our audio editor is Lauren DeLuca.
Speaker:Our graphic designer is Emma Seager.
Speaker:And I'm The Bul Bey.
Speaker:And I'm Kirsten Michelle Cills. Thanks!
Speaker:Thank you! See ya.
