The science of experiencing more than one sense simultaneously
Dr. Sean Day experiences sound differently than the rest of us.
When Dr. Day hears, say, a song by Taylor Swift, he doesn’t just hear the words about breakups with John Mayer or Tom Hiddleston—he can see the music alongside of it. Dr. Day first became aware of this unique sensory experience at the age of 5.
This is called synesthesia—a neurological trait in which a person experiences one sense simultaneously with another. As time went on, Dr. Day became aware that the synesthesia experience extended to other sensory modalities: He realized that every bite of food produces a visual sensation—meaning every time Day eats something, he sees something in front of him.
Different smells will evoke different visual sensations for Dr. Day too. The scent of cinnamon, for instance, might trigger a tingling in his fingertips.
Dr. Day, who holds a Ph.D. in linguistics and currently teaches at the department of Behavioral and Social Sciences at Trident Technical College in Charleston, South Carolina, has spent the entirety of his academic career researching synesthesia, found by some estimates in 4 percent of the world’s population.
“It is not a disease or a condition,” he said. “It is a trait along the same lines as being left-handed. You might be approaching the world differently and you might need some accommodations.”
There are two kinds of synesthetes (that’s the technical term for people who have synesthesia). Those experiencing the more common version of synesthesia do so through concepts, where letters or numbers or days of the week take on colors or flavors or sounds. The less common type of synesthesia is Dr. Day’s, where stimuli in one sensory mode produces a reaction in another sensory mode.
Dr. Day is a rarity in that he experiences the less common type of synesthesia in more than one form.
Dr. Day’s childhood was also rare compared to many young synesthetes of his generation, who often found themselves mocked, isolated, and alone. Growing up in the Midwest in 1960s and 1970s in a scientifically literate family, Dr. Day understood precisely what he had from an early age. He had heard about other forms of synesthesia and was determined to seek out others like himself.
Because members of his family were musicians who traveled widely, Dr. Day would often run across people from other bands and orchestras.
Dr. Day was cognizant that synesthetes were often drawn to the arts. He was also curious. He made sure to always ask if they experienced what he did or if they knew of someone who did.
This ground-level outreach would serve him well as he started his formal research in his early 20s.
“Most kids with synesthesia have major problems in grade school and in high school,” Dr. Day said. “And they’re wondering, ‘Why am I different?”
He said this only fueled his passion for understanding more about the neurotype and understanding how to help those who have it. He asked himself, How can we set things up so they do have the information they need and so they don’t feel like a freak when they’re in junior high school? Answering these questions became his became his life’s work.
Father of modern synesthesia
A general fascination with synesthesia led to the first serious study of the neurotype in the 1900s. That fascination had died out when John Watson and the Behaviorists left their imprint on psychology from the 1930s to the 1960s, which favored the study of direct, observable behavior over inner, subjective psychological experiences. .
As the study of cognitive science took hold, perhaps it was inevitable that the study of synesthesia would be taken up again, that people would make sure that young synesthetes would not remain siloed, their requests for help completely ignored. The question was, who would carry the torc?
Enter Dr. Larry Marks. As a graduate student at Harvard University in the 1960s, Marks’ Ph.D. advisor was no other than Dr. George Armitage Miller, the “Father of the Cognitive Revolution.” It was while working with Dr. Miller that Dr. Marks became interested in synesthesia, that once vibrant topic that had been cobwebbed hundreds of times over for decades.
This renewed interest would crest with Dr. Marks’ June 1975 cover story in Psychology Today.
Titled “The Lucky People with Mixed-Up Senses,” the piece served as a prelude to his groundbreaking 1978 book: The Unity of the Senses: Interrelations Among the Modalities.
Then a faculty member at Yale, Dr. Marks’ work helped re-establish a field of study and validate a way of life. Many also credit him with kicking off the celebration of neurodiversity, though it would be sometime before the term would become part of our shared language.
“One of the problems we face in psychology has been that people get a vision of the world and of people,” said Dr. Marks, now a professor emeritus in epidemiology and psychology at Yale University. “Then they try to pigeonhole everything to fit that scheme.”
In many ways, Unity of the Senses was a callback to the work on synesthesia of that earlier time. But it also integrated the new, cognitive science of the day. Implicitly, Dr. Marks told the world: We’ve moved past the rigidities of the Watson gang. Let’s go! He summarized his approach to Unity as ground-breaking for synthesthetes.
“Now we can go on and talk about the things that are important to people who experience synesthesia,” said Dr. Marks. “This is an essential part of their world. This is how synesthetes experience something—whether it’s conversations or music or art or just everyday life.”
His work used synesthesia to explore the broader human experience, helping us understand how the senses work with one another to provide information about the world. That topic now is often referred to as “multi-sensory integration.” What Dr. Marks didn’t try to do was speculate on the cause. But as new technologies developed with brain scans, new people went looking. And each time, new failures arose.
Dr. Marks believes this speaks to the unique experience of each synesthete.
“Synesthesia is a singular word,” Dr. Marks said. “But just because we have a singular word for synesthesia doesn’t mean it’s one kind of thing. Different forms of synesthesia have some properties in common—a crossing in the modalities or dimensions or domains. That’s true of all forms of synesthesia.”
He continued: “Are different forms of Synesthesia expressions of one thing? Or do they have something in common other than the word we use? I don’t think anybody could currently record the neural traffic in all the pertinent areas of the brain, but [with Artificial Intelligence (AI) and machine learning], we’re getting close. You should be able to look at the activity of very large numbers of neurons and, with enough computational power, figure out what it is that’s being said or heard. An AI computer of the future should be able to tell me, for example, which areas of the brain interact or overlap in their responses when there are synesthetic responses. When, for example, music produces synesthetic visual experiences of color, shape, and movement, it does so because there is, at the same time, corresponding neural activity in the visual system.”
Following the Marks
At present, the scientific community knows a few things about where synesthesia comes from. According to Dr. Richard Cytowic, the author of Wednesday is Indigo Blue and The Man who Tasted Shapes, the hereditary basis for synesthesia is strong. For instance, in a 2002 book titled Synesthesia: A Union of the Senses, Dr. Cytowic found that a child whose parent has synesthesia has a 50 percent chance of inheriting it.
In a recent interview with BrainWise, Dr. Cytowic waxed poetic about that work.
“What’s interesting is that you don’t get just one type of synesthesia,” he said. “But you may end up with multiple types. If you have colored-hearing, you’ve got a 50 percent chance of having a second, third or fourth type of synesthesia.”
Yet, as Dr. Cytowic, said, there is something of the synesthete in all of us.
“We do cross-sensory integration all the time,” Dr. Cytowic said. “Every time we’re lipreading, we’re watching what a person is saying. Sound and sight are always so inter-connected. Even a bad ventriloquist can convince us that the dummy is talking. Even with cinema, we’re convinced that the dialogue is coming from the actor’s mouth on the screen rather than the speakers that are surrounding us. Cross-sensory connection is the rule, not the exception. It’s just that synesthetes are aware of these connections on a conscious level, whereas the rest of us are not.
It took four decades for synesthesia to emerge from a secret you kept hidden from those closest to you to something freely talked about by the creative crowd. In the last few years alone, artists such as Billy Eilish, LL Cool J, Chris Martin, and Lady Gaga have “come out” as synesthetes.
It follows a certain logic. The best artists strike out across all of our senses, emotionally move us in ways that we can never really explain.
Psychologists V. S. Ramachandran and David Brang have argued that synesthesia, and the “cross-wiring” that comes with it, gives synesthetes natural advantages when it comes to sensory processing.
Indeed, without Synesthesia, Pharrell Williams told author Maureen Seaberg, he’d be lost.
“It’s my only reference for understanding,” Seaberg quoted Williams as saying. “I don’t think I would have what some people would call talent and what I would call a gift. The ability to see and feel (this way) was a gift given to me that I did not have to have. And if it was taken from me suddenly, I’m not sure that I could make music. I wouldn’t be able to keep up with it. I wouldn’t have a measure to understand.”
Seaberg herself is also a synesthete. The author of three books on synesthesia, including Fearfully and Wonderfully Made: The Astonishing Science of the Senses, she counts herself as a rare “polysynesthete,” just like Dr. Day She sees the color aubergine with the number 8. She experiences the letter K as teal. She experiences the even rarer mirror touch synesthesia, which causes her to feel the pain or pleasure of other people.
“One thing we’ve found is that synesthetes are really good with metaphor,” Ms. Seaberg said. “Because metaphor is everything in terms of just being an innovative, creative person.
She continued: “I went through all the song lyrics of the people who were self-identifying as synesthetes. The one example that was most crystal clear was Pharrell Williams in his song, ‘Happy, singing, ‘Clap along if you feel like a room without a roof.’ What the heck is that? And yet it makes sense. After you hear this unique and stunning metaphor that’s never been used before, you’re like, ‘Yeah, I might be happy in a room without a roof. I could see the sky, right?’ But who says that? No one. [Only] synesthetes come up with things like that.”
This article has been factchecked by members of NAN’s Publications Committee. For more about that process, click here.
For a closer look at how synesthesia empowers an artist named Penelope Moore, check out this companion piece.
