Can AI Hear Cancer in Your Voice? – Chasing Life with Dr. Sanjay Gupta

Welcome to Chasing Life. You know, you may not notice in your everyday life, but your voice probably says a lot about you. Some voices are very iconic.

No, I am your father.

Recognize him? That’s James Earl Jones, and he’s voicing the infamous Star Wars villain Darth Vader. He also was the powerful voice of Mufasa from The Lion King.

The sun will set on my time here.

He even lent his voice to this network.

This is CNN.

The thing is all voices have something to say, iconic or not, but the point is they don’t just articulate words. They can signal a lot more than that. They can signal how you’re feeling, physically, mentally. They can conjure up memories, just the voice alone. They can even reveal secrets.

I remember I was at a soccer game with my kids, so there was a lot of noise around. She said, hey, how are you? And I said, you’re pregnant. And she said, how do you know?

‘Dr. Yaël Bensoussan is a laryngologist. She is director of the University of South Florida’s Health Voice Center. She is co-leading research there on using AI and the human voice to try and detect diseases and perhaps even treat them.

If you think about your Alexa at home, your devices, Siris, that can integrate vocal biomarkers to help understand your health, that’s really where the future is going in my opinion.

The technology isn’t there yet, but Dr. Bensoussan joins me on the podcast today to talk about what voice can do, why voice health matters, and how exactly your voice can tell you so much about your overall wellbeing. I’m Dr. Sanjay Gupta, CNN’s chief medical correspondent, and this is Chasing Life. How does my voice sound?

Your voice sounds great and healthy. And I listen to voices for a living. So I think you actually sound very healthy.

Good, good. That’s a good place to start the podcast today. How did you get interested in in this area?

‘Uh, so, uh, basically I was a singer and songwriter, so I was always very passionate about voice. I was about 15, uh and one day I was at my singing teacher and she said, you sound like something’s wrong with your voice and I’m uncomfortable to make you continue singing, uh so you should really go see a laryngologist. So I went to see a laryngologist, uh they put a little camera down my throat and they said, You have vocal cord nodules, uh which are little things that are not cancerous. Little areas on the vocal cords and then I was able to go see a speech pathologist to do rehabilitation and get my voice back and then i was able to sing again and then later on I did my studies in speech pathology which is kind of a physiotherapist for the voice so a voice rehabilitation expert and then, I went into med school and I got really interested in surgery so for me it was really life-changing. And I got to understand what it means when you lose your voice or you lose the ability to sing or you loose the ability communicate with people. And that was really powerful to see what it can do to help rehabilitate the voice. So that’s what got me to do what I’m doing today. I always said when I grow up, I wanna run a voice center. And here I am, I’m a grown up I think, and I run a voice center at USF.

So what is your life like now then? You’re singing and you’re a voice surgeon. You’re doing both.

So I don’t sing anymore except in my shower or to my kids, but I’m an ENT surgeon and I specialize in the T of ENT, which is the throat, and I do voice surgery. So I direct this voice center. We work with laryngologists, surgeons like me, and also speech pathologists. We also collaborate with singing teachers, and we see patients with various voice disorders.

So who is your typical patient?

So there’s a lot of different types of typical patients, I would say. So we could see singers, voice professionals, and we talk about voice professionals, not only singers. So people like you, hosts, anchors, teachers are also voice professionals. They use their voice every day for a living. We can see people with vocal cord cancers or vocal cord lesions that are benign or malignant. So cancerous or not cancerous. We see people with neurological diseases that affect their voices. We see people with complications of voice complications of other surgeries. So vocal cord paralysis, for example.

So, you know, when I first started reading about your work, there was this anecdote that was shared that you were talking to someone and from their voice, you were able to tell that they were pregnant. Yes. Is that an accurate story?

It’s an accurate story and I think it’s important to say, so this is a friend of mine who did residency with me. So she’s also a surgeon. So I knew her, right? I remember I was at a soccer game with my kids so there was a lot of noise around. She said, hey, how are you? And I said, you’re pregnant. And she said, how do you know? I said well, you sound really stuffy. You know, your nose sounds really stuff. Also, you don’t call me often. So I’m imagining that you’re gonna announce me something. So obviously you know, it’s not like I took the voice out of nowhere. I knew how old she was, but I could tell just from the sound, because when we’re pregnant, there’s more estrogen in our body. The mucosa of our sinuses of our nose are bigger. So we sound kind of congested. And that’s how I could tell.

‘That that’s that- I mean again i know this is your world so you’re you’re used to this but that that that pretty fascinating i think for the average person to hear that so much could potentially be told from voice. We’re not saying it’s a hundred percent accurate but but the idea that it could be a biomarker as they say. I’ve heard about this, doctor, uh with regard to something like Parkinson’s and there was the studies that came out that said through voice even before someone started to develop physical tremor even before they had a diagnosis or knew that they had symptoms, people could hear changes in the voice. And that sort of made sense to me because your larynx and all that, that may be more sensitive, so you’re starting to pick up things there. But you, and obviously if someone’s sick, congested as you were talking about, but all these other things, tell me about some of these other things that our voice can tell us about ourselves.

Yeah, so I always think about it as three separate categories of the diseases. So the first category is more intuitive things that affect the voice box, the larynx. So if you have a voice box cancer, obviously it’s going to affect your voice, you know, that’s going change your voice. If you have vocal cord paralysis, that is going to change your voice. The second category, what you talked about is things like neurological diseases, like Parkinson’s, right? So it doesn’t necessarily affect the voice box. But the way we speak, it affects our brain. It affects how we control our muscles, so it’s going to affect our speech, and then you can think about things like Alzheimer’s disease or other, you know, having a stroke is going to affect your speech. And then there’s a third category that’s really coming out with more of the AI analysis because AI can analyze a lot more data and more precisely things like diabetes. I cannot tell by listening if somebody has diabetes or not, but people have developed algorithms to hear that. There’s labs working on hypertension, same thing, atrial fibrillation, AFib. So we can not only diagnose specific diseases or screen for specific diseases, but we know that overall health has a lot to do with our voice. So when you pick up the phone and your friend calls you, you can say, hey, you don’t sound good. So you as a listener that is not an expert listener can say you don’t sound right. And then what exactly does it mean, right? Then for example, if we bring that patient to somebody, a lung doctor, they can say, yes, yes, you don’t sound right. It sounds like you have a pneumonia. So that’s what we’re really trying to recreate with artificial intelligence. Kind of taking that expertise from every specialist where we can hear things because we know the disease and really convert the you don’t sound right to why exactly you don’t sound right.

Yeah, and I guess I’ve usually associated that with their mood, right? Sort of more from a mental health standpoint. You sound sad or you don’t sound as enthusiastic. The idea that you could then extrapolate and say, this person has hypertension or has diabetes. That is wonderful if it’s possible because think about the implications. If a patient’s in my office and I’m laying eyes on them, I can often get a good sense. But the idea that I could talk to them on the phone. And start to discern that they might be diabetic or having hypertension. So what do you think this means fundamentally? How good is this potentially as a biomarker? If you measure cholesterol for heart disease, you have very good sort of objective data, obviously physical exam findings of patients in your office. How good or how important could voice be in all this?

It’s really important to understand the use cases of voice biomarkers. As you know, telehealth is now very much used in the world, right? We help people from home. So in that context, if we can converse, and then at the end of our conversation, as a doctor you get a report saying, well, your patient has a high risk of Parkinson, or he sounds really depressed. That’s really helpful. The other really breakthrough that we’re seeing is in ambient scribes. AI is listening to you when you go to the doctor. There’s a microphone that listens to your conversation with the physician and instead of your physician typing the notes and not looking at you, he can actually converse with you or she can converse with you, and then at the end you get a summary saying the patient comes for blah, blah, and this is kind of the note at the end. And the ambient scribes were invented to help with administrative tasks of the physician so that we don’t have to take notes. But now if you combine ambient scribes with the vocal biomarker technology, that becomes super interesting because at the end of your encounter with your physician, then they get a report saying, well, this is the physical exam, this the clinical history, but also from the voice, this what I hear and this is test that you should be doing. So I personally think that’s where the future of vocal biomarker is going and then obviously even at home, right, if you think about your Alexa at home your devices, Siris, that can integrate vocal biomarks to help understand your health, that’s really where the future is going in my opinion.

‘It is so fascinating to think about the fact that obviously you’re communicating via your voice, but the idea that it could also at least be a, in part, part of a diagnosis, um, I, I think is really interesting. You, you, you talked about this, but what are some of the things- so diabetes, hypertension, at least with your friend, you were able to determine that she was pregnant. What are some other things?

Alzheimer’s, there’s a lot of research on Alzheimer’s and even what we called MCI, so mild cognitive impairment can be detected. So obviously Alzheimer’s disease, you can tell when somebody has Alzheimer’s.

‘And just to be clear, that’s not- like with someone with Alzheimer’s, they may not remember words, but that’s not what you’re talking about here. You’re saying from their voice itself, you might be able to develop a biomarker for cognitive impairment.

It’s a very good distinction. So, you’re bringing up the distinction between voice and speech, which is very important. I always say birds can voice, but they cannot speak. So, speech really takes into account the articulations of the word, but also some cognitive implications. So, we can really separate them, but when we talk about how voice can talk about our health, we also include speech. So you’re correct to say it’s also the words we use, how fast we use them, right? So there’s things like rate of the speech. In Parkinson’s, for example, the voice and the speech are different. The voice is monotone. So instead of kind of singing when we talk, like me, we talk on a more monotonous voice. So that’s a vocal biomarker, but then the speech rate is also different. So the speech rate is also slower. If you measure how many words per second, that’s slower. So that a speech biomarkers. So when we get information, we can get information both from the voice itself and from the speech.

Where are we in this discussion then as a medical community? Because you’re talking about a very specific thing and obviously you’re trained. You have a trained voice, you have a trained ear. But as a, as a medical community overall, how much are we starting to think about voice and use voice as a real biomarker?

So there’s two things, are we using it in our everyday life by listening? You know, that part I think people are not necessarily trained, but they can’t be, right, because they don’t hear those things a lot. So that’s when technology comes in. That’s when we have to come and support with technology and where AI can help and then we can develop technology to help give these tools to physicians to say, hey, you’re not used to listening to other tons of voices, but I developed an AI tool that was trained with thousands of voices and I could give it to you and at the end of your encounter they could tell you you know what they hear. We don’t have a product that talks about diagnosis yet because there’s no FDA approval of these products but every company is kind of racing to get this FDA approval. So we’re not far from that.

But what is voice then? Because it strikes me that if you’re trying to train platforms to do this, you could obviously, you could train them on lots and lots of different voices and correlate those voices to different diseases. We think of sound as being these patterns of electrical energy. Is that the same thing that we’re talking about with voice? Is it broken down into specific patterns that are then analyzed and used to diagnose something?

Yes, great question. So you can extract a lot of things from what we call an acoustic signal. So voice is an acoustic signal, right? It’s energy going through the vocal tract. It comes from your lungs. Your vocal folds vibrate, creates a wave of energy that then goes through what we called our resonators. And then we have articulators. So all of that creates an acoustic sound, a waveform. Then you can, in machine learning or in AI, you can analyze the sound in different ways. One method is to change this in what we call a spectrogram image, because AI was really developed to analyze images. So we convert the sound into an image, into a spectrogram, and then the AI analyzes the image. You could also extract what we called features from the sound. So amplitude of the sound, so how loud the sound is, what’s the fundamental frequency? What’s the pitch? And all these things that you can extract, you can also analyze. And to give you a parallel, when I did my studies in speech pathology, I had to listen to a video of a kid who stuttered. And then I had count how many syllables he repeated per word. And then to assess the severity of the disease, I would have to say, well, he repeated three syllables per word. So that’s severe versus two syllables per sentence. So now a software engineer or an AI engineer could tell me, yeah, Yaël give me 15 videos of kids and I can count to you the rate of, of repetition of syllables and I’m, I’ll be done tomorrow, you know, versus that homework took me like a week of listening to videos.

I’m always blown away by some of the examples of how AI is being used in healthcare to really sort of expedite some of the analysis like this. I gotta tell you a quick side story. Years ago, these people presented me with this technology using voice analysis to basically determine stress levels, which I thought was really interesting. And you talk on the phone and then after the phone call was over, you’d get a readout. And basically it was a number, but it was mostly color coded. So you could tell if you were in the green, which meant that you didn’t have much stress, or if you’re in orange or red, which obviously meant more stress. And I got really fascinated by this. I would start to use it and I’d have these phone calls with people. And I remember I was driving home from the hospital once and I called my wife, as I often do as I’m driving home. And we were having, I thought a perfectly good conversation. And then I hung up, and 10, 15 seconds later, I got a readout, and the readout was that I was firing in the red. And it was surprising to me. So I called her back, or I guess I got home, and I said to her, I said, hey, how do you think that conversation went that we had earlier? She goes, no, no it was fine, why do you ask? And I said well, you know. She goes, you had your hospital voice on. I said my hospital voice, what is my hospital? She said it’s very procedural, very matter of fact, very to the point. And I thought it was fascinating, you know, it wasn’t something that I was consciously aware of, but I’ve made a sort of a distinct difference in my own life after sort of getting that feedback. But when I talked to the engineers who created the product, I said, hey, can I fake it? Today, I seem happy, right? Everything is great in life. And they said that was sort of the amazing thing about voice, is that it’s very hard to fake. Because you have these things that are coming from your brainstem, that are going to these muscles in your larynx. Had you heard of that? Is that ring true to you?

It could be, but we have to be very careful. You know, the technology is there, but it’s not perfect. So we have be very carefull with how excited, you know, and how hyped we get about it. What happens when you put a voice that is sick, right? Let’s say you had laryngitis. Would it analyze that you’re stressed or not? And the question becomes, how did they train their model? Did they train there models just with healthy people who are stressed and healthy people who are not stressed? If that’s the case, if you’re sick, it’s gonna say you’re stressed because it’s going to note that there’s a difference. So we have to be careful also in trusting the technology fully. And that really comes with all the ethical question and the trust question in the technology, right? If you’re a doctor and you’re adopting a technology, you want to be sure that it’s accurate. And for it to be accurate, it has to be trained on a lot of different people. If it was just trained on people that were healthy, stressed or not stressed, then if you present it with somebody that has a vocal cord paralysis, of vocal cancer, it’s going to say it’s stress. So that’s where we have to be careful with the technology and that’s why it’s important to train it on lots and lots and lots of data.

Culturally or across different languages, does that seem to make a difference?

It used to when the technology was not developed yet. So you’re correct, obviously there’s different languages that have different tones, different accents. And when we started studying this about five years ago, it mattered, right? You developed an algorithm in English and then you tried it on Indian speakers and it just didn’t work. And now with the technology that’s really evolved and the fact that we can train with a lot more data as well, it seems to be less important.

And then the other question, you know, the other day I had a conversation with this person on the phone. I assumed it was a woman that I was talking to was probably in her 30s. And then I later came to find out she was in her late 70s. And I don’t know, I guess that surprised me. How much does our voice change as we age?

That’s one of the things I get really fascinated about, is what is your voice age?

Yeah.

‘So there are physiological changes with our voice. So specifically when a five-year-old male or a five year old female, you can’t tell the difference. Then when they grow up, obviously male’s voice changes, right? And becomes more masculine. Female voices tend to stay the same until really 20, 25 year old. And then our pitch lowers. As we grow older. And for males, it’s the opposite. When they age, they start going up and the curve at the end of our life touches again. So often, you know, if your 90-year-old grandpa or your 90 year old grandma calls you, they might sound a lot similar than they did when they were, you now, 50.

That’s interesting. So forgive me if you just said this, but what is a older voice sound like then? Is it, I imagine it’s both voice and speech, right? Again, going back to what you were talking about earlier, but what is the 90 year old voice versus a 50 year old voice?

Great question. So let’s separate voice and speech. So our vocal cords, as we age, they’re muscles, and then there’s also a lot of collagen around it, and as we aged, like everything else, there’s some muscle atrophy, there is some loss of collagen. So we’re going to have a breathier voice, okay? So because the vocal cords are not coming together fully, there’s kind of a little bit of a gap when they vibrate. There’s air that escapes, so it’s usually a weaker voice. So when we talk about an older voice people tend to associate it with a weaker voice. So when I have patients that have a vocal cord paralysis, meaning one of their vocal cords not working, there’s a lot of air that escapes, we associate it to our grandma calling. But there is actually a disease that’s called vocal tremor, where people that are 50 year old can get tremor of their voice.

Just real quick, RFK Jr., the Health Secretary, I think everyone has sort of heard his voice at this point. What is going on there?

‘I don’t know because I’m not his doctor so I’m not giving any medical opinion but he sounds like he has spasms in his voice which we can associate with diseases called spasmodic dysphonia. So I’ve never assessed him but I can hear spasms in his voice. It’s very debilitating as you can hear and people have to get different types of treatments. Some people can get botox injections to their vocal cords when they have laryngeal spasms. Some people get voice therapy, for example. And you know, he’s not the only famous person who we can hear a change in his voice: President Biden. President Biden towards the end, the speech was getting slower. We analyzed that and it’s really interesting. I think there was a really nice interview — I don’t remember who did it, but they could put together President Biden’s speech from the beginning of his term and the end of his turn. And his voice sounded a lot older. And people, when they hear that, it’s also interesting the social implication of that, right? When we hear somebody that sounds older, we say, well, you know, he’s not as strong. So that’s a lot of my work also as a voice doctor to make sure to talk about this so people don’t take voice disorders as a weakness. It’s a disability like any other disability. And I could tell you that in a day of clinic, there’s at least two or three patients that cry in my office because our voice really, it’s our identity.

You know, I hadn’t really thought of it in that way. I mean, how much of our identity is wrapped up in our voice. It’s really interesting to think how much of who we are is tied to our voice and to our speech. Where does this go? If I talk to you in five or 10 years from now, how much do you think voice as a biomarker will be a thing?

‘Because of the ambient scribe’s technology really taking a place, I think it’ll be really important what we do with AI agents. We’ll be listened to, right? You’ll have maybe a little robot at your house that you can talk to. The technology is going to listen to us. That’s what we’re trying to do is to help vocal biomarkers take a bigger place in the market by collecting lots of voices from lots of individuals. We’re developing this huge database of voices from patients. And then we also need to make sure they trust us. How do we ethically represent these patients? So that is a huge part also of our work that we do with the Hastings Center and Dr. Ravitzky, Dr. Bélisle-Pippon, that are ethicists, bioethicists that are looking at what are, what, how do we protect patients? Right? Because you can also clone a voice. You can harm someone. You can call them and say, hey, I have your daughter here. And she’s going to talk to you if you don’t give me two million dollars. So these are things that are happening so we can do great things with technology and we can do bad things with technology. So I think to your question, it’s going be very present in the next five years, both in the good and in the bad. And our job is to make sure we fuel innovation for the good of people and our patients, but also protect patients from the bad that the technology can bring.

That’s really interesting. And I also can’t help but think that there’s going to be certain populations of people who are going to benefit from this, what you’re talking about earlier than others, people who may live in more rural areas, who may not have access to physicians, may not be able to get to your office to be able to do that telehealth and, and collect voice biomarker data. In addition to everything else, I think might be really helpful for certain populations.

I think that’s where probably the future is, right? To help people that are remote, that don’t have access. So again, is the AI going to replace a laryngologist in an academic center? Probably not. But there’s not a lot of us. There’s just a few of us, there’s about a couple hundred of us in the country. And people wait often six, eight months to see somebody like me. So if we can get these technologies to people earlier, to know which specialist to find. I think that’s really important, unfortunately, and I think that’s the advocacy part. I cannot tell you enough how many times I get patients in my office and they say, well, I’ve had this for three years but I was told it was reflux or I was it was allergies. And then I diagnosed an advanced laryngeal cancer that if I could have diagnosed it earlier, if somebody has sent it to me earlier, they wouldn’t have that much of a bad cancer. It would have been curable. But I think we have to think broader, right? Three weeks of voice change or more, of anything, could be any other diseases, and that’s important that when you hear, when your voice changes, if it doesn’t go away in three to four weeks, you should seek medical attention.

That’s a great note. Hopefully that’s something that, that’s news people can use for sure. By the way, I have to tell you that if my mom is listening, which she listens to this podcast all the time, she’s gonna be delighted with this podcast because she has an incredibly youthful voice. She’s in her early eighties now, so hopefully she has many, many more years in front of her.

Wonderful. I hope so, too.

Thank you, doctor. Thanks for joining us.

Thank you so much, this was super fun.

That was my conversation with Dr. Yaël Bensoussan, director of the University of South Florida’s Health Voice Center. Thanks so much for listening.