From something as simple as the sound of your breath, Rita Singh, a researcher at Carnegie Mellon University's Language Technologies Institute , can pull information about what you look like and the room from where you are calling.
“Every person's voice is unique, just like your fingerprint and DNA, so we are on our way to converting voice to kind of like a barcode to identify every human,” said Singh, who has been studying voice for decades. “Basically, we are trying to sketch the entire persona of a human and their environment.”
Before you gasp, think about it, Singh said. How many times do you formulate a picture in your mind of a person you are talking to on the phone who you've never seen in real life? And it's easy to tell the difference between a phone call from the car, a room in a house, a large cathedral or, god forbid, a bathroom, right?
Singh said we hear and pick up on a lot of the information already present in a person's voice. Her software uses artificial intelligence and machine learning to take it a step further. She said it was similar to police developing a profile or a sketch of a suspect. She called it artificial intelligence-driven voice forensics.
“We have techniques to automatically discover information in the voice signal that humans cannot discover,” Singh said. “Our hearing is not that good, but that doesn't mean that the information is not there in the voice.
“There's a lot of information in the voice.”
Singh's research has found a perhaps unlikely customer: The U.S. Coast Guard. They've been working together to combat fake distress calls since the end of 2014.
The Coast Guard handles about 16,000 calls for help a year. About 1 percent of those are fake. It's not a huge number — about 150 hoax calls last year — but it's enough to be a concern, said Alan Arsenault of the Coast Guard's Research and Development Center in New London, Conn. And the number of hoax calls is increasing. The Coast Guard has received about 160 fake distress calls already this year on the Great Lakes alone.
The Coast Guard takes every call for help seriously. Arsenault said 99 percent of the time, the Coast Guard is going to launch a search and rescue mission, even if the agency believes the call is fake.
Going after fake calls puts Coast Guard personnel in danger and diverts resources from real emergencies if they arise, Arsenault said.
And it's expensive.
One of the Coast Guard's HC-130s airplanes costs about $15,000 an hour to operate. A search and rescue helicopter can cost $10,000 an hour. Small boats the Coast Guard uses for search and rescue operations cost about $5,000 an hour to run, Arsenault said.
The Coast Guard tries to crack down on prank callers by using direction and location finding technology and by searching Facebook and Twitter.
“We realized that people like to brag on social media or post pictures of helicopters and boats responding to fake calls,” Arsenault said.
But when the Coast Guard heard about Singh's research at CMU — work that hoped to extract information from someone's voice — it got excited. Arsenault called it another piece that the agency can use to put together the puzzle of who the pranksters are. He said Singh's help has gotten the Coast Guard closer to nabbing a few serial fake callers.
“She gives us information about the physical characteristics of the caller, and she also gives us great information about the environment they're calling from. Are they calling from a boat or from an area with a lot of concrete or from a room with a lot of windows,” Arsenault said. “It's been very fruitful.”
The Coast Guard faces two common problems with fake distress calls.
First, there are serial callers — people who make repeated fake calls. Singh can match new calls to older callers, even if they try to disguise their voice or are really drunk, as many prank callers are.
The second problem is fake calls are often short, typically not longer than “Mayday. Mayday. Mayday,” Arsenault said.
That's long enough for Singh. She can pull troves of information from short snippets of sounds.
“There are things about your voice that you simply cannot change, like the sound of your breath,” Singh said. “Any information that we give them narrows the search. We're not at the point where we can say this is what the person looks like, go get him.”
Singh wasn't really sure what she had until the Coast Guard came asking. The Coast Guard came to her with a tape. She listened to it, ran in through her computer programs and was surprised with what she found.
“That was when we realized what we could do and that was when they realized we could do it,” Singh said. “You could actually get a lot of information from sound and it was actually pretty useful.”
Singh said she has helped other law enforcement agencies but couldn't go into detail. She said child abusers who videotape horrific acts with children will often release only the audio from the tapes first in hopes of finding a buyer for the video. Her research can help police find the abusers from just the audio.
“There are so many crimes being uploaded to the web where the perpetrator is not visible but may have made a sound,” Singh said.
Singh also analyzed then candidate Donald J. Trump's voice during the 2016 campaign.
The website www.factcheck.org, run by The Annenberg Public Policy Center, asked Singh to compare a 1991 phone interview with John Miller — who many suspected was Trump masquerading as a publicist to brag about himself, which Trump denied — to a television interview with Trump from around the same time.
“Same person,” Singh told the website, adding that micro-sections of the Miller tape matched the Trump television interview.
Singh said she also believes the president has an issue with his nose that affects his speech.
©2017 The Pittsburgh Tribune-Review (Greensburg, Pa.) Distributed by Tribune Content Agency, LLC.
