Breaking News
September 23, 2018 - HIV and a tale of a few cities
September 23, 2018 - NIH launches clinical trial to test infusions of combination antibodies in people with HIV
September 23, 2018 - Researchers develop new system to detect consumption of synthetic cannabinoids
September 23, 2018 - Vax-Hub to influenze radical change in development and manufacturing of vaccines
September 23, 2018 - People who have slept lesser than seven hours have higher risks of car crashes
September 23, 2018 - an ancient art may work best to prevent falls in old age
September 23, 2018 - Consumption of foods with lower nutritional quality related to increased cancer risk
September 23, 2018 - Patient Health Information Often Shared Electronically
September 23, 2018 - Can machine learning bring more humanity to health care?
September 23, 2018 - Body organs undergo structural changes in response to diet
September 23, 2018 - Genetic polymorphisms linked with muscle injury and stiffness
September 23, 2018 - As states try to rein in drug spending, feds slap down one bold Medicaid move
September 22, 2018 - Why Eczema Is Tougher to Treat for Black Patients
September 22, 2018 - Team reveals that human genome could contain up to 20 percent fewer genes
September 22, 2018 - USC research uncovers previously unknown genetic risk factor for Alzheimer’s disease
September 22, 2018 - Novel method achieves accurate and precise temperature estimation in fat-containing tissues
September 22, 2018 - BSI accredits Oxehealth’s vital signs measurement software as Class IIa medical device
September 22, 2018 - Evolution of psychiatric disorders and human personality traits
September 22, 2018 - Obesity in early puberty doubles asthma risk for boy’s future offspring
September 22, 2018 - World’s most advanced real-time patient monitoring platform receives key US patent
September 22, 2018 - Study explores connection between sexuality and cognitive status in older adults
September 22, 2018 - LSTM partners with TB Alliance to develop novel TB drug regimens
September 22, 2018 - Annual wellness visits improve delivery of preventive services in elderly population
September 22, 2018 - CHMP provides positive opinion to Cabometyx for previously-treated patients with hepatocellular carcinoma
September 22, 2018 - Hispanic communities with high proportions of Hispanics face more cardiovascular-related death
September 22, 2018 - Vici syndrome – Genetics Home Reference
September 22, 2018 - Single-dose drug can shorten flu symptoms by about a day, studies suggest
September 22, 2018 - AMSBIO launches circulating tumor DNA Reference Standards
September 22, 2018 - Sandalwood mimicking odorant could stimulate hair growth in humans
September 22, 2018 - Overlooked immune cells could play a key role in cancer immunotherapy, claims new study
September 22, 2018 - Study reveals prevalence of diagnosed type 1 and type 2 diabetes among American adults
September 22, 2018 - Researchers develop fast detection strategy to know type of virus acquired by patients
September 22, 2018 - Global Prevalence of Insufficient Activity 27.5 Percent
September 22, 2018 - Strategies to protect bone health in hematologic stem cell transplant recipients
September 22, 2018 - Brigham Genomic Medicine program unravels 30 medical mysteries
September 22, 2018 - New system harnesses power of bubbles to destroy dangerous biofilms
September 22, 2018 - Inflammation plays crucial role in preventing heart attacks and strokes, study reveals
September 22, 2018 - Calorie dense, nutrient deficient meals common across the world
September 22, 2018 - Researchers develop technology to study behavior of implants without animal testing
September 22, 2018 - First gut bacteria in newborns may have lasting effect on ability to ward off chronic diseases
September 22, 2018 - Detection of BFD virus in parrots in 8 new countries raises concerns for threatened species
September 22, 2018 - Insulin treatment shows great potential against chronic bowel inflammation
September 22, 2018 - ‘Liking Gap’ Might Stand in Way of New Friendships
September 22, 2018 - Simple factors that can avoid harmful side effects in type 2 diabetes
September 22, 2018 - ALSAM Foundation invests additional $2 million for drug discovery and development projects
September 22, 2018 - Study findings may advance discussion of how to effectively curb human-wildlife conflict
September 22, 2018 - Dopamine neurons may involve in conditions ranging from Parkinson’s disease to schizophrenia
September 22, 2018 - Protein C and Protein S Tests: MedlinePlus Lab Test Information
September 22, 2018 - Obesity and diabetes—two reasons why we should be worried about the plastics that surround us
September 22, 2018 - Concern over fussy eating prompts parents to use non-responsive feeding practices
September 22, 2018 - Novel mathematical approach uncovers existence of unsuspected biological cycles
September 22, 2018 - Cancer Research UK invests £14 million to transform London into cancer biotherapeutics hub
September 22, 2018 - Scientists predict how well the body will fight lung cancer by analyzing immune cell shapes
September 22, 2018 - New outbreak of rare eye disease identified in contact lens wearers
September 22, 2018 - Iterum Initiates SURE 2 and SURE 3 Phase 3 Clinical Trials of IV and Oral Sulopenem in Complicated Urinary Tract and Complicated Intra-abdominal Infections
September 22, 2018 - Research finds divide in dental health accessibility between city and regional areas
September 22, 2018 - Premature babies show better brain development when fed breast milk, finds study
September 22, 2018 - Novel system uses AI to detect abnormalities in fetal hearts
September 22, 2018 - UNC scientists reveal new approach to prevent obesity and diabetes
September 22, 2018 - CWRU receives NIH grant to learn how non-coding genes contribute to spread of colorectal cancer
September 22, 2018 - Scientists better understand influenza virus and how it spreads
September 22, 2018 - Scientists to focus on length of time when a person is alive and healthy
September 22, 2018 - Study shows positive financial impacts of Medicaid expansion for low-income Michigan residents
September 22, 2018 - Innovative approach for developing vaccine against most prevalent human malaria parasite
September 22, 2018 - Study finds excess emissions from diesel cars produced by major auto manufacturers in Europe
September 22, 2018 - Decision aids for parents of children with minor head injuries help communicate with physicians
September 22, 2018 - Research scientist at Kessler Foundation receives $10,000 grant to study aphasia after stroke
September 22, 2018 - New findings on characteristics of Burning Mouth Syndrome
September 22, 2018 - Study sheds light on molecular mechanisms underlying progression of prion diseases
September 22, 2018 - Innovation Fund Denmark supports research project that aims to fight Clostridium difficile diarrhea
September 22, 2018 - Survey estimates caregiving costs for family members
September 22, 2018 - Inhibiting NF-kB improves heart function in a mouse model of Duchenne muscular dystrophy
September 22, 2018 - Introducing new EMR system may affect several aspects of clinic workflow
September 22, 2018 - Study finds why some human genes are more popular with biomedical researchers
September 22, 2018 - Finding epigenetic signature appears to predict inflammation risk in serious type of IBD
September 22, 2018 - Researchers develop light-based technique to measure very weak magnetic fields
September 22, 2018 - UAB researchers study dysfunction of the immune system associated with NSAID carprofen
September 22, 2018 - QIAGEN and DiaSorin launch automated, CE-marked workflow for high-throughput TB screening
September 22, 2018 - EFS checklist provides user-friendly tool for evaluating feeding skills in preterm infants
September 22, 2018 - Family history in blacks, Latinos associated with higher risk of AFib
Study reveals vision’s role in vowel perception

Study reveals vision’s role in vowel perception

image_pdfDownload PDFimage_print
In a study based at Brown University, researchers found that the motion and configuration of a speaker’s lips are key components of the information people gather when distinguishing vowels in speech. In this case the difference is the position of the lips when making the ‘oo’ sound, as in goose, in English and in French. Credit: Masapollo et. al.

For all talkers, except perhaps the very best ventriloquists, the production of speech is accompanied by visible facial movements. Because speech is more than just sound, researchers set out to ascertain the exact visual information people seek when distinguishing vowel sounds.

“An important and highly debated issue in our field concerns what is it that we are attending to in speech—what’s the object of perception?” said lead author Matthew Masapollo, who conducted the research as a postdoctoral scholar at Brown University and is a now at Boston University. “Another question that’s debated is whether speech processing is special and distinct from other kinds of auditory processing since it is not purely an acoustic signal.”

Resolving these questions would improve the scientific understanding of how we perceive speech, Masapollo said. That, in turn, could apply to the design of more intelligible online avatars and physical robots, and could even improve computer recognition of human speech and enhance communication devices for the hearing impaired.

While scads of studies have investigated which audible features of speech are important, Masapollo said, far fewer have looked at which visual components are essential, despite evidence from phenomena as intuitive as lip reading that the sights of speech matter, too.

Through a series of experiments at Brown and McGill University in Montreal reported in the Journal of Experimental Psychology: Human Perception and Performance, Masapollo and colleagues found that when people perceive speech, they closely watch the form and motion of the lips. If either of those cues is missing, their ability to make subtle distinctions between vowel sounds suffers measurably.

“The findings demonstrate that adults are sensitive to the observable shape and movement patterns that occur when a person talks,” said Masapollo, who did the work as a researcher in the lab of senior author James Morgan, a Brown professor of cognitive, linguistic and psychological sciences.

Exploiting differences in speech perception

Earlier this year, Masapollo set the table for the new study when he and co-authors Linda Polka and Lucie Ménard showed in the journal Cognition that people exhibit the same “directional asymmetry” in visually perceiving vowels that they do when hearing vowels: They are better at distinguishing between two versions of the “oo” sound, as in the word “loose,” if the less extremely articulated version occurs first and then the more extreme version second. If the order is switched, they are much less likely to discriminate them—by sight or sound. While these directional effects may seem like a quirky instinct, they reflect a universal bias favoring vowels produced with extreme articulatory maneuvers. Current research is focused on uncovering what salient features or properties of extreme vowels give rise to these perceptual asymmetries.

Researchers employed many visual representations of lip motion to see which essential features really mattered. Masapollo et. al. Credit: Brown University

It turns out that this asymmetry plays out between French and English, being manifest in the bilingual speech of many Canadians. When speaking French, their articulation of “oo” is produced with more visible lip protrusion and tongue positioning than when making the same vowel sound in English.

For the new study, Masapollo realized that this asymmetry in vowel production and perception provided a great opportunity to determine which visual features matter in distinguishing subtle speech differences. He devised and led five experiments to ferret out exactly what visual information was pertinent to this asymmetry.

In the first, with help from Brown graduate student and co-author Lauren Franklin, he employed eye-tracking technology to measure where Brown student volunteers looked when watching videos of a bilingual Canadian woman make “oo” sounds in both French and English. Definitively, people watched the mouth, far more, for instance than the eyes.

But what about the mouth mattered? To determine if motion, rather than simply a particular position, was important, the next experiment presented students with a still frame rather than video. In experiment two, volunteers at McGill tried to distinguish “oo” speech using just still images of the same speaker. Without the cue of motion, the results showed, the asymmetry of French-English or English-French ordering no longer occurred, suggesting that motion is a key component in this instinct of vowel perception.

In the next three experiments, the team continued to investigate which visual aspects of speech perception mattered among groups of Brown or McGill student volunteers. In experiment three, the subjects saw not a face, but an array of four dots in a diamond pattern that moved just like the speaker’s lips did. When the speaker pursed her lips to make the “oo,” the dots moved closer together, for example. Masapollo’s hypothesis was that position and motion might matter together, even if the face isn’t actually represented. In this experiment, people returned to showing the asymmetry suggesting that he was on the right track.

Experiment four was exactly the same but the dot pattern was rotated 45 degrees clockwise, showing more of a square than a diamond. Here the asymmetry didn’t occur, suggesting that the orientation of the dots to represent a speech-making mouth matter. In experiment five, the motion was represented by a sideways figure eight that would move in a manner analogous to the speaker’s lips. There, too, without even an essential form of a mouth, people didn’t show their instinctual asymmetry of vowel perception. Mere motion, without the form and position of a mouth, was not enough.

“Overall, the picture that emerges is that perceptual asymmetries appear to be elicited by optical stimuli that depict both lip motion and configural information,” the authors wrote.

To Masapollo, the results demonstrate that vision makes specific contributions to perceiving speech.

“The findings of the present research suggest that the information we are attending to in speech is multimodal, and perhaps gestural, in nature,” Masapollo said. “Our perceptual system appears to treat auditory and visual speech information similarly.”


Explore further:
When your eyes override your ears: New insights into the McGurk effect

More information:
Matthew Masapollo et al, Asymmetries in unimodal visual vowel perception: The roles of oral-facial kinematics, orientation, and configuration., Journal of Experimental Psychology: Human Perception and Performance (2018). DOI: 10.1037/xhp0000518

Journal reference:
Cognition

Provided by:
Brown University

Tagged with:

About author

Related Articles