Breaking News
March 24, 2019 - Researchers track effects of epigenetic marks carried by sperm chromosomes
March 24, 2019 - AHA News: Family Adopts Three Children With Three Different Heart Conditions
March 24, 2019 - Research into opioid painkillers could provide clues for safer drug development
March 23, 2019 - Lung cancer survivor recounts her lifetime struggles
March 23, 2019 - Radial and femoral approach for PCI achieve similar results in terms of survival
March 23, 2019 - Study sheds light on the optimal timing of coronary angiography in NSTEMI patients
March 23, 2019 - Excess hormones could cause a condition that can lead to blindness in women, study finds
March 23, 2019 - Dramatic shifts in first-time opioid prescriptions bring hope, concern
March 23, 2019 - Antidepressant drugs may not work when neurons are out of shape
March 23, 2019 - TTUHSC El Paso to establish endowed chair in neurology through a major grant
March 23, 2019 - New device approved by FDA for treating patients with moderate-to-severe heart failure
March 23, 2019 - People with peripheral artery disease have lower Omega-3 Index, shows research
March 23, 2019 - Trigger warnings have minimal impact on how people respond to content, shows research
March 23, 2019 - Gilead Announces Data From Two Studies Supporting Further Development of GS-6207, a Novel, Investigational HIV-1 Capsid Inhibitor as a Component of Future Long-Acting HIV Therapies
March 23, 2019 - Selfish genetic elements amplify inflammation and age-related diseases
March 23, 2019 - Study provides new understanding of how the brain recovers from damage caused by stroke
March 23, 2019 - CRISPR/Cas libraries could revolutionize drug discovery
March 23, 2019 - Allergic reaction during pregnancy may alter sexual-development in offspring’s brain
March 23, 2019 - Seeing through a robot’s eyes helps those with profound motor impairments
March 23, 2019 - Recent research shows that ease of breastfeeding after C-section differs culturally
March 23, 2019 - Newly discovered parameters offer more control over efficient release of drugs
March 23, 2019 - ‘De-tabooing’ of abortion- Women would like more support from health care community
March 23, 2019 - Anti-TB drugs can increase susceptibility to Mtb reinfection
March 23, 2019 - New survey indicates need of attention to neglected tropical diseases
March 23, 2019 - Innovative in vitro method to develop easy-to-swallow medicine for children and older people
March 23, 2019 - Sugary drinks could raise risk of early deaths finds study
March 23, 2019 - Lian wins ENGINE grant for stem-cell-based therapy to treat Type 1 diabetes
March 23, 2019 - Overall, Physicians Are Happy and Enjoy Their Lives
March 23, 2019 - Researchers discover how blood vessels protect the brain during inflammation
March 23, 2019 - CDC study shows modest improvement in optimal hospital breastfeeding policy
March 23, 2019 - Family-based prevention program to reduce alcohol use among older teens
March 23, 2019 - Remote monitoring of implanted defibrillators in heart failure patients prevents hospitalizations
March 23, 2019 - Appropriate doffing of personal protective equipment may reduce healthcare worker contamination
March 23, 2019 - Window screens can suppress mosquito populations, reduce malaria in Tanzania
March 23, 2019 - Researchers discover new biomarker for postoperative liver dysfunction
March 23, 2019 - Pregnancy history may be linked to cognitive function in older women, finds study
March 23, 2019 - Study shows ticagrelor is equally safe and effective as clopidogrel after heart attack
March 23, 2019 - FDA Approves First Drug for Postpartum Depression, Zulresso (brexanolone)
March 23, 2019 - New guidelines outline new treatment management for psoriasis
March 23, 2019 - Thermally abused cooking oil may promote progression of breast cancer
March 23, 2019 - High-fructose corn syrup fuels growth of colon tumors in mice
March 23, 2019 - Partnership aims at establishing best practices to promote diversity in clinical trials
March 23, 2019 - New study examines presence of microbes in tap water from residences, office buildings
March 23, 2019 - Early life trauma may affect brain structure, contribute to major depressive disorder
March 23, 2019 - NIH starts clinical trial of drug to treat cravings associated with opioid use disorder
March 23, 2019 - Cervix bacteria, immune factors could be a warning signal of premature birth, reports new research
March 23, 2019 - Worst-ever emergency care performance figures underscore the need to focus on staffing
March 23, 2019 - The Current issue of “The view from here” is concerned with Cancer
March 23, 2019 - Mouse model validates how ‘good’ and ‘bad’ bacteria affect acne
March 23, 2019 - Individual amygdala neurons respond to touch, imagery and sounds
March 23, 2019 - Combination of two topical creams can prevent cancer
March 23, 2019 - Study suggests depression screening when assessing African-Americans for schizophrenia
March 23, 2019 - New electronic support system for choosing drug treatment based on patient’s genotype
March 23, 2019 - First-of-its-kind study provides pregnancy statistics of imprisoned U.S. women
March 23, 2019 - Marinus Pharmaceuticals Initiates Phase 3 Study in Children with PCDH19-Related Epilepsy
March 23, 2019 - Laparoscopy: MedlinePlus Lab Test Information
March 23, 2019 - Shellfish allergies: can they be treated?
March 23, 2019 - Toilet seat heart monitoring system
March 23, 2019 - Researchers identify way to improve common treatment for PTSD
March 23, 2019 - High potency cannabis use linked to psychosis finds study
March 23, 2019 - Evoke Pharma Submits Response to FDA Review Letter for Gimoti NDA
March 23, 2019 - Tracking HIV’s ever-evolving genome in effort to prioritize public health resources
March 23, 2019 - Scientists grow most sophisticated brain organoid to date
March 23, 2019 - ADHD drug raising risk of psychosis
March 22, 2019 - FDA approves brexanolone, first drug developed to treat postpartum depression
March 22, 2019 - Gruesome cat and dog experiments by the USDA exposed
March 22, 2019 - Ball pits used in children’s physical therapy may contribute to germ transmission
March 22, 2019 - Long-term use of inexpensive weight-loss drug may be safe and effective
March 22, 2019 - FDA Approves Sunosi (solriamfetol) for Excessive Daytime Sleepiness Associated with Narcolepsy or Obstructive Sleep Apnea
March 22, 2019 - Anti-Müllerian Hormone Test: MedlinePlus Lab Test Information
March 22, 2019 - Finding the right exercise, diet aids for HIV patients
March 22, 2019 - Health Plans For State Employees Use Medicare’s Hammer On Hospital Bills
March 22, 2019 - Researchers develop new tool for imaging large groups of neurons in living animals
March 22, 2019 - Certain bacteria and immune factors in vagina may cause or protect against preterm birth
March 22, 2019 - Research identifies guidelines for prioritizing hepatitis C treatment in U.S. prisons
March 22, 2019 - Novel breath test could pave new way to non-invasively measure gut health
March 22, 2019 - Pharmaceutical and personal care products may result in new contaminants in waterways
March 22, 2019 - New model could revolutionize the way researchers investigate spread of pathogens
March 22, 2019 - MSU professor receives NSF CAREER grant for biosensor diagnostics
March 22, 2019 - High-fat, high-sugar diet in mouse mothers causes problems in the hearts of offspring
Study reveals vision’s role in vowel perception

Study reveals vision’s role in vowel perception

image_pdfDownload PDFimage_print
In a study based at Brown University, researchers found that the motion and configuration of a speaker’s lips are key components of the information people gather when distinguishing vowels in speech. In this case the difference is the position of the lips when making the ‘oo’ sound, as in goose, in English and in French. Credit: Masapollo et. al.

For all talkers, except perhaps the very best ventriloquists, the production of speech is accompanied by visible facial movements. Because speech is more than just sound, researchers set out to ascertain the exact visual information people seek when distinguishing vowel sounds.

“An important and highly debated issue in our field concerns what is it that we are attending to in speech—what’s the object of perception?” said lead author Matthew Masapollo, who conducted the research as a postdoctoral scholar at Brown University and is a now at Boston University. “Another question that’s debated is whether speech processing is special and distinct from other kinds of auditory processing since it is not purely an acoustic signal.”

Resolving these questions would improve the scientific understanding of how we perceive speech, Masapollo said. That, in turn, could apply to the design of more intelligible online avatars and physical robots, and could even improve computer recognition of human speech and enhance communication devices for the hearing impaired.

While scads of studies have investigated which audible features of speech are important, Masapollo said, far fewer have looked at which visual components are essential, despite evidence from phenomena as intuitive as lip reading that the sights of speech matter, too.

Through a series of experiments at Brown and McGill University in Montreal reported in the Journal of Experimental Psychology: Human Perception and Performance, Masapollo and colleagues found that when people perceive speech, they closely watch the form and motion of the lips. If either of those cues is missing, their ability to make subtle distinctions between vowel sounds suffers measurably.

“The findings demonstrate that adults are sensitive to the observable shape and movement patterns that occur when a person talks,” said Masapollo, who did the work as a researcher in the lab of senior author James Morgan, a Brown professor of cognitive, linguistic and psychological sciences.

Exploiting differences in speech perception

Earlier this year, Masapollo set the table for the new study when he and co-authors Linda Polka and Lucie Ménard showed in the journal Cognition that people exhibit the same “directional asymmetry” in visually perceiving vowels that they do when hearing vowels: They are better at distinguishing between two versions of the “oo” sound, as in the word “loose,” if the less extremely articulated version occurs first and then the more extreme version second. If the order is switched, they are much less likely to discriminate them—by sight or sound. While these directional effects may seem like a quirky instinct, they reflect a universal bias favoring vowels produced with extreme articulatory maneuvers. Current research is focused on uncovering what salient features or properties of extreme vowels give rise to these perceptual asymmetries.

Researchers employed many visual representations of lip motion to see which essential features really mattered. Masapollo et. al. Credit: Brown University

It turns out that this asymmetry plays out between French and English, being manifest in the bilingual speech of many Canadians. When speaking French, their articulation of “oo” is produced with more visible lip protrusion and tongue positioning than when making the same vowel sound in English.

For the new study, Masapollo realized that this asymmetry in vowel production and perception provided a great opportunity to determine which visual features matter in distinguishing subtle speech differences. He devised and led five experiments to ferret out exactly what visual information was pertinent to this asymmetry.

In the first, with help from Brown graduate student and co-author Lauren Franklin, he employed eye-tracking technology to measure where Brown student volunteers looked when watching videos of a bilingual Canadian woman make “oo” sounds in both French and English. Definitively, people watched the mouth, far more, for instance than the eyes.

But what about the mouth mattered? To determine if motion, rather than simply a particular position, was important, the next experiment presented students with a still frame rather than video. In experiment two, volunteers at McGill tried to distinguish “oo” speech using just still images of the same speaker. Without the cue of motion, the results showed, the asymmetry of French-English or English-French ordering no longer occurred, suggesting that motion is a key component in this instinct of vowel perception.

In the next three experiments, the team continued to investigate which visual aspects of speech perception mattered among groups of Brown or McGill student volunteers. In experiment three, the subjects saw not a face, but an array of four dots in a diamond pattern that moved just like the speaker’s lips did. When the speaker pursed her lips to make the “oo,” the dots moved closer together, for example. Masapollo’s hypothesis was that position and motion might matter together, even if the face isn’t actually represented. In this experiment, people returned to showing the asymmetry suggesting that he was on the right track.

Experiment four was exactly the same but the dot pattern was rotated 45 degrees clockwise, showing more of a square than a diamond. Here the asymmetry didn’t occur, suggesting that the orientation of the dots to represent a speech-making mouth matter. In experiment five, the motion was represented by a sideways figure eight that would move in a manner analogous to the speaker’s lips. There, too, without even an essential form of a mouth, people didn’t show their instinctual asymmetry of vowel perception. Mere motion, without the form and position of a mouth, was not enough.

“Overall, the picture that emerges is that perceptual asymmetries appear to be elicited by optical stimuli that depict both lip motion and configural information,” the authors wrote.

To Masapollo, the results demonstrate that vision makes specific contributions to perceiving speech.

“The findings of the present research suggest that the information we are attending to in speech is multimodal, and perhaps gestural, in nature,” Masapollo said. “Our perceptual system appears to treat auditory and visual speech information similarly.”


Explore further:
When your eyes override your ears: New insights into the McGurk effect

More information:
Matthew Masapollo et al, Asymmetries in unimodal visual vowel perception: The roles of oral-facial kinematics, orientation, and configuration., Journal of Experimental Psychology: Human Perception and Performance (2018). DOI: 10.1037/xhp0000518

Journal reference:
Cognition

Provided by:
Brown University

Tagged with:

About author

Related Articles