Breaking News
November 14, 2018 - Study shows novel strategy to reduce breast cancer bone metastasis
November 14, 2018 - Empowering the NHS through Industry Partnerships
November 14, 2018 - One size does not fit all in obesity treatment, study finds
November 14, 2018 - Seeking ways to prevent ‘secondary cataracts’
November 14, 2018 - Change Within the Eye May Be Early Warning for Macular Degeneration
November 14, 2018 - Study of 500,000 people clarifies the risks of obesity
November 14, 2018 - Ultrasound releases drug to alter activity in targeted brain areas in rats | News Center
November 14, 2018 - Umass Amherst researchers battle against youth suicide in rural Alaska Native communities
November 14, 2018 - Cancer stem cells depend on amino acid metabolism, and it’s proving to be their Achilles’ heel
November 14, 2018 - Epigenetic link found between prenatal exposure to maternal smoking and offspring’s cardio-metabolic health
November 14, 2018 - Meditation, music may change biomarkers of cellular aging and Alzheimer’s disease in older adults
November 14, 2018 - Multidisciplinaryresearch teams selected to study age-related brain disorders
November 14, 2018 - The Current issue of “The view from here” is concerned with Informatics
November 14, 2018 - Researchers identify tool to help transgender women have a more authentic voice
November 14, 2018 - Four faculty members appointed to endowed professorships | News Center
November 13, 2018 - Research finds strongest evidence yet that obesity causes depression
November 13, 2018 - Researchers compare stools of breastfed and formula-fed infants
November 13, 2018 - Entasis Therapeutics Announces Zoliflodacin Phase 2 Results Published in The New England Journal of Medicine
November 13, 2018 - Gene changes driving myopia reveal new focus for drug development
November 13, 2018 - $6 million grant to support study of preeclampsia, atherosclerosis links | News Center
November 13, 2018 - Beneficial gut microbes metabolize high-fiber diet to improve heart health in mouse model
November 13, 2018 - Excessive use of social media through visual postings linked to increase in narcissistic traits
November 13, 2018 - Study finds why obesity both fuels cancer growth and helps immunotherapy to kill tumors
November 13, 2018 - Women prefer and invest more in daughters, while men favor sons
November 13, 2018 - With hospitalization losing favor, judges order outpatient mental health treatment
November 13, 2018 - Transgenic rat model may provide new insights into cerebral amyloid angiopathy
November 13, 2018 - Study identifies factors tied to greater risk of advanced liver disease in cystic fibrosis patients
November 13, 2018 - Risk of blindness among premature babies with low levels of blood platelets
November 13, 2018 - A new strategy for combatting antibiotic-resistant infections
November 13, 2018 - Study aims to find which outreach method is more effective at improving cancer screening rates
November 13, 2018 - Insufficient sleep duration linked with unhealthy lifestyle profile among children
November 13, 2018 - IIASA researchers introduce new, simple measure for human wellbeing
November 13, 2018 - Magnetic nanosprings used as targeted drug delivery agents for anticancer therapy
November 13, 2018 - Scientists examine FCMs containing silver nanoparticles
November 13, 2018 - Failed DNA repair triggers chromosomal chaos
November 13, 2018 - Study shows new emerging role of osteopontin in HCV-related hepatocellular carcinoma
November 13, 2018 - Food insecurity during pregnancy linked to severity of neonatal abstinence syndrome
November 13, 2018 - Majority of Americans are concerned about health threat posed by antibiotic resistance
November 13, 2018 - Addition of Elotuzumab Ups PFS in Refractory Multiple Myeloma
November 13, 2018 - Study finds women with pregnancy-related nausea, vomiting use marijuana more
November 13, 2018 - Lethal heart rhythm more likely to be found in patients with common heart failure
November 13, 2018 - Study provides new clues to origin and development of multiple sclerosis
November 13, 2018 - Climate change could pose threat to male fertility
November 13, 2018 - Researchers discover how mitochondria deploy a powerful punch against disease-causing bacteria
November 13, 2018 - AHA: Traumatic Childhood Could Increase Heart Disease Risk in Adulthood
November 13, 2018 - Feeling the Burn? | NIH News in Health
November 13, 2018 - Women’s birth canals in Kenya, Korea, Kansas not the same: study
November 13, 2018 - Fecal microbiota transplantation effective against ICI-associated colitis
November 13, 2018 - New physical activity guidelines released that urge people to “move more”
November 13, 2018 - Angiotensin receptor blockers improve sodium excretion in blacks
November 13, 2018 - New project seeks to address alarming injury rate in youth footballers
November 13, 2018 - Fish oil or omega 3 fatty acid supplements can prevent heart attacks finds study
November 13, 2018 - The Human Heart-in-a-Jar That Could One Day Replace Animal Testing
November 13, 2018 - Treat patients’ partners without a doctor visit
November 13, 2018 - Belgian beer landscape mapped using scientific insights
November 13, 2018 - ‘Master key’ gene has links to both ASD and schizophrenia
November 13, 2018 - Gladstone scientists gain new insights into the aging brain
November 13, 2018 - Drug therapy can improve outcomes for acutely ill heart patients
November 13, 2018 - Three landmark studies provide better understanding of sudden cardiac arrest
November 13, 2018 - Cholesterol control revised in the latest AHA/ACC guidelines
November 13, 2018 - Vulnerable young teenagers urgently need better sex education, say researchers
November 13, 2018 - Breakthrough research reveals how deadly pneumococcus avoids immune defenses
November 13, 2018 - Researchers discover possible path forward in preventing cancers tied to two viruses
November 13, 2018 - Wishes can help pediatric patients to get better over time
November 13, 2018 - Janssen Reports Positive Topline Results for FLAIR Phase 3 Study of a Novel, Long Acting Injectable Two-Drug Regimen for the treatment of HIV-1
November 13, 2018 - Experimental compound reduces Gulf War illness-like behavior in mice
November 13, 2018 - Small-stature in rainforest populations may be linked to cardiac adaptations
November 13, 2018 - Study shows how pneumococci challenge the immune system
November 13, 2018 - Simple cysts can be safely ignored, study finds
November 13, 2018 - First fully personalized tissue implant engineered from patient’s own materials and cells
November 13, 2018 - FDA Approves Keytruda (pembrolizumab) in Combination with Carboplatin and Either Paclitaxel or Nab-Paclitaxel for the First-Line Treatment of Patients with Metastatic Squamous Non-Small Cell Lung Cancer (NSCLC)
November 13, 2018 - Scientists take big step toward finding non-addictive painkiller
November 13, 2018 - Diabetes medication reduces risk of heart failure hospitalization
November 13, 2018 - Achieving high follow-up rates for violently injured patient population is feasible
November 13, 2018 - Shortage of specific gene ‘silencing’ molecules linked with pediatric low-grade gliomas
November 13, 2018 - Abx-Resistant Enterobacteriaceae Tied to Clinical Failure in UTI
November 13, 2018 - US approves first new type of flu drug in 2 decades
November 13, 2018 - Is zinc the link to how we think? Some evidence, and a word of warning
November 13, 2018 - Dispelling taboos, Michelle Obama talks IVF and miscarriage
November 13, 2018 - Medical experts discuss future challenges of healthcare at HSMA’s inaugural conference
AI in medical space must be carefully tested for performance across wide range of populations

AI in medical space must be carefully tested for performance across wide range of populations

image_pdfDownload PDFimage_print

Artificial intelligence (AI) tools trained to detect pneumonia on chest X-rays suffered significant decreases in performance when tested on data from outside health systems, according to a study conducted at the Icahn School of Medicine at Mount and published in a special issue of PLOS Medicine on machine learning and health care. These findings suggest that artificial intelligence in the medical space must be carefully tested for performance across a wide range of populations; otherwise, the deep learning models may not perform as accurately as expected.

As interest in the use of computer system frameworks called convolutional neural networks (CNN) to analyze medical imaging and provide a computer-aided diagnosis grows, recent studies have suggested that AI image classification may not generalize to new data as well as commonly portrayed.

Researchers at the Icahn School of Medicine at Mount Sinai assessed how AI models identified pneumonia in 158,000 chest X-rays across three medical institutions: the National Institutes of Health; The Mount Sinai Hospital; and Indiana University Hospital. Researchers chose to study the diagnosis of pneumonia on chest X-rays for its common occurrence, clinical significance, and prevalence in the research community.

In three out of five comparisons, CNNs’ performance in diagnosing diseases on X-rays from hospitals outside of its own network was significantly lower than on X-rays from the original health system. However, CNNs were able to detect the hospital system where an X-ray was acquired with a high-degree of accuracy, and cheated at their predictive task based on the prevalence of pneumonia at the training institution. Researchers found that the difficulty of using deep learning models in medicine is that they use a massive number of parameters, making it challenging to identify specific variables driving predictions, such as the types of CT scanners used at a hospital and the resolution quality of imaging.

“Our findings should give pause to those considering rapid deployment of artificial intelligence platforms without rigorously assessing their performance in real-world clinical settings reflective of where they are being deployed,” says senior author Eric Oermann, MD, Instructor in Neurosurgery at the Icahn School of Medicine at Mount Sinai. “Deep learning models trained to perform medical diagnosis can generalize well, but this cannot be taken for granted since patient populations and imaging techniques differ significantly across institutions.”

“If CNN systems are to be used for medical diagnosis, they must be tailored to carefully consider clinical questions, tested for a variety of real-world scenarios, and carefully assessed to determine how they impact accurate diagnosis,” says first author John Zech, a medical student at the Icahn School of Medicine at Mount Sinai.

This research builds on papers published earlier this year in the journals Radiology and Nature Medicine, which laid the framework for applying computer vision and deep learning techniques, including natural language processing algorithms, for identifying clinical concepts in radiology reports for CT scans.

Source:

https://www.mountsinai.org/

Tagged with:

About author

Related Articles