Breaking News
January 20, 2019 - New study takes first step toward treating endometriosis
January 20, 2019 - Researchers find how GREB1 gene promotes resistance to prostate cancer treatments
January 20, 2019 - Replacing Sitting Time With Activity Lowers Mortality Risk
January 20, 2019 - A simple, inexpensive intervention makes birth safer for moms and babies in parts of Africa
January 19, 2019 - New anti-inflammatory compound acts as ‘surge protector’ to reduce cancer growth
January 19, 2019 - Significant flaws found in recently released forensic software
January 19, 2019 - New Leash on Life? Staying Slim Keeps Pooches Happy, Healthy
January 19, 2019 - Men and women remember pain differently
January 19, 2019 - Rising air pollution linked with increased ER visits for breathing problems
January 19, 2019 - Study uses local data to model food consumption patterns among Seattle residents
January 19, 2019 - The brain’s cerebellum plays role in controlling reward and social behaviors, study shows
January 19, 2019 - Relationship between nurse work environment and patient safety
January 19, 2019 - Pioneering surgery restores movement to children paralyzed by acute flaccid myelitis
January 19, 2019 - Genetic variants linked with risk tolerance and risky behaviors
January 19, 2019 - New research provides better understanding of our early human ancestors
January 19, 2019 - First-ever tailored reporting guidance to improve patient care and outcomes
January 19, 2019 - 4.6 percent of Massachusetts residents have opioid use disorder
January 19, 2019 - New study suggests vital exhaustion as risk factor for dementia
January 19, 2019 - New antibiotic discovery heralds breakthrough in the fight against drug-resistant bacteria
January 19, 2019 - Ural Federal University scientists synthesize a group of multi-purpose fluorophores
January 19, 2019 - Researchers identify new therapeutic target in the fight against chronic liver diseases
January 19, 2019 - Preparation, characterization of Soyasapogenol B loaded onto functionalized MWCNTs
January 19, 2019 - FDA Approves Ontruzant (trastuzumab-dttb), a Biosimilar to Herceptin
January 19, 2019 - Tobacco use linked with higher use of opioids and sedatives
January 19, 2019 - Study delves deeper into developmental dyslexia
January 19, 2019 - Anti-vaccination movement one of the top health threats in 2019 says WHO
January 19, 2019 - Newly developed risk score more effective at identifying type 1 diabetes
January 19, 2019 - Highly effective protocol to prepare cannabis samples for THC/CBD analysis
January 19, 2019 - Prinston Pharmaceutical Inc. Issues Voluntary Nationwide Recall of Irbesartan and Irbesartan HCTZ Tablets Due to Detection of a Trace Amount of Unexpected Impurity, N-Nitrosodiethylamine (NDEA) in the Products
January 19, 2019 - How does solid stress from brain tumors cause neuronal loss, neurologic dysfunction?
January 19, 2019 - $14.7 million partnership to supercharge vaccine development
January 19, 2019 - Ian Fotheringham receives Charles Tennant Memorial Lecture award
January 19, 2019 - Brain vital signs detect neurophysiological impairments in players with concussions
January 19, 2019 - Lack of job and poor housing conditions increased likelihood of people attending A&E
January 19, 2019 - Novel targeted drug delivery system improves conventional cancer treatments
January 19, 2019 - Rutgers study finds gene responsible for spread of prostate cancer
January 19, 2019 - Complications Higher Than Expected for Invasive Lung Tests
January 19, 2019 - 3-D printed implant promotes nerve cell growth to treat spinal cord injury
January 19, 2019 - Automated texts lead to improved outcomes after total knee or hip replacement surgery
January 19, 2019 - Poor cardiorespiratory fitness could increase risk of future heart attack, finds new study
January 19, 2019 - Drinking soft drinks while exercising in hot weather may increase risk of kidney disease
January 19, 2019 - Formlabs 3D prints anatomical models
January 19, 2019 - Heart-Healthy Living Also Wards Off Type 2 Diabetes
January 19, 2019 - Teaching Kids to Be Smart About Social Media (for Parents)
January 19, 2019 - Metabolite produced by gut microbiota from pomegranates reduces inflammatory bowel disease
January 19, 2019 - Researchers examine how spray from showers and toilets expose us to disease causing bacteria
January 19, 2019 - Behavioral experiments confirm that additional neurons improve brain function
January 19, 2019 - New study compares performance of real-time infectious disease forecasting models
January 19, 2019 - Obesity can be risk factor for developing renal cell carcinoma, confirms study
January 19, 2019 - New regulation designs on cigarette packs direct smokers’ attention to health warnings
January 19, 2019 - QIAGEN receives first companion diagnostic approval in Japan
January 19, 2019 - Study explores role of Dunning-Kruger effect in anti-vaccine attitudes
January 19, 2019 - Newly identified subset of immune cells may be key to fighting chronic inflammation
January 19, 2019 - New immune response regulators discovered
January 18, 2019 - Poor blood oxygenation during sleep predicts chance of heart-related death
January 18, 2019 - First international consensus on the diagnosis and management of fibromuscular dysplasia
January 18, 2019 - Rapid resistance gene sequencing technology can hasten identification of antibiotic-resistant bacteria
January 18, 2019 - Researchers develop artificial enzymatic pathway for synthesizing isoprenoids in E. coli
January 18, 2019 - Scientists advise caution in immunotherapy research
January 18, 2019 - How children across the world develop language
January 18, 2019 - Columbia Medical Student Receives McDonogh Scholarship
January 18, 2019 - Secretive ‘Rebate Trap’ Keeps Generic Drugs For Diabetes And Other Ills Out Of Reach
January 18, 2019 - Plant based diet could be the best option for the planet says commission
January 18, 2019 - New conservation practice could reduce nitrogen from agricultural drainage, study shows
January 18, 2019 - UIC researchers receive $1.7 million NCI grant to study Southeast Asian fruit
January 18, 2019 - New study determines the fate of DNA derived from genetically modified food
January 18, 2019 - Scientists develop new gene therapy that prevents axon destruction in mice
January 18, 2019 - Study finds critically low HPV vaccination rates among younger adolescents in the U.S.
January 18, 2019 - Brain cells involved in memory play key role in reducing future eating behavior
January 18, 2019 - Risk for Conversion of MS Varies With Different Therapies
January 18, 2019 - Investigational cream may help patients with inflammatory skin disease
January 18, 2019 - Medical school news office receives six writing awards | News Center
January 18, 2019 - County By County, Researchers Link Opioid Deaths To Drugmakers’ Marketing
January 18, 2019 - Research reveals risk for developing more than one mental health disorder
January 18, 2019 - Scientists discover a dramatic pattern of bone growth in female mice
January 18, 2019 - Study finds link between lengthy periods of undisturbed maternal sleep and stillbirths
January 18, 2019 - New nuclear medicine method could improve detection of primary and metastatic melanoma
January 18, 2019 - Combination therapy shows high efficacy in treating people with leishmaniasis and HIV
January 18, 2019 - Health Tip: Don’t Ignore Changes in Skin Color
January 18, 2019 - Dietary Recommendations for Healthy Children
AI in medical space must be carefully tested for performance across wide range of populations

AI in medical space must be carefully tested for performance across wide range of populations

image_pdfDownload PDFimage_print

Artificial intelligence (AI) tools trained to detect pneumonia on chest X-rays suffered significant decreases in performance when tested on data from outside health systems, according to a study conducted at the Icahn School of Medicine at Mount and published in a special issue of PLOS Medicine on machine learning and health care. These findings suggest that artificial intelligence in the medical space must be carefully tested for performance across a wide range of populations; otherwise, the deep learning models may not perform as accurately as expected.

As interest in the use of computer system frameworks called convolutional neural networks (CNN) to analyze medical imaging and provide a computer-aided diagnosis grows, recent studies have suggested that AI image classification may not generalize to new data as well as commonly portrayed.

Researchers at the Icahn School of Medicine at Mount Sinai assessed how AI models identified pneumonia in 158,000 chest X-rays across three medical institutions: the National Institutes of Health; The Mount Sinai Hospital; and Indiana University Hospital. Researchers chose to study the diagnosis of pneumonia on chest X-rays for its common occurrence, clinical significance, and prevalence in the research community.

In three out of five comparisons, CNNs’ performance in diagnosing diseases on X-rays from hospitals outside of its own network was significantly lower than on X-rays from the original health system. However, CNNs were able to detect the hospital system where an X-ray was acquired with a high-degree of accuracy, and cheated at their predictive task based on the prevalence of pneumonia at the training institution. Researchers found that the difficulty of using deep learning models in medicine is that they use a massive number of parameters, making it challenging to identify specific variables driving predictions, such as the types of CT scanners used at a hospital and the resolution quality of imaging.

“Our findings should give pause to those considering rapid deployment of artificial intelligence platforms without rigorously assessing their performance in real-world clinical settings reflective of where they are being deployed,” says senior author Eric Oermann, MD, Instructor in Neurosurgery at the Icahn School of Medicine at Mount Sinai. “Deep learning models trained to perform medical diagnosis can generalize well, but this cannot be taken for granted since patient populations and imaging techniques differ significantly across institutions.”

“If CNN systems are to be used for medical diagnosis, they must be tailored to carefully consider clinical questions, tested for a variety of real-world scenarios, and carefully assessed to determine how they impact accurate diagnosis,” says first author John Zech, a medical student at the Icahn School of Medicine at Mount Sinai.

This research builds on papers published earlier this year in the journals Radiology and Nature Medicine, which laid the framework for applying computer vision and deep learning techniques, including natural language processing algorithms, for identifying clinical concepts in radiology reports for CT scans.

Source:

https://www.mountsinai.org/

Tagged with:

About author

Related Articles