Breaking News
August 21, 2018 - National Foundation for Cancer Research receives Safeway Foundation grant
August 21, 2018 - Protein aggregation in neurons linked to gene regulation in Huntington’s disease
August 21, 2018 - Aravive Biologics gains Fast Track Designation for AVB-S6-500 from U.S. FDA
August 21, 2018 - FDA Approves Opdivo (nivolumab) for Certain Patients with Previously Treated Small Cell Lung Cancer
August 21, 2018 - Success of blood test for autism affirmed
August 21, 2018 - Diabetic patients with disrupted sleep may need more time to heal their wounds
August 21, 2018 - AADE honors six educators for achievements in diabetes education
August 21, 2018 - Scientists find two molecules that may combat cancer and chronic infections
August 21, 2018 - Two Strategies for Preventing Diabetes in Minority Patients
August 21, 2018 - Living as a Gallbladder Cancer Survivor
August 21, 2018 - Can we predict the long-term outcome of boys with ADHD?
August 21, 2018 - GBCA creates model for developing scientist-advocate collaborations in cancer research
August 21, 2018 - Healthy diet could help promote healthy cellular aging in women
August 21, 2018 - Researchers develop gene expression predictor for immunotherapy response in melanoma
August 21, 2018 - MDI Biological Laboratory introduces Morris Scientific Discovery Fund for eligible research programs
August 21, 2018 - Micro-flow model reveals complex interactions between the brain’s blood vessels and nerve cells
August 21, 2018 - Study investigates impact of osteoporosis on risk of developing dementia
August 21, 2018 - Federal method fails to detect most stores that sell cigarettes to minors
August 21, 2018 - Workers in open office seating have less stress than those in private offices and cubicles
August 21, 2018 - 1 in 4 in U.S. Has a Disability, CDC Reports
August 21, 2018 - Studies provide new insights into the role of sleep in chronic pain
August 21, 2018 - Study shows that rogue proteins may underlie some ALS and frontotemporal dementia cases
August 21, 2018 - Elevated LDL cholesterol levels linked to higher risk of CVD death in young, healthy people
August 21, 2018 - Measles cases on the rise in Europe
August 21, 2018 - CURE Media Group welcomes CancerCare to Strategic Alliance Partnership Program
August 21, 2018 - Blood management program associated with fewer transfusions in orthopedic patients
August 21, 2018 - Researchers create the world’s first artificial retina
August 21, 2018 - Yale researchers identify racial disparities in prescribing opioids for chronic pain
August 21, 2018 - BOOST-3 clinical trial aims to improve outcomes for severe TBI patients
August 21, 2018 - New study highlights Alzheimer’s herpes link, experts say
August 21, 2018 - Airline crew don’t have significantly elevated risk of thyroid cancer, new study finds
August 21, 2018 - States leverage federal funds to help insurers lower premiums
August 21, 2018 - New badge course explores research around skeletal muscle as an endocrine organ
August 21, 2018 - TG Therapeutics Announces Completion of Target Enrollment in the ULTIMATE Phase 3 Trials in Multiple Sclerosis
August 21, 2018 - Increased levels of human herpesvirus ID’d in Alzheimer’s
August 21, 2018 - To help patients quash pain, researcher develops practical guide for health care providers
August 20, 2018 - Medicine on the front line to be presented at Medical Innovation 2018
August 20, 2018 - Harbour Biomed and Kelun-Biotech collaborate to develop, commercialize anti-PD-L1 antibody
August 20, 2018 - The man who sold America on vitamin D — and profited in the process
August 20, 2018 - Finding the light in antimicrobials
August 20, 2018 - Unique pain program helps surgical patients wean off opioids safely and effectively
August 20, 2018 - Strawberries could mitigate colonic inflammation
August 20, 2018 - FDA Accepts New Drug Application (NDA) to Review Midazolam Nasal Spray, an Investigational Product for the Acute Treatment of Seizure Clusters
August 20, 2018 - Using Facebook to help young adults quit smoking
August 20, 2018 - ‘Liquid biopsy’ predicts lymphoma therapy success within days | News Center
August 20, 2018 - 5 Questions with Jordan Orange, Chair of Pediatrics
August 20, 2018 - New assay may help improve both sarcoma diagnosis and treatment
August 20, 2018 - New information on the brain regions related to metacognition, tactile sense
August 20, 2018 - New class of insect repellents to fight against mosquito-borne diseases
August 20, 2018 - ACA Coverage Gains Include Workers Without Insurance
August 20, 2018 - 3-D printed biomaterials for bone tissue engineering
August 20, 2018 - Current surveillance system does not quickly pick up most listeriosis cases in the EU, study reveals
August 20, 2018 - Prenatal exposure to acute stress can affect cognitive function in children of low-income households
August 20, 2018 - New study examines scope of state policies targeting drug use by pregnant women
August 20, 2018 - Researchers find long-term structural, functional brain abnormalities in individuals with AUDs
August 20, 2018 - Shortage of insurance fraud cops sparks campaign debate
August 20, 2018 - Researchers find STAT3 as therapeutic target for chronic active EBV infection
August 20, 2018 - Health Tip: Keep Diabetic Feet Healthier
August 20, 2018 - FDA approves brain stimulation device for OCD
August 20, 2018 - NYU Langone’s Perlmutter Cancer Center expands Blood and Marrow Transplant Program
August 20, 2018 - New drug shows potential to prevent painful side effect of therapy
August 20, 2018 - RDMD raises $3 million in seed funding to accelerate rare disease research, drug development
August 20, 2018 - Illicit drug use is higher during celebratory events, may be worse than previously thought
August 20, 2018 - Exploring the relationship between fever and cancer incidence
August 20, 2018 - Study reveals how socioeconomic status affects racial, ethnic disparities in childhood cancer survival
August 20, 2018 - Brain tumors trap immune cells needed to fight cancer in the bone marrow, finds research
August 20, 2018 - Three factors that contribute to physician burnout
August 20, 2018 - Babies dependent on opioids need touch, not tech
August 20, 2018 - Understanding How Antibodies Shape the Gut Microbiome
August 20, 2018 - Cara Therapeutics Doses First Patient in Second Pivotal Phase 3 Efficacy Trial of Korsuva (CR845/difelikefalin) Injection in Hemodialysis Patients with Chronic Kidney Disease-Associated Pruritus
August 20, 2018 - Kidney transplant chains more effective in saving lives
August 20, 2018 - Study unravels cellular and molecular mechanisms behind dermal condensate formation
August 20, 2018 - New integrated gene logic-chips could have great value in medical care
August 20, 2018 - FDA Advisory Committee Recommends Approval of Paratek’s Omadacycline
August 20, 2018 - Total, open repairs decline for abdominal aortic aneurysms
August 20, 2018 - Novel system can pinpoint ingestible implants inside the body using wireless signals
August 20, 2018 - Infection rates of high risk oral HPV in England found to be lower than expected
August 20, 2018 - Making robots as valuable and trustworthy assistants for medical therapies
August 20, 2018 - Patients with low-risk blood clots can be better treated at home than at hospital
August 20, 2018 - Passive smoking exposure among kids greatly increases COPD risk late in life
Can Electronic Health Data Solve Medicine’s Reproducibility Problem?

Can Electronic Health Data Solve Medicine’s Reproducibility Problem?

image_pdfDownload PDFimage_print

According to headlines in recent years, coffee causes cancer…and it cures cancer; moderate alcohol consumption improves health…and adults should consume no alcohol for optimum health; drugs for osteoporosis cause esophageal cancer…and don’t cause it. And the list of incompatible results goes on.

“The research literature is filled with examples of studies reporting statistically significant results in conflicting directions, sometimes even when using the same data,” says George Hripcsak, MD, chair of biomedical informatics at Columbia University Vagelos College of Physicians and Surgeons.

George Hripcsak. Photo: Jörg Meyer.

The divergent findings reported in these headlines usually come from observational studies, a research design known for its poor reproducibility. In contrast to randomized controlled trials, which enroll patients before a study begins, observational studies analyze past events.

And though that approach is rational in theory, in practice it’s fraught with problems.

“If you parse the medical literature you find that it’s basically a data dredging machine,” Hripcsak says. “There’s publication bias, when authors and editors tend to publish things that match the answers they were looking for, or that advance their careers, or that benefit the journal.”

With massive troves of electronic health data now available, Hripcsak and his colleagues think the time is right to change the way observational studies are conducted. Instead of performing one study and publishing (or not) one result at a time, electronic health data now allow researchers to answer thousands of questions at once.

“By disseminating all the findings, we not only provide new results on a large scale but also prevent the publication bias we see in the literature,” says Hripcsak. “The reader then sees all the results, not just the ones favored by the authors or editors, and other researchers can see a whole body of work to better judge if the methods are operating as they should.”

Their new approach is described–and used to examine treatments for depression–in a paper published this month in Philosophical Transactions of the Royal Society A by Hripcsak and a team of coauthors from Columbia, the University of California Los Angeles, and Jansen Research and Development.


5,984 studies conducted simultaneously with data from electronic health records

In the new paper, the researchers used unbiased algorithms and enormous databases of anonymous electronic patient records to perform simultaneously all reasonable observational studies about the side effects of depression treatments. The team defined 17 treatments, 272 pairs of combined treatments, and 22 outcomes—generating almost 6,000 research hypotheses and more than 55,000 control questions.

The analysis crunched data from hundreds of millions of patients and ran continuously for about a month on a powerful network of computers. The result was 5,984 calibrated estimates of effects from different treatments.

In a small sample, most of the computer’s results matched findings known from randomized controlled trials.

Reassuringly, the analysis yielded both positive and negative results, in a distribution that appeared to be more consistent with expectations of what fraction of drugs have side effects. In a small sample, most of the computer’s results matched findings known from randomized controlled trials.

Clinicians can use these results in the same way they now use results scattered across observational papers in the literature. “When data from clinical trials are unavailable, clinicians can make well-informed decisions based on evidence gained by our type of high-throughput observational research,” Hripcsak says.


New approach to observational research

Such evidence is more reliable than that contained in current published literature, he adds. In the same paper, the researchers used sophisticated computer algorithms to analyze the results of all published studies on treatments for depression. They found sharp cutoffs in the way effects are reported, pointing to a stark publication bias.

“The absence of negative results makes the published literature highly suspect,” Hripcsak says. “Did you have to do a billion studies to get those positive ones, in which case they’re probably mostly chance, or did you do a small number, in which case the effect is probably real.”

The new analysis yielded both positive and negative results, in a distribution that appeared to be more consistent with expectations of what fraction of drugs have side effects. Image: Schuemie, et al. 10.1098/rsta.2017.0356

Pointing to a graph of all of the new results, Hripcsak explains that “we’re using state of the art methods for each of these dots, and we throw nothing away, we’re transparent” about how the data were processed. Any one of the paper’s thousands of results, in other words, would meet the current standards for publication in a peer-reviewed journal as an individual paper.

Hripcsak hopes the work will point the way toward a new approach to observational research. But that may take some time. Established researchers and publications have been reluctant to accept the idea of automated studies; the paper sat in review at one major journal for a year before being rejected.

With the publication of this proof-of-concept paper, though, the group now plans to proceed through other areas of biomedical research, eventually yielding a trove of results that any scientist or statistically minded physician can mine for insights.



The paper is titled “A systematic approach to improving the reliability and scale of evidence from health care data.”

Other authors: Martijn J. Schuemie (Janssen Research and Development, Titusville, NJ); Patrick B. Ryan (Janssen Research and Development and Columbia University Vagelos College of Physicians and Surgeons); David Madigan (Columbia University); and Marc A. Suchard (University of California, Los Angeles). All authors are members of the Observational Health Data Sciences and Informatics program, an international network of researchers and observational health databases with a central coordinating center housed at Columbia University.

This work was supported in part through the National Science Foundation (IIS1251151 and DMS1264153) and the National Institutes of Health (R01LM06910 and U01HG008680).

The authors declare no conflicts of interest.

Tagged with:

About author

Related Articles