Breaking News
February 22, 2019 - Growing number of cancer survivors, fewer providers point to challenge in meeting care needs
February 22, 2019 - Innovative compound offers a new therapeutic approach to treat multiple sclerosis
February 22, 2019 - $1.5 million grant to develop opioid treatment program for jail detainees
February 22, 2019 - FDA’s new proposed rule would update regulatory requirements for sunscreen products in the U.S
February 22, 2019 - Most Hip, Knee Replacements Last Decades, Study Finds
February 22, 2019 - Wellness problems prevalent among ob-gyn residents
February 22, 2019 - In the Spotlight: “The world is your oyster in geriatrics”
February 22, 2019 - Successful testing of multi-organ “human-on-a-chip” could replace animals as test subjects
February 22, 2019 - Analysis of cervical precancer shows decline in two strains of HPV
February 22, 2019 - Sugary stent eases suturing of blood vessels
February 22, 2019 - From surgery to psychiatry: A medical student reevaluates his motivations
February 22, 2019 - Is New App From Feds Your Answer To Navigating Medicare Coverage? Yes And No
February 22, 2019 - New pacemakers powered by heartbeats could reduce need for surgery
February 22, 2019 - The United States records highest drug overdose death rates
February 22, 2019 - Phase 1 data reinforce safety profile of new drug for treating Duchenne muscular dystrophy
February 22, 2019 - Vitamin D supplementation less effective in the presence of obesity, shows study
February 22, 2019 - Sarepta Announces FDA Acceptance of Golodirsen (SRP-4053) New Drug Application for Patients with Duchenne Muscular Dystrophy Amenable to Skipping Exon 53
February 22, 2019 - An institutional effort to reduce the amount of opioids prescribed following lumbar surgery
February 22, 2019 - Failure to take statins leads to higher mortality rates | News Center
February 22, 2019 - New study explains why some patients report phantom sensations after limb amputation
February 22, 2019 - First motor-controlled heart valves implanted by Mainz University Medical Center
February 22, 2019 - Novel preclinical model mimics persistent interneuron loss seen in preterm infants
February 22, 2019 - Global health burden of glaucoma has increased, study reveals
February 22, 2019 - A holistic approach key to minimize treatment complexity in patients with interstitial lung disease
February 22, 2019 - 1 in 10 middle-aged Chinese adults are at high risk for heart disease, finds study
February 22, 2019 - More than half a million breast cancer patient’s lives saved by improvements in treatment
February 22, 2019 - Study finds no evidence that tougher policies prevent teenage cannabis use
February 22, 2019 - New blood test detects genetic disorders in fetuses
February 22, 2019 - Lower Self-Perception Observed in Children With Amblyopia
February 22, 2019 - Up to 15 percent of children have sleep apnea, yet 90 percent go undiagnosed
February 22, 2019 - Rare pulmonary defect prompts parents’ nationwide search for answers | News Center
February 22, 2019 - Lesbian and bisexual women at greater risk of being overweight, study finds
February 22, 2019 - UQ research may explain why vitamin D is essential for brain health
February 22, 2019 - Heart Attacks Rising Among Younger Women
February 22, 2019 - How your smartphone is affecting your relationship
February 22, 2019 - Orthopaedic surgeon receives prestigious award, $10 million grant | News Center
February 22, 2019 - New sepsis test could save thousands of lives
February 22, 2019 - Cervical cancer could be eradicated by 2100
February 21, 2019 - Sustained smoking cessation can lower risk of seropositive RA
February 21, 2019 - Thousands with chronic UTIs are not receiving the treatment they need
February 21, 2019 - Are teens getting high on social media? The surprising study seeking the pot-Instagram link
February 21, 2019 - Stanford expands biobank services | News Center
February 21, 2019 - Scientists identify link between drinking contexts and early onset intoxication among adolescents
February 21, 2019 - Strong social support may reduce cardiovascular disease risk in postmenopausal women
February 21, 2019 - Rapid expansion of interventions could prevent up to 13 million cases of cervical cancer within 50 years
February 21, 2019 - Motif Bio Receives Complete Response Letter From The FDA
February 21, 2019 - Researchers map previously unknown disease in children
February 21, 2019 - A skeptical look at popular diets: Going gluten-free
February 21, 2019 - Podcast: KHN’s ‘What The Health?’ How Safe Are Your Supplements?
February 21, 2019 - Factors associated with increased risk of developing surgical site infections
February 21, 2019 - Anticipatory signals in eye movements can help measure attentive capacity, learning with greater precision
February 21, 2019 - Study explores daily exposure to indoor air pollutants
February 21, 2019 - Evening exercise does not negatively affect sleep, may also reduce hunger
February 21, 2019 - Artificial intelligence technique can be used to identify alcohol misuse in trauma setting
February 21, 2019 - Overweight, obesity in adolescence associated with increased risk of renal cancer later in life
February 21, 2019 - BGU develops new AI platform for monitoring and predicting ALS progression
February 21, 2019 - Researchers discover a new promising target to improve HIV vaccines
February 21, 2019 - Brief Anesthesia in Infancy Does Not Mar Neurodevelopment
February 21, 2019 - Gaming system helps with autism diagnosis
February 21, 2019 - Heart Disease: Six Things Women Should Know
February 21, 2019 - More States Say Doctors Must Offer Overdose Reversal Drug Along With Opioids
February 21, 2019 - Researchers explore case studies focused on industries that kill more people than employed
February 21, 2019 - Only half of GP practice buildings are fit for purpose
February 21, 2019 - Intense exercise, fasting and hormones can enhance waste-protein removal, study shows
February 21, 2019 - Scientists can monitor brain activity to predict epileptic seizures few minutes in advance
February 21, 2019 - Study quantifies hepatic and intestinal mRNA expression of Ugt isoforms in rats
February 21, 2019 - ‘Apple-Shaped’ Body? ‘Pear-Shaped’? Your Genes May Tell
February 21, 2019 - Can we repair the brain? The promise of stem cell technologies for treating Parkinson’s disease
February 21, 2019 - Trump Plan To Beat HIV Hits Rough Road In Rural America
February 21, 2019 - PENTAX Medical introduces new electrosurgical and argon plasma coagulation platforms
February 21, 2019 - Trump plan to beat HIV hits rough road in rural America
February 21, 2019 - Eating blueberries every day could help decrease blood pressure
February 21, 2019 - ‘No Second Chances’ report calls for new measures to combat cardiovascular disease in Australia
February 21, 2019 - Mayo clinic researchers discuss local case studies of leprosy
February 21, 2019 - Scientists demonstrate key role of salt in allergic immune reactions
February 21, 2019 - Experts propose revising the criteria for diagnosis of Parkinson’s disease
February 21, 2019 - The med student and the machine
February 21, 2019 - Hey, Hey! Ho, Ho! Is Striking For School Nurses The Way To Go?
February 21, 2019 - Latest research encourages children to move out and learn through physical activity
February 21, 2019 - Proper oral hygiene and regular visits to dentist can promote heart health
Can Electronic Health Data Solve Medicine’s Reproducibility Problem?

Can Electronic Health Data Solve Medicine’s Reproducibility Problem?

image_pdfDownload PDFimage_print

According to headlines in recent years, coffee causes cancer…and it cures cancer; moderate alcohol consumption improves health…and adults should consume no alcohol for optimum health; drugs for osteoporosis cause esophageal cancer…and don’t cause it. And the list of incompatible results goes on.

“The research literature is filled with examples of studies reporting statistically significant results in conflicting directions, sometimes even when using the same data,” says George Hripcsak, MD, chair of biomedical informatics at Columbia University Vagelos College of Physicians and Surgeons.

George Hripcsak. Photo: Jörg Meyer.

The divergent findings reported in these headlines usually come from observational studies, a research design known for its poor reproducibility. In contrast to randomized controlled trials, which enroll patients before a study begins, observational studies analyze past events.

And though that approach is rational in theory, in practice it’s fraught with problems.

“If you parse the medical literature you find that it’s basically a data dredging machine,” Hripcsak says. “There’s publication bias, when authors and editors tend to publish things that match the answers they were looking for, or that advance their careers, or that benefit the journal.”

With massive troves of electronic health data now available, Hripcsak and his colleagues think the time is right to change the way observational studies are conducted. Instead of performing one study and publishing (or not) one result at a time, electronic health data now allow researchers to answer thousands of questions at once.

“By disseminating all the findings, we not only provide new results on a large scale but also prevent the publication bias we see in the literature,” says Hripcsak. “The reader then sees all the results, not just the ones favored by the authors or editors, and other researchers can see a whole body of work to better judge if the methods are operating as they should.”

Their new approach is described–and used to examine treatments for depression–in a paper published this month in Philosophical Transactions of the Royal Society A by Hripcsak and a team of coauthors from Columbia, the University of California Los Angeles, and Jansen Research and Development.


5,984 studies conducted simultaneously with data from electronic health records

In the new paper, the researchers used unbiased algorithms and enormous databases of anonymous electronic patient records to perform simultaneously all reasonable observational studies about the side effects of depression treatments. The team defined 17 treatments, 272 pairs of combined treatments, and 22 outcomes—generating almost 6,000 research hypotheses and more than 55,000 control questions.

The analysis crunched data from hundreds of millions of patients and ran continuously for about a month on a powerful network of computers. The result was 5,984 calibrated estimates of effects from different treatments.

In a small sample, most of the computer’s results matched findings known from randomized controlled trials.

Reassuringly, the analysis yielded both positive and negative results, in a distribution that appeared to be more consistent with expectations of what fraction of drugs have side effects. In a small sample, most of the computer’s results matched findings known from randomized controlled trials.

Clinicians can use these results in the same way they now use results scattered across observational papers in the literature. “When data from clinical trials are unavailable, clinicians can make well-informed decisions based on evidence gained by our type of high-throughput observational research,” Hripcsak says.


New approach to observational research

Such evidence is more reliable than that contained in current published literature, he adds. In the same paper, the researchers used sophisticated computer algorithms to analyze the results of all published studies on treatments for depression. They found sharp cutoffs in the way effects are reported, pointing to a stark publication bias.

“The absence of negative results makes the published literature highly suspect,” Hripcsak says. “Did you have to do a billion studies to get those positive ones, in which case they’re probably mostly chance, or did you do a small number, in which case the effect is probably real.”

The new analysis yielded both positive and negative results, in a distribution that appeared to be more consistent with expectations of what fraction of drugs have side effects. Image: Schuemie, et al. 10.1098/rsta.2017.0356

Pointing to a graph of all of the new results, Hripcsak explains that “we’re using state of the art methods for each of these dots, and we throw nothing away, we’re transparent” about how the data were processed. Any one of the paper’s thousands of results, in other words, would meet the current standards for publication in a peer-reviewed journal as an individual paper.

Hripcsak hopes the work will point the way toward a new approach to observational research. But that may take some time. Established researchers and publications have been reluctant to accept the idea of automated studies; the paper sat in review at one major journal for a year before being rejected.

With the publication of this proof-of-concept paper, though, the group now plans to proceed through other areas of biomedical research, eventually yielding a trove of results that any scientist or statistically minded physician can mine for insights.



The paper is titled “A systematic approach to improving the reliability and scale of evidence from health care data.”

Other authors: Martijn J. Schuemie (Janssen Research and Development, Titusville, NJ); Patrick B. Ryan (Janssen Research and Development and Columbia University Vagelos College of Physicians and Surgeons); David Madigan (Columbia University); and Marc A. Suchard (University of California, Los Angeles). All authors are members of the Observational Health Data Sciences and Informatics program, an international network of researchers and observational health databases with a central coordinating center housed at Columbia University.

This work was supported in part through the National Science Foundation (IIS1251151 and DMS1264153) and the National Institutes of Health (R01LM06910 and U01HG008680).

The authors declare no conflicts of interest.

Tagged with:

About author

Related Articles