Breaking News
December 13, 2018 - New study reveals yin-yang personality of dopamine
December 13, 2018 - Long-term Benefit of Steroid Injections for Knee Osteoarthritis Challenged
December 13, 2018 - Adding new channels to the brain remote control
December 13, 2018 - In the Spotlight: A different side of neuroscience
December 13, 2018 - Medical Marvels: Using immunotherapy for melanoma that spread to the brain
December 13, 2018 - Puzzles do not keep dementia away finds study
December 13, 2018 - New mouse model shows potential for rapid identification of promising muscular dystrophy therapies
December 13, 2018 - Study reveals urban and rural differences in prenatal exposure to essential and toxic elements
December 13, 2018 - New collaborative partnership in quest of novel antibiotics
December 13, 2018 - Single tau molecule holds clues to help diagnose neurodegeneration in its earliest stages
December 13, 2018 - AHA Scientific Statement: Low Risk of Side Effects for Statins
December 13, 2018 - What Is Acute Flaccid Myelitis?
December 13, 2018 - How bereaved people control their thoughts without knowing it
December 13, 2018 - Health care democratization underway, according to 2nd annual Stanford Medicine Health Trends Report | News Center
December 13, 2018 - Going Beyond a Single Color
December 13, 2018 - London-based startup launches ‘’ aiming to clean up CBD industry
December 13, 2018 - Loss of tight junction barrier protein results in gastric cancer development
December 13, 2018 - Novel way to efficiently deliver anti-parasitic medicines
December 13, 2018 - RKI publishes new data on disease prevention and utilization of medical services
December 13, 2018 - High-tech, flexible patches sewn into clothes could help to stay warm
December 13, 2018 - Restoring Hair Growth on Scarred Skin? Mouse Study Could Show the Way
December 13, 2018 - Probiotic use may reduce antibiotic prescriptions, researchers say
December 13, 2018 - Drug repositioning strategy identifies potential new treatments for epilepsy
December 13, 2018 - Chronic rhinitis associated with hospital readmissions for asthma and COPD patients
December 13, 2018 - Food poisoning discovery could save lives
December 13, 2018 - Cloned antibodies show potential to treat, diagnose life-threatening fungal infections
December 13, 2018 - Exercise may reduce colorectal cancer risk after weight loss
December 13, 2018 - Russian scientists create hardware-information system for brain disorders treatment
December 13, 2018 - Moderate alcohol consumption linked with lower risk of hospitalization
December 13, 2018 - Nurturing Healthy Neighborhoods | NIH News in Health
December 13, 2018 - Rise in meth and opioid use during pregnancy
December 13, 2018 - Researchers gain new insights into pediatric tumors
December 13, 2018 - FSU study finds racial disparity among adolescents receiving flu vaccine
December 13, 2018 - Study investigates attitudes toward implementation of ‘sex as a biological variable’ policy
December 13, 2018 - Drug cocktail induces cancer cell death by switching off energy supply
December 13, 2018 - Baculovirus virion completely eliminates liver-stage parasites in mouse model
December 13, 2018 - Researchers create noninvasive technology that detects when nerve cells fire
December 13, 2018 - Allen Institute for Immunology to partner with CU Anschutz to understand dynamics of human immune system
December 13, 2018 - Inability to do daily living tasks delays discharge of mental health patients
December 13, 2018 - Treating patients with hypertension induced albuminuria
December 13, 2018 - New substance could improve efficacy of established breast cancer treatments
December 13, 2018 - Scientists develop new stem cell line to study conversion of stem cells into muscle
December 13, 2018 - Re-programming the body’s energy pathway boosts kidney self-repair
December 13, 2018 - Research findings could help improve treatment of anxiety and post-traumatic stress disorders
December 13, 2018 - The Microbiome Movement announce Microbiotica as official industry partner
December 13, 2018 - New study reveals potential benefits of cEEG monitoring for infant ICU patients
December 13, 2018 - Whole-body imaging PET/MRI offers information to guide treatment options for prostate cancer
December 13, 2018 - International investigators fight against the negative campaign on benzodiazepines
December 13, 2018 - Targeting biochemical pathway may lead to new therapies for alleviating symptoms of anxiety disorders
December 13, 2018 - FDA Approves Tolsura (SUBA®-itraconazole capsules) for the Treatment of Certain Fungal Infections
December 13, 2018 - Are scientists studying the wrong kind of mice?
December 13, 2018 - Computer memory: A scientific team builds a virtual model of a key brain region
December 13, 2018 - Visual inspection alone is insufficient to diagnose skin cancer
December 13, 2018 - Paternal grandfather’s access to food associated with grandson’s mortality risk
December 13, 2018 - Our brain senses angry voices in a flash, study shows
December 13, 2018 - PM2.5 Exposure Linked to Asthma Rescue Medication Use
December 13, 2018 - Can’t exercise? A hot bath may help improve inflammation, metabolism, study suggests
December 13, 2018 - Can artificial intelligence help doctors with the human side of medicine?
December 13, 2018 - Virginia Tech and UC San Diego researchers team up to develop nonopioid drug for chronic pain
December 13, 2018 - NIH offers support for HIV care and prevention research in the southern United States
December 12, 2018 - Activating brain region could revive the urge to socialize among opioid addicts
December 12, 2018 - Relationship impairment appears to interfere with seeking mental health treatment in men
December 12, 2018 - Sleep, Don’t Cram, Before Finals for Better Grades
December 12, 2018 - Effective treatments for urticarial vasculitis
December 12, 2018 - Gun violence is a public health issue: One physician’s story
December 12, 2018 - The Science of Healthy Aging
December 12, 2018 - Yes to yoghurt and cheese: New improved Mediterranean diet
December 12, 2018 - Researchers uncover a number of previously unknown insecticide resistance mechanisms
December 12, 2018 - Regulating the immune system’s ‘regulator’
December 12, 2018 - In breaking bad news, the comfort of silence
December 12, 2018 - Study finds upward link between alcohol consumption and physical activity in college students
December 12, 2018 - FDA issues warning letter to Zhejiang Huahai Pharmaceutical involved in valsartan recall
December 12, 2018 - Weight history at ages 20 and 40 could help predict patients’ future risk of heart failure
December 12, 2018 - Presence of antiphospholipid antibodies tied to first-time MI
December 12, 2018 - DNA analysis finds that stethoscopes are teaming with bacteria
December 12, 2018 - New study could help inform research on preventing falls
December 12, 2018 - Women and men with heart attack symptoms receive different care from EMS
December 12, 2018 - Disrupted biological clock can contribute to onset of diseases, USC study shows
December 12, 2018 - New publications generate controversy over the value of reducing salt consumption in populations
December 12, 2018 - New data from TAILORx trial confirms lack of chemo benefit regardless of race or ethnicity
Can Electronic Health Data Solve Medicine’s Reproducibility Problem?

Can Electronic Health Data Solve Medicine’s Reproducibility Problem?

image_pdfDownload PDFimage_print

According to headlines in recent years, coffee causes cancer…and it cures cancer; moderate alcohol consumption improves health…and adults should consume no alcohol for optimum health; drugs for osteoporosis cause esophageal cancer…and don’t cause it. And the list of incompatible results goes on.

“The research literature is filled with examples of studies reporting statistically significant results in conflicting directions, sometimes even when using the same data,” says George Hripcsak, MD, chair of biomedical informatics at Columbia University Vagelos College of Physicians and Surgeons.

George Hripcsak. Photo: Jörg Meyer.

The divergent findings reported in these headlines usually come from observational studies, a research design known for its poor reproducibility. In contrast to randomized controlled trials, which enroll patients before a study begins, observational studies analyze past events.

And though that approach is rational in theory, in practice it’s fraught with problems.

“If you parse the medical literature you find that it’s basically a data dredging machine,” Hripcsak says. “There’s publication bias, when authors and editors tend to publish things that match the answers they were looking for, or that advance their careers, or that benefit the journal.”

With massive troves of electronic health data now available, Hripcsak and his colleagues think the time is right to change the way observational studies are conducted. Instead of performing one study and publishing (or not) one result at a time, electronic health data now allow researchers to answer thousands of questions at once.

“By disseminating all the findings, we not only provide new results on a large scale but also prevent the publication bias we see in the literature,” says Hripcsak. “The reader then sees all the results, not just the ones favored by the authors or editors, and other researchers can see a whole body of work to better judge if the methods are operating as they should.”

Their new approach is described–and used to examine treatments for depression–in a paper published this month in Philosophical Transactions of the Royal Society A by Hripcsak and a team of coauthors from Columbia, the University of California Los Angeles, and Jansen Research and Development.


5,984 studies conducted simultaneously with data from electronic health records

In the new paper, the researchers used unbiased algorithms and enormous databases of anonymous electronic patient records to perform simultaneously all reasonable observational studies about the side effects of depression treatments. The team defined 17 treatments, 272 pairs of combined treatments, and 22 outcomes—generating almost 6,000 research hypotheses and more than 55,000 control questions.

The analysis crunched data from hundreds of millions of patients and ran continuously for about a month on a powerful network of computers. The result was 5,984 calibrated estimates of effects from different treatments.

In a small sample, most of the computer’s results matched findings known from randomized controlled trials.

Reassuringly, the analysis yielded both positive and negative results, in a distribution that appeared to be more consistent with expectations of what fraction of drugs have side effects. In a small sample, most of the computer’s results matched findings known from randomized controlled trials.

Clinicians can use these results in the same way they now use results scattered across observational papers in the literature. “When data from clinical trials are unavailable, clinicians can make well-informed decisions based on evidence gained by our type of high-throughput observational research,” Hripcsak says.


New approach to observational research

Such evidence is more reliable than that contained in current published literature, he adds. In the same paper, the researchers used sophisticated computer algorithms to analyze the results of all published studies on treatments for depression. They found sharp cutoffs in the way effects are reported, pointing to a stark publication bias.

“The absence of negative results makes the published literature highly suspect,” Hripcsak says. “Did you have to do a billion studies to get those positive ones, in which case they’re probably mostly chance, or did you do a small number, in which case the effect is probably real.”

The new analysis yielded both positive and negative results, in a distribution that appeared to be more consistent with expectations of what fraction of drugs have side effects. Image: Schuemie, et al. 10.1098/rsta.2017.0356

Pointing to a graph of all of the new results, Hripcsak explains that “we’re using state of the art methods for each of these dots, and we throw nothing away, we’re transparent” about how the data were processed. Any one of the paper’s thousands of results, in other words, would meet the current standards for publication in a peer-reviewed journal as an individual paper.

Hripcsak hopes the work will point the way toward a new approach to observational research. But that may take some time. Established researchers and publications have been reluctant to accept the idea of automated studies; the paper sat in review at one major journal for a year before being rejected.

With the publication of this proof-of-concept paper, though, the group now plans to proceed through other areas of biomedical research, eventually yielding a trove of results that any scientist or statistically minded physician can mine for insights.



The paper is titled “A systematic approach to improving the reliability and scale of evidence from health care data.”

Other authors: Martijn J. Schuemie (Janssen Research and Development, Titusville, NJ); Patrick B. Ryan (Janssen Research and Development and Columbia University Vagelos College of Physicians and Surgeons); David Madigan (Columbia University); and Marc A. Suchard (University of California, Los Angeles). All authors are members of the Observational Health Data Sciences and Informatics program, an international network of researchers and observational health databases with a central coordinating center housed at Columbia University.

This work was supported in part through the National Science Foundation (IIS1251151 and DMS1264153) and the National Institutes of Health (R01LM06910 and U01HG008680).

The authors declare no conflicts of interest.

Tagged with:

About author

Related Articles