Breaking News
October 20, 2018 - Eye discovery to pave way for more successful corneal transplants
October 20, 2018 - New analysis examines the importance of location in the opioid crisis
October 20, 2018 - Green filters increase reading speed for children with dyslexia
October 19, 2018 - Bariatric Sx Cuts Macrovascular Complications in Obesity, T2DM
October 19, 2018 - Better assessments for early age-related macular degeneration
October 19, 2018 - Visible and valued: Stanford Medicine’s first-ever LGBTQ+ Forum | News Center
October 19, 2018 - Understanding of metal-free enzymes used by bacteria could lead to new effective antibiotics
October 19, 2018 - Beckman Coulter Life Sciences announces new research-focused website
October 19, 2018 - Study finds link between refined soluble fibers, gut microbiota and liver cancer
October 19, 2018 - Social media reduces risk of depression among seniors with pain
October 19, 2018 - Newly developed synthetic DNA molecule may one day be used as ‘vaccine’ for prostate cancer
October 19, 2018 - Preoperative weight loss may not provide health benefits after surgery
October 19, 2018 - U.S. Birth Rates Continue to Drop as Age of New Moms Rises
October 19, 2018 - New technology can keep an eye on babies’ movements in the womb
October 19, 2018 - Juul e-cigarettes pose addiction risk for young users | News Center
October 19, 2018 - Gene sequencing reveals crucial molecular aspects of Trypanosoma brucei
October 19, 2018 - New DNA vaccine strategy protects mice against lethal challenge by multiple H3N2 viruses
October 19, 2018 - Study shows close link between cytokine interleukin-1ß and obesity-promoted colon cancer
October 19, 2018 - Muscle mass plays a critical role in health, shows research
October 19, 2018 - Study finds undiagnosed prediabetes in many infertile men
October 19, 2018 - The Current issue of “The view from here” is concerned with Nanotherapeutic strategies
October 19, 2018 - Delay in replacing the Pap smear with HPV screening is costing lives
October 19, 2018 - Physicians battle pediatric diseases of ear, nose, throat in Zimbabwe | News Center
October 19, 2018 - Researchers investigate why some cancers affect only young women
October 19, 2018 - Drugmakers funnel millions to lawmakers; a few dozen get $100,000-plus
October 19, 2018 - Unselfish people tend to have more children and receive higher salaries
October 19, 2018 - New findings reveal potential cellular players in tumor microenvironment
October 19, 2018 - Human brain cell transplant offers insights into neurological conditions
October 19, 2018 - Parental education associated with increased family health care spending
October 19, 2018 - New statistical method estimates long- and short-term risk of recurrence of breast cancer in US women
October 19, 2018 - Father’s exposure to nicotine may cause cognitive deficits in descendants
October 19, 2018 - Could we prevent Alzheimer’s disease by treating herpes?
October 19, 2018 - Nurse-led care can be more successful in managing gout
October 19, 2018 - Trump administration, pharma exchange verbal volleys on drug-price transparency
October 19, 2018 - Duke researchers find way to detect blood doping in athletes
October 19, 2018 - Many primary care doctors are still prescribing sedative drugs for older adults
October 19, 2018 - Finger length can predict sexuality in women say researchers
October 19, 2018 - Study finds differences in side-effects experienced by male and female OG cancer patients
October 19, 2018 - Few Seniors Who Self-Harm Referred for Mental Health Care
October 19, 2018 - Don’t sweat the sweet stuff
October 19, 2018 - URMC researchers discover new approach to deliver therapeutics to the brain
October 19, 2018 - Speech Pathology Australia raises awareness about Developmental Language Disorder
October 19, 2018 - Middlemen suppliers can increase drug prices and hospital bills, say Johns Hopkins researchers
October 19, 2018 - Survey finds high prevalence of HTLV-1 infection among teens and adults in Gabon
October 19, 2018 - Bliss funds research to find whether parental touch can help alleviate pain in premature infants
October 19, 2018 - Human neurons employ highly compartmentalized signaling, study shows
October 19, 2018 - Ultromics expands multiple clinical trials for coronary heart disease to the U.S.
October 19, 2018 - $11 million NIH grant for Clemson University helps launch new center for musculoskeletal research
October 19, 2018 - A new approach identified to control Zika virus, dengue fever
October 19, 2018 - Head Blows Without Concussion May Not Damage Brain, Study Claims
October 19, 2018 - US opioid use not declined, despite focus on abuse and awareness of risk
October 19, 2018 - Next-generation RNA sequencing technology sheds new light on human mitochondrial diseases
October 19, 2018 - UT Southwestern biochemist receives 2019 Breakthrough Prize in Life Sciences for innate immunity discovery
October 19, 2018 - The immune system also plays a key role in day-to-day function of healthy organs
October 19, 2018 - New tool may reveal how the brain structure impacts brain activity, human behavior
October 19, 2018 - Trump Administration announces ‘Winning on Reducing Food Waste’ initiative
October 19, 2018 - For-profit nursing home residents more likely to experience health issues caused by substandard care
October 19, 2018 - Incidence of stroke has risen steadily among marijuana users, show studies
October 19, 2018 - Conceptual framework proposed to examine role of exercise in multiple sclerosis
October 19, 2018 - Near infrared spectroscopy technique for accurate evaluation of chondral injuries
October 19, 2018 - Scientists receive $5.1 million grant to develop stem cell-based therapy for blinding retinal conditions
October 19, 2018 - Shorter physician encounters associated with antibiotic prescribing
October 19, 2018 - In the Spotlight: Enjoying research and exploring opportunities
October 19, 2018 - Physical activity lowers cardiovascular mortality risk in frail older adults
October 19, 2018 - New imaging tool helps visualize how sound-induced vibrations travel through the ear
October 19, 2018 - Key insights into the application, production of bioactive materials
October 19, 2018 - New urea sorbent could speed up the development of wearable artificial kidney
October 19, 2018 - Intensive care patients’ muscles less able to use fats for energy
October 19, 2018 - FDA Advisory Committee Recommends Approval of Dsuvia for the Treatment of Moderate-to-Severe Acute Pain
October 19, 2018 - 48,XXXY syndrome – Genetics Home Reference
October 19, 2018 - Physical exercise improves the elimination of toxic proteins from muscles
October 19, 2018 - How a new system improved wait times for Stanford kidney transplant patients
October 19, 2018 - Nutrition has bigger positive impact on bone mass and strength than exercise
October 19, 2018 - Study finds lack of progress in media representation of nurses over last 20 years
October 19, 2018 - Many people have trouble understanding differences between OCD and OCPD
October 19, 2018 - New family planning app found to be as effective as modern methods
October 19, 2018 - Gastric Banding, Metformin Similar for Improving Glycemia
October 19, 2018 - Physiologist publishes findings on the role of the protein titin in muscle contraction
October 19, 2018 - What digital health companies need to do to succeed
October 19, 2018 - N. Carolina Sees Alarming Spike in Heart Infections Among Opioid Users
Can Electronic Health Data Solve Medicine’s Reproducibility Problem?

Can Electronic Health Data Solve Medicine’s Reproducibility Problem?

image_pdfDownload PDFimage_print

According to headlines in recent years, coffee causes cancer…and it cures cancer; moderate alcohol consumption improves health…and adults should consume no alcohol for optimum health; drugs for osteoporosis cause esophageal cancer…and don’t cause it. And the list of incompatible results goes on.

“The research literature is filled with examples of studies reporting statistically significant results in conflicting directions, sometimes even when using the same data,” says George Hripcsak, MD, chair of biomedical informatics at Columbia University Vagelos College of Physicians and Surgeons.

George Hripcsak. Photo: Jörg Meyer.

The divergent findings reported in these headlines usually come from observational studies, a research design known for its poor reproducibility. In contrast to randomized controlled trials, which enroll patients before a study begins, observational studies analyze past events.

And though that approach is rational in theory, in practice it’s fraught with problems.

“If you parse the medical literature you find that it’s basically a data dredging machine,” Hripcsak says. “There’s publication bias, when authors and editors tend to publish things that match the answers they were looking for, or that advance their careers, or that benefit the journal.”

With massive troves of electronic health data now available, Hripcsak and his colleagues think the time is right to change the way observational studies are conducted. Instead of performing one study and publishing (or not) one result at a time, electronic health data now allow researchers to answer thousands of questions at once.

“By disseminating all the findings, we not only provide new results on a large scale but also prevent the publication bias we see in the literature,” says Hripcsak. “The reader then sees all the results, not just the ones favored by the authors or editors, and other researchers can see a whole body of work to better judge if the methods are operating as they should.”

Their new approach is described–and used to examine treatments for depression–in a paper published this month in Philosophical Transactions of the Royal Society A by Hripcsak and a team of coauthors from Columbia, the University of California Los Angeles, and Jansen Research and Development.


5,984 studies conducted simultaneously with data from electronic health records

In the new paper, the researchers used unbiased algorithms and enormous databases of anonymous electronic patient records to perform simultaneously all reasonable observational studies about the side effects of depression treatments. The team defined 17 treatments, 272 pairs of combined treatments, and 22 outcomes—generating almost 6,000 research hypotheses and more than 55,000 control questions.

The analysis crunched data from hundreds of millions of patients and ran continuously for about a month on a powerful network of computers. The result was 5,984 calibrated estimates of effects from different treatments.

In a small sample, most of the computer’s results matched findings known from randomized controlled trials.

Reassuringly, the analysis yielded both positive and negative results, in a distribution that appeared to be more consistent with expectations of what fraction of drugs have side effects. In a small sample, most of the computer’s results matched findings known from randomized controlled trials.

Clinicians can use these results in the same way they now use results scattered across observational papers in the literature. “When data from clinical trials are unavailable, clinicians can make well-informed decisions based on evidence gained by our type of high-throughput observational research,” Hripcsak says.


New approach to observational research

Such evidence is more reliable than that contained in current published literature, he adds. In the same paper, the researchers used sophisticated computer algorithms to analyze the results of all published studies on treatments for depression. They found sharp cutoffs in the way effects are reported, pointing to a stark publication bias.

“The absence of negative results makes the published literature highly suspect,” Hripcsak says. “Did you have to do a billion studies to get those positive ones, in which case they’re probably mostly chance, or did you do a small number, in which case the effect is probably real.”

The new analysis yielded both positive and negative results, in a distribution that appeared to be more consistent with expectations of what fraction of drugs have side effects. Image: Schuemie, et al. 10.1098/rsta.2017.0356

Pointing to a graph of all of the new results, Hripcsak explains that “we’re using state of the art methods for each of these dots, and we throw nothing away, we’re transparent” about how the data were processed. Any one of the paper’s thousands of results, in other words, would meet the current standards for publication in a peer-reviewed journal as an individual paper.

Hripcsak hopes the work will point the way toward a new approach to observational research. But that may take some time. Established researchers and publications have been reluctant to accept the idea of automated studies; the paper sat in review at one major journal for a year before being rejected.

With the publication of this proof-of-concept paper, though, the group now plans to proceed through other areas of biomedical research, eventually yielding a trove of results that any scientist or statistically minded physician can mine for insights.



The paper is titled “A systematic approach to improving the reliability and scale of evidence from health care data.”

Other authors: Martijn J. Schuemie (Janssen Research and Development, Titusville, NJ); Patrick B. Ryan (Janssen Research and Development and Columbia University Vagelos College of Physicians and Surgeons); David Madigan (Columbia University); and Marc A. Suchard (University of California, Los Angeles). All authors are members of the Observational Health Data Sciences and Informatics program, an international network of researchers and observational health databases with a central coordinating center housed at Columbia University.

This work was supported in part through the National Science Foundation (IIS1251151 and DMS1264153) and the National Institutes of Health (R01LM06910 and U01HG008680).

The authors declare no conflicts of interest.

Tagged with:

About author

Related Articles