Not all high-quality tumor biomarker assays are highly congruent or have clinical relevance, results from two separate comparison studies showed.
In the first, laboratory-developed tests (LDTs) and FDA-approved companion diagnostics (FDA-CDs) both demonstrated a high degree of accuracy and excellent performance overall for three oncology analytes in College of American Pathologists (CAP) proficiency testing (PT), researchers said.
Pathological evaluation of tumors comparing LDTs and FDA-CDs in 6,897 proficiency responses showed that both test types exceeded 97% accuracy across all comparable molecular oncology proficiency testing samples for BRAF, EGFR, and KRAS genes, according to Annette S. Kim, MD, PhD, of Brigham and Women’s Hospital in Boston, and colleagues.
“These findings support both the excellent and equivalent performance of both LDTs and FDA-CDs in clinical diagnostic testing,” the study authors wrote in JAMA Oncology.
However, a comparison of laboratories’ pre-analytic practices showed more than 60% were using FDA-CDs outside the bounds of FDA-approved methods. Although participants said they made modifications to the FDA-CDs to increase the flexibility of clinical practice, this effectively rendered the assays LDTs, the study authors noted.
“Both FDA-CD and LDT laboratories accept a wide range of specimen preparations, as well as tumor types,” Kim and colleagues said. “Although this flexibility is advantageous for patient care, it is important to recognize that the use of specimens other than formalin-fixed paraffin-embedded samples of the specified tumor type for the FDA-CDs is off-label, resulting in reclassification of the assay as an LDT.”
Laboratories used the assay off-label by accepting specimens with lower tumor content than required for the approved FDA-CD assay, the study revealed. Other practice modifications included accepting unapproved specimen and tumor types, and not quantifying DNA before performing the FDA-CD assay.
In the study, PT responses were included for 2,524 BRAF (14 PT samples), 2,216 EGFR (11 PT samples), and 2,157 KRAS (10 PT samples). Approximately 15% of these assays were performed using commercially available FDA-CD kits and 85% were analyzed as LDTs.
There were no significant differences between LDTs and the FDA-CD acceptability rates for KRAS. The FDA-CDs performed less well than the LDTs for BRAF p.V600K analysis (CD 66.1% versus LDT 88.0%), and the LDT assay was inferior for EGFR p.L861Q mutation (CD 100% versus LDT 91%).
Noting the comparable performance of the two assays, the greater clinical relevance of LDTs and the “significant off-label use of FDA-CDs,” it might make more sense to reclassify FDA-CDs as LTDs, the researchers suggested.
In an accompanying editorial, Daniel F. Hayes, MD, of the University of Michigan Comprehensive Cancer Center, Ann Arbor, noted that a tumor biomarker test that has analytical validity but lacks clinical utility can’t be introduced into routine practice. “Therefore, approval of a tumor biomarker test by the FDA does not necessarily imply that it should be ordered or used to direct patient care,” he said.
In the era of precision oncology medicine, clinicians need to be knowledgeable about tumor biomarkers as well as the tests used to measure them, he added.
Although Hayes called the CAP proficiency testing “terrific” and said it “should make all of us who are clinicians sleep better at night,” he also pointed out that the study included only pathologists participating in the CAP proficiency testing.
“You should hope that your patient’s sample was tested in one of these laboratories and not in one that did not include these proficient pathologists,” he commented.
The number of tested specimens was low, Hayes said, who also zeroed in on the fact that more than half of the laboratories using FDA-CDs didn’t follow manufacturer’s instructions.
“Although it is reassuring that different laboratories using different techniques and assays usually came up with the same results for BRAF, EGFR, and KRAS, these results cannot be extrapolated to other, more complex tumor tests,” he concluded.
Wide Inter-Test Variability
In a related research letter in the same issue, Gonzalo Torga, MD, and Kenneth J. Pienta, MD, of Johns Hopkins University School of Medicine in Baltimore, reported findings from a small study showing very low patient-paired sample congruence between two liquid commercial biopsy tests. Both genetic tests had self-reported high accuracy, specificity, and sensitivity to detect and quantify tumor-specific alteration, and both laboratories were CAP-accredited for cfDNA next-generation sequencing.
However, in 40 patients with metastatic prostate cancer, 6 blood samples were unevaluable. In the remaining 34 samples, complete concordance between the two assays was observed in 9 (35%). The assays were partially concordant in six samples of the remaining 25, but in the last 16 samples, there was no concordance between the assays at all.
“These data cannot determine which test is more accurate but suggest that reported gene alterations will not be the same across different platforms, raising the specter that patients could potentially receive different treatments depending on the cfDNA platform,” the study authors said.
These results are “disturbing,” said Hayes, and have obvious implications for every clinician ordering laboratory-developed tests. Although LTDs have made important contributions to patient care, a lot of uncertainty remains about their accuracy and basis for use, Hayes added.
“I have often said that ‘a bad tumor biomarker test is as bad as a bad drug.’ Caveat emptor is not a good way to practice medicine, and I for one will continue to call for collegial and constructive measures to improve, and in many cases further regulate, tests that are so important for our patients.”
Lead study author Kim reported a relationship with Aushon Biosciences and a number of study co-authors also reported relationships with industry. Second study authors Torga and Pienta disclosed no potential conflicts of interest. Editorialist Hayes reported relationships with OncImmune, Inbiomotion, Merrimack Pharmaceuticals, Eli Lilly Company, Janssen, Menarini/Silicon bioSystems, Puma Biotechnology, Pfizer, AstraZeneca, and Menarini/Silicon bioSystems.