Slides and Transcript
Slide 1 of 15
So far in these talks, we've seen ways that pharmacogenetic testing can be useful but in this talk we're going to explore a mystery. Why do so many clinical trials of these tests fail?
Slide 2 of 15
A few years ago, the American Psychiatric Association commissioned a panel to evaluate pharmacogenetic tests. And in 2022, their committee chair, Charles Nemeroff, told the audience at an APA meeting, “There is no evidence to support that commercial pharmacogenetic tests can improve patient care.” Ironically, he made that statement while receiving an award named in honor of the psychiatrist who pioneered genetic tests, the late David Mrazek. But wait, if the US Food and Drug Administration recommends some of these tests, why did the American Psychiatric Association come out against them?
References:
- Zagorski, N. (2022). Charles Nemeroff Receives 2022 Mrazek Award. Psychiatric News, 57(7). https://doi.org/10.1176/appi.pn.2022.07.7.32
Free Files
Download PDF and other files
Success!
Check your inbox, we sent you all the materials there.
Slide 3 of 15
To understand that, you need to know the difference between genetic tests and pharmacogenetic panels. A genetic test is a single test whose results are relevant to certain medications, those drug-gene interactions that the FDA considers actionable and which we reviewed in an earlier talk. There's no controversy there. A pharmacogenetic panel though is different. These tests gather dozens of genes and use proprietary algorithms to make recommendations about nearly all psychiatric meds, not just the actionable ones.
References:
- Zagorski, N. (2022). Charles Nemeroff Receives 2022 Mrazek Award. Psychiatric News, 57(7). https://doi.org/10.1176/appi.pn.2022.07.7.32
- Bousman, C. A., & Dunlop, B. W. (2018). Genotype, phenotype, and medication recommendation agreement among commercial pharmacogenetic-based decision support tools. The Pharmacogenomics Journal, 18(5), 613-622. https://doi.org/10.1038/s41397-018-0027-33
Slide 4 of 15
The problem is that they mix irrelevant genes with relevant ones and water it down further by including drugs that have no actionable interactions. So Dr. Nemeroff is right. These pharmacogenetic panels have been tested in well-designed, industry-sponsored trials involving thousands of patients, and the results don't look good. I just said that pharmacogenetic panels do not work. But if you've been to a dinner program sponsored by the testing company, you may have heard the opposite.
References:
- Zagorski, N. (2022). Charles Nemeroff Receives 2022 Mrazek Award. Psychiatric News, 57(7). https://doi.org/10.1176/appi.pn.2022.07.7.32
Free Files
Download PDF and other files
Success!
Check your inbox, we sent you all the materials there.
Slide 5 of 15
To understand why, we need to remember that we live in the age of genetics and with that comes a good dose of hype that can bias the study if it was not blinded correctly. In the early studies of genetic panels, these panels appeared effective. Patients got better when their antidepressants were chosen based on the genetic results, much better than treatment as usual. But in those studies, the doctors or the patients were not blind to the test results.
References:
- Baum, M. L., Widge, A. S., Carpenter, L. L., McDonald, W. M., Cohen, B. M., Nemeroff, C. B., & American Psychiatric Association (APA) Workgroup on Biomarkers and Novel Treatments. (2024). Pharmacogenomic Clinical Support Tools for the Treatment of Depression. American Journal of Psychiatry, 181(7), 591-607. https://doi.org/10.1176/appi.ajp.20230657
Slide 6 of 15
Patients got excited to take a medication that matches their genes, and doctors might even convey a little more enthusiasm if they prescribe a medication that fits with a genetic panel. Around 2016, the company started blinding the participants and the results were no longer positive, none, including trials from GeneSight's Assurex, Genomind's Genecept, NeuroIDgenetix and Neuropharmagen. But many of these trials are still presented on company brochures as supporting the tests. What's going on here?
References:
- Baum, M. L., Widge, A. S., Carpenter, L. L., McDonald, W. M., Cohen, B. M., Nemeroff, C. B., & American Psychiatric Association (APA) Workgroup on Biomarkers and Novel Treatments. (2024). Pharmacogenomic Clinical Support Tools for the Treatment of Depression. American Journal of Psychiatry, 181(7), 591-607. https://doi.org/10.1176/appi.ajp.20230657
Free Files
Download PDF and other files
Success!
Check your inbox, we sent you all the materials there.
Slide 7 of 15
To understand that, we need to learn a little more about statistics, and I promise it won't be too painful. A clinical trial is trying to look for solid patterns in a cloud of statistical noise. To do this kind of work honestly, we need to start by assuming that everything we see is random noise and then try to prove the opposite for a single result. That's called the primary outcome. And we identify that primary outcome before we gather all the data. That is the right way to run a trial.
References:
- Greden, J. F., Parikh, S. V., Rothschild, A. J., Thase, M. E., Dunlop, B. W., DeBattista, C., Conway, C. R., Forester, B. P., Mondimore, F. M., Shelton, R. C., Macaluso, M., Li, J., Brown, K., Gilbert, A., Burns, L., Jablonski, M. R., & Dechairo, B. (2019). Impact of pharmacogenomics on clinical outcomes in major depressive disorder in the GUIDED trial: A large, patient- and rater-blinded, randomized, controlled study. Journal of Psychiatric Research, 111, 59-67. https://doi.org/10.1016/j.jpsychires.2019.01.0037
Slide 8 of 15
The wrong way is to gather all the data and then look at dozens of outcomes and to find one that fits a pattern and looks positive because when you do that it's a high probability that those secondary outcomes were positive just by random chance alone. That's called data fishing. Here's an analogy. Suppose you have 1000 colored beads and you shake them up in a box. The resulting pattern is random. But if you stare long enough, you might see some meaningful shapes in them – a winding road, a tree, a circle here and there. I wouldn't make too much of that. But suppose you said in advance, I predict that when these beads have been shaken a circle of yellow beads will appear near the center, if your words came true then I would be really impressed.
References:
- Greden, J. F., Parikh, S. V., Rothschild, A. J., Thase, M. E., Dunlop, B. W., DeBattista, C., Conway, C. R., Forester, B. P., Mondimore, F. M., Shelton, R. C., Macaluso, M., Li, J., Brown, K., Gilbert, A., Burns, L., Jablonski, M. R., & Dechairo, B. (2019). Impact of pharmacogenomics on clinical outcomes in major depressive disorder in the GUIDED trial: A large, patient- and rater-blinded, randomized, controlled study. Journal of Psychiatric Research, 111, 59-67. https://doi.org/10.1016/j.jpsychires.2019.01.003
Free Files
Download PDF and other files
Success!
Check your inbox, we sent you all the materials there.
Slide 9 of 15
That's why it's important to state the primary outcome before you shake up those beads or run that clinical trial. And that is not what the company brochures do. They cherry-pick secondary outcomes and put them on the cover of the brochures to make it look like the tests do more than they do.
References:
- Greden, J. F., Parikh, S. V., Rothschild, A. J., Thase, M. E., Dunlop, B. W., DeBattista, C., Conway, C. R., Forester, B. P., Mondimore, F. M., Shelton, R. C., Macaluso, M., Li, J., Brown, K., Gilbert, A., Burns, L., Jablonski, M. R., & Dechairo, B. (2019). Impact of pharmacogenomics on clinical outcomes in major depressive disorder in the GUIDED trial: A large, patient- and rater-blinded, randomized, controlled study. Journal of Psychiatric Research, 111, 59-67. https://doi.org/10.1016/j.jpsychires.2019.01.0039
Slide 10 of 15
But let's play devil's advocate and imagine what if those secondary results really weren't just a random signal, what if they really represented the real truth about the test. Even then, they are not very impressive. With GeneSight's secondary measures, you would need to test 19 patients with depression to bring one to remission, which means it has a number needed to treat of 19. If their sample was limited to the minority of patients who actually had genetic variations that inform their medication choice, those figures improve slightly. Then you need to test 13 patients to bring 1 to remission.
References:
- Greden, J. F., Parikh, S. V., Rothschild, A. J., Thase, M. E., Dunlop, B. W., DeBattista, C., Conway, C. R., Forester, B. P., Mondimore, F. M., Shelton, R. C., Macaluso, M., Li, J., Brown, K., Gilbert, A., Burns, L., Jablonski, M. R., & Dechairo, B. (2019). Impact of pharmacogenomics on clinical outcomes in major depressive disorder in the GUIDED trial: A large, patient- and rater-blinded, randomized, controlled study. Journal of Psychiatric Research, 111, 59-67. https://doi.org/10.1016/j.jpsychires.2019.01.00310
Free Files
Download PDF and other files
Success!
Check your inbox, we sent you all the materials there.
Slide 11 of 15
The reason behind these disappointing numbers might have to do with their reliance on pharmacodynamic genes. These genes are supposed to tell us how the brain responds to medications, but there is scant evidence that any of them actually do that. In the next video, I'll show you how that played out with the most famous of the pharmacodynamic genes, the short arm of the serotonin receptor.
References:
- Bousman, C. A., Jaksa, P., & Pantelis, C. (2017). Systematic evaluation of commercial pharmacogenetic testing in psychiatry: a focus on CYP2D6 and CYP2C19 allele coverage and results reporting. Pharmacogenetics and Genomics, 27(11), 387-393. https://doi.org/10.1097/FPC.0000000000000303
- Preskorn, S. H. (2016). New laboratory tests in psychiatry: What should mental health practitioners know? Journal of Psychiatric Practice, 22(4), 308-312. https://doi.org/10.1097/PRA.0000000000000165
Slide 12 of 15
Another problem might lie in the accuracy of these genetic panels. In a study where samples were sent to different direct to consumer laboratories, they found inconsistency between the companies in the specific genotypes that they came up with as well as in the metabolic phenotypes, meaning whether they thought a patient had a poor, intermediate or rapid metabolizer status. And the medications that they recommended or cautioned against were very different. The FDA has gone so far as to issue cease and desist letters to some of these companies.
References:
- Bousman, C. A., Jaksa, P., & Pantelis, C. (2017). Systematic evaluation of commercial pharmacogenetic testing in psychiatry: a focus on CYP2D6 and CYP2C19 allele coverage and results reporting. Pharmacogenetics and Genomics, 27(11), 387-393. https://doi.org/10.1097/FPC.0000000000000303
- U.S. Food and Drug Administration. (2019, April 4). FDA issues warning letter to genomics lab for illegally marketing genetic test that claims to predict patients' responses to specific medications. https://tinyurl.com/2fn8f6sw12
Free Files
Download PDF and other files
Success!
Check your inbox, we sent you all the materials there.
Slide 13 of 15
While genotype panels may become helpful in the future, just the lack of standardization alone is enough reason to avoid them. Instead, I recommend ordering specific genes from reliable laboratories when they are indicated.
References:
- Bousman, C. A., Jaksa, P., & Pantelis, C. (2017). Systematic evaluation of commercial pharmacogenetic testing in psychiatry: a focus on CYP2D6 and CYP2C19 allele coverage and results reporting. Pharmacogenetics and Genomics, 27(11), 387-393. https://doi.org/10.1097/FPC.0000000000000303
Slide 14 of 15
Let's recap the key points of this talk. The early trials of pharmacogenetic panels looked promising for depression, but when the patients were blinded to the use of the test, the benefits lost their statistical significance. This is likely because there are only a few drug-gene interactions that are clinically meaningful, and most of them concern rare side effects like cardiac arrhythmias rather than general efficacy or tolerability.
Free Files
Download PDF and other files
Success!
Check your inbox, we sent you all the materials there.
Slide 15 of 15
