 |
|
Science and Ethics of Sham Surgery
A Survey of Parkinson Disease Clinical Researchers
Scott Y. H. Kim, MD, PhD;
Samuel Frank, MD;
Robert Holloway, MD, MPH;
Carol Zimmerman, RN;
Renee Wilson, MA;
Karl Kieburtz, MD, MPH
Arch Neurol. 2005;62:1357-1360.
ABSTRACT
| |
Background Sham surgery is used in neurosurgical clinical trials in Parkinson disease (PD) but remains controversial. The controversy may be compounded when gene-transfer technologies are tested in sham surgical trials.
Objective To determine the perspective of PD clinical researchers on the science and ethics of sham-surgery controls when used to test novel interventions such as gene transfer for PD.
Design Internet survey eliciting both quantitative and qualitative responses.
Participants Investigator members of the Parkinson Study Group.
Results Overall response rate was 103 (61.3%) of 168 researchers. A large majority (97%) of PD clinical researchers believe sham-surgery controls are better than unblinded controls for testing the efficacy of neurosurgical interventions such as gene transfer for PD. Half of the researchers believe an unblinded control efficacy trial would be unethical because it may lead to a falsely positive result. A minority (less than 22%) believe that an invasive sham condition that involves penetration of brain tissue is justified.
Conclusion It appears unlikely that the PD clinical research community will perceive future neurosurgical interventions for PD, such as gene-transfer therapies, as truly efficacious unless a sham-control condition is used to test it.
INTRODUCTION
What is the appropriate control condition for testing a neurosurgical intervention for the treatment of Parkinson disease (PD)? The use of placebo or sham-surgery controls—in which, for example, burr holes are created without further experimental intervention—in randomized trials of surgical interventions is becoming more common in PD research1-4 but not without controversy.5 The controversy may be further magnified when the intervention involves gene-transfer therapies, an area that has received its own share of ethical scrutiny.6 A phase 1 study of gene transfer for PD is already under way,7 and other studies will likely follow. Thus, we may soon be facing the question of how best to conduct a gene-transfer efficacy trial for PD.
Aside from questions surrounding informed consent, the controversy over sham-surgery controls involves 2 distinct but related issues. The first is whether a randomized clinical trial with a sham-surgery control is the scientifically optimum method to test novel neurosurgical interventions for PD. Because of significant placebo responses in PD treatment trials,8-9 many argue that a sham-control trial is the scientific ideal10-12 while others believe alternative designs are adequate.13 Second, is a sham-control condition ethically permissible in PD research? Specifically, are the risks and burdens to the subjects in randomized clinical trials with sham-surgery controls reasonable in relation to the potential benefit to the subjects and to society, as current federal regulations (45 CFR 46.111.a.2) require? Given that the degree of invasiveness varies with the kind of sham condition at issue, which sham conditions optimally minimize the risk of harm to subjects without sacrificing scientific value?
We surveyed experts in PD clinical research regarding their views about the science and ethics of an efficacy study testing a hypothetical novel gene-transfer technique with promising phase 1 results.
METHODS
PARTICIPANTS
We surveyed the voting investigator members of the Parkinson Study Group (PSG), a nonprofit cooperative group of PD experts in the United States and Canada. The PSG consists of investigators who are employed or otherwise engaged by academic or research institutions. The PSG currently has approximately 85 sites, including more than 350 active investigators, coordinators, and scientists. At the time of the survey, there were 168 voting investigator members.
SURVEY INSTRUMENT
The survey was conducted over the Internet and can be reviewed online at http://www.supersurvey.com/customers/mbh/rochester/. The hypothetical scenario consisted of 2 main topics. First, the respondents were asked to suppose that, following preliminary rodent and primate studies, a successful phase 1 study of a novel gene-transfer therapy for PD had been conducted on 8 subjects. They were told that the experiment appeared to reduce a subject’s awake time spent in the "off" state, "as well as reducing time spent with troubling dyskinesias. There were no significant short-term (6-month) negative effects." Second, they were asked to compare 2 potential randomized control trial designs for a follow-up study involving 50 subjects, the goal of which was to test the efficacy and safety of the gene-transfer therapy. The sham-control study design would compare gene-transfer surgery plus optimal pharmacological treatment vs sham surgery plus optimal pharmacological treatment (blinded study), in which sham surgery consisted of drilling 2 full-thickness burr holes. The open-control study would compare gene-transfer surgery plus optimal pharmacological treatment vs optimal pharmacological treatment alone (unblinded study). For the sham-control study, the respondents were told that if the gene-transfer surgery turned out to be safe and effective, "the participants who undergo sham surgery will be offered the gene transfer procedure using the predrilled holes."
The survey questions are presented in Table 1, Table 2, and Table 3. Except for the questions regarding quantitative estimates of efficacy and false-positive rates, the respondents were invited to explain their answers in open-ended format. An earlier version of the survey was pretested at 2 scientific meetings.
|
|
|
|
Table 1. Scientific Questions About Appropriate Control Condition
|
|
|
|
|
|
|
Table 2. Permissibility of Open and Sham Controls
|
|
|
|
|
|
|
Table 3. Are Various Invasive Options for Sham Condition Justified?
|
|
|
SURVEY PROCEDURE
The survey was conducted from April 5, 2004, through April 26, 2004. The e-mail list obtained from the PSG contained more than the investigator voting members, but it was decided a priori that the primary analysis (presented in this article) would include the investigator voting members only. The respondents were sent 2 additional reminder e-mails. The e-mails contained an embedded link to the survey and a unique password. We used an online survey company (SuperSurvey; Tercent, Inc, Lake Oswego, Ore). The survey took 10 to 15 minutes to complete.
ANALYSIS
We tabulated frequency data for categorical answers and means for quantitative answers. We analyzed the qualitative data from the open comments as follows. We (S.K. and R.W.) read through all the comments together and devised a coding scheme that we (R.H. and S.F.) later modified. Three of us independently coded each comment; the final code was assigned when at least 2 of 3 of us agreed. The few that did not get agreement at least from 2 of 3 of us were resolved by joint discussion. Some respondents made lengthy comments that required more than 1 code per comment.
HUMAN SUBJECTS
The University of Michigan (Ann Arbor) and the University of Rochester (Rochester, NY) reviewed the protocol and deemed this survey exempt from institutional review board review.
RESULTS
Of the 168 eligible individuals, 103 responded to the survey, for a 61.3% overall response rate. Not everyone answered every question, but no question had a response rate lower than 55%. Mean ± SD age was 48 ± 8.3 years, and 77% were men. Most respondents made comments, for a total 439 comments; 2 or better agreement among 3 coding judges occurred in 91% of the comments.
Table 1 summarizes the responses to scientific questions surrounding the use of sham surgery as compared with open controls. The scientific preference for sham-surgery controls was nearly unanimous (91/94 or 96.8%). The most common comments were that a sham-control design better controls for placebo effect (57/79 or 72% of comments mentioned this), that a sham-control design has less potential for bias (13/79 or 16%), and that results of an efficacy study would not be definitive without a sham control (9/79 or 11%).
Almost every respondent (96.8%) felt that an open-control study risked greater likelihood of falsely concluding that an ineffective intervention was effective. The respondents felt that an open-control study would involve, on average, a 33.0% false-positive rate. Also, these investigators felt that based on the hypothetical vignette of the phase 1 study results, there was a 40.3% chance that the gene-transfer intervention would in fact turn out to be "truly effective."
Table 2 summarizes the responses regarding the ethical permissibility of the sham-control trial and of the open-control trial. One half of the respondents answered that an open-control design should probably not or definitely not be allowed. Of those who held this view and made comments, 24 (61%) of 39 respondents were concerned about either false-positive results (33%, 13/39) or inconclusive results (28%, 11/39), and 10 (26%) of 39 respondents felt that despite the lower risk profile of an open-control study, the risks of an open-control design were not justified because of the decrease in scientific benefits.
As for the sham-control study, there was strong (93.5%) support for its permissibility. Various reasons were given in the comments: it is the only way to determine true efficacy (13/67, 19%), general statements to the effect that it is a better design (15/67, 22%), risks to subjects are outweighed by benefits to society and science (12/67, 18%), and the risk-benefit ratio for subjects is acceptable (7/67, 10%). Another 8 (12%) of 67 comments reflected a conditional approval: they would be in favor of allowing sham-control design but only after further preliminary testing.
Different types of sham surgery have varying degrees of invasiveness.1-4 What do the respondents believe about whether various types of sham surgery are justified by the potential benefits to science and society? The responses split along the line of whether or not the sham condition would involve penetration of the brain matter (Table 3). About 90% of the respondents felt the risks of burr holes without penetration of the dura were justifiable. A minority (less than 22%) of respondents believe that the risks associated with sham conditions that involve brain penetration are justifiable. These 2 groups differed in their open-ended comments. Those who are reluctant to endorse sham surgeries beyond burr holes were mainly concerned about placebo effects due to expectancy effects in the patient and bias in the clinical raters. Nearly all comments (53/55, 96%) made by those who do not favor sham conditions that penetrate the brain matter reflected this point: "The primary goal is to guard against placebo effects, investigator bias, and similar sources of bias. Insertion of a probe and related manipulations, which would considerably raise the chance of a serious complication (hemorrhage), are really controls for nonspecific effects of surgery. Preclinical work should be sufficient to defuse these concerns." However, those in support of more invasive sham conditions felt that nonspecific clinical effects of brain manipulation should be controlled for as well. Twelve (57%) of 21 comments made by those in favor of going beyond burr holes reflected this position, eg, "Any of these [sham procedures] might be scientifically justified if the appropriate question were seriously raised (eg, does empty viral vector affect symptoms of PD?)."
COMMENT
A vast majority of PD investigators believe that a sham-control design is scientifically superior to an open-control design in efficacy trials of gene transfer for PD. A similarly large majority believe that the sham-surgery control condition is ethically permissible while half of the respondents question the ethics of an open-control trial. The respondents showed considerable concern about false-positive results and seemed to agree that the sham-control condition is needed to definitively show that the intervention is effective.
These sentiments may represent a shift from a decade ago14 that is at least in part due to the recent history of placebo-controlled surgical trials in PD: clinical experience and early open trials seemed to indicate benefit, only to be proven ineffective and even possibly harmful when tested in sham-surgery controlled trials.1-4 Prior to these negative studies, cell-transplant procedures as treatments for PD had been offered at many centers throughout the world.10
There is disagreement about the degree of invasiveness of the sham condition that is justifiable. Most seem to believe that controlling for placebo effects and potential bias is scientifically sufficient. One attractive feature of the sham-control condition of a recent cell transplant trial was the minimization of risk by not penetrating the dura.10 Since then, however, a placebo condition involving penetration into the brain (indwelling intraventricular catheter with periodic placebo injections) has been used.4 The question of degree of invasiveness of the sham condition should receive more attention and discussion.
What were the underlying quantitative assumptions behind the respondents’ views? Despite being told that a preceding phase 1 trial had been successful, the respondents’ estimate of the likelihood that the gene transfer described in the survey was "truly effective" was 40%. This figure is comparable with data reported by the Food and Drug Administration regarding drug development trials and does not seem unduly optimistic.15 Our respondents would view a positive open-control study to have a one-third chance of being a false-positive study. Given that the usual randomized clinical trial is powered to tolerate a 5% chance of falsely concluding that an experimental intervention is effective, the degree of skepticism about open-control efficacy trials is notable.
This survey has limitations. The PSG is predominantly US-based, and the results may not reflect the views of clinical researchers in the rest of the world.14 We also did not investigate whether the researchers’ responses were related to their own experience with conducting sham surgical trials.
Although scientists may have special insight into the scientific design and the degree of risk that may exist in sham surgical trials, the value-based policy judgments about the degree of subject and societal benefit needed to outweigh the research risks to subjects involves multiple stakeholders. In the future, it will be important to compare the perspective of the scientists with the perspectives of potential research subjects (as special stakeholders in the debate) and the general public (representing a broader societal perspective) to aid our society’s attempt to reach an informed ethical consensus on this important issue.
AUTHOR INFORMATION
Correspondence: Scott Y. H. Kim, MD, PhD, Bioethics Program, 300 North Ingalls St, 7C27, Ann Arbor, MI 48109 (scottkim{at}umich.edu).
Accepted for Publication: February 4, 2005.
Author Contributions: Study concept and design: Kim, Frank, Holloway, Zimmerman, and Kieburtz. Acquisition of data: Frank, Wilson, and Kieburtz. Analysis and interpretation of data: Kim, Frank, Holloway, Wilson, and Kieburtz. Drafting of the manuscript: Kim, Frank, Holloway, and Wilson. Critical revision of the manuscript for important intellectual content: Kim, Frank, Holloway, Zimmerman, Wilson, and Kieburtz. Obtained funding: Kim, Holloway, and Kieburtz. Administrative, technical, and material support: Frank, Zimmerman, Wilson, and Kieburtz. Study supervision: Kim and Holloway.
Funding/Support: This study was supported as a subproject of U54-NS45309 from the National Institutes of Health, Bethesda, Md (Howard Federoff, Principal Investigator). Dr Kim is also supported by grant MH64172 from the National Institutes of Health.
Acknowledgment: We thank the members of the Parkinson Study Group who generously participated in this survey.
Author Affiliations: Department of Psychiatry, Bioethics Program, Center for Behavioral and Decision Sciences in Medicine, University of Michigan, Ann Arbor (Dr Kim); Department of Neurology, Boston University, Boston, Mass (Dr Frank); Department of Neurology, University of Rochester, Rochester, NY (Drs Kieburtz and Holloway, Ms Wilson, and Ms Zimmerman).
REFERENCES
| |
1. Freed C, Greene P, Breeze R, et al. Transplantation of embryonic dopamine neurons for severe Parkinson's disease. N Engl J Med. 2001;344:710-719.
FREE FULL TEXT
2. Freeman TB, Watts R, Hauser RA, et al. A prospective, randomized, double-blind, surgical placebo-controlled trial of intrastriatal transplantation of fetal porcine ventral mesencephalic tissue (Neurcell-PD) in subjects with Parkinson's disease [abstract]. Exp Neurol. 2002;175:426.
3. Olanow C, Goetz CG, Kordower J, et al. A double-blind controlled trial of bilateral fetal nigral transplantation in Parkinson's disease. Ann Neurol. 2003;54:403-413.
FULL TEXT
|
ISI
| PUBMED
4. Nutt JGM, Burchiel K, Comella C, et al, ICV GDNF Study Group. Randomized, double-blind trial of glial cell line-derived neurotrophic factor (GDNF) in PD. Neurology. 2003;60:69-73.
FREE FULL TEXT
5. Macklin R. The ethical problems with sham surgery in clinical research. N Engl J Med. 1999;341:992-996.
FREE FULL TEXT
6. Marshall E. Gene therapy on trial. Science. 2000;288:951-957.
FREE FULL TEXT
7. Grady D, Kolata G. Gene therapy used to treat patients with Parkinson's. New York Times. August 19, 2003; Available at: http://www.nytimes.com. Accessed July 12, 2005.
8. McRae C, Cherin E, Yamazaki TG, et al. Effects of perceived treatment on quality of life and medical outcomes in a double-blind placebo surgery trial. Arch Gen Psychiatry. 2004;61:412-420.
FREE FULL TEXT
9. Goetz CG, Janko K, Blasucci L, Jaglin J. Impact of placebo assignment in clinical trials of Parkinson's disease. Mov Disord. 2003;18:1146-1149.
FULL TEXT
|
ISI
| PUBMED
10. Freeman T, Vawter D, Leaverton P, et al. Use of placebo surgery in controlled trials of a cellular-based therapy for Parkinson's disease. N Engl J Med. 1999;341:988-992.
FREE FULL TEXT
11. Albin RL. Sham surgery controls: intracerebral grafting of fetal tissue for Parkinson's disease and proposed criteria for use of sham surgery controls. J Med Ethics. 2002;28:322-325.
FREE FULL TEXT
12. Miller FG. Sham surgery: an ethical analysis. Am J Bioeth. 2003;3:41-48.
ISI
| PUBMED
13. Boer GJ, Widner H. Clinical neurotransplantation: core assessment protocol rather than sham surgery as control. Brain Res Bull. 2002;58:547-553.
FULL TEXT
|
ISI
| PUBMED
14. Cohen J. New fight over fetal tissue grafts. Science. 1994;263:600-601.
FREE FULL TEXT
15. Food and Drug Administration Center for Drug Evaluation and Research. From Test Tube to Patient: Improving Health Through Human Drugs. Rockville, Md: Food and Drug Administration; 1999.
RELATED ARTICLE
Double-Blind, Placebo-Controlled Trials for Surgical Interventions in Parkinson Disease
C. Warren Olanow
Arch Neurol. 2005;62(9):1343-1344.
EXTRACT
| FULL TEXT
THIS ARTICLE HAS BEEN CITED BY OTHER ARTICLES
Long-term fetal cell transplant in Huntington disease: Stayin' alive
Frank and Biglan
Neurology 2007;68:2055-2056.
FULL TEXT
Studying neurosurgical implants for Parkinson disease: A question of design
Prehn et al.
Neurology 2006;67:1503-1505.
ABSTRACT
| FULL TEXT
A randomized trial of deep-brain stimulation for Parkinson's disease.
Deuschl et al.
NEJM 2006;355:896-908.
ABSTRACT
| FULL TEXT
What is the risk of sham surgery in Parkinson disease clinical trials? A review of published reports
Landau et al.
Neurology 2006;66:1788-1789.
FULL TEXT
Double-Blind, Placebo-Controlled Trials for Surgical Interventions in Parkinson Disease
Olanow
Arch Neurol 2005;62:1343-1344.
FULL TEXT
|
