Issue Navigator

Volume 07 No. 05
Earn CME
Accepted Papers

Special Articles

Revisiting Evidence-Based Guidelines: Not Such a Nightmare

R. Nisha Aurora, M.D.1; Rochelle S. Zak, M.D.2; Sanford H. Auerbach, M.D.3; Kenneth R. Casey, M.D.4; Susmita Chowdhuri, M.D.5; Anoop Karippot, M.D.6; Rama K. Maganti, M.D.7; Kannan Ramar, M.D.8; David A. Kristo, M.D.9; Sabin R. Bista, M.D.10; Carin I. Lamm, M.D.11; Timothy I. Morgenthaler, M.D.8
1Johns Hopkins University, Department of Medicine, Baltimore, MD; 2Sleep Disorders Center, University of California, San Francisco, San Francisco, CA; 3Boston University School of Medicine, Boston, MA; 4Cincinnati Veterans Affairs Medical Center, Cincinnati, OH; 5Sleep Medicine Section, John D. Dingell VA Medical Center, Detroit, MI; 6Penn State University Hershey Medical Center, Hershey, PA and University of Louisville School of Medicine, Louisville, KY; 7Barrow Neurological Institute at Saint Joseph's, Phoenix, AZ; 8Mayo Clinic, Rochester, MN; 9University of Pittsburgh, Pittsburgh, PA; 10University of Nebraska Medical Center, Omaha, NE; 11Children's Hospital of NY – Presbyterian, Columbia University Medical Center, New York, NY

Dr. Cranston and colleagues should be commended for the thorough job they did expanding upon the foundation of the article, “Best Practice Guide for the Treatment of Nightmare Disorder in Adults.”1 The authors have 3 criticisms of the article: (1) the methodology was used inconsistently at updated time points; (2) the article was missing references that were within the search criteria and therefore not reported; and (3) the search database used was insufficient. In the following sections, these concerns are addressed.

Our initial search was conducted in December 2007 and then updated in March 2009 to include all articles indexed and available at that time. Since the cut-off date was March 2009, a number of articles that Cranston and colleagues found missing from our paper were not included. In response to one of our external reviewer's comments, another electronic search was conducted in February 2010 using the keywords “anxiety dreams.” The results yielded 19 articles, which were manually refined to evaluate all inclusion criteria and limits, including discussion of a treatment or therapy. However, there were no additional papers that met all of the inclusion criteria. Thus, the search was in fact conducted consistently at updated time points.

Table 1 lists the articles cited by Cranston et al., along with the reasons for not including them in our review. One article was in our database but inadvertently omitted from the analysis (Davis and Wright 20072); Krakow et al. (19953) was superseded by another publication by the same research group on the same subjects (Krakow et al. 19964), so it was discussed but not included in the tally of supporting evidence; two did not have keywords used in our search criteria (Thompson et al. 2008,5 Aukst-Margetic et al. 20046); four were published after the cut-off date of March 2009 (Fraser 2009,7 Lu et al. 2009,8 Harb et al. 2009,9 Swanson et al. 200910); one was excluded because it was a letter to the editor (Moore and Krakow 200711); and three (Fraser 2009,7 Harb et al. 2009,9 Ginsberg 200312) were not in PubMed (of which two (Fraser 2009,7 Harb et al. 20099) were published after the cut-off date).

ArticleTreatmentLevelReason for Exclusion
Thompson 2008Prazosin4Not found using the keyword search
Ginsberg 2003Prazosin4Not in PubMed
Aukst-Margetic 2004Levomepromazine4Not found using the keyword search
Fraser 2009Nabilone4Not in PubMed; published after date cut-off
Krakow 1995IRT2Was cited in the paper as a level 2 study; it was not cited in the first paragraph as it was superseded by Krakow 1996, a level 3 paper but it was included in the discussion of the level 3 paper
Lu 2009IRT4Published after date cut-off
Moore 2007IRTNALetter to the Editor
Harb 2009IRT4Not in PubMed and published after the cut-off date
Davis 2007ERRT2Inadvertently excluded
Swanson 2009ERRT4Published after cut-off date

table icon
Table 1

(more ...)

TermLevelEvidence LevelsExplanation
Recommended / Not recommendedA1 or 2Assessment supported by a substantial amount of high quality (Level I or II) evidence and/or based on a consensus of clinical judgment
Suggested / Not SuggestedB1 or 2—few studies
3 or 4—many studies and expert consensus
Assessment supported by sparse high grade (Level I or II) data or a substantial amount of low-grade (Level III or IV) data and/or clinical consensus by the task force
May be considered / Probably should not be consideredC3 or 4Assessment supported by low grade data without the volume to recommend more highly and likely subject to revision with further studies

table icon
Table 2

(more ...)

Of the three articles from Table 1 that could have resulted in a recommendation change, only one was incorrectly omitted. Davis and Wright was in our evidence table, graded and extracted, but was inadvertently missed in the analysis (Davis and Wright 2007).2 We agree with the authors that it should have been included. This oversight on our part had the greatest effect, as the recommendation for Exposure, Relaxation, and Rescripting Therapy (ERRT) would likely have been increased from a Level C to a Level B. We regret this omission and believe that a correction should be issued to address this error.

The other two papers were not included in our final analysis for the following reasons. Aukst-Margetic et al. 20046 was not found using our keyword search. Had this paper been included, there may have been a Level C recommendation based on the use of levomepromazine. The other paper (Fraser 20097), which discusses nabilone, was published after the March 2009 cut-off date. This latter paper was not in PubMed, but would have been picked up by using PsycINFO, if the search had been performed at a later date.

There were 3 articles that were not in PubMed, but were picked up by other search databases. In terms of our paper's conclusions, however, one article would have been excluded because it was published after March 2009 (Harb et al. 20099), and it did not affect the level of recommendation; one would have been included (Ginsberg 200312) but did not affect the level of recommendation; and the other, as mentioned above, may have supported the use of nabilone, but would not have been included as it was published after the cut-off date (Fraser 20097). This information does not justify using only one database but merely points out that the fundamental conclusions of our paper would not have been substantially altered by including the additional database given the cut-off date. Nonetheless, we would have added a discussion of nabilone, had the cut-off been later and we would like to thank Cranston et al. for pointing this out.

In addition, Cranston et al., commented that we “clustered treatment evidence” for PTSD-associated and idiopathic nightmares. In each recommendation, we included the type of nightmare disorder studied and only used “nightmare disorder” in the instances in which the study populations were noted either to have both PTSD-associated and idiopathic nightmares or the type of nightmare disorder was not specified. We agree that the two disorders may be preferentially responsive to different types of treatment. We also recognize that at times it is clinically difficult to differentiate between the two.

Interestingly, Cranston et al. used the search methodology described in our paper and, with the exceptions noted above, formed the same evidence tables that we did. To some extent, this is perhaps remarkable when one considers that the final evidence tables contained only 4% of those returned by the computerized search, and that the two lists were 98% homologous. The AGREE instrument13 for appraising guidelines suggests that “The criteria for selecting the evidence are clearly described.” We seem to have hit the mark on this item. On the other hand, a recent task force commissioned by the Institute of Medicine recommended that when performing systematic reviews one should “access an array of information sources that provide both published and unpublished research reports.”14 In this regard, we agree that searching more databases is a desirable strategy that should be targeted. However, a balance must be struck between allocating additional resources and the gains attained with searching additional databases. With respect to this best practice paper, adding the additional database would not have changed our conclusions substantially.

In summary, the approach utilized by Cranston and colleagues would have resulted in an increase in the level of recommendation only for ERRT from a Level C to a Level B. Nonetheless, we appreciate the time, effort, and commitment devoted by the authors in replicating and expanding the search and analyses for “Best Practice Guide for the Treatment of Nightmare Disorder in Adults.” The excellent work by Cranston and colleagues gave us the opportunity to reevaluate our process and methods.


The Standards of Practice Committee members have indicated no financial conflicts of interest.



Aurora RN, Zak RS, Auerbach SA, et al., authors. Best practice guide for the treatment of nightmare disorder in adults. J Clin Sleep Med. 2010;6:389–401. [PubMed Central][PubMed]


Davis J, Wright D, authors. Randomized clinical trial for treatment of chronic nightmares in trauma-exposed adults. J Trauma Stress. 2007;20:123–33. [PubMed]


Krakow B, Kellner R, Pathak D, Lambert L, authors. Imagery reharsal treatment for chronic nightmares. Behav Res Ther. 1995;33:837–43. [PubMed]


Krakow B, Kellner R, Pathak D, Lambert L, authors. Long term reduction of nightmares with imagery rehearsal treatment. Behav Cogn Psychother. 1996;24:135–48


Thompson CE, Taylor FB, McFall ME, Barnes RF, Raskind MA, authors. Nonnightmare distressed awakenings in veterans wiht posttraumatic stress disorder: response to prazosin. J Trauma Stress. 2008;21:417–20. [PubMed]


Aukst-Margetic B, Margetic B, Tosic G, Billc-Prcic A, authors. Levomepromazine helps reduce sleep problems in patients with PTSD. Eur Psychiatry. 2004;19:235–6. [PubMed]


Fraser GA, author. The use of a synthetic cannabinoid in the management of treatment-resistant nightmares in posttraumatic stress disorder (PTSD). CNS Neurosci Ther. 2009;15:84–8. [PubMed]


Lu M, Wagner A, Van Male L, Whitehead A, Boehnlein J, authors. Imagery rehearsal therapy for posttraumatic nightmares in U.S. veterans. J Trauma Stress. 2009;22:236–9. [PubMed]


Harb GC, Cook JM, Gehrman PR, Gamble GM, Ross RJ, authors. Post-traumatic stress disorder nightmares and sleep disturbance in Iraq war veterans: a feasible and promising treatment combination. J Aggress Maltreat Trauma. 2009;18:516–31


Swanson LM, Favorite TK, Horin E, Amedt JT, authors. Imagery rehearsal therapy for posttraumatic nightmares in U.S. veterans. J Trauma Stress. 2009;22:236–9. [PubMed]


Moore BA, Krakow B, authors. Imagery rehearsal therapy for acute posttraumatic nightmares among combat soldiers in Iraq. Am J Psychiatry. 2007;164:683–4. [PubMed]


Ginsberg DL, author. Prazosin reduces nightmares in posttraumatic stress disorder. Prim Psychiatry. 2003;10:24


The AGREE Collaboration. Appraisal of Guidelines for Research – Evaluation (AGREE) Instrument. 2001.


IOM (Institute of Medicine). Finding What Works in Health Care: Standards for Systematic Reviews. 2011. Washington, DC: The National Academies Press;