A recent article by Sniderman and Furberg, “Why Guideline Making Requires Reform”  is a remarkably comprehensive over-view of the pitfalls of evidence based guidelines.  The article points out that guidelines are often mistaken for science when in fact their recommendations may be no more than “expert opinions”—which, of course, are subject to full spectrum of human failings, including bias, group think, and even conflicts-of-interests.   If this sounds like part of the problem with the IDSA guidelines, it should.  IDSA’s 2006 Lyme Guidelines contain 71 recommendations, of which 38 are based on no more than opinion.  

Some of the other highlights from the Sniderman article include:

Importantly, even when it is known that areas of legitimate controversy will be covered, there is often no attempt to ensure that all sides will have reasonable opportunity to present their evaluation of the evidence and participate in the decisionmaking process. . . .

On many, if not most, issues the evidence, no matter how extensive, remains incomplete. For example, even though the general conclusion that statins reduce the risk of vascular events is incontestable, many specifics remain debatable, such as how intensive therapy should be, when therapy should be started for different clinical situations, and which markers should be used to monitor therapy. Because gaps in the evidence are inevitable, they must be filled in with judgments, and judgments tend to preserve previous positions. Thus, what is to be decided is often already decided with the selection of the deciders. . . .

In summary, evidence is complex and incomplete. Therefore, when the evidence warrants, guidelines should respect diversity of views. Guidelines must be directed only to the interests of patients and not to those who profit from them. Failure to reform the guideline process risks replacing one authority-based system with another, whereas the core objective should be to strengthen an evidence-based approach to improve clinical care.

Sniderman suggests a number of reforms to the guideline development process, including:

Third, reports should not be issued unanimously unless all members fully agree to all sections.  Alternate interpretations and viewpoints should be recorded and issued along with the majority opinions. Fourth, posting an almost final version on the Internet and inviting commentary is an attractive model. This helps ensure that where legitimate differences of scientific opinion exist, there is an opportunity for exchange before final decisions are taken. Fifth, before publication, guidelines should undergo independent scientific review. The journal editor should present the criticisms and suggestions that result from the reviewers to the panel for its responses, and may require revision of the guideline document, as appropriate. The editor also should consider co-publishing alternate points of view as necessary.