Create and Revise Program Materials - Intervention Research

63 Sarah Zlotnik, MSW, MPH, is a co-author of Chapter 4 .

Typically, manuals specify program objectives and activities. In some programs (e.g., the Casey Family Program described in Chapter 1 ), the selection of activities is guided by needs or risk assessment that is used in matching content to program participants. In other programs, a single intervention is implemented. In many manuals, activities and other program content are speciﬁ ed in sessions or lessons, which may include scripted discussion, demonstration, learning exercises, or role-plays.

Session materials often contain illustrated handouts to be used in appli-cation drills or homework assignments that can be completed between sessions. Some manuals include process tips such as suggested ways for handling interpersonal conﬂ ict in group-based interventions. In addi-tion, manuals often contain content on providing services in alternative settings (e.g., schools, after-school programs, neighborhood centers, hospitals, or community clinics), including decision rules to guide the application of content in different kinds of settings.

Step 2 of intervention research is wholly concerned with the creation and revision of program materials, such as treatment manuals. Although manuals are reﬁ ned across Steps 3, 4, and 5 of the intervention research process, the bulk of the work in manual development occurs in Step 2. In this chapter, we describe the process of developing intervention manuals, and discuss issues involved in the use of written materials in practice.

Although we describe the process of manual development as a sequence of activities, manual development is iterative and recursive. Develop-ment does not proceed in a steady progression to a ﬁ nal product. It often involves reconceptualization and rewriting. Sometimes, the end product bears little resemblance to the initial drafts.

Variation in Practice Manuals

Typically, manuals are characterized as guides that spell out a program theory and practice content; however, manuals vary signiﬁ cantly in content and length. Some manuals focus on principles or beliefs related to speciﬁ c models of practice. These kinds of principle-driven interventions tend to leave the content and sequencing of intervention activities to the practi-tioner. Some manuals are barely more than compilations of suggested

activities and only offer lists of resources to be used as need arises. These manuals lack prescriptive clarity and a problem focus. A key feature of a good treatment manual is—in our view—a detailed description of core practice activities and a prescribed course of action (we note exceptions in Chapter 5 where principle-driven manuals, like Multisystemic Treatment, are coupled with extensive training and supervision). Written materials that lack this kind of detailed description are best classiﬁ ed as guides or resources rather than program manuals. Resource guides do not clearly deﬁ ne a program, and thus they leave us vulnerable to the black box conundrum.

To be sure, even detailed manuals differ in the extent to which they are prescriptive. They vary concerning ﬂ exibility of implementation (i.e., the degree to which practitioners are encouraged to adapt content);

speciﬁ cation of a program theory (e.g., the amount of text allocated to discussion of mediators, logic models, and theories of change); descrip-tion of techniques (e.g., presentadescrip-tion of sample dialogue that may be used by practitioners); and provision of implementation guidelines (e.g., description of strategies to enhance attendance, decision rules for excluding disruptive clients). Furthermore, manuals differ in terms of the relative importance placed on literature reviews. Some manuals jump quickly to practice objectives and activities, whereas others contain extensive theoretical and conceptual content.

Elements of Treatment Manuals

Both researchers and practitioners broadly and imprecisely use the term manual . Manual is sometimes interchanged with other terms that more accurately describe practice tools. For example, the term curriculum is frequently used to refer to manuals in which practice activities are psy-choeducational in nature and involve didactic processes. A manual may be described as a series of practice protocols (standardized procedure guidelines for a speciﬁ c area of practice), which enumerate steps to reach particular goals, whether general or speciﬁ c. In addition, manuals are sometimes described as practice guidelines ; however, our understanding is that practice guidelines are more general decision-making tools based

on research evidence and expert practitioner consensus. Practice guide-lines aid in the selection of interventions appropriate for a target popula-tion and a targeted outcome (Howard and Jenson 1999 ; Proctor and Rosen 2003 ). In the literature on evidence-based practice and intervention research, there is no widely accepted deﬁ nition of the term manual . Thus, for the purposes of this text, we deﬁ ne manuals as guides to practice that describe a problem, a program theory, practice objectives, and program content .

History of Manualized Interventions

Intervention manuals originally developed as research tools to counter-act the black box problem, and they gradually seeped into prcounter-actice, particularly cognitive-behavioral practice (Addis 1997 ). Growing out of behavioral and cognitive therapy research, the trend toward manual-based practice emerged in the late 1960s (Luborksy and DeRubeis 1984 ).

An early proponent of manuals, Joseph Wolpe (1969) developed some of the ﬁ rst manualized interventions as part of his work on anxiety-related disorders.

In part, the trend favoring manuals was a response to controversial ﬁ ndings that emerged in the 1950s and 1960s about the ineffectiveness of psychosocial interventions, such as those represented in the Cambridge-Somerville Youth Project. By the late 1970s, research studies had begun to show that therapy was often better than no treatment (Luborsky, Singer, and Luborsky 1975 ), but intervention processes and outcomes were poorly measured. Too often, interventions were only vaguely described using terms such as casework, in-home treatment, or struc-tural family therapy. This lack of speciﬁ city regarding clinical techniques frustrated both researchers and practitioners. The dearth of detailed information was especially frustrating when research ﬁ ndings were positive and there was interest in using programs in community agencies. As a result, researchers began to focus on more clearly delineating therapeutic strategies (Addis 1997 ).

Efforts grew to specify the components of interventions and to demonstrate the efﬁ cacy of speciﬁ c treatment modalities for clinical

problems (e.g., Beck, Rush, Shaw, and Emery 1979 ). The development of manuals—and indeed intervention research—was fueled by the perspec-tive that a crucial task of practice research was to describe “what works for whom.” In addition, other forces have inﬂ uenced the development of manualized interventions. In particular, legislative reforms have pressed third-party insurers (i.e., programs or organizations that provide reim-bursements for health care) to require practitioners to provide interven-tions with strong evidence bases. In placing a premium on replication of

“best practices,” these reforms have accelerated the use of manuals.

These developments notwithstanding, the increasing use of manuals in practice has been the subject of considerable debate. Those who favor manual-based intervention cite beneﬁ ts such as the ability of manuals to help transfer acquired knowledge (Galinsky, Terzian, and Fraser 2006 ).

Proponents argue that manuals increase the quality of services by making it easier to replicate evidence-based services, and they cite manuals as a key vehicle in disseminating best practices (Chambless and Hollon 1998 ).

From this perspective, manuals also contribute to clinical training and supervision, and they facilitate greater consistency in the delivery of services across practitioners with different educational backgrounds (Dobson and Hamilton 2002 ). In addition, because manuals clarify intervention processes, manuals also strengthen inferences about the outcomes of services (Wilson 1996 ). Furthermore, manuals increase accountability because they make it possible to monitor the extent to which an implemented intervention is congruous with the written pro-gram materials (Luborsky and DeRubeis 1984 ).

Conversely, criticism of manualized interventions abounds. Generally, objections to manual-based interventions arise from concerns about the complexities of practice, the need for ever-responsive clinical adapta-tions, and an overall perspective that manuals discount practice experi-ence. Speciﬁ cally, some critics have argued that manuals try to reduce to a prescribed routine what is essentially an art form (Addis, Wade, and Hatgis 1999 ). Outside the cognitive behavioral ﬁ eld, practitioners have given manuals a lukewarm reception (Addis and Krasnow 2000 ; Kendall 1998 ). Those opposed to manuals hold that the multidimensionality of everyday living situations, organizational processes, and community

inﬂ uences produce complexities that defy manualized treatment (Fonagy 1999 ). Indeed, the problems confronted by practitioners are often cited as more challenging than those confronted by researchers who test manualized interventions (Abrahamson 1999 ; Foxhall 2000 ).

For example, practitioners must deal with all clients, whereas researchers often establish sampling criteria to screen out challenging cases, such as those with high comorbidity, those who failed previous interventions, and those with compromised social or environmental supports (Chorpita 2002 ; Luborsky 1999 ). Critics note also that manuals can lead to a “cook-book approach,” which yields a mechanistic and myopic intervention that devalues practice wisdom and precludes the use of a dynamically changing intervention that is therapeutically reactive to clients’ needs (Garﬁ eld 1996 ; Wilson 1996 ). In addition, opponents of manuals con-tend that the use of manualized interventions is time-consuming and demands extensive training and ongoing supervision (Najavits, Weiss, Shaw, and Dierberger 2000 ).

In response to these criticisms, advocates of treatment manuals have acknowledged that using a manualized approach with clients who have multiple problems presents a challenge, but such challenges can be accommodated by most interventions and are not insurmountable (Carroll and Nuro 2002 ). Well-tested and carefully designed manuals of-ten provide guidelines for varied intervention activities, for the use of adjunctive interventions, and for adaptations determined by client needs (see, e.g., DePanﬁ lis and Dubowitz 2005 ). Moreover, proponents of manuals have counterargued that if the use of manuals is time-consuming or requires additional training, this may be part of the cost of improving practice outcomes. Advocates have supported this point by noting that most manuals are developed, tested, and reﬁ ned as part of the process in intervention research. Therefore, manual-based interventions that have produced positive effects in research should take time to master because they usually provide a template for doing practice in a different way. Changing practice almost always involves learning new skills, and the investment of time is worthwhile if it improves outcomes.

In our perspective, the crux of this debate is the premium placed on intervention research in which program materials are developed and

evaluated systematically. Part of this systematic development includes changes that are made based on data collected during four stages of manual development that are embedded within the steps of intervention research. Described below, each stage of manual development serves a different development purpose, ranging from the initial creation of pro-gram materials to the adaptation of materials in different settings. From start to ﬁ nish in intervention research, manuals are modiﬁ ed based on feedback and critical review—ﬁ rst during formulation, then in revision during pilot tests, next in reﬁ nement during efﬁ cacy and effectiveness tests, and ﬁ nally in translation and adaptation for other cultures (e.g., when manuals are extrapolated to new populations). Embedded within intervention research is a design process that involves constant ﬁ ne-tuning of manuals to improve their ﬁ t with current practice and environ-mental exigencies.

With that said, it is worth noting that it is only when manuals are developed systematically that we can argue with conﬁ dence that manual-based interventions improve practice. Not all manuals are manual-based on research. In our view, manual development must be conjoined with research whenever possible, and integrated into a process that involves conﬁ rming and reﬁ ning program components based on the data. When developed in this way, manuals prescribe practice innovations that are likely to improve outcomes.

Stages in Development of Program Manuals and Materials

Intervention research is characterized by interplay between generative processes—used in creating program materials, and evaluative processes—

used in estimating the impact of program materials. As noted earlier, interpretative and creative processes are involved in transforming program theory into intervention objectives and content. Often innovative, these processes yield the design of an intervention, including practice activities, materials used for screening and recruitment, and training protocols. In contrast, evaluative processes, which are rooted in the critical traditions of science, provide information on the extent to which programs do what

they are intended to do. In intervention research, program formulation and program evaluation are interwoven. The two interact to produce a program of known dimensions and with known outcomes.

Although many conceptualizations of program evaluation can be found (e.g., Rossi et al . 2003 ), few include the development and reﬁ ne-ment of the intervention itself. The inclusion of program developne-ment is a central feature of intervention research. Figure 4.1 shows the four stages of development of program materials that stretch across the ﬁ ve steps of intervention research. These four stages are: (1) formulation , (2) revision , (3) differentiation , and (4) translation and adaptation .

To be sure, the development of program materials can be conceptu-alized in a variety of ways. In Chapter 2 , we described Carroll and Nuro’s (2002) three-phase model: (1) developing and testing a ﬁ rst draft, (2) adding content to guide implementation, and (3) reﬁ ning content for alternative settings. Based on our work and recent advances in transla-tional research, we now propose four stages that further elaborate activi-ties ranging from the initial design of an intervention to its extension to new settings and populations. Because it is literally impossible to test

Figure 4.1 Four stages in the development of program materials integrated across the ﬁ ve steps in intervention research.

Stage 4

interventions on every population, we must assume that evidence-based interventions will be used in cultures and settings where they have not been tested. That is, interventions will be extrapolated to populations that appear similar to those in which programs were developed but, nonetheless, for which there are no data regarding program effectiveness.

When programs are extrapolated, the essential features of a program are usually preserved. However, at the same time, program content must be translated and adapted to have cultural congruence with the new popula-tion. As is demonstrated in Chapter 6 , these translational research processes are being given increased attention. In part, evidence-based practice is rooted in the notion that these processes of translation and adaptation will maintain the features of interventions that make them effective while tailoring program content for cultural relevance. This is a tall order, and we address the challenge by proposing four stages in the design and development of program materials.

Each of the four stages is deﬁ ned by a set of activities that leads to a new set of activities. Shown in Figure 4.1 , each stage is integrated with the ﬁ ve steps of intervention research. Although the bulk of program formu-lation occurs in Step 2 of intervention research (i.e., Create and Revise Program Materials), program objectives and content derive from pro-gram theory that is developed as the ﬁ rst step of intervention research.

The double-sided arrow for formulation stretches across Steps 1 and 2 to indicate that the foundations for manuals come from the identiﬁ cation of malleable mediators in a program theory. Other stages in the develop-ment of program materials are linked to evaluative processes in interven-tion research. For example, over time and based on data from pilot studies, efﬁ cacy trials, and larger effectiveness trials, program materials are reﬁ ned and then differentiated for various settings and populations.

In the sections that follow, the core aspects of the process of developing program materials are described for each stage. The four stages focus exclusively on the development of manuals and other program materials.

In contrast, as shown in Figure 2.1, the ﬁ ve steps of intervention research include program design and evaluation processes. Although this chapter focuses on Step 2 of intervention research, it elaborates on the develop-ment of program materials by highlighting formulation, revision,

differentiation, and translation/adaptation activities that occur across all steps of intervention research.

Stage 1: Formulation of Program Manuals and Materials

Stage 1 in the development of treatment manuals and materials draws on the reading and the research that has been done to Specify the Problem and Develop Program Theory (Step 1 of intervention research). Outlined in Table 4.1 , the formulation of program manuals progresses from Description of Problem, to a Program Rationale, to the Program Theory, to a Program Format, and ﬁ nally to Session Content.

The formulation of a program is founded on a clear speciﬁ cation of a social or health problem, a rationale for intervention, and theory for program development. The latter includes breaking down the problem to identify its context, the factors that give rise to the problem (i.e., risk factors), the factors that suppress the problem (i.e., protective factors), and relevant theories or perspectives that may help to explain the prob-lem (see, e.g., DePanﬁ lis and Dubowitz 2005 ). These elements are summarized in logic models and theories of change. Taken together, they provide a rationale for a new intervention.

Format of Manual

However, understanding a problem and having a program theory are not enough. They are building blocks. In Stage 1 of manual development, the researcher must select a format for the delivery of the intervention. This format selection involves deciding on intervention content, logically ordering the content, and integrating the content with a delivery mecha-nism, such as provision by a worker in face-to-face meetings, provision via the Internet in self-paced learning modules, or provision by a class-room teacher as an integrated aspect of a school curriculum. Clearly linked to mediators (i.e., factors that are targeted to bring about change), session or unit content must be developed and sequenced. For some interven-tions, the researcher will also develop between-session content. This can involve creating homework assignments, application exercises, or interac-tive projects (e.g., prescribed family outings or discussions). Finally, when

Section Content Areas Considerations to increase? How strong are the data?

• Do rates vary by race/ethnicity? By gender? By income?

By rural/urban? By other factors?

• Does the public consider the problem to be important? on providers, e.g., agency policies or practice standards?

• Is the change strategy feasible in the current sociopolitical environment and in real-world practice? Can it work?

• What is innovative about the program theory (e.g., targets newly identiﬁ ed mediator, employs new delivery mechanism)?

(Continued)

Section Content Areas Considerations

(e.g., review of previous or current session content, review of homework, sharing) • Nature of between-session activities • Guidelines for delivering the intervention,

e.g., integration with practice standards, funding mechanisms, best practices • Incentives for participation in activities or

attendance

• Provision of environmental supports

(e.g., provision of meals, child care, transporta-tion to enhance participatransporta-tion and attendance)

• Is the intervention targeted toward the individual, family, link to program theory

• Content and activities for each session

developing a format and content, the researcher must consider the compatibility with the expected venue in which the program will be implemented. This process involves integrating content with practice standards, agency policies, funding strategies, and other contextual factors that are likely to inﬂ uence the delivery and, ultimately, the adoption of an intervention. In this sense, we begin to consider dissemination (Step 5 in intervention research) early in the process of program formulation.

Let’s take the Making Choices intervention as an example. Making Choices, a primary prevention intervention for elementary-school chil-dren, is based on social information processing (SIP) theory (Fraser et al . 2000 ). The Making Choices program is intended to reduce antisocial and aggressive behavior by strengthening children’s social skills. As noted earlier, the key mediator is thought to be limited social problem solving skills. SIP theory provides a framework for conceptualizing a social skills

在文檔中 Intervention Research (頁 76-118)