an inTeRlink feature article

Efficacy research: An opportunity for reflection on what we do

By Kenneth E. Mobily, University of Iowa

Introduction

The purpose of this brief note is to reflect on the problem of efficacy research in therapeutic recreation (TR). I do not plan to enter into a dialogue about whether this or that intervention is "therapy", nor do I plan to enter into a discussion concerning whether recreation as an intervention is useful as a means or an end. Certainly, both of these topics are of considerable importance, have occupied a measure of my time in the past, and are apt to do more of the same in the future.

Instead, in the following I focus on the problems that plague TR efficacy research. I also suggest some directions researchers and practitioners may wish to pursue to learn more about the effects of their interventions, regardless of whether one calls such interventions therapy, leisure education, recreation participation, or something else altogether.

Efficacy research

The first problem facing efficacy research efforts in TR is that so often researchers fail to conceptualize their problem adequately. By this I mean that we gravitate toward variables that are theoretically vague. The result is empirically weak studies. And to the surprise and dismay of many, interventions based on poor conceptualization have a negligible effect on the end points of interest.

One obvious cause of this problem is to begin with a weak idea; everything that follows will then be corrupted by the flaw. Although emotionally safer to find harbor in the latest "pop" health fad or join the choir in singing the merits of one chic intervention after another, such concepts often lack the substance and specificity necessary for sound research. Pop variables tend to be global, all-inclusive, and offer simple answers to complex problems--this, of course, is their appeal. "If we all could only ____________ (fill in the blank), then everything would be 'OK' and we could demonstrate TR's efficacy." Most researchers know that good research is not that easy, especially when the wealth of TR research, by definition, must be field-based (see Coyle, Kinney, & Shank, 1993 for a review of the problems of field-based research in TR).

Sometimes pop variables are disguised and difficult to detect. For example, some derivations of health promotion may be considered pop variables. With apologies to the rigorous research in health promotion, some in the TR profession have latched on to health promotion and championed it as if it were some earth-shattering revelation instead of the "old wine in a new bottle" that it really is. TR has always been concerned with prevention of secondary disabilities, impairments, and complications associated with a primary disorder (e.g., prevention of decubiti that are the frequent sequelae of spinal cord injuries). TR has always tried to help clients live in as healthy a manner as possible (i.e., enhance quality of life). The result again is weak research designs and non-significant findings in efficacy research.

Sometimes poor conceptualization begins with a sound theory.

Learned helplessness is an apt example. For over 20 years, the TR profession has eulogized about, capitalized on, and re-conceptualized the theory of learned helplessness. But unless the research was carefully designed, the rationale behind the research amounts to claiming the "rain causes us to get wet"! I submit that when learned helplessness is invoked in a cavalier fashion, because it has been fashionable to do so, the conceptualization and the research that follows is likely to fail. If everything and anything causes learned helplessness, and practically anything can cure it--everything "gets wet". The architects (Peterson, Maier, & Seligman, 1993, p. 9) of the theory even issue a caution about the conceptual trap awaiting would-be researchers, "We believe that learned helplessness has been overused and applied promiscuously to situations that do not bear a convincing resemblance to pure cases1".

Furthermore, Peterson, Maier, & Seligman (1993) wisely acknowledge that learned helplessness is somewhat a product of the times. The era of personal control is characterized by an economy that caters to personal idiosyncrasy, by the shift in efficacy from social institutions to individuals, and by historically unprecedented prosperity in the United States. These social forces created an environment well-suited for a psychology of personal control. Control and helplessness are means for us to explain our world (and to conduct research), just as gods were used to explain the world by ancient cultures. But like any good idea, there are limits. Learned helplessness is not the final or complete answer to explain all aspects of human behavior. Neither is every intervention in TR an antidote for learned helplessness.

This does not mean that learned helplessness is useless anymore than it means that TR cannot be an effective intervention for it. What it does mean is that researchers have to be very focused and precise with their measures of learned helplessness, and have good reasons (theory) for believing that interventions X, Y, or Z might help address learned helplessness and/or augment perceived control.

The report of Shary and Iso-Ahola (1989) is an example of well designed research pertaining to learned helplessness in the TR literature. But the encouraging results of this study and a few others may have given rise to the indiscriminate use of learned helplessness and its progeny (e.g., perceived control, self-efficacy) by well-meaning but ill-prepared authors in the TR literature. For a time, it seemed as though every other TR research report and practically every theory in the TR literature had to pay its figurative respects to learned helplessness. Almost all research reports, theses, and dissertations seemed to add another intervention to the list of those that would be effective with learned helplessness and its sequelae. Such discourses and designs were more often than not ill-conceived and casual, rather than systematic and causal. And we got what we deserved, a litany of non-significant to marginally significant findings.

Worse yet are the initiatives into less productive theoretical and empirical areas, such as recent discussions of stress related research. The stress model is certainly a useful one, no doubt exists about that. Questions do, however, arise about the wisdom of making it the crux of TR efficacy research. Inspection of models of stress quickly reveals that everything effects everything else (see Russoniello, 1997, p. 472).

At a foundation level, recognition of the fact that physiological and psychological entities interact and affect one another is essential to understanding the dynamics and breath of interventions. Complete understanding of health and illness requires an appreciation of stress theory. But if the models are correct, then a change in one variable (i. e., a treatment) may affect many systems. The end result is a nonsignificant effect in an end-point variable, such as diminished secretion of stress hormones.

Another problem with global, theoretical models, such as stress, is that they rarely show a direct effect of TR interventions on outcome variables. So many variables correlated with TR intervention accompany these sorts of models that only associations can be shown. Correlations are valuable as a beginning, but in the long run are only the poor stepchildren to direct, cause-effect relationships.

A second and related problem for efficacy research is that some of the interventions used by TR tend to have diffuse effects. This means that when an intervention affects many systems the results are difficult to detect, and often barely discernible. The generalized effects of a treatment mask the specific change in the variable of interest, such as in depressive symptoms. If TR interventions are designed to address depressive symptoms, but also affect anxiety, self-esteem, self-concept, life satisfaction, and other mood states, then the effectiveness of the intervention is "spread out" across many different target variables. The active ingredient in some interventions is simply too general. At least this certainly seems to be the case with the stress model, on both theoretical and empirical fronts.

Instead, I suggest the use of more specific change agents and interventions. Some have already been tried, others have to be discovered or re-discovered.

1. Aquatic programs seem to be very effective in producing direct and significant changes in important functional variables. Beneficial changes in muscular strength, endurance, cardiovascular function, and joint mobility have been associated with water-based interventions. Note that the theoretical leap necessary is only a short hop rather than a heroic leap of faith across a yawning valley of confounding variables, co-morbid conditions, and co-linear relationships. The fact that when a person with a joint disease is immersed in water, allowed to "warm up", and move through a range of motion and against the resistance of the water itself makes the functional improvements that result less surprising. Moreover, the cause-effect relationship (between aquatic exercise and functional outcomes) is clear and convincing. The results have been convincing because the changes are theoretically linked to the intervention in a clear and direct manner.

2. Exercise has emerged as an intervention more often in TR programs of late. Findings are usually quite pronounced and rather unequivocal--mild to moderate recreational exercise has produced significant and beneficial changes in a host of functional variables. From reducing the risk of falls in older adults to slowing the rate of physical deterioration in patients with multiple sclerosis, exercise has shown itself to be a very powerful intervention.

And I think it is time to stop tip-toeing around PT and OT about the use of exercise as an intervention in TR. PT uses exercise prescriptions, usually for the short term and usually in a clinical situation. Notwithstanding, research has shown that many of the benefits of exercise are associated with chronic participation and compliance over a sustained period of time. Research has also indicated that even minimal exercise can produce dramatic effects in sedentary groups, such as most persons with disabilities and chronic conditions. Third, exercise intensity does not have to be risky for functional improvement to occur. Mild to moderate exercise has produced remarkable improvements in functional measures.

Clearly TR should not use exercise in the prescriptive manner of the PT. Neither should TR Specialists use exercise that is risky and intensive, e.g., heavy exercise with post-myocardial infarction patients. Customary precautions, such as securing physician clearance and only inducing exercise intensities that are mild to moderate come with this re-discovered territory for intervention. Let's face it, other therapies are not as interested in sustained exercise over the long term. For this kind of adherence to a program to occur, it has to be fun and intrinsically rewarding--it has to be recreational.

3. Instructional variables can be translated into TR interventions. Cueing clients in to relevant stimuli in the environment, using feedback about the correctness of an action or behavior, and making activity objectives clear are simple but time-honored techniques that have been proven effective in education. The same strategies should be effective in leisure education; except the learning activity is recreational, or a recreation-related skill, instead of seventh grade mathematics.

The problem has been that TR researchers often look for change in the wrong places, among variables less central to the point of leisure education (e.g., self-esteem). Instead, researchers should focus on the point of the educational experience as an outcome measure--the acquisition of knowledge, skills, and behaviors specific to the intervention, If the point of the instruction is to teach tennis skills, then one should expect some assessment of tennis skill as an outcome.

Hedrick's work (1985;1986) serves as an excellent example of focused and specific research on leisure education. Not only did he examine tennis skill acquisition, but also the perceptions of competence of the subjects with disabilities and their able-bodied coactors. The logic is compelling and succinct, did the subjects learn anything; did they think they learned it well; and did significant others think they learned it well?

Should we be surprised when leisure education does not produce a change in self-esteem, when the point of instruction is the acquisition of skill or knowledge? Settle instead for outcomes closer to home and more logically tethered to the "cause"--leisure education. Indeed, self-esteem may change as a result. But self-esteem is so global and so volatile that we should not be shocked when lessons based in leisure education produce no effect.

Furthermore, TR Specialists should not think that they have let someone down, or be disappointed if a skill is acquired in lieu of a monumental change in personality or affect. A colleague once remarked that clients take many years to develop their problems (e.g., low self-esteem), should we be surprised when a two week intervention fails to make a big difference?

4. I am not an expert in behaviorism, but positive reinforcements appear to be very effective in inducing favorable changes in clients. The work of Stuart Schleien, John Dattilo and their associates has clearly demonstrated the effectiveness of simple, positive reinforcers in inducing learning and the acquisition of positive social skills and behaviors. Those interested in efficacy research should follow their lead and apply similar strategies to other populations.

A third problem with efficacy research in TR is that many times publications and presentations fail to provide enough information about the subjects involved in the studies. Although a case study is no substitute for an experiment, at least the case study provides sufficient information about the client served. In reviewing the literature, the lack of specific information about subjects involved in TR research is striking. Effective interventions are hamstrung by insufficient information on who the intervention was successful with. More likely than not, subjects are described in vague terms: "orthopedically impaired", "mobility impaired", "neuro-muscular disorders", "depressed" (clinical depression is different than depressive symptoms), "mentally ill", and so on. How can the efficacy of an intervention be understood, placed in proper context, applied to the relevant persons with disabilities, without adequate description of the subjects? Who exactly were the subjects? Persons with spinal cord injuries (SCI); at what level; what functions were retained; what was the demographic profile (e.g., age, gender, education, etc.) of the persons involved in the intervention?

Furthermore, when disability groups are mixed and only generally described, sorting out significant effects is an impossible task. Suppose an aquatic-based exercise program was effective in inducing an average of 20% increase in range of motion at the shoulder in three planes of movement for subjects described only as "mobility impaired". Did the effect hold equally for all different disabilities within the aggregation of subjects? Or, as is more than likely the case, was the intervention more effective with some impairments than others? This sort of finding is doubly confounding because the practitioner reading the report of the study may well apply the procedure to all sorts of mobility impairments.

An aquatics program may have been especially effective for subjects with multiple sclerosis, but not those with muscular dystrophy, rheumatoid arthritis, or SCI. The findings reported for a mixed subject pool also mask the extent of effectiveness because the gain from the intervention is spread over several diagnoses, only some of which are responsive to aquatic-based exercise.

Furthermore, if the intervention was most effective for persons with multiple sclerosis, was it effective for all levels of impairment? Multiple sclerosis is classified according to functional capability using the Expanded Disability Status Scale (EDSS). Low scores translate into minimal impairment; whereas higher scores indicate more functional loss. Interventions using exercise have been most effective with subjects with low EDSS scores. Findings pertaining to the use of exercise with higher EDSS scores have been mixed.

Conclusion

I note the frequent calls from various corners in the TR community for the development of protocols. Assuming for a moment that the development of protocols is a wise course to follow, then how can protocols be developed unless some changes in the manner in which TR research is conducted are effected? In summary, those changes include the following:

1. Use precise interventions and outcomes that are theoretically and empirically sound;

2. Limit the use of variables that diffuse the effects of interventions and of interventions that produce diffuse effects; and

3. Describe clients the interventions apply to thoroughly in research reports, protocols, case studies and the like.

References

Coyle, C. P., Kinney, W. P., & Shank, J. W. (1993). Trials and tribulations in field-based research in therapeutic recreation. In M. J. Malkin, & C. Z. Howe (Eds.), Research in therapeutic recreation: Concepts and methods, pp. 207-232. State College, PA: Venture.

Hedrick, B. N. (1985). The effect of wheelchair tennis participation and mainstreaming upon the perceptions of competence of physically disabled adolescents, Therapeutic Recreation Journal, 19 (2), 34-46.

Hedrick, B. N. (1986). Wheelchair sport as a mechanism for altering the perceptions of the nondisabled regarding their disabled peers' competence. Therapeutic Recreation Journal, 20 (2), 72-84.

Peterson, C., Maier, S. F., & Seligman, M. E. P. (1993). Learned helplessness: A theory for the age of personal control. New York, NY: Oxford University Press.

Russoniello, C. (1997). Behavioral medicine: A model for therapeutic recreation. In D. M. Compton (Ed.), Issues in therapeutic recreation (2nd edition), pp. 461-487. Chicago, IL: Sagamore.

Shary, J. M., & Iso-Ahola, S. E. (1989). Effects of a control-relevant intervention on nursing home residents' perceived competence and self-esteem. Therapeutic Recreation Journal, 23 (1), 7-16.

1. By "pure cases" the authors mean situations that meet the criteria for learned helplessness based on contingency, cognition, and behavior. They mean complete cases that satisfy the three criteria for learned helplessness.

All rights reserved.

Ads - NCTRC CTRS Exam Secrets - Recreation Therapy Store - Study For Tests - Cram for Tests - Therapy & Rehabilitation Crafts and Products - Beads from S&S Crafts - Activity Stuff Store - NRPA Study Guide - Danny Pettry CEU Program - NCTRC Study Guide

Sing Along With John DVDs: Need an update to your facility's DVD library?
These DVDs provide a breath of musical fresh air for your residents.
A wide variety of music! Go to singalongwithjohndvds.com