ABSTRACT
Purpose: To determine whether practitioner-endorsed and practitioner-suggested items for a patient-centered complementary and alternative medicine (CAM) outcomes questionnaire that became the Self-Assessment of Change instrument (developed under the NIH-NCCAM grant R01AT0033214) differ by CAM discipline.
Methods: In-depth semi-structured interviews (n=24), one focus group (n=4), and two group discussions (n=108) were conducted with a convenience sample of CAM practitioners, including naturopathic physicians, Traditional Chinese Medicine practitioners, massage therapists, homeopaths, and energy healers. Qualitative analysis focused on patient outcomes practitioners said they observed in their practice. Outcome items and related comments by practitioners were further distilled for similarities and contrasts by practitioner type. These were then organized into the following pre-existing domains established by the research team: Physical, Cognitive, Emotional, Social, Spiritual, and Whole Person (outcomes that transcend any single outcome domain or construct). Overarching themes that encompassed outcome items were derived when possible.
Results: Considerable overlap was observed among practitioners from different CAM disciplines with regard to types of patient outcomes assessed.
Conclusions: Practitioner data assembled from multiple sources suggest outcome measures derived from a single CAM tradition are likely to overlap considerably with other CAM systems, with some non-generalizable items possible. These data enhance our understanding of diverse issues in questionnaire development, and suggest the need for a few tailored items to capture outcomes for specific CAM disciplines.
INTRODUCTION
Generally speaking, clinical research on complementary and alternative medicine (CAM) interventions tends to study patients in conventional medical terms, using conventionally-defined medical outcomes.1–3 This remains largely true even within the field of integrative medicine research, which studies the incorporation of complementary approaches into mainstream healthcare.4–7 Diverse CAM systems share a common goal of restoring health to the “whole person,” and standard outcome measures, including questionnaires such as Health Related Quality of Life (HRQoL), do not adequately capture the range of patient-centered outcomes observed in both CAM clinical trials and practice.8
These changes have been described as improvements in overall well-being; energy; clarity of thought; emotional and social functioning; lifestyle patterns; inner life; and spiritual connection.9–16 In addition, investigators have further noted that a subset of responders to CAM approaches undergo a major transformative change that involves a sense of “unstuckness” from unhealthy repetitive patterns of thought or behavior, and results in a comprehensive transformation of their way of being in the world.17,18
The Self-Assessment of Change (SAC) questionnaire19 was developed under an NIH-NCCAM (National Center for Complementary and Alternative Medicine, now the National Center for Complementary and Integrative Health) investigator-initiated grant. The goal was specifically to help capture and measure outcomes experienced as a result of CAM therapeutic interventions that are frequently missed or dismissed by researchers and clinicians but have a great impact for patients. The SAC is referred to as a patient-centered questionnaire, even though it is technically assessing patient-reported outcomes (PROs), because unlike the majority of PRO instruments,20 patients were involved in its development at every stage.
The SAC covers outcomes that patients might deliberately seek or expect, and is also able to capture unexpected, or emergent outcomes. Unexpected or emergent outcomes refer to changes that were unimaginable before the intervention; for example, a change in a patient’s sense of self that transforms her or his capacity to engage in life, and which reverberates at multiple dimensions of being (physical, psychological, and spiritual), to create greater whole-being resilience. It uses a retrospective pre-test design commonly used to evaluate learning outcomes in educational and training research. The retrospective pre-test design controls for the phenomenon of response shift bias, whereby, for example, what a construct such as “ease” or “open hearted” meant before an intervention radically shifts post treatment.21–23
The SAC questionnaire is the result of a lengthy, rigorous and highly systematic development process, which derived initial content from the actual language used by patients (in several pooled peer-reviewed CAM trials), who spontaneously reported types of changes they experienced as a result of CAM therapies that could be classified as transformational. Content validity for questionnaire items and refinement of the final format were achieved by several iterations of psychometric testing through cognitive interviews with a culturally diverse cohort of CAM users.24 The SAC is unusual in that it was designed and tested specifically in populations of CAM users.25 It was also designed for clinical research purposes, but may be applicable for a multilevel exploration of patient outcomes (along with other mainstream outcome measures) in clinical practice and across a broader patient population base. CAM and integrative medicine providers and clinical investigators are invited to learn more about using this instrument and to access the collaborative network of colleagues sharing information and experiences at http://www.selfassessmentofchange.org
This paper reports on a preliminary study that was conducted before the psychometric phase of questionnaire testing had begun, which contributed to questionnaire content via a different research question. It was observed that CAM practitioners assess physical, mental, and emotional signs and symptoms in concert to guide diagnosis and therapeutic choices. The goal of treatment is often wellness, harmony, balance, and flourishing, not just absence of pathology.9–18 It has also been noted that the way a system of medicine interprets health and the etiologies of illness or disharmony directly influences that system’s conceptualization and organization.26 We wondered, therefore, whether such differences in interpretation might also have an impact on practitioner goals for patient healing. This led to the framing of the research question: Do practitioners of different CAM disciplines have differing definitions of health that lead them to focus on different patient outcomes? To initially explore this question, CAM practitioners were interviewed regarding the list of outcomes that comprised the final draft content at this stage of questionnaire development. Our goal was to determine whether a sufficient overlap exists among different CAM systems for a single outcomes questionnaire to have broad applicability in both research and clinical settings, or would different outcomes questionnaires be necessary for different CAM disciplines. The results of the qualitative analysis are reported here.
METHODS
In-depth semi-structured interviews, one focus group, and two group discussions were conducted in person with a diverse group of CAM practitioners in Arizona, Florida, Georgia, and Oregon. Practitioners were drawn from the following CAM disciplines: Naturopathic medicine, Traditional Chinese Medicine (TCM), Chiropractic, Massage therapy, Homeopathy, and Energy healing. For the interviews and focus group, practitioners received the draft items list 24 hours in advance of meeting with the interviewer. Practitioners were asked to review the list and think about the kinds of changes they look for in their patients during the course of treatment. The word “changes” was used rather than “outcomes” as the former is a more clinical and the latter a more research-oriented term. During the interviews and focus group, practitioners were systematically guided through each section of the draft items list, and asked to compare what was there with what they actually observe in practice. In the large discussion groups, which occurred before the draft items list had been generated, practitioners were asked to describe extemporaneously the kinds of changes they consider important in their patients.
The specific question used to start the discussion was:
- I’d like you to describe for me the kinds of outcomes or changes that you look for in your patients that are indications for you that your patient is getting well, or healthier, or moving in the direction of flourishing. It may be helpful for you to think of a particular patient or patients as a starting point, and then talk about the kinds of changes you look for or hope to see in a broader sense.
DATA ANALYSIS
The in-depth semi-structured interviews (n=24) and one focus group (n=4) were transcribed and open-coded in Atlas-ti (qualitative data analysis and research software). Coding focused on patient outcomes/changes practitioners said they observed in their clinical practices. Transcripts from two group discussions with naturopathic doctors (n=100) and TCM practitioners (n=8) elicited free-text outcomes/changes practitioners said they focused on when treating patients. Sometimes the outcome coded was the “negative” state practitioners hoped to see shift, for example, Anger/Frustration or Irritability. Next, open codes were collapsed to create more theoretical codes. These theoretical codes were then sorted according to the following pre-existing domains established by the research team and based on a combination of operationally defined components of HRQoL,27 and the definition of HRQoL used by the FDA28: Physical, Cognitive, Emotional, Social, Spiritual, and Whole Person (outcomes that transcend any single outcome domain or construct). Outcome items and related comments by practitioners were compared for themes that indicated similarities and contrasts by practitioner type.
RESULTS
Practitioners from all disciplines focused on several similar outcomes in the Physical domain, particularly the alleviation of pain and discomfort; improvement in sleep; and improvement in energy (see Table 1).
Table 1: Patient outcomes in the Physical domain by CAM practitioner type.
Domain | Practitioner type | |||||
---|---|---|---|---|---|---|
Chiropractic (2 practitioners) | Energy Healing (7 practitioners) | Homeopathy (3 practitioners) | Massage Therapy (9 practitioners) | Naturopathic Medicine (3 practitioners and 100 in group discussion) | Traditional Chinese Medicine (4 focus group practitioners and 8 in group discussion) | |
Physical | Pain/discomfort Coping Weighed down |
Pain/discomfort Energy Sleep Overall physical well-being Coping |
Energy Sleep Eating habits Sensation Sexual energy |
Pain/discomfort Physical effects of treatment Overall physical well-being Coping (physical flavor) Stressed/relaxed Energy Sleep |
Pain/discomfort Sleep Taking control of diet and exercise Balance |
Pain/discomfort Sleep Weighed down Inactive Uncoordinated |
Practitioners from all disciplines focused on several common outcomes in the Psychological (Cognitive/Emotional) domains, in terms of both alleviation of negative states as well as movement toward positive emotional states (see Table 2).
Table 2: Patient outcomes in the Psychological domain (Cognitive/Emotional) by CAM practitioner type.
Domain | Practitioner type | |||||
---|---|---|---|---|---|---|
Chiropractic (2 practitioners) | Energy Healing (7 practitioners) | Homeopathy (3 practitioners) | Massage Therapy (9 practitioners) | Naturopathic Medicine (3 practitioners and 100 in group discussion) | Traditional Chinese Medicine (4 focus group practitioners and 8 in group discussion) | |
Cognitive | Cloudiness / muddled | Cloudiness / muddled | Struggle Satisfied |
Cloudiness / muddled Pessimistic |
Balance | Cloudiness / muddled Indecisive |
Emotional | Fear Anger Depressed Hope Happiness Coping (Emotional flavor) Self-blame/guilt |
Fear Anger/frustration Anxiety Hope Happiness |
Frustrated Irritable Shattered Hope Joyful |
Fear Anger/frustration Irritability Anxiety Hope Happiness Coping (Emotional flavor) Numb/depressed Trusting one’s body Impassioned Forgiving |
Fear Depressed Hope Enjoyment Coping Self-blame/guilt Balance |
Fear Depressed Hope Coping Insecure |
Practitioners from all disciplines included improvement in social interactions and relationships as an outcome they assess. Naturopathic physicians, Energy healers, and Homeopaths focused on outcomes that were defined by the Spiritual domain with relevance to feeling one’s life has meaning (see Table 3).
Table 3: Patient outcomes in the Social and Spiritual domains by CAM practitioner type.
Domain | Practitioner type | |||||
---|---|---|---|---|---|---|
Chiropractic (2 practitioners) | Energy Healing (7 practitioners) | Homeopathy (3 practitioners) | Massage Therapy (9 practitioners) | Naturopathic Medicine (3 practitioners and 100 in group discussion) | Traditional Chinese Medicine (4 focus group practitioners and 8 in group discussion) | |
Social | Feeling alone | Feeling alone | Changed relationships | Feeling alone | Feeling alone Strained relationships Balance |
Feeling alone |
Spiritual | Meaningful life | Meaningful life | Meaningful life Balance |
Practitioners from all disciplines assessed numerous outcomes in common that were captured by the Whole Person domain; outcomes that transcend any single outcome domain or construct and may have repercussions throughout all domains (see Table 4).
Table 4: Patient outcomes in the Whole Person (WP) domain by CAM practitioner type.
Domain | Practitioner type | |||||
---|---|---|---|---|---|---|
Chiropractic (2 practitioners) | Energy Healing (7 practitioners) | Homeopathy (3 practitioners) | Massage Therapy (9 practitioners) | Naturopathic Medicine (3 practitioners and 100 in group discussion) | Traditional Chinese Medicine (4 focus group practitioners and 8 in group discussion) | |
Whole Person |
Creative Trapped/stuck Self awareness Balance Feeling alive Connected Coping (WP flavor) Different experience of self |
Creative/imaginative Trapped/stuck Self awareness Connected Present Resilience |
Empowered Self awareness Sensitivity Perceptual change Overall improvement Easier life |
Empowered Trapped/stuck Self awareness Balance Feeling alive Connected Coping (WP flavor) Self-confidence Rigid Awake Renewed |
Empowered Trapped/stuck Self awareness Balance Zest for life Connected Balance |
Trapped/stuck Self-care Different experience of self |
DISCUSSION
The overall purpose of this study, which was conducted within the development process of the SAC instrument, was to determine whether a sufficient overlap exists among different CAM systems for a single outcomes questionnaire to have broad applicability in both research and clinical settings. Early-phase qualitative data analysis indicated considerable similarities and congruence among practitioners from different CAM disciplines with regard to types of patient outcomes assessed.
Multiple outcomes overlapped in the Physical domain, particularly the alleviation of pain and discomfort; improvement in sleep; and improvement in energy. Massage therapists imbued “coping” and “well-being” with a sense of body-centeredness. Homeopaths took into account the quality of “sensation” patients experienced; and Naturopathic physicians focused on achieving “balance” in the physical domain.
Multiple outcomes were also common to all CAM disciplines in the Psychological (Cognitive/Emotional) domains, in terms of both alleviation of negative states as well as movement toward positive emotional states. Chiropractors distinguished coping with an emotional flavor from coping physically. Massage therapists covered a wide range of emotional states in their purview of outcomes. Naturopathic physicians again focused on achieving balance in the Psychological domain.
Practitioners from all disciplines examined changes in capacity for social interaction and relationships when assessing patients’ health. Naturopathic physicians, Energy healers, and Homeopaths also focused on outcomes that were defined by the Spiritual domain with relevance to feeling one’s life has meaning. Numerous outcomes overlapped in the Whole Person domain (outcomes that transcend any single domain or construct and may have repercussions throughout all domains). These outcomes focused in particular on moving from a “stuck” state to a greater connection with oneself, and the changes, such as renewal, resilience, and ease that result from that shift.
It is possible that for some CAM disciplines the language of a questionnaire may not capture the nuances of patient experience from the practitioner’s perspective. For example, massage therapists are interested in emotional well-being but also framed it in physical, body–mind-connection terms (such as “trusting one’s body”) rather than by using affective language alone. Naturopathic physicians worked with the concept of achieving balance throughout all domains. These findings indicate the likelihood of being able to use a single outcomes instrument with a set of core questions, which can also be tailored when necessary to capture outcomes language for specific CAM disciplines.
STUDY LIMITATIONS
This study did not use a representative or evenly distributed sample of practitioners. In addition, practitioners sometimes responded to items in our item list rather than with their own spontaneous ideas.
CONCLUSION
It has been observed that the complexity of research methods needs to match the complexity of the human experience of healing.29 CAM clinical research often studies patients defined in conventional medical terms, and uses conventional medical outcome measures. However, it can be said that CAM clinicians interact with and treat a different body from the one generally studied by mainstream medicine. Physical, mental, and emotional signs and symptoms are assessed together to guide the therapeutic approach.9–16 In addition, the goal of CAM is not simply the absence of pathology, but the cultivation of wellness, harmony, and flourishing, which may include changes that could be classified as whole-person.17,18 The concept of patient-centered outcomes for CAM approaches, therefore, requires a whole-person and transformative focus in addition to conventional medical outcome measures.
Practitioner data assembled from multiple sources suggest patient outcomes derived from a single CAM tradition overlap considerably with other CAM systems, and may also have a few non-generalizable items. The many points of convergence in our data indicate that the SAC questionnaire would likely capture patient outcomes assessed by practitioners from diverse CAM traditions. These data enhance our understanding of both the array of issues involved in questionnaire development and the complexity of the phenomenon that is human health and healing. Further qualitative research and psychometric evaluation is recommended to determine the outcomes CAM and integrative medicine practitioners look for in their patients that may not be generalizable across therapeutic approaches.
The author declares she has no competing interests. She also affirms that this paper was accepted for publication in accordance with standard peer-review procedures.
This work was supported by the following National Institutes of Health, National Center for Complementary and Alternative Medicine grants: 2 T32AT001287, 1R01AT003314, and 1U01AT002570. In addition, the author is indebted to Cheryl Ritenbaugh, PhD, MPH and Mary Koithan, PhD under whose direction and vision this work took place.
- Ritenbaugh C,
Verhoef M,
Fleishman S, et al.
Whole systems research: a discipline for studying complementary and alternative medicine.
- Verhoef M,
Lewith G,
Ritenbaugh C, et al.
Complementary and alternative medicine whole systems research: beyond identification of inadequacies of the RCT.
- Fonnebo V,
Grimsgaard S,
Walach H, et al.
Researching complementary and alternative treatments – the gatekeepers are not at home.
- Bell IR,
Caspi O,
Schwartz GE, et al.
Integrative medicine and systemic outcomes research: issues in the emergence of a new model for primary health care.
- Verhoef M,
Mulkins A,
Boon H.
Integrative health care: how can we determine whether patients benefit? - Maizes V,
Rakel D,
Niemiec C.
Integrative medicine and patient-centered care.
- https://nccih.nih.gov/about/strategic-plans/2016/Objective-1-Advance-Fundamental-Science-Methods-Development#strategy-3 Accessed October 6, 2017.
- Verhoef M,
Vanderheyden L,
Dryden T, et al.
Evaluating complementary and alternative medicine interventions: in search of appropriate patient-centered outcome measures.
- Gould A,
MacPherson H.
Patient perspectives on outcomes after treatment with acupuncture.
- Bell IR,
Koithan M,
Gorman MM,
Baldwin CM.
Homeopathic practitioner views of changes in patients undergoing constitutional treatment for chronic disease.
- Paterson C,
Britten N.
Acupuncture for people with chronic illness: combining qualitative and quantitative outcome assessment.
- Schulman D.
The unexpected outcomes of acupuncture: case reports in support of refocusing research designs.
- Elder C,
Ritenbaugh C,
Mist S, et al.
Randomized trial of two mind-body interventions for weight-loss maintenance.
- White M,
Verhoef M,
Davison BJ, et al.
Seeking mind, body and spirit healing – why some men with prostate cancer choose CAM (complementary and alternative medicine) over conventional cancer treatments.
- Sutherland EG,
Ritenbaugh C,
Kiley SJ, et al.
An HMO-based prospective pilot study of energy medicine for chronic headaches: whole-person outcomes point to the need for new instrumentation.
- Hsu C,
Bluespruce J,
Sherman K,
Cherkin D.
Unanticipated benefits of CAM therapies for back pain: an exploration of patient experiences.
- Mulkins AL,
Verhoef M.
Supporting the transformative process: experiences of cancer patients receiving integrative care.
- Koithan M,
Verhoef M,
Bell IR, et al.
The process of whole person healing: “unstuckness” and beyond.
- Ritenbaugh C,
Nicht M,
Nichter MA, et al.
Developing a patient-centered outcome measure for complementary and alternative medicine therapies I: defining content and format.
- Weldring T,
Smith SMS.
Patient-reported outcomes (PROs) and patient-reported outcome measures (PROMs).
- Howard GS.
Response-shift bias: a problem in evaluating interventions with pre/post self-reports.
- Rockwell SK,
Kohn H.
Post-then-pre evaluation: measuring behavior change more accurately.
- Drennan J,
Hyde A.
Controlling response shift bias: the use of the retrospective pre-test design in the evaluation of a master’s programme.
- Thompson JJ,
Kelly KL,
Ritenbaugh C, et al.
Developing a patient-centered outcome measure for complementary and alternative medicine therapies II: refining content validity through cognitive interviews.
- Eton DT,
Bauer BA,
Sood A, et al.
Patient-reported outcomes in studies of complementary and alternative medicine: problems, solutions, and future directions.
- Levin J.
Esoteric healing traditions: a conceptual overview.
- Ferrell BR,
Dow KH,
Grant M.
Measurement of the quality of life in cancer survivors.
- Guidance for Industry.
Patient-reported outcome measures: use in medical product development to support labeling claims.
- O’Malley PG.
Studying optimal healing environments: challenges and proposals.