PDF

Defining Health: Do Practitioners from Different Complementary and Alternative Medicine (CAM) Disciplines Focus on Different Patient Outcomes?

ABSTRACT

Purpose: To determine whether practitioner-endorsed and practitioner-suggested items for a patient-centered complementary and alternative medicine (CAM) outcomes questionnaire that became the Self-Assessment of Change instrument (developed under the NIH-NCCAM grant R01AT0033214) differ by CAM discipline.

Methods: In-depth semi-structured interviews (n=24), one focus group (n=4), and two group discussions (n=108) were conducted with a convenience sample of CAM practitioners, including naturopathic physicians, Traditional Chinese Medicine practitioners, massage therapists, homeopaths, and energy healers. Qualitative analysis focused on patient outcomes practitioners said they observed in their practice. Outcome items and related comments by practitioners were further distilled for similarities and contrasts by practitioner type. These were then organized into the following pre-existing domains established by the research team: Physical, Cognitive, Emotional, Social, Spiritual, and Whole Person (outcomes that transcend any single outcome domain or construct). Overarching themes that encompassed outcome items were derived when possible.

Results: Considerable overlap was observed among practitioners from different CAM disciplines with regard to types of patient outcomes assessed.

Conclusions: Practitioner data assembled from multiple sources suggest outcome measures derived from a single CAM tradition are likely to overlap considerably with other CAM systems, with some non-generalizable items possible. These data enhance our understanding of diverse issues in questionnaire development, and suggest the need for a few tailored items to capture outcomes for specific CAM disciplines.

INTRODUCTION

Generally speaking, clinical research on complementary and alternative medicine (CAM) interventions tends to study patients in conventional medical terms, using conventionally-defined medical outcomes.13 This remains largely true even within the field of integrative medicine research, which studies the incorporation of complementary approaches into mainstream healthcare.47 Diverse CAM systems share a common goal of restoring health to the “whole person,” and standard outcome measures, including questionnaires such as Health Related Quality of Life (HRQoL), do not adequately capture the range of patient-centered outcomes observed in both CAM clinical trials and practice.8

These changes have been described as improvements in overall well-being; energy; clarity of thought; emotional and social functioning; lifestyle patterns; inner life; and spiritual connection.916 In addition, investigators have further noted that a subset of responders to CAM approaches undergo a major transformative change that involves a sense of “unstuckness” from unhealthy repetitive patterns of thought or behavior, and results in a comprehensive transformation of their way of being in the world.17,18

The Self-Assessment of Change (SAC) questionnaire19 was developed under an NIH-NCCAM (National Center for Complementary and Alternative Medicine, now the National Center for Complementary and Integrative Health) investigator-initiated grant. The goal was specifically to help capture and measure outcomes experienced as a result of CAM therapeutic interventions that are frequently missed or dismissed by researchers and clinicians but have a great impact for patients. The SAC is referred to as a patient-centered questionnaire, even though it is technically assessing patient-reported outcomes (PROs), because unlike the majority of PRO instruments,20 patients were involved in its development at every stage.

The SAC covers outcomes that patients might deliberately seek or expect, and is also able to capture unexpected, or emergent outcomes. Unexpected or emergent outcomes refer to changes that were unimaginable before the intervention; for example, a change in a patient’s sense of self that transforms her or his capacity to engage in life, and which reverberates at multiple dimensions of being (physical, psychological, and spiritual), to create greater whole-being resilience. It uses a retrospective pre-test design commonly used to evaluate learning outcomes in educational and training research. The retrospective pre-test design controls for the phenomenon of response shift bias, whereby, for example, what a construct such as “ease” or “open hearted” meant before an intervention radically shifts post treatment.2123

The SAC questionnaire is the result of a lengthy, rigorous and highly systematic development process, which derived initial content from the actual language used by patients (in several pooled peer-reviewed CAM trials), who spontaneously reported types of changes they experienced as a result of CAM therapies that could be classified as transformational. Content validity for questionnaire items and refinement of the final format were achieved by several iterations of psychometric testing through cognitive interviews with a culturally diverse cohort of CAM users.24 The SAC is unusual in that it was designed and tested specifically in populations of CAM users.25 It was also designed for clinical research purposes, but may be applicable for a multilevel exploration of patient outcomes (along with other mainstream outcome measures) in clinical practice and across a broader patient population base. CAM and integrative medicine providers and clinical investigators are invited to learn more about using this instrument and to access the collaborative network of colleagues sharing information and experiences at http://www.selfassessmentofchange.org

This paper reports on a preliminary study that was conducted before the psychometric phase of questionnaire testing had begun, which contributed to questionnaire content via a different research question. It was observed that CAM practitioners assess physical, mental, and emotional signs and symptoms in concert to guide diagnosis and therapeutic choices. The goal of treatment is often wellness, harmony, balance, and flourishing, not just absence of pathology.918 It has also been noted that the way a system of medicine interprets health and the etiologies of illness or disharmony directly influences that system’s conceptualization and organization.26 We wondered, therefore, whether such differences in interpretation might also have an impact on practitioner goals for patient healing. This led to the framing of the research question: Do practitioners of different CAM disciplines have differing definitions of health that lead them to focus on different patient outcomes? To initially explore this question, CAM practitioners were interviewed regarding the list of outcomes that comprised the final draft content at this stage of questionnaire development. Our goal was to determine whether a sufficient overlap exists among different CAM systems for a single outcomes questionnaire to have broad applicability in both research and clinical settings, or would different outcomes questionnaires be necessary for different CAM disciplines. The results of the qualitative analysis are reported here.

METHODS

In-depth semi-structured interviews, one focus group, and two group discussions were conducted in person with a diverse group of CAM practitioners in Arizona, Florida, Georgia, and Oregon. Practitioners were drawn from the following CAM disciplines: Naturopathic medicine, Traditional Chinese Medicine (TCM), Chiropractic, Massage therapy, Homeopathy, and Energy healing. For the interviews and focus group, practitioners received the draft items list 24 hours in advance of meeting with the interviewer. Practitioners were asked to review the list and think about the kinds of changes they look for in their patients during the course of treatment. The word “changes” was used rather than “outcomes” as the former is a more clinical and the latter a more research-oriented term. During the interviews and focus group, practitioners were systematically guided through each section of the draft items list, and asked to compare what was there with what they actually observe in practice. In the large discussion groups, which occurred before the draft items list had been generated, practitioners were asked to describe extemporaneously the kinds of changes they consider important in their patients.

The specific question used to start the discussion was:

  • I’d like you to describe for me the kinds of outcomes or changes that you look for in your patients that are indications for you that your patient is getting well, or healthier, or moving in the direction of flourishing. It may be helpful for you to think of a particular patient or patients as a starting point, and then talk about the kinds of changes you look for or hope to see in a broader sense.

DATA ANALYSIS

The in-depth semi-structured interviews (n=24) and one focus group (n=4) were transcribed and open-coded in Atlas-ti (qualitative data analysis and research software). Coding focused on patient outcomes/changes practitioners said they observed in their clinical practices. Transcripts from two group discussions with naturopathic doctors (n=100) and TCM practitioners (n=8) elicited free-text outcomes/changes practitioners said they focused on when treating patients. Sometimes the outcome coded was the “negative” state practitioners hoped to see shift, for example, Anger/Frustration or Irritability. Next, open codes were collapsed to create more theoretical codes. These theoretical codes were then sorted according to the following pre-existing domains established by the research team and based on a combination of operationally defined components of HRQoL,27 and the definition of HRQoL used by the FDA28: Physical, Cognitive, Emotional, Social, Spiritual, and Whole Person (outcomes that transcend any single outcome domain or construct). Outcome items and related comments by practitioners were compared for themes that indicated similarities and contrasts by practitioner type.

RESULTS

Practitioners from all disciplines focused on several similar outcomes in the Physical domain, particularly the alleviation of pain and discomfort; improvement in sleep; and improvement in energy (see Table 1).

Table 1: Patient outcomes in the Physical domain by CAM practitioner type.

Domain Practitioner type
Chiropractic (2 practitioners) Energy Healing (7 practitioners) Homeopathy (3 practitioners) Massage Therapy (9 practitioners) Naturopathic Medicine (3 practitioners and 100 in group discussion) Traditional Chinese Medicine (4 focus group practitioners and 8 in group discussion)
Physical Pain/discomfort
Coping
Weighed down
Pain/discomfort
Energy
Sleep
Overall physical
well-being
Coping
Energy
Sleep
Eating habits
Sensation
Sexual energy
Pain/discomfort
Physical effects of treatment
Overall physical well-being
Coping (physical flavor)
Stressed/relaxed
Energy
Sleep
Pain/discomfort
Sleep
Taking control of
diet and exercise
Balance
Pain/discomfort
Sleep
Weighed down
Inactive
Uncoordinated

 

Practitioners from all disciplines focused on several common outcomes in the Psychological (Cognitive/Emotional) domains, in terms of both alleviation of negative states as well as movement toward positive emotional states (see Table 2).

Table 2: Patient outcomes in the Psychological domain (Cognitive/Emotional) by CAM practitioner type.

Domain Practitioner type
Chiropractic (2 practitioners) Energy Healing (7 practitioners) Homeopathy (3 practitioners) Massage Therapy (9 practitioners) Naturopathic Medicine (3 practitioners and 100 in group discussion) Traditional Chinese Medicine (4 focus group practitioners and 8 in group discussion)
Cognitive Cloudiness / muddled Cloudiness / muddled Struggle
Satisfied
Cloudiness / muddled
Pessimistic
Balance Cloudiness / muddled
Indecisive
Emotional Fear
Anger
Depressed
Hope
Happiness
Coping (Emotional flavor)
Self-blame/guilt
Fear
Anger/frustration
Anxiety
Hope
Happiness
Frustrated
Irritable
Shattered
Hope
Joyful
Fear
Anger/frustration
Irritability
Anxiety
Hope
Happiness
Coping (Emotional flavor)
Numb/depressed
Trusting one’s body
Impassioned
Forgiving
Fear
Depressed
Hope
Enjoyment
Coping
Self-blame/guilt
Balance
Fear
Depressed
Hope
Coping
Insecure

 

Practitioners from all disciplines included ­improvement in social interactions and relationships as an outcome they assess. Naturopathic physicians, Energy healers, and Homeopaths focused on outcomes that were defined by the Spiritual domain with relevance to feeling one’s life has meaning (see Table 3).

Table 3: Patient outcomes in the Social and Spiritual domains by CAM practitioner type.

Domain Practitioner type
Chiropractic (2 practitioners) Energy Healing (7 practitioners) Homeopathy (3 practitioners) Massage Therapy (9 practitioners) Naturopathic Medicine (3 practitioners and 100 in group discussion) Traditional Chinese Medicine (4 focus group practitioners and 8 in group discussion)
Social Feeling alone Feeling alone Changed relationships Feeling alone Feeling alone
Strained relationships
Balance
Feeling alone
Spiritual Meaningful life Meaningful life Meaningful life
Balance

 

Practitioners from all disciplines assessed numerous outcomes in common that were captured by the Whole Person domain; outcomes that transcend any single outcome domain or construct and may have repercussions throughout all domains (see Table 4).

Table 4: Patient outcomes in the Whole Person (WP) domain by CAM practitioner type.

Domain Practitioner type
Chiropractic (2 practitioners) Energy Healing (7 practitioners) Homeopathy (3 practitioners) Massage Therapy (9 practitioners) Naturopathic Medicine (3 practitioners and 100 in group discussion) Traditional Chinese Medicine (4 focus group practitioners and 8 in group discussion)
Whole
Person
Creative
Trapped/stuck
Self awareness
Balance
Feeling alive
Connected
Coping (WP flavor)
Different experience of self
Creative/imaginative
Trapped/stuck
Self awareness
Connected
Present
Resilience
Empowered
Self awareness
Sensitivity
Perceptual change
Overall improvement
Easier life
Empowered
Trapped/stuck
Self awareness
Balance
Feeling alive
Connected
Coping (WP flavor)
Self-confidence
Rigid
Awake
Renewed
Empowered
Trapped/stuck
Self awareness
Balance
Zest for life
Connected
Balance
Trapped/stuck
Self-care
Different experience of self

 

DISCUSSION

The overall purpose of this study, which was conducted within the development process of the SAC instrument, was to determine whether a sufficient overlap exists among different CAM systems for a single outcomes questionnaire to have broad applicability in both research and clinical settings. Early-phase qualitative data analysis indicated considerable similarities and congruence among practitioners from different CAM disciplines with regard to types of patient outcomes assessed.

Multiple outcomes overlapped in the Physical domain, particularly the alleviation of pain and discomfort; improvement in sleep; and improvement in energy. Massage therapists imbued “coping” and “well-being” with a sense of body-centeredness. Homeopaths took into account the quality of “sensation” patients experienced; and Naturopathic physicians focused on achieving “balance” in the physical domain.

Multiple outcomes were also common to all CAM disciplines in the Psychological (Cognitive/Emotional) domains, in terms of both alleviation of negative states as well as movement toward positive emotional states. Chiropractors distinguished coping with an emotional flavor from coping physically. Massage therapists covered a wide range of emotional states in their purview of outcomes. Naturopathic physicians again focused on achieving balance in the Psychological domain.

Practitioners from all disciplines examined changes in capacity for social interaction and relationships when assessing patients’ health. Naturopathic physicians, Energy healers, and Homeopaths also focused on outcomes that were defined by the Spiritual domain with relevance to feeling one’s life has meaning. Numerous outcomes overlapped in the Whole Person domain (outcomes that transcend any single domain or construct and may have repercussions throughout all domains). These outcomes focused in particular on moving from a “stuck” state to a greater connection with oneself, and the changes, such as renewal, resilience, and ease that result from that shift.

It is possible that for some CAM disciplines the language of a questionnaire may not capture the nuances of patient experience from the practitioner’s perspective. For example, massage therapists are interested in emotional well-being but also framed it in physical, body–mind-connection terms (such as “trusting one’s body”) rather than by using affective language alone. Naturopathic physicians worked with the concept of achieving balance throughout all domains. These findings indicate the likelihood of being able to use a single outcomes instrument with a set of core questions, which can also be tailored when necessary to capture outcomes language for specific CAM disciplines.

STUDY LIMITATIONS

This study did not use a representative or evenly distributed sample of practitioners. In addition, practitioners sometimes responded to items in our item list rather than with their own spontaneous ideas.

CONCLUSION

It has been observed that the complexity of research methods needs to match the complexity of the human experience of healing.29 CAM clinical research often studies patients defined in conventional medical terms, and uses conventional medical outcome measures. However, it can be said that CAM clinicians interact with and treat a different body from the one generally studied by mainstream medicine. Physical, mental, and emotional signs and symptoms are assessed together to guide the therapeutic approach.916 In addition, the goal of CAM is not simply the absence of pathology, but the cultivation of wellness, harmony, and flourishing, which may include changes that could be classified as whole-person.17,18 The concept of patient-centered outcomes for CAM approaches, therefore, requires a whole-person and transformative focus in addition to conventional medical outcome measures.

Practitioner data assembled from multiple sources suggest patient outcomes derived from a single CAM tradition overlap considerably with other CAM systems, and may also have a few non-generalizable items. The many points of convergence in our data indicate that the SAC questionnaire would likely capture patient outcomes assessed by practitioners from diverse CAM traditions. These data enhance our understanding of both the array of issues involved in questionnaire development and the complexity of the phenomenon that is human health and healing. Further qualitative research and psychometric evaluation is recommended to determine the outcomes CAM and integrative medicine practitioners look for in their patients that may not be generalizable across therapeutic approaches.

The author declares she has no competing interests. She also affirms that this paper was accepted for publication in accordance with standard peer-review procedures.

This work was supported by the following National Institutes of Health, National Center for Complementary and Alternative Medicine grants: 2 T32AT001287, 1R01AT003314, and 1U01AT002570. In addition, the author is indebted to Cheryl Ritenbaugh, PhD, MPH and Mary Koithan, PhD under whose direction and vision this work took place.

  1. Ritenbaugh C,
    Verhoef M,
    Fleishman S, et al.
    Whole systems research: a discipline for studying complementary and alternative medicine.
    Altern Ther Health Med..
    2003; 9(4):32–6.
  2. Verhoef M,
    Lewith G,
    Ritenbaugh C, et al.
    Complementary and alternative medicine whole systems research: beyond identification of inadequacies of the RCT.
    Complement Ther Med.
    2005; 13(3):206–12.
  3. Fonnebo V,
    Grimsgaard S,
    Walach H, et al.
    Researching complementary and alternative treatments – the gatekeepers are not at home.
    BMC Med Res Methodol.
    2007; 7:7.
  4. Bell IR,
    Caspi O,
    Schwartz GE, et al.
    Integrative medicine and systemic outcomes research: issues in the emergence of a new model for primary health care.
    Arch Intern Med.
    2002;162(2):13340.
  5. Verhoef M,
    Mulkins A,
    Boon H.
    Integrative health care: how can we determine whether patients benefit? J Altern Complement Med.
    2005; 11(Suppl 1):S57–65.
  6. Maizes V,
    Rakel D,
    Niemiec C.
    Integrative medicine and patient-centered care.
    Explore.
    2009; 5(5):277–89.
  7. https://nccih.nih.gov/about/strategic-plans/2016/Objective-1-Advance-Fundamental-Science-Methods-Development#strategy-3 Accessed October 6, 2017.
  8. Verhoef M,
    Vanderheyden L,
    Dryden T, et al.
    Evaluating complementary and alternative medicine interventions: in search of appropriate patient-centered outcome measures.
    BMC Complement Altern Med..
    2006; 6:6–38.
  9. Gould A,
    MacPherson H.
    Patient perspectives on outcomes after treatment with acupuncture.
    J Altern Complement Med.
    2001; 7(3):261–8.
  10. Bell IR,
    Koithan M,
    Gorman MM,
    Baldwin CM.
    Homeopathic practitioner views of changes in patients undergoing constitutional treatment for chronic disease.
    J Altern Complement Med..
    2003; 9(1):39–50.
  11. Paterson C,
    Britten N.
    Acupuncture for people with chronic illness: combining qualitative and quantitative outcome assessment.
    J Altern Complement Med.
    2003; 9(5):671–81.
  12. Schulman D.
    The unexpected outcomes of acupuncture: case reports in support of refocusing research designs.
    J Altern Complement Med..
    2004; 10(5):785–9.
  13. Elder C,
    Ritenbaugh C,
    Mist S, et al.
    Randomized trial of two mind-body interventions for weight-loss maintenance.
    J Altern Complement Med.
    2007; 13(1):67–78.
  14. White M,
    Verhoef M,
    Davison BJ, et al.
    Seeking mind, body and spirit healing – why some men with prostate cancer choose CAM (complementary and alternative medicine) over conventional cancer treatments.
    Integr Med Insights.
    2008; 3:1–11.
  15. Sutherland EG,
    Ritenbaugh C,
    Kiley SJ, et al.
    An HMO-based prospective pilot study of energy medicine for chronic headaches: whole-person outcomes point to the need for new instrumentation.
    J Altern Complement Med.
    2009; 15(8):819–26.
  16. Hsu C,
    Bluespruce J,
    Sherman K,
    Cherkin D.
    Unanticipated benefits of CAM therapies for back pain: an exploration of patient experiences.
    J Altern Complement Med.
    2010; 16(2):157–63.
  17. Mulkins AL,
    Verhoef M.
    Supporting the transformative process: experiences of cancer patients receiving integrative care.
    Integr Cancer Ther.
    2004; 3(3):230–7.
  18. Koithan M,
    Verhoef M,
    Bell IR, et al.
    The process of whole person healing: “unstuckness” and beyond.
    J Altern Complement Med.
    2007; 13(6):659–68.
  19. Ritenbaugh C,
    Nicht M,
    Nichter MA, et al.
    Developing a patient-centered outcome measure for complementary and alternative medicine therapies I: defining content and format.
    BMC Complement Altern Med.
    2011; 11:135.
  20. Weldring T,
    Smith SMS.
    Patient-reported outcomes (PROs) and patient-reported outcome measures (PROMs).
    Health Serv Insights.
    2013; 6:61–8.
  21. Howard GS.
    Response-shift bias: a problem in evaluating interventions with pre/post self-reports.
    Eval Rev..
    1980; 4(1):93–106.
  22. Rockwell SK,
    Kohn H.
    Post-then-pre evaluation: measuring behavior change more accurately.
    J Ext..
    1989; 27(2):2FEA5.
  23. Drennan J,
    Hyde A.
    Controlling response shift bias: the use of the retrospective pre-test design in the evaluation of a master’s programme.
    Assess Eval High Educ..
    2008; 33(6):699–709.
  24. Thompson JJ,
    Kelly KL,
    Ritenbaugh C, et al.
    Developing a patient-centered outcome measure for complementary and alternative medicine therapies II: refining content validity through cognitive interviews.
    BMC Complement Alt Med.
    2011; 11:136.
  25. Eton DT,
    Bauer BA,
    Sood A, et al.
    Patient-reported outcomes in studies of complementary and alternative medicine: problems, solutions, and future directions.
    Explore..
    2011; 7(5):314–9.
  26. Levin J.
    Esoteric healing traditions: a conceptual overview.
    Explore..
    2008; 4(2):101–12.
  27. Ferrell BR,
    Dow KH,
    Grant M.
    Measurement of the quality of life in cancer survivors.
    Qual Life Res.
    1995; 4:523–31.
  28. Guidance for Industry.
    Patient-reported outcome measures: use in medical product development to support labeling claims.
    Federal Regist.
    2009; 74(35):65132–3.
  29. O’Malley PG.
    Studying optimal healing environments: challenges and proposals.
    J Altern Complement Med.
    2005; 11 suppl 1:S17–22.

DOI: https://doi.org/10.14200/jrm.2017.6.0107

You can also access full issues of the Journal of Restorative Medicine here.