Volume 44 Number 5 2007
Pages 659 — 664
Televideo assessment using Functional Reach Test and European Stroke Scale
Susan E. Palsbo, PhD;1* Stephen J. Dawson, PT;2 Lynda Savard, PT;3 Marc Goldstein, EdD;4 Andrew Heuser, PT2
Abstract — This study explored the equivalence of physical function assessment by physical therapists (PTs) during face-to-face and remote administration of the European Stroke Scale (ESS) and the Functional Reach Test (FRT) to 26 subjects with a history of stroke. Patients were randomized to remote or face-to-face administration groups. Each patient was simultaneously rated by both the face-to-face and remote PTs. The PTs were blinded to each other's results. Equivalence was set at the 95% limits of agreement. When the face-to-face PT directed the patient, the two PTs reported equivalent values in more than 90% of the patients for the FRT and for all ESS components, with the exception of gait (83%) and maintaining leg position (85%). When the remote PT directed the patient, the two PTs reported equivalent values in more than 90% of the patients for the FRT and more than 83% for all ESS components. Televideo assessment of function by PTs is substantially equivalent to a face-to-face encounter.
Key words: European Stroke Scale, functional reach, physical function assessment, physical therapy, rehabilitation, remote assessment, stroke, telemedicine, telerehabilitation, televideo.
Email: spalsbo@gmu.edu
A growing number of pilot studies on telemedicine extend the physical rehabilitation encounter from the clinic to distant sites, including the home [1-2]. Initial telerehabilitation studies explored videoconferenced clinician-clinician consultations [3], orthotic assessments [4], emailed consultations with attached video recordings of children with complex movement or postural disorders [5], and gait analysis for poststroke subjects [6]. Several research programs are investigating the use of robotics and force-feedback [7-10], while others are exploring the use of haptic gloves for assessing muscle tone and movement during exercise [7]. Italian researchers developed and evaluated a virtual-reality system connected to a three-dimensional motion-tracking device in a 4-week trial for upper-limb poststroke rehabilitation [10-11]. Some case studies report successful mentoring of onsite therapists by a remote specialist physical therapist (PT) in the treatment of patients with traumatic brain injury or stroke [12-14].
In addition to these case studies, we identified only two additional published studies on equivalence of remote assessments of function and one randomized trial on delivering therapy to restore motor function. The equivalence study by Nitzkin et al. investigated three clinics (knee, neck, and back) and compiled four matched pairs of serial observations in each clinic (13, 11, and 10 data points, respectively) [15]. The subjects were young adults recuperating from athletic injuries. The percentage of scores that were in exact or similar agreement was 81, 96, and 91, respectively. The equivalence study by Durfee et al. evaluated the equivalence of joint range of motion, manual muscle test, sit-to-stand, forward reach, and timed up and go tests between remote and onsite assessments. In this study, the therapists scored the patients the same, with the exception of a few cases of screen-based range of motion measurements [16]. However, the therapists were not randomized and the test subjects were sham patients. The randomized trial was a televideo-guided 6-week exercise program for 21 patients recovering from total knee replacement. This study found no significant difference between face-to-face and remote guides [17].
Our long-range goal is to define evidence-based parameters for "true telemedicine" sessions in which the patient is seen by the therapist for the purpose of assessment, therapy, reassessment, and discussion of results [18]. Evidence-based findings can inform the development of practice guidelines, define suitable candidates for teletherapy, and devise training for PTs and physiatrists [19]. We also wanted to explore the feasibility of teletherapy for people whose functional impairment and/or lack of transportation preclude them from traveling to a distant clinic for an appointment. We decided to test the hypothesis that remote assessment is equivalent to a face-to-face assessment as the first step in the patient management process. If the assessments are equivalent, we can proceed to develop randomized controlled trials for providing remote physical therapy.
In this initial study, we assessed patients with a history of stroke because it is frequently seen in rehabilitation settings and several well-validated assessment tools for stroke are visually based. Assessment of physical function after a stroke is an important component of diagnosis and recovery. Rehabilitation professionals have a growing interest in both remote stroke diagnosis and remote poststroke therapy because many stroke patients have co-occurring cognition or mobility impairments that compromise their ability to keep therapy appointments [20-24].
This study was a randomized double crossover agreement design and included fixed pairs of onsite and remote PT evaluators. The study was designed to test two effects. First, we tested the effect of using televideo equipment to assess physical function; i.e., does assessment with televideo equipment bias the measure in one direction or the other (first crossover)? Second, we tested the effect of using televideo equipment to direct the patient through the assessment exercise (second crossover). We selected measurement tools that had known physical function psychometric properties that were highly relevant to stroke patients and could be administered with minimal adaptations over televideo equipment. Our procedures enhanced interrater reliability and eliminated serial correlation bias by the raters.
Two PTs were recruited at each of the two rehabilitation hospitals: the Sister Kenny Rehabilitation Institute in Minneapolis, Minnesota, and the INTEGRIS Jim Thorpe Rehabilitation Hospital in Oklahoma City, Oklahoma. All PTs had at least 2 years experience using televideo to support onsite PTs for a wide variety of patient assessments. The four PTs and the principal investigator conducted a videoconference to agree on a protocol and a verbal script for the physical assessments. Each PT also completed human study subject protection courses and ensured the subjects gave informed consent.
This study used a convenience sample. The participating PTs extended personal invitations to eligible patients who had already scheduled appointments. Study eligibility criteria were 18 years of age, history of stroke, currently receiving outpatient or inpatient physical therapy, ability to understand spoken English well enough to complete the measurement process, and ability to comprehend and sign the informed consent documents. Candidates with uncorrected visual deficits (e.g., visual-spatial neglect or reduced field of view) were excluded because they would be unable to see the entire video screen. Each participant was given a $5 gift card to a local variety store. This study was conducted in 2004, prior to JRRD Clinical Trial registration requirements, and was approved by the institutional review boards at the MedStar Research Institute, INTEGRIS, and the Sister Kenny Rehabilitation Institute.
A total of 18 men and 8 women between the ages of 25 and 81 (median age of 64) were recruited at the two study sites. Time since stroke ranged from 2 months to 15 years, with a mean of 2.7 years. Table 1 shows the functional ability recorded by the face-to-face therapist; patients were distributed across the entire range of most measures except consciousness and comprehension. Since the telerehabilitation application envisioned by this study is outpatient therapy at home, we focused recruitment efforts on outpatients; however, we ultimately had to include some inpatients to meet recruitment goals.
We could not locate published criteria for selecting telemedicine equivalency measures; therefore, we devised our own: the measures had to (1) be appropriate and relevant to people with stroke, (2) have known psychometric properties (high coefficients of validity and reliability) published in peer-reviewed literature, (3) be widely used in research and clinical practice, (4) be visually based (the therapist could conduct the measurement without touching the patient), and (5) be completed within 30 minutes.
We selected two measures: the Functional Reach Test (FRT) [25] and the European Stroke Scale (ESS) [26]. The FRT is "the maximal distance one can reach forward beyond arm's length while maintaining a fixed base of support in the standing position" and has good concurrent and predictive validity [27-28]. The ESS consists of 14 measures designed for patients with middle cerebral artery stroke.
The research protocol eliminated bias unrelated to the use of the televideo equipment. Serial correlation bias was eliminated by the therapists conducting the assessments simultaneously. That is, when the patient was positioned in front of the televideo equipment, the patient was scored at the same time by both the remote and the face-to-face PT. This is a departure from most previous studies that presented patients serially or when a single rater conducted both assessments, once face-to-face and once over video.
The face-to-face administration bias was eliminated by randomizing patients at the two study sites into one of two groups at each site. The first group was directed through the assessment by the onsite PT, and the second group was directed through the assessment by the remote PT. None of the patients was touched by the PTs.
We administered the FRT by replicating Duncan et al.'s method [27]. The subject stood adjacent to a large yard stick that was at shoulder height. We constructed an enlarged paper yardstick so the remote therapist could view it over the video screen. The therapists recorded the start (position 1) and end (position 2) measurements, and each subject performed five trials. The remote or face-to-face PT asked the subject to reach as far forward as possible without losing his or her balance. Functional reach was defined as the mean difference between the first and second positions over the last three trials. Patients were guarded by a physical therapy technician.
We administered the ESS by asking the study participant to sit on an examination table. The administering PT directed the participant to perform the activities on the scale. At no time did the face-to-face therapist touch the patient. Instead, a physical therapy technician guarded the subject and performed the tasks that required positioning, such as "actively lifting the patient's leg such that the thigh forms an angle of 90º with the bed."
The videoconferencing equipment was a Polycom SoundStation (Polycom, Inc, Pleasanton, California), and data were transferred at a transmission speed of 384 kB/s. Our original intention was to include a pair of therapists at each study site, 800 miles apart. However, we had scheduling conflicts and other practical challenges when conducting this study, which we describe elsewhere [29]. As a consequence, face-to-face and remote evaluations were conducted within the same healthcare delivery system. The transmissions went offsite several miles to a sister hospital and were then rerouted back to the bridge and another room, just as if we had connected to a more distant site. Thus, the degrading effects of the network on the evaluation process were taken into account. PTs recorded the data during the encounter. Data were analyzed with Microsoft Excel 2002 (Microsoft Corp, Redmond, Washington). Following Bland and Altman [30-31], we set the criterion limit of agreement at 95 percent.
Table 2 presents the percentage of patients who scored within the 95 percent limits of agreement and percent exact agreement. The therapists reported equivalent scores for at least 80 percent of the patients for all measures, regardless of which therapist directed the patient through the testing. A Wilcoxon signed rank test examined the results of the face-to-face and remote therapist administration. No significant difference was found in the results (Z = - 0.239, p > 0.05). We found no scoring bias based on the location of the therapist administering the assessment.
The published interrater reliability values of the ESS scale and FRT show that the percentage agreement between remote and face-to-face assessment administration exceeded what would be expected from simultaneous paired face-to-face measures (Table 1). Nevertheless, questions are raised by the slightly lower agreement percentages for several measures when the patient was directed by the remote therapist. This result suggests a slight negative effect on the patient's understanding of the therapist's direction when televideo was used. If therapists were allowed to deviate from this highly scripted assessment, this negative effect might disappear. Regardless, the percentage of equivalent measures when the remote therapists administered the assessments exceeded 83 percent for all measures, and exact agreement was noted for four measures. Whether or not these differences are clinically meaningful is a question best left to the patient's therapist.
This study had several methodological improvements over previous telerehabilitation studies. First, we established measurement tool selection criteria that emphasized the use of tools for the population being studied (in our case, stroke). In addition, the tools had very high interrater reliability statistics, which helped us ensure that measured differences (if any) were true differences between the patients rather than differences in how two therapists measured the patients.
A second improvement was the simultaneous scoring by the remote and face-to-face evaluators that eliminated the problems of patient fatigue or warm-up and serial correlation bias. The third improvement was comparison of remote and face-to-face administration of the examination. The fourth was use of a more appropriate test of agreement than the Pearson correlation coefficients or intraclass correlation coefficients that are commonly used in telemedicine studies.
Since this is was pilot study, we cannot generalize to all visually based televideo physical therapy assessment tools. A larger trial should be conducted in a more naturalistic environment, evaluate patients across a larger spectrum of functional impairment origins and ability, assign random pairs of therapists, include patients unknown to the therapists, and exchange therapists between face-to-face and remote assessment. Repeating the trial with a slower Internet speed [3-4,32] or less-expensive and more widely available equipment, such as the video cellular telephone, would also be useful [33].
We found that a PT can accurately assess changes in a patient's physical function as measured by the FRT and the ESS when assessment is administered by televideo and that the assessment will be substantially equivalent to a face-to-face assessment. Moreover, a remote PT can direct patients with a wide range of functional ability through a standardized assessment tool. Our findings warrant continued development of telemedicine physical therapy assessment and treatment, thereby improving access to physical therapy for patients whose functional limitations or transportation barriers prevent them from attending an in-person clinical therapy session.
Dr. Palsbo is now with the George Mason University Center for the Study of Chronic Illness and Disability.
This material was based on work supported by the "Effectiveness Measures for Telerehabilitation," Robert Wood Johnson Foundation Health e-Technologies Initiative, RWJF 49143; the National Rehabilitation Hospital, Washington, DC; INTEGRIS Jim Thorpe Rehabilitation Hospital, Oklahoma City, Oklahoma; the Sister Kenny Institute, Minneapolis, Minnesota; the American Physical Therapy Association, Alexandria, Virginia; and the U.S. Department of Education, National Institute on Disability and Rehabilitation Research, Rehabilitation Engineering and Research Center, Center on Telerehabilitation, grant H133E990007-00C.
The authors have declared that no competing interests exist.
[PMID: 11331045]
[PMID: 10761773]
[PMID: 17041824]
[PMID: 10168280]
[PMID: 15273029]
[PMID: 12537905]
Go to TOP
Last Reviewed or Updated
Tuesday, October 9, 2007 2:14 PM