Reliability and Validity of Korean Mann Assessment of Swallowing Ability(K-MASA) A Dissertation Submitted to the Department of Occupational Therapy an

Transcription

1 Reliability and Validity of Korean Mann Assessment of Swallowing Ability(K-MASA) Jong-Chi Oh The Graduate School Yonsei University Department of Occupational Therapy

2 Reliability and Validity of Korean Mann Assessment of Swallowing Ability(K-MASA) A Dissertation Submitted to the Department of Occupational Therapy and the Graduate School of Yonsei University in partial fulfillment of the requirements for the degree of Doctor of Philosophy Jong-Chi Oh December 2013

3 This certifies that the dissertation of Jong-Chi Oh is approved. Thesis Supervisor: Ji-Hyuk Park Min-Ye Jung Eun-Young Yoo Ki-Yeon Chang Teak-Young Lee The Graduate School Yonsei University December 2013

4 Acknowledgements I owe the greatest debt of gratitude to my advisor, Prof. Ji-hyuk Park, who has supported me unreservedly throughout this work from topic selection to completion of this paper. My sincere hope is to become a successful researcher doing justice to his great teachings. My sincere thanks also go to Prof. Min-ye Jung and Prof. Eun-young Yoo who have provided me with valuable guidance and encouragement all along the path to this manuscript to make it a worthy work, and Prof. Ki-yeon Jang and Prof. Teak-young Lee for their insightful comments and encouragement as members of my examination committee. I also wish to express my sincere gratitude to Bum-sun Kwon, Department Head of Dongguk University Ilsan Hospital, who helped me so effectively with subject recruitment and data collection, as well as Prof. Jin-woo Park, Prof. Ho-jun Lee, Team Leader Myoeng-ryoel Choi, and Chi-hoon Oh and Ho-jin Hong at the Department of Occupational Therapy for their persistent efforts in helping me during the data collection. My sincere thanks are also due to Ji-hyun Choi, Vice President of the Korean Academy of Dysphagia Rehabilitation, for her contribution to my research with her competent work of translation and back-translation as well as the translation validation process, Prof. Ji-young Baik, Prof. Tae-hyun Cha, Prof. Hee-soon Woo, Prof. Sung-yoon Won, Won-il Song, Kyoung-Chul Min, Sang-min Seo, Su-jung Lee, Bo-un Choi, Hyo-jun Yang, Sang-gyoon Park, Tu-kyu Kim, and the Academy members my apologies for not

5 naming them one by one who have always supported and motivated me. My heart-felt thanks extend to the other side of the Pacific, to Min-soo Park and Jeong-eun Kim for their participation in the back-translation process. Particular appreciation is expressed to Prof. Yong-kyoung Kim of Kyungdong University Department of Korean Language Education for taking his precious time to polish the Korean expressions in this manuscript. Last but not least, I would like to thank Joo-hyun Lee, Hae-yoon Jung, Young-jo Kim, and all the other graduate students who have supported me with various kinds of practical help and advice for smooth progress of my paper, although they are always devoted to their studies from dawn until dusk at the Graduate School of the Department of Occupational Therapy. Thank you.

6 Table of Contents List of Figures iv List of Tables ⅴ List of Appendices ⅵ Abstract ⅶ Introduction 1 Methods 6 1. Participants Stroke Dysphagic Patients 6 2. Instruments Mann Assessment of Swallowing Ability(MASA) Content Validity Questionnaire Videofluoroscopic swallowing examination(vfse) Videofluoroscopic dysphagia scale(vds) Study Procedure Tranlation Content Validity 19 i

7 Content Validity Verification Committee Swallowing Examination Validity Concurrent Validity Reliability Internal Consistency Test-Retest Reliability Inter-Rater Reliability Statistical Analysis 27 Results Translation Content validity Concurrent validity Reliability Internal Consistency Test-Retest Reliability Inter-Rater Reliability 37 Discussion Translation 39 ii

8 2. Validity Reliability 45 Conclusion 48 References 49 Abstract in Korean 55 Appendix 58 iii

9 List of Figures Figure 1. Study process 17 iv

10 List of Tables Table 1. Inclusion and exclusion criteria 7 Table 2. Demographic characteristics of the participants 8 Table 3. Demographic characteristics of the respondents participating in the context validity investigation 20 Table 4. Demographic characteristics of the patient participating in the testretest reliability 24 Table 5. Demographic characteristics of the patient participating in the interrater reliability 26 Table 6. Tranlation verification results of K-MASA 29 Table 7. Responses to questions on the content validity questionnaire 31 Table 8. Content validity index(cvi) of K-MASA 32 Table 9. Concurrent validity of K-MASA 34 Table 10. Interrater, test-retest reliability of K-MASA 38 v

11 List of Appendices Appendix 1. Mann Assessment of Swallowing Ability(MASA) Instruction Manual 58 Appendix 2. Korean Mann Assessment of Swallowing Ability(K-MASA) Instruction Manual 71 Appendix 3. Clinical Diagnostic Criteria for Dysphagia and Aspiration of MASA 83 Appendix 4. Clinical Diagnostic Criteria for Dysphagia and Aspiration of K-MASA 84 Appendix 5. Question Investigation for the identification of content validity of K-MASA 85 Appendix 6. Videofluoroscopic Dysphagia Scale 96 vi

12 ABSTRACT Reliability and Validity of Korean Mann Assessment of Swallowing Ability Jong-Chi Oh Dept. of Occupational Therapy The Graduate School Yonsei University The aims of this study was to establish the psychometric properties of the Korean Mann Assessment of Swallowing Ability (K-MASA) including validity (content validity, concurrent validity), reliability(test-retest reliability, inter-rater reliability, and internal consistency). MASA was translated and back-translated by 2 bilingual experts. Original MASA and back-translated MASA were compared by native speaker. Understanding of K- vii

13 MASA was verified by 2 swallowing rehabilitation experts and scholar of Korean literature. 10 occupational therapists who have treated dysphagic patients more than 3 years completed the content validity questionnaire about the K-MASA. The results of the content validity suggested that the dysphagia professionals content review supported the content validity of each item of the K-MASA for use in individuals with stroke dysphagic patients. Concurrent validity was assessed using Pearson s correlation coefficient analysis about K-MASA results versus Videofluoroscopic Dysphagia Scale (VDS) which administered to 54 stroke dysphagic patients. Significant correlation was existed between K-MASA and VDS (correlation coefficient= -.509). There were significant correlation between subjective judgement about dysphagia and VDS (correlation coefficient= -.650). Inter-rater reliability and test-retest reliability was calculated with intraclass coefficients (ICCs) and internal consistency was measured on Cronbach s alpha. In testretest reliability and inter-rater reliability of score scale, ICCs (2, 1) were high (.98,.99). Test-retest reliability of subjective judgement about dysphagia and aspiration were high (1.00, 1.00). Inter-rater reliability of subjective judgement about dysphagia and aspiration were also high (.95,.83). For internal consistency, Cronbach s alpha for K-MASA 24 items was.92. In conclusion, the results of present study indicate that the K-MASA holds adequate content validity, reliability, and concurrent validity for measuring swallowing ability of viii

14 Korean stroke dysphagic patients clinically. K-MASA would contribute to more systematic management of swallowing problems and efficient therapeutic service. Key Words : Deglutition, Dysphagia, Screening, Stroke, Swallowing ix

15 Introduction The clinical swallowing examination (CSE) is a method for evaluating whether a subject has dysphagia, which uses various clinical methods such as swallowing difficultyrelated history taking, observation of the anatomical structures of the oral cavity, pharynx, and larynx, sensory and motor functions, behavior, level of consciousness, speechlanguage ability, and trial swallowing (Mann & Lenius, 2008). A CSE is commonly employed as the first test for dysphagia. Performing costly instrumental examinations on all patients suspected of dysphagia may incur unnecessary disadvantages in terms of time and costs. It is therefore useful for suspected acute dysphagia patients to undergo this first screening test with clinical methods to identify high-risk cases of dysphagia or aspiration for secondary test via instrumental examination (Martino, Pron, & Diamant, 2000). The CSE is divided into single- and multiple-item screening methods. The latter combines various factors associated with aspiration. The single-item screening involves the patient swallowing a large volume of water and measuring the swallowing speed (Bours, Spyer, Lemmens, Limburg, & de Wit, 2009; Daniels & Huckabee, 2008). The multiple-item screening involves combined testing of any of the following items to confirm or rule out aspiration or dysphagia: anatomical state of the oral cavity and pharynx, pre- and post-swallowing voice changes, gag reflex, oropharyngeal sensory and motor function, cough reflex, and respiratory function (Daniels & Huckabee, 2008)

16 Because the CSE is performed without using costly testing devices to observe the interior of the body, it has the advantages of being cost-effective, noninvasive, and less time-consuming compared to instrumental examination (Mann & Lenius, 2008). Additionally, the CSE can be used to assess other risk factors that impair the safety of the swallowing function other than aspiration (Westergren, 2006). The Mann Assessment of Swallowing Ability (MASA) is one of the clinical assessment tools capable of performing integrative assessment of such factors. The MASA is a comprehensive clinical assessment tool designed to assess the extent of oropharyngeal dysphagia. It is a standardized clinical assessment tool integrating a variety of aspiration-related items with high reliability and validity. It provides a systematic method that allows clinicians to evaluate patients swallowing ability and serves as a standardized communication tool among medical professionals regarding the degree of functional impairment. The assessment item set of MASA comprises highvalidity items selected from the empirical assessment items, which have been traditionally used in swallowing rehabilitation settings, via literature review related to clinical assessment. These 24 selected items associated with swallowing ability are constructed on a score scale in the order of swallowing phases (Mann, 2002). In the MASA, an operational definition is given to each of its 24 items, and a detailed explanation is given to each task to be used for testing the patient s swallowingrelated performance. The score scheme has a distinctive construct that allows independent assessment of the score scale and total score of all items as well as ordinal risk rating - 2 -

17 regarding dysphagia and aspiration risks. Furthermore, based on the evaluation results of 128 acute stroke patients, patients were grouped according to the level of severity of dysphagia and aspiration and each group was given a specific cut-off score. This score serves as the reference value for decisions regarding the patient s degree of dysphagia (Mann, 2002). The MASA item and total scores can also be used as a basis for follow-up observation of the subject s swallowing ability and inter-subject comparison of swallowing ability. The validity of this test was verified by comparing the mean MASA score with the degree of aspiration and dysphagia of the 128 acute stroke patients as assessed by videofluoroscopic swallowing examination (VFSE) (Antonios et al., 2010; Gonzalez- Fernandez, Sein, & Palmer, 2011), and a high rate of inter-rater concordant response was established with respect to subjective judgment (Mann, 2000). The MASA serves as concurrent validity in the development of a clinical dysphagia assessment tool, not only because it is the first standardized clinical assessment tool but also owing to its advantage of quantifying the degree of swallowing ability (Antonios et al., 2010; Carnaby, Crary, 2013; Crary, Mann, & Groher, 2005; Edmiaston et al., 2010). It is also used in many dysphagia-related studies as a clinical assessment tool to clinically confirm or rule out the subject s dysphagia and to measure the degree of dysphagia (Gonzalez-Fernandez et al., 2011; Hasan, Al-Shimmery, & Taha, 2010). While the use of the CSE as a diagnostic tool for confirming or ruling out aspiration may be limited by its nature of not being as accurate as instrumental examination, it can - 3 -

18 derive information that cannot be obtained from an instrumental examination performed in the laboratory environment in a limited timeframe. Therefore, combining the results of instrumental examination with those of clinical examination may be conducive to establishing a comprehensive and effective treatment plan (Daniels & Huckabee, 2008). In order for the CSE to be able to play such a role, it is crucial to establish a high level of reliability and an adequate level of validity by verifying the correlation between its results and those of instrumental examination, which can detect aspiration with accuracy (Daniels, Anderson, & Willson, 2012). Unlike most CSE tests performed to ascertain the presence or absence of aspiration, the MASA can assess the cumulative variables influencing swallowing from the preoral phase to the pharyngeal stage and has the advantage of evaluating the changes occurring in the subject at any time owing to the score scale of the 24 swallowing-related items (Mann, 2002). Therefore, the MASA is likely to fulfill the function of replacing or complementing VFSE, which can be used only at a limited frequency owing to radiographic exposure. Now that the use of the MASA for stroke patients is standardized, extensive research has been conducted to investigate its applicability to other patients such as those with neurological disease and cancer (Carnaby & Crary, 2013; Gonzalez- Fernandez et al., 2011). This paper presents a Korean version of the MASA (K-MASA), the first standardized CSE test widely used throughout the world, produced through the process of translation and back-translation. The present study aims to verify the reliability and - 4 -

19 validity of K-MASA, thus proving that it is a useful assessment tool that will enable domestic professionals working in swallowing rehabilitation settings to clinically evaluate the state of patients with dysphagia

20 Methods 1. Participants 1.1. Stroke Dysphagic Patients In order to verify the reliability and validity of the K-MASA, the Korean version of the MASA, a clinical tool for the assessment of swallowing ability, 54 stroke dysphagic patients (20 90 years) were recruited using the convenience sampling method among the inpatients at the rehabilitation medicine department of a university hospital located in Goyang-si, Gyunggi-do. Table 1 outlines the inclusion and exclusion criteria for subject selection. Voluntary informed consent was obtained from each of the participants after an explanation of the study purpose, procedure, and possible side effects during the trial. If patients could not draft the consent themselves because of disease or other reasons, their legal guardians provided consent for their participation. The mean subject age was 58.6 ± 15.1 years, and the mean period of illness and average Korean Mini Mental State Examination (MMSE-K) score were 10.9 ± 12.3 months and 25.4 ± 1.5, respectively. All subjects satisfied the inclusion and exclusion criteria presented in Table 1. The subjects demographic characteristics are outlined in Table 2. This study was conducted under the approval of the Yonsei University Wonju Institutional Review Board

21 Table 1. Inclusion and exclusion criteria Inclusion Criteria 1. Stroke confirmed by neurologist through CT or MRI 2. Oropharyngeal dysphagia confirmed by a VFSE 3. Have sufficient cognitive function to follow the instruction(mmse score 24) 4. Show medically stable status 5. Men and women between 20 and 90 years or age Exclusion Criteria 1. Dysphagia of other known disease - 7 -

22 Table 2. Demograhphic characteristics of the participants Number(N) (N= 54) Percent(%) Gender Male Female Age Group Stroke Type Infarction Hemorrhage Lesion Site Rt cortical Lt cortical Brainstem Cerebellum Mixed Mean Duration of Illness(Month) 10.9 ± 12.3 Mean MMSE-K(Score) 24.5 ±

23 2. Instruments In this study, the MASA was used for the Clinical Swallowing Examination (CSE) of stroke dysphagic patients, and VFSE was performed to ensure an objective evaluation of the subjects actual swallowing abilities. The VFSE results were scored on the videofluoroscopic dysphagia scale (VDS), consisting of 14 items with a maximum attainable score of 100. The original MASA in English was translated to Korean and its content validity, which can determine whether the 24 MASA items are suitable for the evaluation of the swallowing ability of dysphagic patients, was established by an expert group via 5-point scale questionnaire

24 2.1. Mann Assessment of Swallowing Ability(MASA) The MASA was developed by Giselle Mann in 2002 as a comprehensive assessment tool for the clinical evaluation of oropharyngeal dysphagia. The MASA consists of 24 clinical items arranged in the order of normal swallowing phases and designed to evaluate the following three key factors related to the subject s swallowing ability: 1) oral motor and sensory elements and preceding skills necessary for swallowing; 2) functional evaluation of swallowing; and 3) diet recommendation and predictive risk rating for possible impairment of swallowing integrity. Each assessment item is provided with the definition of the given item and explanation about the tasks used for eliciting the subject s reaction. Each evaluation takes about minutes for a patient who is moderately impaired. The MASA test requires several objects, such as a tongue depressor, flashlight, gloves, food of different consistencies, and portable water. The MASA enables quantitative measures reflecting the extent of damage regarding each item. The measured score of each item is converted into a weighted five- or 10-point score scale, and the individual scores of the 24 items are summed to yield a final MASA score/200. This score serves as a reference value for follow-up assessment of the changes in the state of each patient or inter-patient differences in the degree of dysphagia. Upon completion of the 24-item evaluation, the observer s subjective judgment (Mann, 2002) is additionally generated as the ordinal risk rating for aspiration or dysphagia

25 According to a study conducted by the primary author of MASA, Cronbach s alpha coefficient for the entire test is The concordance rate between the subjective judgments of two independent observers, as confirmed with kappa statistics, was reported to be 0.82 for dysphagia and 0.75 for aspiration. The accuracy rates of the CSE score scale for dysphagia and aspiration, with the reference value set at 180 points, were 71% and 93% for sensitivity and 72% and 55% for specificity, respectively. The accuracy rates of the subjective judgments for dysphagia and aspiration were 73% and 93% for sensitivity and 89% and 63% for specificity, respectively (Mann, 2002). The original MASA is presented in Appendix

26 2.2. Content Validity Questionnaire The expert group s assessment of the K-MASA was scored on a 5-point Likert scale (very inappropriate, inappropriate, neutral, appropriate, and very appropriate). The questionnaire was designed for the respondents to provide an assessment of appropriate or inappropriate to each test item and to give reasons for considering any inappropriate. The content validity questionnaire is presented in Appendix

27 2.3. Videofluoroscopic Swallowing Examination(VFSE) VFSE is recognized as the gold standard for the evaluation of a subject s overall swallowing ability from the oral phase to the esophageal phase while swallowing food of different consistencies via irradiation of the subject s anatomical structures associated with swallowing (Logemann, 1998). In this study, the degree of a subject s swallowing ability was evaluated using VFSE (Sonialvision, Shimadzu, Japan), which was administered by a rehabilitation physiatrist and a radiologist in the fluoroscopic X-ray room of the radiology department. The subject was asked to sit upright and the evaluator fixed the subject s head with their hands. The distance between the X-ray tube and the subject s head was maintained at 1.5 meters. The subject was asked to swallow 5 ml of 35% w/v liquid barium, 5 ml of Yoplait, and mashed sweet pumpkin, thrice, respectively. The test was stopped when excessive aspiration was observed. Side-angle fluoroscopic images were captured using the Picture Archiving and Communication System (PACS), a digital imaging and communication system used in medicine. The capture rate was 30 frames/s, and the frame measured pixels

28 2.4. Videofluoroscopic Dysphagia Scale(VDS) The VDS evaluates the degree of oropharyngeal dysphagia using VFSE findings (Appendix 6). This scale consists of 14 variables of the oral and pharyngeal phases, which can yield numerical data on the swallowing ability of stroke patients. A score proportional to the aspiration relevance is assigned to each itemized variable ranging from 3 to 13.5 points. The distribution range of the total VDS score is between 0 and 100 points; the more severe the damage to the swallowing ability, the higher the score (Han, Paik, Park, & Kwon, 2008). Among the oral phase items, the evaluator assigns lip sealing, bolus formation, mastication, and tongue-to-palate contact one of the three following possible scores: 0 = normal; 1 = inappropriate; and 2 = incapable. Apraxia, tongue thrust, and piecemeal deglutition are evaluated at four levels: 0 = none; 1 = mild; 2 = moderate; and 3 = severe. Pre-swallow leakage into the pharynx and oral cavity residue are measured at four levels: 0 = none; 1 = <10%; 2 = 10 50%; and 3 = >50%. Oral transit time is also measured. The pharyngeal phase items that the evaluator observes or measures are the triggering of pharyngeal swallowing in the pharyngeal phase, laryngeal elevation, vallecular residue, pyriform sinus residue, coating of the pharyngeal wall after swallowing, repeated swallowing, and pharyngeal transit time. Vallecular and pyriform sinus residue are classified into four levels: 0 = none; 1 = <10% on a two-dimensional surface; 2 = 10 50% of the surface area; and 3 = >50%). Aspiration is evaluated at three

29 levels: 1 = no aspiration; 2 = laryngeal penetration; and 3 = aspiration. The inter-rater reliability of the VDS is reported to be 0.56 (Kim et al., 2012). The VDS is presented in Appendix

30 3. Study Procedure In this study, the MASA developed for CSE in stroke dysphagic patients was translated to Korean, and the validation of the appropriateness of its back-translation as well as its reliability and validity were established according to the following procedure. The first stage consisted of the translation of the MASA, back-translation, and validation of comprehensibility. The second stage included content validity verification of the K-MASA as a dysphagia assessment tool. In the third stage, in order to verify the reliability and validity of the final version of the K-MASA, VFSE and the K-MASA were administered to the dysphagic inpatients between April and July 2013 at a university hospital located in Goyang-si, Gyunggi-do. VFSE was performed by a rehabilitation physiatrist and radiologist and the K-MASA test was administered by two occupational therapists with 7 years of experience in swallowing rehabilitation. To ensure the reliability of the K-MASA test, the researcher lead a 3-hour training session for the therapists on the testing and interpretation skills, and the evaluation skills were sufficiently trained through practice. Fifty-four dysphagic patients participated in the study. The fourth and fifth stages involved the verification of the reliability and validity of the K-MASA, respectively

31 1. Translation and Adaptation of K-MASA Translation Back-translation Verification of translation suitability 2 Bilingual occupational therapists 2 Bilingual occupational therapists Survey for 2 occupational therapists and scholar of Korean literature 2. Verification of Content Validity Content Validity Questionnaire Survey for 10 occupational therapists 3. Swallowing Examination K-MASA, VFSE 54 stroke dysphagic patients 4. Verification of Reliability Internal Consistency Test-Retest Reliability Inter-Rater Reliability 54 stroke dysphagic patients 10 stroke dysphagic patients 19 stroke dysphagic patients 5. Verification of Validity Concurrent validity(vds) 54 stroke dysphagic patients Fig. 1. Study process

32 3.1. Translation The first stage of this study was to translate the English original MASA to Korean and implement its cultural adaptation. In December 2012, the researcher was given the approval from the author of the MASA (Giselle Carnaby) to translate and use it for research purposes. The translation was performed in compliance with the internationally recognized guidelines (Guillemin, Bombardier, & Beaton, 1993) by two bilingual persons educated in occupational therapy without any prior experience of using the MASA on a collaborative basis. The translated MASA was back-translated by two different bilingual persons educated in occupational therapy without any prior experience of using the MASA on a collaborative basis. The back-translation was subjected to comparative check against the original MASA by a native English speaker who was educated in occupational therapy. When discrepancies between the original and back-translation were discovered, the same translation and back-translation process was repeated until the two versions were confirmed to conform entirely. This checked and rechecked preliminary version of the K-MASA was subjected to a practical comprehensibility test by two occupational therapists with MA degrees and at least 5 years of work experience. They corrected and complemented the less-comprehensible portions without any loss or distortion of the intended meanings. The final K-MASA version was completed after the final linguistic verification by a scholar of Korean literature. Appendices 1 and 2 present the full versions of the original MASA and K-MASA, respectively

33 3.2. Content Validity In order to verify the content validity of the K-MASA, 10 professionals, either occupational therapists with at least 3 years of experience of working with stroke dysphagic patients or professors of the occupational therapy department, were recruited using the convenience sampling method. The researcher explained the objective of the study and the administration methods and scoring of the K-MASA for 1 hour and distributed the K-MASA manual and content validity questionnaire (Appendix 5) constructed with a five-point Likert scale, asking the professionals to complete it. The number of responses corresponding to appropriate and very appropriate (out of the options of very inappropriate, inappropriate, neutral, appropriate, and very appropriate) was divided by the total number of responses, which yielded the item content validity index (I-CVI) and the scale s content validity index (S-CVI) (Polit, Beck, & Owen, 2007) Content Validity Verification Committee The committee members demographic characteristics are outlined in Table 3. The male-to-female gender ratio was 7:3. None of the committee members participated in the development of K-MASA. Five were working in a university hospital, two in a general hospital, and three were university professors. The mean age of the committee was 34.2 ± 4.13 years, and their mean work experience in swallowing rehabilitation therapy was 7.5 ± 1.72 years

34 Table 3. Demographic characteristics of the respondents participating in the context validity investigation (N= 10) Number (N) Percent (%) Gender Male 7 70 Female 3 30 Age (Years) Occupation Occupational Therapist 7 70 Professor 3 30 Academic Degree Bachelor Degree 4 40 Master Degree 4 40 Doctorate Degree 2 20 Practice Field University Hospital 5 50 Professional Experience Length (Years) General Hospital 2 20 University

35 3.3. Swallowing Examination Both VFSE, an instrumental screening tool, and the K-MASA, a CSE, were administered to 54 stroke dysphagic patients in order to test the subjects swallowing ability. VFSE was administered after the dysphagic patients satisfying the inclusion criteria of this study had given their consent to participate in the study. To minimize the effect of neurological recovery on the changes in subjects swallowing ability (Mann & Hankey, 2001), the two aforementioned K-MASA-trained occupational therapists administered the K-MASA to the subjects within 24 hours of VFSE administration (Martino et al., 2009). The subjects involved in the test-retest reliability process received the first K-MASA test and received it for the second time from the same therapist within 2 hours (Massey & Jedlicka, 2002). The subjects involved in the inter-rater reliability process received the first K-MASA test from one therapist and received the same test for the second time within 2 hours from the other therapist

36 3.4. Validity Concurrent validity Fifty-four stroke dysphagic patients participated in the verification process of the concurrent validity of the K-MASA. To verify the concurrent validity of the K-MASA, an occupational therapist with over 100 cumulated VFSE analysis hours scored the VFSE findings in accordance with the scoring standard of the VDS(Han et al., 2008) while watching the subjects swallowing images as often as necessary in video and frame by frame Reliability Internal Consistency The internal consistency for the verification of the reliability of the K-MASA was tested on the 54 stroke dysphagic patients. Two occupational therapists with 7 years of work experience in dysphagia rehabilitation determined the Cronbach s alpha correlation coefficient on the basis of the scores of the individual items obtained from the K-MASA test at the department of occupational therapy

37 Test-Retest Reliability Ten stroke dysphagic patients participated in the test-retest reliability of the K- MASA. The first test was performed during the period of internal consistency testing and the retest was performed within 2 hours by the same therapist. The results from both tests were then compared. The subjects mean age was 49.9 ± 14.5 years, and the male-tofemale ratio was 8:2. The demographic characteristics of the subjects are outlined in Table

38 Table 4. Demograhphic characteristics of the patients participating in the testretest reliability (N= 10) Number(N) Percent(%) Gender Male 8 80 Female 2 20 Age Group Stroke Type Infarction 2 20 Hemorrhage 8 80 Lesion Site Rt cortical 2 20 Lt cortical Brainstem 2 20 Cerebellum Mixed 6 60 Mean Duration of Illness(Month) 15.5 ± 8.2 Mean MMSE-K(Score) 25.9 ±

39 Inter-Rater Reliability Nineteen stroke dysphagic patients participated in the inter-rater reliability of the K- MASA. The first test was performed during the period of internal consistency testing and the second test was administered by a different therapist within 2 hours. The results from both tests were then compared. The subjects mean age was 54.4 ± 13.6 years and the male-to-female ratio was 11:8. The demographic characteristics of the subjects are outlined in Table

40 Table 5. Demograhphic characteristics of the patients participating in the interrater reliability (N= 10) Number(N) Percent(%) Gender Male Female Age Group Stroke Type Infarction Hemorrhage Lesion Site Rt cortical Lt cortical Brainstem Cerebellum Mixed Mean Duration of Illness(Month) 3.3 ± 1.6 Mean MMSE-K(Score) 25.2 ±

41 4. Statistical Analysis SPSS 18.0 (SPSS Inc. Chicago, IL, USA) was used for statistical analyses. The internal consistency of the K-MASA was verified using Cronbach s alpha, and the correlation between K-MASA and VDS for the verification of the concurrent validity of the K-MASA was calculated using Pearson s correlation coefficient. Both test-retest and inter-rater reliabilities were verified using the intraclass correlation coefficient (ICCs). All statistical analyses were considered statistically significant when p <

42 Results 1. Translation The comprehensibility test of the K-MASA generated through the process of translation and back-translation led to changes of inadequate terms and phrases and adaptation of examples to Korean situations within the scope of keeping the original intention intact. Of the 24 items constituting the score scale of the K-MASA, for the items in which tongue coordination is required, it was necessary to find the words in the Korean language that solicit the same part of the tongue instead of translating the example sentences. Thus, the sentence Take Tim to tea was replaced with a Korean sentence requiring a successive tongue-tip movement, Tazani tazorl tago dallin da (Tarzan is riding an ostrich), and the sentence Can you keep Katie clean? was replaced with a Korean sentence requiring a successive tongue-base movement Camera case ga quae qeutada. (Camera case is clean). Apart from such adaptations to evoke the same effects as the original wordings, all single-syllable phonemes were only transliterated. All K-MASA items were grammatically checked by a Professor of the Department of Korean Literature from a university of Korean language education; the results are listed in Table

43 Table 6. Tranlation verification results of K-MASA Item Before modification After modification 1.Alertness 2. Cooperation 6. Dysphasia 10. Lip Seal 11. Tongue Movement 12. Tongue Strength Appendix 4. for aspiration 각성상태또는각성수준이오르락내리락함변동이심한협조- 다양한동시적자극에산만해짐일반적인언어기능손상이다양한언어양식에서나타남소리반복과말과제동안양입술기능을관찰한다. 혀로입의양쪽모서리를건드리게한후, 양가쪽움직임을번갈아반복시킨다. 명백한한쪽의약화음식덩이의기도진입의경미한위험성을암시음식덩이의기도진입의중등도의위험성을암시기도위협의상당한위험성을암시 각성수준이높다낮다함일관되지못한협조- 다양하고동시적인자극에산만해짐언어기능손상이다양하게나타남소리를반복하게하고언어과제를수행하는동안양입술기능을관찰한다. 혀로입의양쪽모서리를건드리게한후, 이를반복하게한다. 확실한한쪽의약화음식덩이가기도로진입할수있는위험성이경미함을암시음식덩이가기도로진입할수있는위험성이중등도임을암시기도진입의위험성이매우높음을암시

44 2. Content Validity Table 7 outlines the results of the responses of the 10 occupational therapy professionals, including therapists with experience ( 3 years) in treating stroke dysphagic patients, to the appropriateness of the 24 CSE items of the K-MASA. The content validity of the 24 items was verified as CSE items in the item content validity index (I-CVI) and scale s content validity index (S-CVI). As shown in Table 8, the former distributed within the range of and the latter was

45 Table 7. Responses to questions on the content validity questionnaire (N= 10) Item Very good Good Neutral Bad Very bad N(%) N(%) N(%) N(%) N(%) 1. Alertness 6(60) 2(20) 2(20) Cooperation 6(60) 3(30) 1(10) Auditory comprehension 4(40) 5(50) 1(10) Respiration 6(60) 2(20) 2(20) Respiratory rate(for swallow) 6(60) 3(30) 1(10) Dysphasia 5(50) 3(30) 2(20) Dyspraxia 4(40) 4(40) 2(20) Dysarthria 5(50) 4(40) 1(10) Saliva 7(70) 3(30) Lip seal 6(60) 2(20) 2(20) Tongue movement 7(70) 3(30) Tongue strength 6(60) 4(40) Tongue coordination 8(80) 2(20) Oral preparation 7(70) 2(20) 1(10) Gag 6(60) 3(30) 1(10) Palate 6(60) 2(20) 2(20) Bolus clearance 7(70) 3(30) Oral transit 5(50) 4(40) 1(10) Cough reflex 7(70) 3(30) Voluntary cough 6(60) 4(40) Voice 4(40) 5(50) 1(10) Trache 8(80) 2(20) Pharyngeal stage 7(70) 1(10) 2(20) Pharyngeal response 4(40) 6(60)

46 Table 8. Content Validity Index(CVI) of K-MASA (N= 10) Item Item Content Validity Index 1. Alertness Cooperation Auditory comprehension Respiration Respiratory rate(for swallow) Dysphasia Dyspraxia Dysarthria Saliva Lip seal Tongue movement Tongue strength Tongue coordination Oral preparation Gag Palate Bolus clearance Oral transit Cough reflex Voluntary cough Voice Trache Pharyngeal stage Pharyngeal response 1.00 Scale s Content Validity Index

47 3. Concurrent validity Pearson s correlation analysis was performed to verify the correlation between the findings of the K-MASA test administered to the 54 stroke dysphagic patients and those of the VFSE-derived VDS(Table 9). There was a significant negative correlation (r = ; p = 0.000) between the total scores of the K-MASA and VDS, as was between the VDS results and the total scores of the K-MASA items directly related to swallowing itself, namely, the oral preparatory phase, bolus clearance, oral transit, pharyngeal phase, and pharyngeal response (r = ; p = 0.000). A significant negative correlation (r = ; p = 0.000) was also verified between the VDS results and those of the subjective judgment about the presence dysphagia in the subjects

48 Table 9. Concurrent validity of K-MASA (N= 54) Correlation coefficient p-value K-MASA total score VDS score ** Swallowing related 5 items score VDS score ** Subjective judgement(dysphagia) VDS score ** Subjective judgement(aspiration) VDS score ** **p<

49 4. Reliability 4.1. Internal Consistency The internal consistency of the items was verified by Cronbach s alpha. The internal consistency of the items used for the verification of the reliability of the K-MASA as test items evaluating swallowing ability using the total scores of the K-MASA test administered to the 54 stroke dysphagic patients was verified to be

50 4.2. Test-Retest Reliability The test-retest reliability of the K-MASA test administered to 10 stroke dysphagic patients were calculated with ICCs. The test-retest reliability of the 24 K-MASA items was verified to be 0.98, and that of the subjective judgments about dysphagia and aspiration were both Table 10 shows the results of the verification of the ICCs for each item that was performed in order to verify their test-retest reliability. The test-retest reliability of each of the 24 items constructed on the score scale was distributed between 0.76 and

51 4.3. Inter-Rater Reliability The inter-rater reliability of the K-MASA administered to 19 stroke dysphagic patients was calculated with ICCs. The inter-rater reliability of the 24 K-MASA items was verified to be 0.99, and that of the subjective judgments about dysphagia and aspiration were verified to be 0.95 and 0.83, respectively. Table 10 shows the results of the verification of the ICCs for each item. The inter-rater reliability of each of the 24 was distributed between 0.70 and

52 Table 10. Interrater, test-retest reliability of K-MASA Item Interrater reliability (N= 19) Test-retest reliability (N= 10) 1. Alertness Cooperation Auditory comprehension Respiration Respiratory rate(for swallow) Dysphasia Dyspraxia Dysarthria Saliva Lip seal Tongue movement Tongue strength Tongue coordination Oral preparation Gag Palate Bolus clearance Oral transit Cough reflex Voluntary cough Voice Trache Pharyngeal stage Pharyngeal response K-MASA 24 items

53 Discussion 1. Translation In this study, internationally recognized guidelines (Guillemin et al., 1993) were applied to the translation method for completing the Korean version of the MASA. Evaluation of the qualitative aspects of a translation while taking into account the cultural context is crucial when translating questions into a language belonging to a different cultural zone (Peters & Passchier, 2006). To ensure this, the final version of the K-MASA was completed by translation, back-translation, a check against the original version, and verification by a Korean literature scholar

54 2. Validity To verify the content validity of the K-MASA, 10 occupational therapy professionals with long-standing ( 3 years) involvement in stroke dysphagia treatment were convened to test the necessity of each of the 24 items as a clinical tool for testing swallowing ability. The validity of all 24 CSE items of the K-MASA as a clinical testing tool were verified. The creator of the MASA verified its content validity by verifying whether it is composed with items suitable for clinical testing of swallowing ability by performing a related literature review (Mann, 2002). In this study, the content validity of the K-MASA items was verified by involving a dysphagia rehabilitation expert group via questionnaire survey on content validity by item and by computing the CVI from the survey results. The item-level index (item content validity index [I-CVI]) was verified to range between 0.80 and 1.00, and the scale-level index (scale s content validity index [S-CVI]), was verified to be In general, content validity is considered high if the item level exceeds 0.78 and the scale-level index exceeds 0.90 (Polit et al., 2007). Therefore, the K-MASA used in this study is considered to have a high content validity as a clinical swallowing test tool. VFSE was used for the verification of the concurrent validity of the K-MASA administered to the subjects of this study. As another instrumental examination to objectively evaluate the swallowing ability of the subject, fiberoptic endoscopic examination of swallowing (FEES) may also be used, but this method has a narrow test

55 angle and a limited possibility of evaluating the function of swallowing-related structures such as the upper esophageal sphincter or laryngeal elevation (Kim & Han, 2006). In this study, VFSE was therefore chosen as a standard diagnostic method because it can effectively evaluate not only dysphagia or absence thereof but also the structural dysfunction and movement of oral cavity, pharynx, and esophagus (Lee et al., 2009). A significant negative correlation (r = /-0.546) was found between the total K- MASA score scale of items from 1 to 24 and the VDS as well as between the five items directly associated with swallowing (oral preparatory phase, bolus clearance, oral transit, pharyngeal phase, and pharyngeal response). A significant negative correlation (r = /-0.642) was also verified between the raters subjective judgments about the degree of dysphagia and the VDS results as well as the raters subjective judgments about the degree of aspiration and the VDS results. Such negative correlation is presumably attributable to the fact that the higher the MASA score, the higher the swallowing integrity (Mann, 2002) while the higher the VDS score, the higher the severity of dysphagia (Han et al., 2008). The reason for the medium-level correlation between the K-MASA and VFSEderived VDS is assumed to be attributable to the different methods used for observing the subjects swallowing ability between the K-MASA and VFSE. While VFSE can observe only the functions directly associated with swallowing itself, the K-MASA evaluates additional items influencing swallowing with relatively low direct relevance to swallowing itself, such as the consciousness level. Considering this inter-item difference

56 in direct relevance to swallowing, the correlation between the five K-MASA items directly related to swallowing and the VDS was separately tested and verified to be higher than the correlation between the total score K-MASA and the VDS. The swallowing ability assessed through VFSE is not always identical to that observed during actual meal taking. A study by Robbins et al. (cited in Perry & Love, 2001) revealed that the subjects who showed swallowing apraxia during VFSE could actually take meals in real daily situations. Given that no clinical screening or swallowing test can have the same sensitivity or specificity for abnormal swallowing variables or aspiration as VFSE, it is difficult to present any definite explanation for the discrepancy between instrumental and clinical examinations (Perry & Love, 2001). Compared to CSE, VFSE has the advantages of evaluating swallowing mechanisms and tracking the effects of compensatory techniques during the evaluation. However, the information provided is dependent on operating procedures and the training of assessors to interprete results. As for limitations of VFSE, it can be noted that its use depends on patients willingness to undergo an instrumental examination, the availability of the instrument itself depending on the situation of healthcare providers, and that it has a limited possibility to be generally accepted in patients actual living situations. Owing to patient radiographic exposure during the examination, its repeat frequency is limited. The fact that continuous attention should be paid to the patient s posture to ensure an efficient examination implies that patients confirmed to be safe in the VFSE test may still be at risk of aspiration in real-life situations (Ramsey, Smithard, & Kalra, 2003). Although

57 aspiration is not detected during the VFSE test in certain patients, they can still aspirate while eating in real situations; therefore, their adverse prognostic effects should be considered as poor as those of the patients who showed aspiration during the VFSE test (Smithard et al., 1998). Furthermore, silent aspiration may have caused the difference between the VFSE test results and the K-MASA results in this study. While aspiration generally elicits symptoms such as cough in normal conditions, dysphagic patients have a high rate of silent aspiration, i.e., they are likely to show no evident symptoms. Silent aspiration was reported to be present in 15 39% of total stroke patients (Leder, Suiter, & Green, 2011) and 7 73% of dysphagic patients concomitant with aspiration (Ramsey et al., 2003). With CSE alone, it is often difficult to judge the presence of aspiration in cases of silent aspiration because its estimation of swallowing physiology is limited to observing the apparent reactions or conditions of the patients. Likewise, in a study by Gonzalez- Fernandez (2011) in which the MASA was administered to dysphagic patients with various diseases other than stroke, no substantial difference was found in the total MASA score between the aspiration and non-aspiration groups confirmed as such in the VFSE test. Despite such inter-tool differences, a statistically significant medium-level correlation was demonstrated between the VDS and total K-MASA score as well as five individual swallowing-related K-MASA items administered to stroke dysphagic patients, which implies that the K-MASA can indirectly reflect a patients VFSE-confirmed swallowing ability. Furthermore, the K-MASA can evaluate various swallowing-related

58 factors undetectable with VFSE. Therefore, the K-MASA may be utilized between the initial instrumental examination, such as VFSE testing, to determine the cause and mechanism of each patient s dysphagia and the follow-up test, for a continuous regular clinical follow-up assessment of the patient s condition by applying it at each treatment or at an interval of once or twice a week (Mann, 2002). During this period, if patient improvement or exacerbation is clinically observed, instrumental examination may be additionally administered in order to confirm the condition or determine the cause by observing the swallowing mechanism. In previous MASA-related studies, the VFSE test results, which were categorized depending on the presence or absence of aspiration (aspiration and non-aspiration) or degree of dysphagia (normal, mild, moderate, and severe), served as the basis for a criterion reliability test by comparing them with the MASA score scale or subjective scale (Antonios et al., 2010; Mann, 2002). In this study, the concurrent validity tests of the results of the K-MASA score scale and subjective scale were conducted using the 100- point VDS scale scored on the basis of the VFSE test commonly used for verifying the condition of dysphagic patients. This test procedure yielded a significant linear correlation between the K-MASA score scale and subjective scale results and those of the VFSE test, which is recognized as the gold standard for the evaluation of dysphagia. This study is significant in that it is the first to verify the correlation between the VDS, the VFSE-derived score scale, and the MASA, thus proving that the MASA can be used as an

59 effective follow-up tool for the objective observation of the dysphagic condition of patients confirmed by the VFSE. 3. Reliability The internal consistency marks the reliability of an item in terms of its relevance with other items. The higher the inter-item correlation, the higher the coefficient of the internal consistency reliability, which is indicative of the degree of homogeneity of a test, with all its subscales and items testing the same characteristic or ability (Lee, 2012). The verification of the reliability of the internal consistency of the 24 K-MASA items administered to the 54 stroke dysphagic patients using Cronbach s alpha was Compared to total-scale internal consistency of 0.88 as revealed in the study of the creator of the MASA (Mann, 2002) and 0.94 for the Modified MASA as reported in another study (Antonios et al., 2010), the total-scale internal consistency of the K-MASA of 0.92 as confirmed in this study is similar to those confirmed in previous studies. Therefore, the 24 items constituting the K-MASA are considered to be suitable for evaluating the subjects swallowing abilities. In this study, the test-retest reliability and inter-rater reliability were confirmed using ICCs. The test-retest reliability of the original MASA is not presented. The test-retest