Journal of the Ergonomics Society of Korea Vol. 31, No. 2 pp.271-279, April 2012 http://dx.doi.org/10.5143/jesk.2012.31.2.271 Classification of Three Different Emotion by Physiological Parameters Eun-Hye Jang 1, Byoung-Jun Park 1, Sang-Hyeob Kim 1, Jin-Hun Sohn 2 1 BT Convergence Technology Research Department, Electronics and Telecommunications Research Institute, Daejeon, 305-700 2 Department of Psychology/Brain Research Institute, Chungnam National University, Daejeon, 305-765 ABSTRACT Objective: This study classified three different emotional states(boredom, pain, and surprise) using physiological signals. Background: Emotion recognition studies have tried to recognize human emotion by using physiological signals. It is important for emotion recognition to apply on human-computer interaction system for emotion detection. Method: 122 college students participated in this experiment. Three different emotional stimuli were presented to participants and physiological signals, i.e., EDA(Electrodermal Activity), SKT(Skin Temperature), PPG(Photoplethysmogram), and ECG (Electrocardiogram) were measured for 1 minute as baseline and for 1~1.5 minutes during emotional state. The obtained signals were analyzed for 30 seconds from the baseline and the emotional state and 27 features were extracted from these signals. Statistical analysis for emotion classification were done by DFA(discriminant function analysis) (SPSS 15.0) by using the difference values subtracting baseline values from the emotional state. Results: The result showed that physiological responses during emotional states were significantly differed as compared to during baseline. Also, an accuracy rate of emotion classification was 84.7%. Conclusion: Our study have identified that emotions were classified by various physiological signals. However, future study is needed to obtain additional signals from other modalities such as facial expression, face temperature, or voice to improve classification rate and to examine the stability and reliability of this result compare with accuracy of emotion classification using other algorithms. Application: This could help emotion recognition studies lead to better chance to recognize various human emotions by using physiological signals as well as is able to be applied on human-computer interaction system for emotion recognition. Also, it can be useful in developing an emotion theory, or profiling emotion-specific physiological responses as well as establishing the basis for emotion recognition system in human-computer interaction. Keywords: Emotion, Physiological signals, Classification, Discriminant analysis 1. Introduction 정서란내적또는외적자극에의한감정의변화와생리적활성도의변화로표출되는정신적상태를의미하는데, 이러 한내 / 외적자극에의한변화는자율신경계의조절에의해발생하는생리적반응의변화를수반한다. 따라서정서상태의변화는심장박동, 체온변화, 피부전도도의변화등과같은생리적변화를반영하는신호들의측정을통하여인식이가능하다. 정서인식분야에서정서를분석하기위한연구는 Corresponding Author: Jin-Hun Sohn. Department of Psychology/Brain Research Institute, Chungnam National University, Daejeon, 305-765. E-mail: jhsohn@cnu.ac.kr Copyright@2012 by Ergonomics Society of Korea(pISSN:1229-1684 eissn:2093-8462). All right reserved. cc This is an open-access article distributed under the terms of the Creative Commons Attribution Non-Commercial License(http://creativecommons.org/licenses/by-nc/3.0/), which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.
272 Eun Hye Jang Byoung Jun Park Sang Hyeob Kim Jin Hun Sohn JESK 표현된정서를토대로한음성, 얼굴표정, 동작, 제스처, 그리고언어등에서부터시작되었으나 (Choi and Woo, 2005), 표현된정서와실제정서와의불일치성, 사람에따른표현의차이로인한판단의불명확성등의한계점들이지적되면서, 다양한생체신호를이용하여인간의정서를인식하는방법에대하여활발히연구되고있다 (Tefas et al., 2001). 생체신호를이용한정서인식은비침습적방법으로센서에의해비교적간단하게신호를획득할수있고사회, 문화적차이에덜민감하다는장점이있다 (Drummond and Quah, 2001). 또한인간의정서상태와생체반응은강한상관을가진것으로알려져있다. 대부분의정서연구에서는기쁨, 슬픔, 분노, 공포, 혐오와놀람의기본정서와관련된생체반응을확인하고이들을구분하고자하였다 (Ax, 1953; Boiten, 1996; Shinha and Parsons, 1996; Alaoui-Ismaili et al., 1997; Kanade et al., 2000; Palomba et al., 2000; Picard et al., 2001; Stemmler, 2004; Stephens et al., 2010). Kreibig(2010) 는 134개의선행연구들을검토하여기본정서와생리반응과의관련성을보고하였으나, ' 놀람 ' 정서와관련된생리반응은논문수가적어어떠한결론도내리지못하였고, 기본정서이외의통증, 무료함등의기타정서와관련된생리반응을보고한연구도많지않다. 통증, 무료함등을포함한정서연구는대부분호흡패턴 (de Melo et al., 2010) 또는얼굴표정과같은단일지표에의한결과이다. 그러나통증과같은기타정서또한피부전도반응, 근전도반응, 심박율, 코티졸반응과도관련이있기때문에다양한생체신호를활용하여정서를구분하는작업이필요하다 (Flor et al., 2002; Jolliffe and Nicholas, 2004). 생체반응을측정하기위한대부분의방법은뇌, 심장, 근육, 그리고피부에서발생하는전기신호를기록하는것으로, 심장의활성화를측정하는심전도 (Electrocardiogram; ECG), 근육의활동을측정하는근전도 (Electromyogram; EMG), 피부표면의땀선활동을측정하는피부전도도 (Electrodermal Activity; EDA) 등이대표적이다. 정서인식연구의대부분은피부전도반응과심박율을핵심적인생리지표로사용하고있으나, 정서에의해조절되는생리신호는다양하기때문에이들지표를정서분류의대표적지표로사용하는것은문제가될수있다. 따라서보다최근에는심전도, 피부전도도, 호흡등의다양한생체신호에서세부지표들을추출하여정서를구분하는연구들이보고되고있다 (Picard et al., 2001; Peter and Herbon, 2006; Broek et al., 2009; Chanel et al., 2009; Kreibig, 2010). 최근 HCI 연구자들의생체신호에기반한정서인식결과는대부분이사진이나영화필름을보고음악을듣거나또는비디오게임을하는동안유도된정서를의도적으로표현하 는실험상황에서획득된것으로, 기본정서나쾌 / 불쾌또는각성수준에따른정서분류를시도하여 70% 이상의정서인식률을보고하고있다. 이들연구는다수의생체신호를이용하여 SFFS(Sequential Floating Forward Search), FP (Fisher Projection), DFA(Discriminant Function Analysis), KNN(k-Nearest Neighbor algorithm), MBP(Marquardt Backpropagation), BNT(Bayesian Networks), LDF (Linear Discrimination Function), SVM(Support Vector Machines), MLP(Multilayer Perceptron Network), RT (Regression Tree) 등의다양한알고리즘으로정서를분류하고있으나, 현재까지어떠한알고리즘이가장좋은결과를제시하는지에대해서는아직풀리지않은과제이기때문에 (Arroyo-Palacios & Romano, 2008), 보다정확한정서분류를위한방법론및알고리즘개발을위한연구들이계속수행되고있다 (Healy, 2000; Picard, Vyzas, & Healey, 2001; Nasoz, Alvarez, Lisetti, Finkelstein, 2003; Alpaydin, 2004; Haag, Goronzy, Schaich & Williams, 2004; Wagner, Kim, & Andre, 2005). 본연구에서는기존의선행연구에서다루지않은통증, 놀람과무료함의세가지정서를구분하고자하였다. 이들세정서는인간이실생활에서빈번하게경험하는정서들이지만, 이들정서에대한정서 -특정적자율신경계반응은거의보고된바없다. 통증 (Pain) 은정서의여부에대하여논란이있으나, 신체적, 생리적그리고심리적장애를동반하는개인적이고사적인통감으로 (Mannheimer and Lampe, 1984), 조직이손상되거나상처가발생하였음을알려주는역할뿐아니라 (Sherrington, 1906), 인체가손상받지않게보호하려는반응형태를포함하는정서로, 유용한보호차원의신호이다 (Sternbach, 1978). 국제통증협회 (International Association for the Study of Pain) 의정의에따르면, 실제적혹은잠재적조직손상에수반되는불유쾌한감각및정서적경험으로 (Bowsher, 1990; Merskey and Bogduk, 1994), 전적으로주관적인개인경험이며인격, 기대, 암시, 과거통증경험과같은개인의심리적특성이나사회문화적환경등여러요인에의해크게영향을받는다 (Lee and Bai, 2005). 놀람 (Surprise) 은예기치못한사건을경험하였을때, 긍정적이든부정적이든새롭거나익숙하지않은자극에대해나타나는갑작스런정서적상태로서, 중성, 쾌또는불쾌의정서가 (Valence) 모두를가질수있다 (Coon, 2004). 자연스럽고무의식적인놀람은주로아주짧은시간동안에만표현되며, 공포, 기쁨또는당혹스러움과같은정서뒤에즉시뒤따를수도있다. 반면매우강하거나오랜동안지속되는놀람은충격을고려될수있다. 본연구에서는 ' 깜짝놀람 (Startle)' 에가까운정서로한정하여부적정서가를가지는
Vol. 31, No. 2. 2012. 4. 30 Classification of Three Different Emotion by Physiological Parameters 273 놀람정서를유발하였다. 무료함 (Boring) 은단조롭고지겹거나부족한자극을지각할때느껴지는정서로 (Fisher, 1993), 개인이현재활동에대해흥미가부족하거나집중하는데어려움을느낄때에도나타나는불쾌하고일시적인정서상태며, 인지적주의과정과관련된정서적경험으로부적정서에속한다 (Leary et al., 1996). 무료함은원하는일부활동에방해를받을때, 원하지않는활동에관여하도록강요받을때, 또는불확실한이유로단순히일부활동이나상황에계속관여할수없게될때와같이주의를기울이는데있어문제를가진다 (Cheyne et al., 2006). 또한본연구에서는이들정서를분류하기위하여판별분석 (Discriminant Function Analysis; DFA) 을이용하였다. 판별분석은선형판별함수를사용하여데이터신호를분류하는통계적방법으로 (Nicol, 1999), 6가지이상의정서분류에서 80% 이상의인식률을가지는것으로선행연구들을통해보고된바있다 (Healey, 2000; Picard, 2001; Nasoz et al., 2003). 2. Method 만 19~25 세의남녀대학생 122 명이본연구에참여하였다 ( 남 55명, 연령 22.9±1.8세 ; 여 67명, 연령 21.7±2.3 세 ). 사용된자극은각정서를유발하기위한 1~1.5 분길이의시청각동영상으로 (Figure 1), 통증유발자극은 '+' 화면에지압기와커프를사용한압박, 놀람유발자극은절규장면과소리, 무료함유발자극은 '+' 화면에 1-10 까지의숫자를반복하는소리로구성되었다. 이들자극은정서를유발하기에적합한자극인지를검증하기위하여예비실험을통해타당성과효과성이검증되었다. 타당성은실험자가의도한정서와실제실험참여자에게유발된정서와의일치성, 효과성은실험참여자가경험한정서의강도를의미한다. 이들은각각통증자극 97.3% 의타당성과 4.96±1.34 의효과성, 놀 람자극 94.1% 의타당성, 6.12±1.14 의효과성, 그리고무료함자극 86.0% 의타당성, 5.23±1.36의효과성을가졌고평균 92.5%(100% 만점 ) 의타당성과 5.43 점 (7점만점 ) 의효과성을보이는것으로나타났다. 실험참여자가실험실에입실하고충분한안정기를거치는동안, 실험자는실험에대한전반적인소개를하고생체신호측정을위하여실험참여자의신체에전극을부착한다. 실험이시작되면 1분동안의안정상태를측정한후, 하나의정서자극이제시되는동안생체신호를측정한다. 사용된생체신호지표는피부전도도, 피부온도 (Skin Temperature; SKT), 혈류맥파 (Photoplethysmogram; PPG), 심전도이었다. 자극제시가끝나면자극에대한평가를실시하고 1분이상의충분한휴식시간을가졌다. 이과정은각정서유발을위해총 3번반복되었고, 정서자극은실험참여자마다무선적으로제시되었다. 모든생체신호는 MP150(Biopac Systems Inc., USA) 을이용하여 256Hz 샘플링주파수로기록되었고 ( 예외로, 심전도만 1,000Hz 샘플링주파수로기록됨 ), 측정된신호는 Matlab 2009b(Mathworks, USA) 로구현한소프트웨어로분석하였다. 생체신호는자극제시전측정한안정상태 30 초와실험자가정서를느꼈다고자기보고한장면에해당하는정서상태 30초동안을분석하였고, 총 27개의특성치를추출하였다. 각생체신호에서추출된특성치는 Table 1과같다. Table 1. The features extracted from physiological signals Physiological signal EDA SKT PPG ECG SCL, NSCR, mean SCR Feature mean SKT, maximum SKT, sum of negative SKT, sum of positive SKT mean PPG Time domain Frequency domain mean RRI, std RR, mean HR, RMSSD, NN50, pnn50, SD1, SD2, CSI, CVI LF, HF, nlf, nhf, LF/HF ratio Figure 1. The example of emotional stimuli 피부전도도 (EDA) 에서는 100Hz 로다운샘플링후, 평균피부전도도인피부전도수준 (SCL), SCR 의수 (NSCR) 와 SCR의평균 (mean SCR) 을특성치로추출하였다. 피부온도 (SKT) 에서는 30초동안의평균피부온도 mean SKT, 피부온도의최대값 maximum SKT, 감소하는구간의피부온도 sum of negative SKT와증가하는피부온도의합 sum of positive SKT 를추출하였다. 혈류맥파 (PPG) 에서추출된 mean PPG는평균혈류맥파를계산한값이다.
274 Eun Hye Jang Byoung Jun Park Sang Hyeob Kim Jin Hun Sohn JESK 심전도 (ECG) 에서추출된 mean RRI는심전도신호로부터 R피크와그다음 R피크간의평균시간, std RR은 RR 간격의표준편차, mean HR은평균심박율, RMSSD 는연속적인 NN 간격 (RRI) 의평균제곱제곱근, NN50은 50msec 보다긴차이를보이는연속적인 NN 간격의개수, pnn50 은전체 RRI를 NN50로나눈백분율을의미한다. SD1, SD2는각각단기간의심박변이도 (Short Term HRV) 와장기간의심박변이도 (Long Term HRV) 성분을반영한다. CSI(Cardiac Sympathetic Index) 와 CVI(Cardiac Vagal Index) 는각각교감, 부교감신경활동을반영하는지표로, 심장교감활동인덱스 CSI는 4SD2/4SD1, 미주신경활동인덱스 CVI는 log10(4sd1*4sd2) 에의해추출되었다. 주파수영역분석법을통하여 LF, HF, nlf, nhf와 LF/HF ratio 가추출되었는데, LF는 0.04~0.15Hz의주파수범위, HF는 0.15~0.4Hz 의주파수범위에서의적분치, nlf, nhf는각각정규화된성분이다. LF/HF ratio 는교감신경의활동을정량적으로나타내는 LF와미주신경의활동을반영하는 HF 간의비를나타난다. 세정서를구분하기위하여이들추출된 23개의특성치는분석된각정서상태 30초동안의신호값에서안정상태 30초동안의신호값을뺀차이값으로판별분석에적용되었다. Fisher 에의해고안된판별분석 (Discriminant Function Analysis; DFA) 은선형판별함수 (Linear Discriminant Analysis; LDA) 라고도하며, 주성분분석 (Principal Com- ponent Analysis, PCA) 과더불어대표적인특징벡터차원의축소기법중의하나이다. 주성분분석이데이터를최적으로표현하는입장에서데이터를축소하여각차원의상관관계를줄이는것이라면, 판별분석은데이터를최적으로분류하기위하여데이터에대한특징벡터의차원을축소하는방법으로집단간분산과집단내분산의비율을최대화시키는선형변환방식이다 (Duda et al., 2000). 즉, 집단간의편차는최대로, 집단내편차는최소로하여데이터를쉽게나눌수있고집단또한쉽게분리할수있도록한다. 3. Results 3.1 Difference of physiological responses among three different emotions SPSS 15.0 버전의통계분석프로그램의대응표본 T검증 (paired T-test) 을이용하여 23개특성치에대한각각의안정상태와정서상태간평균차이를검증한결과는 Table 2와같다. 무료함정서의경우, SCL, NSCR, mean SCR, sum of negative SKT, mean RRI, std RR, mean HR, SD2 과 CVI 에서안정상태와정서상태간의유의한차이가있었다. 통증정서에서는 SCL, NSCR, mean SCR, sum of Table 2. The results of difference between baseline and emotional states using paired t-test Emotions Boredom Pain Surprise Physiological parameters t-score p-value t-score p-value t-score p-value SCL 2.59 *.012 5.53 ***.000 14.36 ***.000 NSCR 3.55 ***.001 11.64 ***.000 10.75 ***.000 Mean SCR 2.68 **.009 8.45 ***.000 7.45 ***.000 Mean SKT 0.20.839-1.05.296 2.04 *.045 Maximum SKT 0.02.987-1.52.131 1.46.148 Sum of negative SKT -2.49 *.015-9.93 ***.000-4.62 ***.000 Sum of positive SKT -1.75.085-5.86 ***.000-4.84 ***.000 Mean PPG 0.93.355 2.66 **.009-4.64 ***.000 Mean RRI -3.11 **.002-0.44.659-4.29 ***.000 std RR 2.00 *.049 2.97 **.004 5.43 ***.000 Mean HR 3.00 **.004 0.93.355 3.32 **.001 RMSSD 1.31.194 3.21 **.002 3.45 **.001 NN50-0.16.875 4.19 ***.000 5.95 ***.000 pnn50-0.42.675 4.10 ***.000 4.72 ***.000 SD1 1.11.270 3.09 **.003 3.68 ***.000
Vol. 31, No. 2. 2012. 4. 30 Classification of Three Different Emotion by Physiological Parameters 275 Table 2. The results of difference between baseline and emotional states using paired t-test (Continued) Emotions Boredom Pain Surprise Physiological parameters t-score p-value t-score p-value t-score p-value SD2 2.07 *.041 2.71 **.008 5.73 ***.000 CSI 0.65.519-1.30.196 5.56 ***.000 CVI 1.68.097 4.10 ***.000 9.66 ***.000 LF 1.48.142 2.78 **.007 1.49.140 HF -0.28.780 1.80.075 1.63.107 nlf 0.03.973 0.82.414 0.85.397 nhf -0.03.973-0.82.414-0.85.397 LF/HF ratio 0.66.512-0.08.934-0.55.581 * p <.05, ** p <.01, *** p <.001 negative SKT, sum of positive SKT, mean PPG, std RR, RMSSD, NN50, pnn50, SD1, SD2, CVI와 LF에서유의한차이를보였고, 놀람정서에서는 SCL, NSCR, mean SCR, mean SKT, sum of negative SKT, sum of positive SKT, mean PPG, std RR, mean HR, RMSSD, NN50, pnn50, SD1, SD2, CSI 와 CVI 에서통계적으로유의한차이를보였다. 이러한변화는정서상태동안의생리반응이안정상태와는다르다는것을의미한다. 또한정서간생리반응의차이를확인하기위하여각특성치의정서상태에서안정상태를뺀차이값을이용하여일원분산분석 (one-way ANOVA) 을실시하였다. 그결과, SCL, mean SCR, mean SKT, mean HR, mean PPG에서정서간에유의한차이가있는것으로나타났다 (Table 3). 이들정서간차이를구체적으로확인하기위하여 LSD 사후검증을실시하였다. SCL, mean SCR과 mean PPG는세정서모두유의한차이를보였고 (Figure 2, 3, 6), mean SKT 는무료함이통증에비하여유의하게변화량이크게나타났고 (Figure 4), HR은놀람이무료함과통증에비하여유의하게큰변화를보였다 (Figure 5). 또한 mean PPG 는통증과놀람에서무료함에비하여유의하게큰것으로나타났다 (Figure 6). Table 3. The result of one-way ANOVA among three emotions SCL mean SCR mean SKT mean HR mean PPG SS df MS F Sig. Between 34.03 2 17.02 294.1.000 Within 30.90 534.06 Total 64.93 536 Between 1010.89 2 505.45 277.54.000 Within 972.50 534 1.82 Total 1983.39 536 Between.66 2.33 2.71.068 Within 65.24 534.12 Total 65.90 536 Between 2720.90 2 1360.45 31.37.000 Within 23158.88 534 43.37 Total 25879.78 536 Between.47 2.24 48.60.000 Within 2.58 534.01 Total 3.05 536 3.2 Result of discrimination function analysis 23개특성치들이세정서를얼마나정확하게분류할수있는지확인하기위하여정서상태와안정상태간의차이값을이용한판별분석을실시하였다. 먼저 122 명의데이터중에서정서자극에대한평가결과에근거하여해당정서를느끼지않았다고보고한사람들의데이터를제외시키고데이터의신뢰성을위하여정규분포곡선상의상 / 하위 20% 를 Figure 2. The difference of SCL among three emotions
276 Eun Hye Jang Byoung Jun Park Sang Hyeob Kim Jin Hun Sohn JESK Figure 3. The difference of mean SCR among three emotions Figure 6. The difference of mean PPG among three emotions Table 4. The results of discrimination frequency and accuracy using DFA Predicted emotion Boredom Pain Surprise N (%) Boredom 69(89.5%) 6(7.9%) 2(2.6%) 76(100%) Pain 20(23.5%) 65(76.5%) 0(0.0%) 85(100%) Surprise 6(7.4%) 3(3.7%) 72(88.9%) 91(100%) Figure 4. The difference of mean SKT among three emotions 4. Conclusion Figure 5. The difference of mean HR among three emotions 제외한나머지데이터를분석한결과, 23개의특성치에의해분류된세가지정서에대한전체판별율은 84.7% 이었다. 각정서에대한예측값은 Table 3과같다. 각정서별판별율은무료함 89.5%, 놀람 88.9%, 통증 76.5% 의순이었다. 인간의판단과선택등의인지과정에정서가미치는영향이강하기때문에정서를이해하는일은매우중요하다 (Lerner and Keltner, 2000). 특히정서를이해하기위한방법의하나로정서와자율신경계활동간의관계를밝히는일은정서이론을개발, 검증하고인간 -컴퓨터혹은인간 - 기계상호작용방법을개발하는데필요한작업이다 (Eom et al., 2011). 이를위하여본연구에서는시청각자극을활용하여기존연구에서다루지않은무료함, 통증과놀람정서를유발하고, 생체반응의변화를확인하여세가지정서를구분하고자하였다. 각정서조건동안의생체반응은안정상태에비하여유의하게변화하였다. 무료함정서동안에는피부전기활동, 피부온도와심혈관반응 (mean RRI, std RR과 mean HR) 에서안정상태에비해유의한활성화가있었다. 이는무료함을유발하는자극이심혈관계활동에어느정도영향을미칠뿐아니라, 피부온도와심박율의증가와관련이있다는선행연구결과와일치한다 (Harrison et al., 2000; Sohn et al., 2001).
Vol. 31, No. 2. 2012. 4. 30 Classification of Three Different Emotion by Physiological Parameters 277 통증정서가유발되는동안피부전기활동, 피부온도, 혈류맥파와모든심전도지표에서유의하게활성화되었다. 선행연구에서는통증의정서적차원을감각적차원과구분하려고하였으나, 자기보고식평가결과두차원이다르지않으며정서적측면이통증자극에의하여유발되는심리반응과생체반응과관련있음을보고하였다 (Fernandez and Turk, 1992; Rainville et al., 1999; Chapman et al., 2001). 통증의정서적측면은피부전도반응, 심박율, 혈압등의자율신경계각성과밀접한상관을가지고있으며 (Sokolov, 1990) 피부전도반응에의해가장잘예측될뿐아니라 (Chapman et al., 2001), 통증의불쾌함은심박율의증가와유의한정적상관이있다 (Rainville et al., 1999). 놀람정서에서는심박변이도를제외한모든생리반응에서유의한활성화가나타났다. 놀람정서와자율신경계반응간의관련성을보고한선행연구는많지않고 (Ekman et al., 1983; Levenson et al., 1990), 대부분의경우, 정서분류를위한타정서와의비교를목적으로한것이어서 (Kim et al., 2004; Nasoz et al., 2004; Stephens et al., 2010), 놀람-특정적생체반응결과는명확하게나타나있지않다 (Kreibig, 2010). 놀람과관련된생리반응은통계적으로유의하지는않으나심박율이증가하는특성을가진다 (Ekman et al., 1983; Levenson et al., 1990). 최근연구에서나타난놀람정서에따른생리반응은심혈관반응의활성화로, 말초혈관을수축시키고심박율을증가시키는교감신경계의반응특성과심박율변산성 (std RR과 RMSSD) 을증가시키는부교감신경계의동시활성화가능성을언급하고있는데 (Eom et al., 2011), 본연구결과에서나타난심혈관반응의유의한활성화를지지하는결과이다. 또한정서간차이검증결과, 무료함은 mean SKT 에서유의한증가를보여다른두정서와뚜렷이구분되었다. 통증과놀람은두정서모두 SCL, mean SCR, mean PPG에서유의한증가를보였으나, 놀람의경우, mean HR에서큰증가를보여통증과구분되었고, 무료함과는모든지표에서유의한차이를보였다. 통증은 mean PPG 에서큰변화를보여무료함과뚜렷하게구분되었다. 무료함에서 mean SKT의증가는선행연구들에서도나타나는데, 피부온도는정신적노동, 스트레스나공포등에의해극도로감소하며, 이완, 무료함과수면동안에는증가한다 (Miura, 1931; Helson and Quantius, 1934). 통증정서가유발되는동안에는특히 SCL과 mean SCR 이유의하게증가하고 mean PPG에서감소하였다. SCL 과 mean SCR은교감신경계의부신수질 (sympatheticadrenal-medullary, SAM) 시스템의활동과관련이있는데, 이는통증의진행과관련되어있다 (Storm, 2008). 또한 mean PPG는편두통과같은통증의효과 ( 강도 ) 와유의한 상관을가지는데 (Allen and Mills, 1982), mean PPG의증가는심혈관계를조절하는교감신경계의활성화가억압되는것을의미한다. 따라서본연구에서의 mean PPG의증가는교감신경계의활성화를반영하며, 통증에의한생리반응의특징은 SAM의활성화와말초혈관을수축시키는교감신경계의활성화라고할수있다. 놀람에서도 SCL과 mean SCR은유의하게증가하였는데, 이는땀선의활동에따른교감신경계의활성화를의미한다 (Hassett, 1978). Mean HR의유의한증가는선행연구의결과와도일치하는데 (Ekman et al., 1983; Levenson et al., 1990; Eom et al., 2011), 이는놀람과다른두정서를구분하는반응이다. 또한 mean PPG 의유의한감소는혈관수축에의해나타나는현상으로, α-아드레날린성자극에의한것으로여겨진다. 따라서놀람과관련된생리반응은땀선의활동과심박율을증가시키고말초혈관을수축시키는교감신경계의반응의활성화로특징지을수있다. 또한무료함, 통증과놀람의세정서를분류하기위하여 23개생리반응의특성치를이용하여판별분석을실시한결과 84.7% 의정확도를확인할수있었다. 이는선행연구에서도보고된바없는결과이며, 다른기본정서의판별율에도뒤지지않는결과이다. 그러나이결과의타당성과신뢰성을확보하기위해서는추후연구를통하여신경회로망, SVM 등과같은여러알고리즘을적용한결과와의비교가필요하며, 최근시도되고있는얼굴표정, 음색, 제스처등의다양한정보를결합한다중모달리티인터페이스를통하여복합적인정서상태의인식률을비교하는것도보다의미있는결과를도출할수있을것으로사료된다. 나아가생리반응특성치를통하여이들정서가다른기본정서와도판별가능한지를확인하는작업, 나아가정서의패턴분류를위한표준화작업도필요하다. 현재정서인식분야에서생체신호를이용한정서인식은시작단계에있으며, 생체신호의이점을충분히활용하기위해서는사용된정서모델, 생체신호의패턴을확인하기위해사용된자극, 사용된생리반응지표, 분석을위한특징점과정서의패턴인식과분류를위한모델의영역에서표준화가이루어져야한다는제한점을가진다 (Arroyo-Palacios and Romano, 2008). 본연구결과는실제로인간이경험하지만검증되지않은정서들의생체반응을확인하고이들정서를분류하였다는점에서의의가있다. 이는생체신호처리에의한인간- 컴퓨터상호작용기술에서정서인식연구의기초자료로활용가능하며, 정서인식및분류를위한표준화작업에도기여할수있을것이다. 또한본연구에사용된데이터는최대한정서를자연스럽게유발하도록하여획득되었기때문에실생활에적용가능한정서인식시스템개발에적용할수있을것으로생각된다.
278 Eun Hye Jang Byoung Jun Park Sang Hyeob Kim Jin Hun Sohn JESK Acknowledgements This research was supported by the Converging Research Center Program through the Converging Research Headquarter for Human, Cognition and Environment funded by the Ministry of Education, Science and Technology(No. 2011K000655 and 2011K658). References Allen, R. A. and Mills, G. K., "The effects of unilateral plethysmographic feedback of temporal artery activity during migraine head pain", Journal of Psychosomatic Research, 26, 133-140, 1982. Arroyo-Palacios, J. and Romano, D. M., "Towards a standardization in the use of physiological signals for affective recognition systems", Proceedings of Measuring Behavior 2008, 2008. Bowsher, D., "Physiology and Pathophysiology of Pain", Journal of British Medical Acupuncture Society, 7, 17-20, 1990. Broek, van den E. L., Janssen, J. H. and Westerink, J. H. D. M., "Guidelines for Affective Signal Processing (ASP): From Lab to Life", 3rd International Conference on Affective Computing & Intelligent Interaction, 2009. Chanel, G., Kierkels, J. J. M., Soleymani, M. and Pun, T., "Short-term emotion assessment in a recall paradigm", International Journal of Human-Computer Studies, 67, 607-627, 2009. Chapman, C. R., Nakamura, Y., Donaldson, G. W., Jaconson, R. C., Bradshaw, D. H., Flores, L. and Chpman, C. N., "Sensory and Affective Dimensions of Phasic Pain Are Indistinguishable in the Self-Report and Psychophysiology of Normal Laboratory Subjects", The Journal of Pain, 2(5), 279-294, 2001. Cheyne, J. A., Carriere, J. S. A. and Smilek, D., "Absent-mindedness: Lapses in conscious awareness and everyday cognitive failures", Consciousness and Cognition, 15 (3), 578-592, 2006. Choi, A. and Woo, W., "Feature extraction for emotion analysis based on physiological signal", KHCI 2005, 1(pp. 624-629), 2005. Coon, D., Introduction to Psychology: Gateways to Mind and Behavior (10th Edi.), Philadelphia: Thomson Wadsworth, 2004. Duda, R. O., Hart, P. E. and Stork, D. G., Pattern Classification, 2nd ed., Wiley-Interscience, 2000. Ekman, P., Levenson, R. W. and Friesen, W. V., "Autonomic nervous system activity distinguishes among emotions", Science, 221, 1208-1210, 1983. Eom, J. S., Park, H. J., Noh, J. H. and Sohn, J. H., "Cardiovascular response to surprise stimulus", Korean Journal of the Science of Emotion and Sensibility, 14(1), 147-156, 2011. Ekman, P. and Friesen, W. V., Unmasking the face, Englewood Cliffs, NJ: Prentice Hall, Inc., 1975. Fisher, C. D., "Boredom at work: A neglected concept", Human Relations, 46, 395-417, 1993. Flor, H., Knost, B. and Birbaumer, N., "The role of operant conditioning in chronic pain: an experimental investigation Pain", Pain, 95(1-2), 111-118, 2002. Gunes, H. and Piccardi, M., "Affect Recognition from Face and Body: Early Fusion vs. Late Fusion", IEEE Transactions on Systems, Man, and Cybernetics, 1-7, 2007. Harrison, L. K., Carroll, D., Burns, V. E., Corkill, A. R., Harrison, C. M., Ring, C. and Drayson, M., "Cardiovascular and secretory immunoglobulin A reactions to humorous, exciting, and didactic film presentations", Biological Psychology, 52, 113-126, 2000. Hassett, J., A primer of psychophysiology, San Francisco: W. H. Freeman and Company, 1978. Helson, H. and Quantius, L., "Changes in skin temperature following intense stimulation", Journal of Experimental Psychology, 77, 20, 1934. Jolliffe, C. D. and Nicholas, M. K., "Verbally reinforcing pain reports: an experimental test of the operant model of chronic pain", Pain, 107(1-2), 167-175, 2004. Kim, K., Bang, S. and Kim, S., "Emotion recognition system using shortterm monitoring of physiological signals", Medical and Biological Engineering and Computing, 42, 419-427, 2004. Leary, M. R., Rogers, P. A., Canfield, R. W. and Coe, C., "Boredom in interpersonal encounters: Antecedents and social implications", Journal of Personality and Social Psychology, 51, 968-975, 1986. Lee, S. J. and Bai, S. J., "New Trend of Pain Evaluation by Brain Imaging Devices", Korean Journal of the Science of Emotion and Sensibility, 8(4), 365-374, 2005. Lerner, J. S. and Keltner, D, "Beyond valence: Toword a model of emotion-specific influences of judgment and choice", Cognition & Emotion, 14, 473-493, 2000. Levenson, R. W., Ekman, P. and Friesen W. V., "Voluntary facial action generates emotion-specific autonomic nervous system activity. Psychophysiology", 27, 363-384, 1990. Mannheimer, J. S. and Lampe, N., Clinical transcutaneous electrical nerve stimulation, FA Davis, Philadelphia. 1984. Merskey, H. and Bogduk, N., Classification of Chronic Pain, IASP Task Force on Taxonomy. Seattle, IASP Press, 1994. Miura, U., "The effect of variations in relative humidity upon skin temperature and sense of comfort", American Journal of Hygiene, 13, 432, 1931. Nasoz, F., Alvarez, K., Lisetti, C. L. and Finkelstein, N., "Emotion recognition from physiological signals using wireless sensors for presence technologies", Cognition, Technology and Work, 6, 4-14, 2004. Peter, C. and Herbon, A., "Emotion representation and physiology assignments in digital systems", Interacting with Computers, 18, 139-170, 2006. Picard, R. W., Vyzas, E. and Healey, J., "Toward Machine Emotional Intelligence: Analysis of Affective Physiological State", IEEE Transactions of Pattern Analysis and Machine Intelligence, 23(10), 2001.
Vol. 31, No. 2. 2012. 4. 30 Classification of Three Different Emotion by Physiological Parameters 279 Rainville, P., Carrier, B., Hofbauer, R. K., Bushnell, M. C. & Duncan, G. H., "Dissociation of sensory and affective dimensions of pain using hypnotic modulation", Pain, 82, 159-171, 1999. Sohn, J. H., Sokhadze, E. and Watanuki, S., "Eletrodermal and cardiovascular manifestations of emotions in children", Journal of Physiological Anthropology and Applied Human Science, 55-64, 2001. Sokolov, E. N., "The orienting response, and future directions of its development", The Pavlovian Journal of Biological Science, 25, 142-150, 1990. Sternbach, R. A., The Psychology of Pain, Raven Press, New York, 1978. Storm, H., "Changes in skin conductance as a tool to monitor nociceptive stimulation and pain", Current Opinion in Anaesthesiology, 21, 796-804, 2008. Sylvia D. Kreibig, "Autonomic Nervous System Activity in Emotion: A Review", Biological Psychology, 2010. Tefas, A., Kotropoulos, C. and Pitas, I., "Using support vector machines to enhance the performance of elastic graph matching for frontal face authentication", IEEE Transactions of Pattern Analysis and Machine Intelligence, 23(7), 735-746, 2001. Byoung-Jun Park: bj_park@etri.re.kr Highest degree: PhD, Department of Electrical Engineering, Wonkwang University Position title: Senior Member of Engineering Staff, BT Convergence Technology Research Department, Electronics and Telecommunications Research Institute Areas of interest: Computational Intelligence, Pattern Recognition, Cognition Convergence Sang Hyeob Kim: shk1028@etri.re.kr Highest degree: PhD, Department of Apply Physics, Tohoku University Position title: Principle Member of Engineering Staff, BT Convergence Technology Research Department, Electronics and Telecommunications Research Institute Areas of interest: Cognition Convergence, Emotion Recognition Jin-Hun Sohn: jhsohn@cnu.ac.kr Highest degree: PhD, Department of psychology, the University of Korea Position title: Professor, Department of Psychology, Chungnam National University Areas of interest: Brain Science, Neuroscience, Electro Physiology Date Received : 2011-04-15 Author listings Date Revised : 2012-03-16 Date Accepted : 2012-03-16 Eun-Hye Jang: cleta4u@etri.re.kr Highest degree: PhD, Department of Psychology, Chungnam National University Position title: Senior Member of Engineering Staff, BT Convergence Technology Research Department, Electronics and Telecommunications Research Institute Areas of interest: Emotion Recognition, Cognition Convergence