A preliminary study on usability evaluation of the navigational equipment in ship handling simulator Jun Hyuk Jang, Seung-bin Oh, Jin Hyoung Park, Hongtae Kim Korea Institute of Ocean Science & Technology, Daejeon, Korea 305-343 ABSTRACT Objective: This study is to analyze the use of the navigational equipment usage frequency and pattern using ship simulator under certain circumstances. Background: Usability Evaluation of navigational equipment is very broad. Navigational equipment in the form of a single device is not being used in the actual sailing conditions. The entire mission system usability evaluation should be considered as an important factor rather than individual. It is possible to analyze the characteristics of sailors' attention and cognitive process using eye tracking system. Method: Ten participants took part in this study, and simulation test was performed. Target area is Pusan north port. Subjects were requested to evaluate the navigational equipment in the simulator. Results: According to the usage rate analysis, the result shows similar pattern between expert group and novice group. However, the expert group used the radar equipment frequently. Conclusion: It is possible to evaluate the navigational equipment using eye tracking. Future study will be performed using improved scenarios and the navigational equipment on the simulator usability evaluation. Application: This study will contribute to the education and training of ship handling simulator system. Keywords: usability evaluation, eye tracking system, usage frequency, usage pattern, navigational equipment 1. Introduction 해양사고의발생원인중상당부분이인적요인에의한것이라고여러연구보고서를통하여보고되고있다. 영국의 UK P&I 클럽 (1993) 의보고서에의하면모든해양사고의 62 % 가인적요인으로인하여발생하였다고보고하고있으며, 미국해안경비대 (1995) 의보고서는모든해양사고중 70 % 에이르는사고의원인이인적요인에기인한다고보고하였다. 또한국제해사기구 (IMO ; International Maritime Organization) 에서는해양사고의약 80 % 가직 / 간접적으로인적요인에의한것이라고밝혔다. 이러한인적요인에의한사고를감소시키기위하여 IMO를비롯한여러연구기관등에서연구및조사가활발하게이루어지고있다. 해양분야에서의인적요인은 1987년 3월 Heraldof Free Enterprise 호의전복사고, 1990년 4월 Scandinavian Star 호의화재사고등인적요인으로인한대형해양사고가잇따르면서 IMO가해양사고에서인적요인의역할에주목하는계기가되었으며, 이에따라 1991년 IMO 해사안전위원회 (MSC) 와해양환경보호위원회 (MEPC) 가공동작업반을구성하여해양사고에서인적요소의역할에대하여논의를 시작하면서 IMO에서인적요인에대한이슈가본격화되기시작하였다 (Beck & Kim, 2009). 특히사고의사전적예방측면에서선교의성능뿐만아니라익숙함의부족으로인한사고방지를위하여선박운항자관점에서의다양한접근방법이시도되고있다. 즉, 선박해양분야에서인간공학적고려여부가매우중요한이슈가되었다. ISO 9241에서는사용성 (Usability) 은특정한목적을성취하기위한상황에서사용자에의해어떤제품을사용할때, 사용자가제품에대해서가지는효과성 (Effectiveness), 효율성 (Efficiency) 그리고만족도 (Satisfaction) 에대한것으로정의하고있다. 해양분야에서또한선박운항과관련하여항해의특성을고려한사용성평가가필요하다. 일본은 NAV 56차와 NAV 57차에걸쳐장비의사용성평가를위한방법론을제안하였다. 항해장비의사용성평가의범위와방법은매우다양하며, 특히사용자와직무에대한고려뿐만아니라실제사용되는환경도고려해야할중요한요소중하나이다. 또한항해장비는실제항해상황에서단일장비의형태로단독으로사용되는것이아니라복합적으로사용되기때문에전체선교시스템에대한사용성평가도중요한요소로서고려되어야한다. 시선추적장비는 1950년대 Poynter Institute에서실험심리
학분야에처음도입되었으나, 1970년대에들어와인간의주목도측정에사용되어오고있다. 항공기와차량등의교통분야와광고분야, 예술분야등에서다양하게활용되고있다 (Nho, 2012). 이후많은종류의시선추적기법이개발되어지속적으로연구가이루어지고있다. 시선추적장비를활용한객관적인인지과정에대한자료획득과운항자의시선움직임분석이가능할것으로판단된다. 특히 e-navigation의사용성평가등사용자요구사항분석에도활용가능할것으로판단된다. 본연구에서는현재선박해양분야의인간공학적다양한시도중사용성평가를위한기초연구로써선박운항시뮬레이터에서시선추적장비를활용하여항해장비의사용빈도와사용패턴을분석을하고자한다. 2. Method 2.1 Subjects 피실험자는초보자그룹과항해경험이있는경력자그룹으로나누어실시하였다. 초보자그룹은해기사실습생 5 명, 경력자그룹은항해경력 3년이상의항해사 5명을대상으로한국해양과학기술원의 Full mission 시뮬레이터를이용하여실험을실시하였다. 피실험자의일반적인특성은아래의 Table 1과같다. 초보자그룹의평균나이는 23.4세이며, 경력자그룹의평균나이는 31세이다. 항해경력은초보자그룹의경우참가자모두 1년의승선경력이있었으며, 경력자그룹의경우평균 3.8년의승선경력이있었다. 또한경력자그룹의경우대부분항해사 2급이상의면허를소지하고있었다. Table 1. Characteristics of subjects Subjects Average age(years) Career(years) Novice 23.4 1 experienced 31 3.8 2.2 Equipment 실험을위해한국해양과학기술원에서개발한전기능선교시뮬레이터 (Full Mission Bridge Simulator) 를이용하였다. 이시뮬레이터는선원의교육및훈련, 해상교통안전성평가, 해양사고원인분석및대책수립그리고선박의운동특성연구를위한목적으로활용되고있으며, 실제선박의선교와유사하게 ARPA Radar, ECDIS, DMDSS, Gyro Compass 등의항해장비가설치되어있다. 아래 Figure 1 (a) 에서는선박 시뮬레이터실험환경을보여준다. 피실험자의시선움직임을측정을위해헤드마운트타입의 Dikablis Eye Tracking 장비를이용하였다 (Figure 1(b)). (a) (b) Figure 1. Simulator and eye tracking equipment 2.2 Scenario 시뮬레이션의대상해역은부산북항이며, 대상해역의평면배치도와시뮬레이션시나리오의타선에관한경로정보는 Figure 2(a) 와같으며, 2(b) 는실험장면을보여준다. 시나리오에는항해상황중자주발생하는 Crossing 상황과 Head on 상황이포함되어있으며, 결과에서는이러한항해상황에따른시선의움직임을분석하였다. 시뮬레이션시간은약 20분정도소요되었다. (a) (b) Figure 2. Experimental environment 2.2 Procedure 실험을수행하기에앞서피실험자들에게연구의목적, 실험시주의사항및실험절차를설명한후실험을실시하였다. 실험전후설문지를작성하게하였는데, 실험전에는피실험자들에게 5점척도로 Radar의중요도와사용빈도에대하여평가하도록하였다. 실험참가자들의시선을분석하기위하여시선추적장비를활용하였고, 실험중모든장면은카메라로녹화하였으며, 녹화된영상은실험결과분석을위한보조수단으로활용되었다. 실험후에는시뮬레이터를활용한운항훈련과관련한불편사항및개선점, 그리고실선운항과의차이점등을자유기술형태로설문하였다. Figure 3은본연구의실험절차를보여준다.
Subjects prior education Questionnaire Eye-tracking Calibration Simulator based experiment (a) Novice Survey the subjective discomfort Figure 3. Procedure 3. Results 3.1 Eye movement 피험자들이시뮬레이터운항동안시선움직임에대한결과를분석하였다. 크게시선의움직임을견시, ODD (Overhead Data Display), 그리고 Radar 장비의주시시간으로나눌수있으며, 이를상대적사용비율로나타내면 Figure 4 와같다. 초보자들은견시와 Radar 사용비율이비슷하나, 숙련자들은복잡한운항환경 (Crossing) 상황에직면했을때, ODD의사용비율은거의차이가없으나, Radar 에서많은정보들을확인하면서운항하는것으로나타났다. Figure 4. Result of motion chart Heat map 분석결과에서도초보자의견시와 Radar 사용비율은거의비슷하게나타났으나, 숙련자의경우에 Radar의사용비율이상대적으로높은것을확인할수있었다. Figure 5는 Heat map 결과를보여준다. (b) Experienced Figure 5. Result of heat map 3.2 Questionnaire 실험전피실험자들에게 Radar의중요도와사용빈도에대하여평가하도록하였다. 실험에사용된설문지는 IMO Safety of Navigation의 Sub-Committee에서일본이제안한항해장비사용성평가관련문서를기초로작성되었으며, ARPA Radar 사용성평가를위한 4가지직무에대하여실험참가자들이해당기능들을 5점척도로평가하도록하였다. Table 2는설문에대한결과를보여준다. ARPA Radar 기능중초보자는 Mode change, Range change, EBL (Electronic cursor), VRM (Variable Range Marker), Off-centre display 순으로중요하다고응답하였고, Target data reading, Range change, Off-centre display, EBL (Electronic cursor), VRM (Variable Range Marker) 순으로사용빈도가높다고응답하였다. 숙련자는 Range change, Target data reading, Off-centre display, Trail display, Vector mode change 순으로중요하다고응답하였고, Range change, Target data reading, Off-centre display, Trail display, VRM (Variable Range Marker) 순으로사용빈도가높다고응답하였다. 설문조사결과초보자와숙련자의중요도와사용빈도면에서응답한기능들사이에는별다른차이점을보이지않으나, Trail display 기능에대해서만차이를보였다.
Item Table 2. Results of questionnaire Contents Practice operation of buttons and trackball Importance Frequency N E N E Mode change 5 3.4 4 2.7 Range change 4.8 5 4.2 4.9 Off-centre display 4.6 4.4 4.2 4.4 Practice of basic operations of ARPA Trail display 4 4.4 3.2 4.4 Target data reading 4.6 4.9 4.6 4.9 EBL (Electronic cursor) 4.8 4 4.2 3.7 VRM (Variable Range Marker) 4.8 3.9 4.2 4 Capture / Capture release 4.2 2.8 3.8 2.8 Vector mode change 4.6 4.3 3.6 3.6 Guard zone setting 3.6 3.1 2 2.9 Capture and decision making for collision avoidance (1) Capture and decision making for collision avoidance (2) Guard zone setting and alarm handling The following operations to be done manually by button operation. 1. Capture a target ship and make decision on collision avoidance 4.4 3.4 4 3.1 2. After making decision, execute capture release 4.4 3.3 4 2.9 Same operations as Task 2 to be done manually by trackball operation. 1. Capture a target ship and make decision on collision avoidance 4.4 3.7 4.2 3.7 2. After making decision, execute capture release 4.4 3.6 4.2 3.4 Set the guard zone, capture a target ship, and make decision on collision avoidance, under the conditions where multiple alarms are ringing. 4.4 3.3 3.8 3 실험후피실험자들에게주관적불편도에대한설문조사를분석한결과, 장비친숙도에대한지적이많았다. 실험에사용한시나리오구성에관해서는선박통항량이실제보다다소많다고느끼고있었으며, 이러한상황으로인하여타선박의정보를충분히확인하지못하고, 견시위주의운항을하게되었던점을지적하였다. 이와더불어상대선박과의의사소통을할수없는 (VHF 장비를사용하지못하여 ) 실제항해와의차이점을지적하였다. 4. Conclusion 본연구는사용성평가의기초연구로서선박운항시뮬레이터에서시선추적장비를활용하여항해장비의사용빈도와사용패턴을분석하였다. 피험자들의시선움직임을견시, ODD (Overhead Data Display), 그리고 Radar 장비의주시시간으로나누어분석하였다. 상대적사용비율을분석한결과, 초보자들은견시와 Radar 사용비율은비슷하나, 숙련자들은복잡한운항환경에직면했을때, ODD의사용비율에서는거의차이가없으나, Radar에서는더많은정보를확인하면서운항하는것으로나타났다. Heat map 분석결과에서도숙련자의경우, Radar의사용비율이상대적으로높은것을확인할수있었다. Radar 장비에대한중요도와사용빈도에대한설문조사결과, 초보자와숙련자의응답에는별다른차이점을보이지않았다. 주관적불편도에대한설문조사결과, 장비의친숙도에대한지적이많았는데, 이는피실험자들이실제항해에서사용하고있는장비와본실험에서사용하였던장비가상이하여불편을초래한것으로보인다. 실험에사용한시나리오구성에관해서는선박통항량이실제보다다소많다고느끼고있었으며, 이러한상황으로인하여견시위주의운항을하게되었던점
을지적하였다. 또한, 시뮬레이터상에서자선의조종특성과성능을파악하지못하고실험에임한점과시뮬레이션실험과실제운항에서의거리감등감각의차이로운항의어려움을호소는경우가많았다. 추후연구에서는특정상황별전체선교내에서의항해장비의사용에대한연구가필요할것으로보인다. Yeo Mi, A Study on Facade Design Applying Gaze Frequency-Focusing on Eye-Tracking Experiment, Kyungsung University master s thesis, 2013. Author listings Acknowledgements The contents of this paper are the results of the research project of KIOST(The Development of Human Factors Experimental Evaluation Techniques for Maritime Traffic Safety Assessment) and Ministry of Oceans and Fisheries of Korea (Development of prevention and management technology for human-related marine accident). References America, http://www.poynter.org Choi Min Young, A Study on the Method of Eye Tracking Analysis Based on the Properties in Visual Perception of User -With Emphasis on the Development of Analysis-Framework for Product Design, Journal of korean society of design science, 16(4), 197-206, 2003. IMO, NAV 56/8/9 Usability assessment methodology for navigational equipment, 2010. IMO, NAV 57/INF.7 Preliminary Draft Guidelines for Usability Evaluation of Navigational Equipment, 2011. IMO, NAV 57/INF.8 Sample summary report of usability test, 2011. IMO, NAV 58/WP.6 Report of the Working Group, 2012. ISO 9241-11, Guidance on usability, 1998. Jinsu Beck, Min-Jong Kim, A Study on Human Element in the Maritime Domain, navigation and port research, Vol.2009, No.2, 267-268, 2009. Kim Byung-Joo, Integrated analysis system of perception, recognition, behavior for web usability test, KIAST master s thesis, 2006. Ki-won Sung, Kun-pyo Lee, Development of integrated analysis model for eyegaze analysis. Journal of Korean society of design science, Vol.56, 2004. Korean Maritime Safety Tribunal, Marine accident statistics, 2011. Nho Jae-Mann, The Development of Visual Cognition Analysis Model for Mobile Web using Eye-tracking Techniques, Kongju National University master s thesis, 2012. UK P&I Club., Analysis of major claims. London: Author, 1993. USCG., Prevention through people, Quality Action Team Report, Washington D.C.:USCG, 1995. Jun-Hyuk Jang: jang@kiost.ac Highest degree: Ph.D Department of Industrial Engineering, Dong-A University Position title: Researcher, Korea Institute of Ocean Science & Technology (KIOST) Areas of interest: Occupational safety, Maritime Human Factors Seung-bin Oh: ohseungbin@kiost.ac Highest degree: MS, Department of Industrial Engineering, Hannam University Position title: Researcher, Korea Institute of Ocean Science & Technology (KIOST) Areas of interest: Maritime Human Factors, Human Sensibility Jin Hyoung Park: jin.h.park@kiost.ac Highest degree: PhD, Department of Computer Science, KAIST Position title: Senior researcher, Korea Institute of Ocean Science & Technology (KIOST) Areas of interest: Computer Simulation, Safety & Reliability, Software Quality Assurance(SQA), Maritime Human Factors Hongtae Kim: hongtae.kim@kiost.ac Highest degree: Ph.D Department of Industrial Engineering, Korea University Position title: Principal Researcher, Korea Institute of Ocean Science & Technology (KIOST) Areas of interest: Maritime Human Factors, Human Safety & Reliability