한수지 49(2), 224-233, 2016 Original Article Korean J Fish Aquat Sci 49(2),224-233,2016 인공신경망에의한 6 개어종의음향학적식별 이대재 * 부경대학교해양생산시스템관리학부 Acoustic Identification of Six Fish Species using an Artificial Neural Network Dae-Jae Lee* Division of Marine Production System Management, Pukyong National University, Busan 48513, Korea The objective of this study was to develop an artificial neural network (ANN) model for the acoustic identification of commercially important fish species in Korea. A broadband echo acquisition and processing system operating over the frequency range of 85-225 khz was used to collect and process species-specific, time-frequency feature images from six fish species: black rockfish Sebastes schlegeli, black scraper Thamnaconus modesutus [K], chub mackerel Scomber japonicus, goldeye rockfish Sebastes thompsoni, konoshiro gizzard shad Konosirus punctatus and large yellow croaker Larimichthys crocea. An ANN classifier was developed to identify fish species acoustically on the basis of only 100 dimension time-frequency features extracted by the principal components analysis (PCA). The overall mean identification rate for the six fish species was 88.5%, with individual identification rates of 76.6% for black rockfish, 82.8% for black scraper, 93.8% for chub mackerel, 90.6% for goldeye rockfish, 96.9% for konoshiro gizzard shad and 90.6% for large yellow croaker, respectively. These results demonstrate that individual live fish in well-controlled environments can be identified accurately by the proposed ANN model. Key words: Fish species identification, Time-frequency image, Artificial neural network, Principal components analysis, Confusion matrix 서론,.,., (Woillez et al., 2012) echo trace (Fernandes, 2009), echogram (Tasgarakis et al., 2015), echo (simmonds et al., 1996; Lee et al., 2015; Lee, 2015a; Lee, 2015b).,., (Lee et al., 2015; Lee, 2015a; Lee, 2015b). chirp - echo (set), (artificial neural network, ANN),., (accuracy),, (activation function), http://dx.doi.org/10.5657/kfas.2016.0224 Korean J Fish Aquat Sci 49(2) 224-233, April 2016 This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial Licens (http://creativecommons.org/licenses/by-nc/3.0/) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited. Received 18 March 2016; Accepted 29 March 2016 *Corresponding author: Tel: +82. 51. 629. 5889 Fax: +82. 51. 629. 5885 E-mail address: daejael@pknu.ac.kr Copyright 2016 The Korean Society of Fisheries and Aquatic Science 224 pissn:0374-8111, eissn:2287-8815
인공신경망에의한 6 개어종의음향학적식별 225, (weight).,.,,, (Lee et al., 2016). 3 (principal components analysis, PCA) (Zuo et al., 2006; Santo, 2012). chirp 3 - echo (Imberger and Boashash, 1986; Dong and Cui, 2012; Sui et al., 2007) 35 75 echo. 2,625., - echo PCA matrix,.,,,,, 6 16 96 85-225 khz - echo,, PCA matrix,. 재료및방법 어류에의한시간 - 주파수 echo 이미지의데이터차원축소,,,,, 6 85-225 khz 16 96 - echo (Lee et al., 2016)., chirp echo (dorsal aspect) 25 2.5, 320 (16 20 / ),, 6 1,920 - echo. 6 chirp echo SPWVD (smoothed pseudo-wigner-ville distribution) -, 0-0.3 ms 85-225 khz i (column) j (row) (pixel) (N = i j ) (cutting). - I 1, [ p 11 p 12 p 1j ] p I 1 = 21 p 22 p 2j (1) p i1 p i2 p ij, column F 1 F 1 = {p 11 p 12 p 1j p 21 p 22 p 2j p i1 p i2 p ij } (2). - M, S M S M = [F 1, F 2, F 3 F M ] T (3)., [*] T [*] (transposed matrix)., S M PCA, W, [ w 11 w 12 w 1N ] w W = 21 w 22 w 2N (4) w N1 w N2 w NN.,., K (W K ) [ w 11 w 12 w 1N ] w W k = 21 w 22 w 2N (5) w K1 w K2 w KN. K PCA W K N - K -., S input S input = W K S M (6).
226 이대재 다중퍼셉트론인공신경망과어류의시간 - 주파수이미지데이터의학습 (neuron, node),., (multilayer perceptron network, MLP), MLP Fig. 1. Fig. 1 j i x i, (connection weight) w ji, j (bias) b j, Fig. 1 MLP., (activation function) (7) (Saad et al., 2007: Bai et al., 2009; Latha el al., 2009). y j = f (net j ) = f ( w ji x i + b j ) d j=1, net j j,,. f, y j j net j,., Fig. 1 k j y i, (7) w kj, j (bias) b k net k, f ( net k ) z k. z k = f (net k ) = f ( w kj y i + b k ) h j=1 z k t k,, (9) (error threshold). E = (8) 1 m 1 { t - z 2 (t k - z k ) 2 = (9) 2 k=1 2, Fig. 1 (9) E (Bai et al., 2009; Latha el al., 2009). MLP (error backpropagation, BP) (supervised learning),,., w b, w (old ) b (old ) w (new) b (new) (10) (11) (Latha el al., 2009; Pinjare and Arun Kumar, 2012). w (new) = w (old ) + w (10) b (new) = b (old ) + b (11) Fig. 1. Architecture of multilayer perceptron (MLP) neural network with one hidden layer and basic block of error back-propagation process. The architecture represents a three-layered MLP with h neurons in the hidden layer and d neurons in the input layer corresponding to the time-frequency echo patterns of 6 fish species. The m nodes in the output layer indicate the m different fish species to be predicted. Matlab (tool) newff (feed-forward backpropagation network) (Demuth et al., 2009),, Fig. 1 BP traingdx (gradient descent with variable learning rate and momentum)., (9) (performance function) E, (mean square error, MSE) (Kuruvilla and Gunavathi, 2014). traingdx Table1 (Shilbayeh et al., 2013). Table 1 matlab newff net, train net net.trainfcn net.trainparam net, (return) net
인공신경망에의한 6 개어종의음향학적식별 227 Table 1. The function traingdx parameters Training parameters Description net.performfcn = mse net.trainparam.goal = 0.001 net.trainparam.show = 20 net.trainparam.epochs = 5000 net.trainparam.mc = 0.95 Mean square error Performance goal Epochs between displays Maximum number of epochs to train Momentum constant sim., (activation) hyperbolic tangent, tan-sigmoid -1 +1 (Bai et al., 2009). 결과및고찰 어종별학습용및테스트용이미지의특징적인패턴 6, 96 - echo.,., 320 - echo 80%(256 )., 20%, Fig. 2. Fig. 2 (a) (b) chirp echo Fig. 2. Original train image sets for black rockfish Sebastes schlegeli consisting of 320 color-scale images each with a matrix dimension of 35 75 pixels in the frequency and time domains. These SPWVD images were obtained from the broadband echoes by 16 live individuals over the frequency range of 85 to 225 khz. (a) The 256 train images, for the supervised learning of ANN, corresponding to 80% of 320 images were selected by random process. (b) The 64 test images corresponding to 20% of 320 images were used as an image set for identifying the target fish species.
228 이대재 신호에 대한 모든 시간-주파수 echo 이미지는 모두 RGB 영상 패턴이다. 실제 이들 이미지는 35 75 픽셀의 gray 이미지 패 턴으로 변환되어 인공 신경망의 입력 데이터로서 사용하였다. Fig. 2(a)는 조피볼락에 대한 학습용 이미지 데이터 셋이고, Fig. 2(b)는 테스트용 이미지 데이터 셋이다. 본 연구에서는 각 이미 지 패턴의 윤곽(contour), 스펙트럼의 구조 및 형상, 이미지 픽 셀의 진폭변동, echo 에너지가 집중되는 영역, echo 신호의 출 현과 소멸 구간 등의 차이 등에 주목하여 분석을 행하였다. 이 들 각 어종에 대한 시간-주파수 이미지 정보를 비교, 분석한 결 과, 각 이미지 상호간에 매우 유사한 공통점이 있음을 알 수 있 었다. 즉, 저주파수 영역에서 고주파수 영역을 향해 일정한 기울 기를 갖는 다양한 형상의 echo contour 패턴이 연속하여 출현하 였다. 이들 contour 패턴의 형상은 조사 대상으로 한 6개의 어종 에 있어 각각 서로 다른 양상을 나타내었다. 또한, 이들 각 이미 지 contour 패턴에서 feature 성분이 강하게 집중되어 나타나는 영역이 서로 다른 점으로부터 이들의 스펙트럼 성분들은 어종 에 매우 의존적인 특징을 갖고 있음을 알 수 있었다. 따라서, 본 연구에서는 각 어종에 대한 시간-주파수 이미지 패 턴 속에 내포되어 있는 어종 의존적인 특징들을 어떻게 추출하 여 이것을 어종식별인자로서 활용할 것인가에 초점을 두고 연 구를 수행하였다. Fig. 2에 나타낸 SPWVD 기법을 이용하여 얻어진 각 어종당 320개의 35 75 픽셀 이미지에 대한 2차원 feature 패턴을 학습 용과 데스트용으로 분류하여 재성형(reshaping)한 결과는 Fig. 3의 (a) 및 (b)와 같다. Fig. 3(a)는 Fig. 2(a)의 학습용 이미지를 대상으로 주파수 축의 35개 픽셀 column을 서로 체인 형으로 1 번부터 35번까지 순서대로 결합시켜 2,625 픽셀을 갖는 새로운 이미지를 생성한 후, 이들 학습용의 256개의 이미지 전체를 이 미지 번호 순으로 배열한 결과이다. 한편, Fig. 3(b)는 Fig. 2(b) 의 테스트용 이미지 셋을 대상으로 Fig. 3(a)에서와 같은 재성 형 처리를 수행하여 얻은 64개의 새로운 이미지 패턴을 이미 지 번호 순으로 배열하여 나타낸 결과이다. Fig. 3의 (a) 및 (b) 에서 종축은 이미지 데이터의 번호이고, 횡축의 1-2,625는 주 파수 축의 35개 column (75 픽셀/column)을 서로 체인 형으로 순서대로 연결하여 생성한 2,625픽셀의 번호이다. 이들 이미지 feature 패턴에는 각 어종 고유의 생물학적, 형태학적, 음향학 적 성질 등을 나타내는 어종 의존적인 정보들이 내포되어 있다 (Gavrovska et al., 2010; Han and Kim, 2010). 따라서, 이들 이 미지 패턴으로부터 각 어종이 갖는 고유의 특징적인 픽셀 패턴 및 변동성 등을 얼마나 정확하게 정량적으로 추출해 내는가는 Fig. 3. (a) A reshaped image set of 256 train images for the supervised learning of ANN (Fig. 2a). (b) A reshaped image set of 64 test images for identifying the target fish species (Fig. 2b). Each matrix for train and test images of black rockfish Sebastes schlegeli was reshaped as a set of 2,625-dimensional feature pixels by concatenating the 35 columns of the 35 75 image matrix.
인공신경망에의한 6 개어종의음향학적식별 229. Fig. 3 (a) (b) 16, 1,000, 1,000-2,300., 2,300.,., Fig. 3 (a) (b) 2,625, 256, 256 2,625.,., PCA 6., Fig. 3(a) PCA (5) PCA W K., Fig. 3(b)., feature space,,, feature space,,.,,. 시간 - 주파수이미지데이터의차원축소에따른어종식별율의변화 시간 - 주파수이미지데이터의차원축소 6 256 -,., 1 - echo 75 35 2,625.,.,,,. PCA PCA,, Fig. 4. Fig. 4 16 320 - echo Fig. 4. A lower dimensional representation of the reduced feature matrix for the time-frequency echo pattern of black rockfish Sebastes schlegeli as a function of the number of principal components (eigenvectors). By only considering the first 250 eigenvectors of 2,526 eigenvectors, the dimensionality of the image feature space (matrix) was greatly reduced. The supervised learning of ANN for identifying the fish species was accomplished by only using a truncated set of the first 100 eigenvectors.
230 이대재 256 - echo PCA echo., - echo 75 35 2,625, PCA 250 PCA W 250. (S input ) W 250, S input echo feature., Fig. 4 PCA., - echo, PCA,. Fig. 4 PCA,, ( ) -, -. Fig. 4 PCA 50, 250., 6 feature Fig. 4. PCA 계수 matrix 의차원에따른어종식별율의변화 PCA, Fig. 4 PCA 75, 100, 125, 150, 175, 200,,,,,, Table 2. Table 2 75 64. Table 2-75, 100, 125, 150, 175, 200,,,,,,, 86.7, 87.0, 87.8, 87.5, 86.7 87.5%. echo 125 87.8%, 75 175 86.7%, 1.1%., 125,,,,,,, 81.3, 89.1, 89.1, 87.5, 89.1 90.6%,,., Table 2,, 3 75,. 3,., - echo 100, 100,., 6, 100, 100, 6 3. 최적의인공신경망모델구축및어종의음향학적식별 Fig. 4 Table 2,, 100, 100, 6 Table 2. Confusion matrix of species classification rates (%) by artificial neural network with one hidden layer of 75 neurons as a function of the number of reduced feature matrix dimensions by PCA Fish species Number of reduced feature dimensions 75 100 125 150 175 200 Black rockfish 81.3 79.7 81.3 81.3 79.7 79.7 Black scraper 92.2 85.9 89.1 93.8 90.6 92.2 Chub mackerel 87.5 90.6 89.1 90.6 87.5 90.6 Goldeye rockfish 84.4 82.8 87.5 84.4 82.8 84.4 Konoshiro gizzard shad 87.5 89.1 89.1 87.5 87.5 87.5 Large yellow croaker 87.5 93.8 90.6 87.5 92.2 90.6 Overall mean classification rate (%) 86.7 87.0 87.8 87.5 86.7 87.5
인공신경망에의한 6 개어종의음향학적식별 231 3,,,,, 6 Table 3. Table 3 - echo 64 (256 ) 25%., 64,,,,, 76.6%, 82.8%, 93.8%, 90.6%, 96.9%, 90.6%, 88.5%.,., 3 7.8%, 4.7%., 7.4%, 4.7%. 4.7%, 4.7%., Table 3,, PCA tuning., - echo,., -, echo.,,, (Foote, 1980; Clay and Horne, 1994; Jaffe, 2006; Nesse et al., 2009; Stanton et al., 2010; Fassler et al., 2012). (epoch) 5,000, MSE 0.001 6, MSE Fig. 5. Fig. 5 5,000 Fig. 5. Performance curve of the artificial neural network with the input layer of 100 neurons, one hidden layer of 100 neurons and the output layer of 6 neurons used in identifying the target species based on the time-frequency images obtained from the broadband echoes of six fish species. Table 3. Confusion matrix of species classification results by the artificial neural network with one hidden layer of 100 neurons and a set of the reduced 100-dimensional feature components by PCA based on the time-frequency images of broadband echoes for six fish species Actual class Fish species Black rockfish Black scraper Chub mackerel Predicated class Goldeye rockfish K. gizzard shad L. yellow croaker Total C. rate 1 (%) Black rockfish 49 2 5 3 2 3 64 76.6 Black scraper 1 53 1 3 4 2 64 82.8 Chub mackerel 0 1 60 0 3 0 64 93.8 Goldeye rockfish 3 2 0 58 1 0 64 90.6 K. gizzard shad 0 0 1 1 62 0 64 96.9 L. yellow croaker 2 0 0 2 2 58 64 90.6 Total 55 58 67 67 67 63 384 88.5 1 The C. rate represent the percentage of successful classification. The bottom-right corner provides the overall mean classification rate (88.5%).
232 이대재 MSE 0.001 0.024,, 6 Table 3 88.5%., Fig. 5,. Fig. 5 MSE, epoch epoch. 2,625 PCA 100.,, 88.5%.,,.. 사사 2013 ( ) ( NRF-2013R1A1A2004378). References Bai Y, Zhang H and Hao Y. 2009. The performance of the backpropagation algorithm with varying slope of the activation function. Chaos, Solutions Fractals 40, 69-77. Clay CS and Horne JK. 1994. Acoustic models of fish: The Atlantic cod (Gadus morhua). J Acoust Soc Am 96, 1161-1668. Demuth H, Beale M and Hagan M. 2009. Neural Network Toolbox 6 User s Guide. The MathWorks Inc, Massachusetts, USA, 84-226. Dong Y and Cui Y. 2012. Analysis of a new joint time-frequency distribution of suppressing cross-term. Res J Appl Sci Eng Technol 4, 1580-1584. Fassler SMM, Fernandes PG, Semple SIK and Brierley AS. 2009. Depthe-dependent swimbladder compression in herring Clupea haengus obserbed using magnetic resonance imaging. J Fish Bio 74, 296-303. http://dx.doi.org/ 10.1111/j.1095-8649.2008.02130.x. Fernandes PG. 2009. Classification trees for species identification of fish-school echotraces. ICES J Mar Sci 66, 1073-1080. http://dx.doi.org/10.1093/icesjms/fsp060. Foote KG. 1980. Importance of the swimbladder in acoustic scattering by fish: A Comparison of gadoid and mackerel target strengths. J Acoust Soc Am 67, 2084-2089. Gavrovska AM, Paskas MP and Reljin IS. 2010. Determination of morphologically characteristic PCG segments from spectrogram image. Teflor J 2, 74-77. Han SK and Kim HT. 2010. Efficient radar target recognition using a combination of range profile and time-frequency analysis. Progress Electrom Res 108, 131-141. Imberger J and Boashash B. 1986. Application of the Wigner- Ville distribution to temperature gradient microstructure: A new technique to study small-scale variations. J Physic Oceanography 16, 1997-2012. Jaffe JS. 2006. Using multi-angle scattered sound to size fish swimbladders. ICES J Mar Sci 63, 1397-1404. http://dx.doi. org/ 10.1016/j.icesjms.2006.04.024. Kuruvilla J and Gunavathi K. 2014. Lung cancer classification using neural networks for CT images. Computer Methods Programs Biomedicine 113, 202-209. Latha P, Ganesan L and Annadurai S. 2009. Face recognition using neural networks. Signal Processing: An International J 3, 153-160. Lee DJ, Kang HY and Kwak MS. 2015. Analysis and classification of broadband acoustic echoes from individual live fish using the pulse compression technique. Korean J Fish Aquat Sci 48, 207-220. http://dx.doi.org/10.5657/ KFAS.2015.0207. Lee DJ. 2015a. Time-frequency analysis of broadband acoustic scattering from chub mackerel Scomber japonicas, goldeye rockfish Sebestes thompsoni, and fat greenling Hexagrammos otakii. Korean J Fish Aquat Sci 48, 221-232. http:// dx.doi.org/10.5657/kfas.2015.0221. Lee DJ. 2015b. Changes in the orientation and frequency dependence of target strength due to morphological differences in the fish swim bladder. Korean J Fish Aquat Sci 48, 233-243. http://dx.doi.org/10.5657/kfas.2015.0233. Lee DJ, Kang HY and Pak YY. 2016. Time-frequency feature extraction of broadband echo signals from individual live fish for species identification. Kor J Fish Aquat Sci 49, 214-223. http://dx.doi.org/10.5657/kfas.2016.0214. Nesse TL, Hobek H and Korneliussen RJ. 2009. Measurement of acoustic-scattering spectra from the whole and pars of Atlantic mackerel. ICES J Mar Sci 66, 1169-1175. http:// dx.doi.org/ 10.1093/icesjms/fsp087. Pinjare SL and Arun Kumar M. 2012. Implementation of neural network back propagation training algorithm on FPGA. International J Computer Appl 52, 1-7. Saad MHM, Nor MJM, Bustami FRA and Ngadiran R. 2007.
Classification of heart abnormalities using artificial neural network. J Appl Sci 7, 820-825. Santo RdE. 2012. Principal component analysis applied to digital image compression. Eistein 10, 135-139. Shilbayeh NF, Alwakeel MM and Naser MM. 2013. An efficient neural network for recognition gestural Hindi digits. American J Appl Sci 10, 938-951. Shui PL, Shang HY and Zhao YB. 2007. Instantaneous frequency estimation based on directionally smoothed pseudo- Wegner-Ville distribution bank. IET Radar Sonar Navig 1, 317-325. http://dx.doi.org/10.1049/rsn:20060123. Simmons EJ, Armstong F and Copland PJ. 1996. Species identification using wideband backscattering with neural network and discriminant analysis. ICES J Mar Sci 53, 189-195. Stanton TK, Chu D, Jech JM and Irish JD. 2010. New broadband methods for resonance classification and high-resolution imagery of fish with swimbladders using a modified commercial broadband echosounder. ICES J Mar Sci 67, 365-378. http://dx.doi.org/10.1093/icesjms/fsp262. Tsagarakis K, Giannoulaki, M, Pyrounaki M and Machias A. 2015. Species identification of small pelagic fish schools by means of hydroacoustics in the Eastern Mediterranean Sea. Medit Mar Sci 16, 151-161. http://dx.doi.org/10.12681/ mms.799. Woillez M, Ressler PH and Wilson CD. 2012. Multifrequency species classification of acoustic- trawl survey data using semi-supervised learning with class discovery. J Acoust Soc Am 131, EL184-EL190. http://dx.doi. org/10.1121/1.3678685. Zuo W, Zhang D and Wang K. 2006. Bidirectional PCA with assembled matrix distance metric for image recognition. IEEE Trans Sys Man Cyber 36, 863-872. 인공신경망에의한 6 개어종의음향학적식별 233