2: CNN (Jaeyoung Kim et al.: Experimental Comparison of CNN-based Steganalysis Methods with Structural Differences) (Regular Paper) 24 2, 2019 3 (JBE Vol. 24, No. 2, March 2019) https://doi.org/10.5909/jbe.2019.24.2.315 ISSN 2287-9137 (Online) ISSN 1226-7953 (Print) CNN a), a), b) Experimental Comparison of CNN-based Steganalysis Methods with Structural Differences Jaeyoung Kim a), Hanhoon Park a), and Jong-Il Park b).. CNN CNN. CNN., CNN.,. Abstract Image steganalysis is an algorithm that classifies input images into stego images with steganography methods and cover images without steganography methods. Previously, handcrafted feature-based steganalysis methods have been mainly studied. However, CNN-based objects recognition has achieved great successes and CNN-based steganalysis is actively studied recently. Unlike object recognition, CNN-based steganalysis requires preprocessing filters to discriminate the subtle difference between cover images from stego images. Therefore, CNN-based steganalysis studies have focused on developing effective preprocessing filters as well as network structures. In this paper, we compare previous studies in same experimental conditions, and based on the results, we analy ze the performance variation caused by the differences in preprocessing filter and network structure. Keyword : Image steganography, CNN-based steganalysis, preprocessing filter, CNN structure, experimental comparison a) (Department of Electronic Engineering, Pukyong National University) b) (Department of Computer Science) Corresponding Author : (Hanhoon Park) E-mail: hanhoon_park@pknu.ac.kr Tel: +82-51-629-6225 ORCID: https://orcid.org/0000-0002-6968-4565. This work was supported by the research fund of Signal Intelligence Research Center supervised by Defense Acquisition Program Administration and Agency for Defense Development of Korea. Manuscript received December 31, 2018; Revised February 18, 2019; Accepted February 18, 2019. Copyright 2019 Korean Institute of Broadcast and Media Engineers. All rights reserved. This is an Open-Access article distributed under the terms of the Creative Commons BY-NC-ND (http://creativecommons.org/licenses/by-nc-nd/3.0) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited and not altered.
(JBE Vol. 24, No. 2, March 2019). (steganography) 3. 3. (,,, ),. LSB(least significant bits) [8] PVD(pixel valued differencing) [9], Multiway- PVD [10-11], Layered-PVD [37]. PVD Multiway-PVD (cover) (stego). [38]. (LSB, PVD, Multiway-PVD, Layered-PVD, ) (steganalysis). HUGO [12], WOW [13], S-UNIWARD [14] (, ). (handcrafted). ALE [15], SPAM [16,39], SRM [17]. SPAM 1 LSB // 2 (co-occurrence matrix) SVM(support vector machine) [26]. SRM(spatial rich model) /. 10000 SVM FLD(Fisher s linear discriminant) [18] (random forest) (ensemble) [19]. CNN(convolutional neural network) [20-22,30] CIFAR [31], ILSVRC [32]. AlexNet [21] [33] CNN. VGG [30] CNN. [1-7]. CIFAR 10 CNN. CNN CNN (pooling) (stride).
2: CNN (Jaeyoung Kim et al.: Experimental Comparison of CNN-based Steganalysis Methods with Structural Differences),,., CNN BOSSBase 1.01 [23] (, payload ),,.. CNN 2 CNN,. 1. CNN(convolutional neural network) CNN (fully connected) CNN. 2-5 20. 3 2 6. CNN. 1. BOSSBase 1,000 1LSB(bpp = 0.4), S-UNIWARD(bpp = 0.4) Fig. 1. Histogram of neighboring pixel value differences of 1,000 cover images in BOSSBase data sets and their stego images made by 1LSB(bpp = 0.4) and S-UNIWARD(bpp = 0.4). CNN. CNN CIFAR- 10. (a) 2. 1. (a), (b) Fig. 2. Layers in 1D-input neural network. (a) Fully connected layer, (b) convolutional layer (b)
(JBE Vol. 24, No. 2, March 2019)..,,, 1 ( 3 ).. 4. Xu-Net [2] CNN Fig. 4. CNN proposed in Xu-net [2] 3. CNN.,,, CNN, A = {0, 3, 1, 2}, B = {2, 5, 2, 1} Fig. 3. CNN parameters, Two CNNs with different padding sizes, filter sizes, stride, and filter channels, A = {0, 3, 1, 2}, B = {2, 5, 2, 1} 3 CNN Xu-net [2] 5 2 ( ) ABS(absolute) [2], BN(batch normalization) [24] pooling [25] ( 4 ). 1.1 BN(Batch normalization) (supervised learning). (global minima) (gradient decent methods).. 8 0~1. [34,35]. 2, 3.
2: CNN (Jaeyoung Kim et al.: Experimental Comparison of CNN-based Steganalysis Methods with Structural Differences) 5. ABS BN /. (a), (b) ABS, BN Fig. 5. Histogram before/after applying ABS and BN. (a) input, (b) after ABS, (c) after BN 4. 5 /. 1.2 (pooling).,,,,. 2... 2. (Image steganography) LSB. HUGO, WOW, S-UNIWARD 3. WOW S-UNIWARD Daubechies 8 LPF(low pass filter), HPF(high pass filter) 6 3,,... WOW,,, S-UNIWARD. 7 payload=0.4.
(JBE Vol. 24, No. 2, March 2019) 6. WOW S-UNIWARD 3 Fig. 6. Three filters used in WOW and S-UNIWARD (a) (b) (c) 7. S-UNIWARD WOW. (a), (b) S-UNIWARD, (c) WOW Fig. 7. Secret information insertion regions by S-UNIWARD and WOW. (a) Cover image, (b) S-UNIWARD, (c) WOW 1 LSB. 3. (Image steganalysis). ALE, SPAM. 1. CNN ([1-7] ). LSB, PVD ALE, SPAM. HUGO, WOW, S-UNIWARD 10000 SRM. SVM FLD
2: CNN (Jaeyoung Kim et al.: Experimental Comparison of CNN-based Steganalysis Methods with Structural Differences). CNN. 1 HPF (high pass filter) SRM. CNN,,,, CNN.. CNN SC(slelection-channel) [36] CNN,, CNN 5. 10 [2-3,5-6] 4 BN 4 BN. [2] 1 HPF [3] [2] 3 HPF. [4] [5] 30 SRM [6] 3.. 5 (WOW S-UNIWARD) (payload), CNN. 3.1 3.2. 1. CNN [2] Xu-net 5 2. 5 HPF. HPF ABS(absolute activation) BN. 2 5 ABS 0. tanh 0. [3] Xunet. 7, 8 Xu-net 2. [4] SRM. SRM 30. SRM TLU(truncated linear unit) 0 ( 8. (a) ). 30 8 1. [5] Xu-net BN, ABS [4] SRM TLU. SRM T=3, T=2 TLU BN. [6] SRM, SRM, Gabor [27] 3 16, 14, 16. Gabor 8,,. ReST-net 0~ 8 or 16. 3 3 10 (e). 5
(JBE Vol. 24, No. 2, March 2019) (a) (b) (c) 8. 4. (a) TLU, (b) ReLU, (c) tanh, (d) sigmoid Fig. 8. 4 activation functions. (a) TLU, (b) ReLU, (c) tanh, (d) sigmoid (d) 2, 4 DAM(diverse activation module) ( 9 ). DAM 8 (b), (c), (d) 3 ReLU, tanh, sigmoid. CNN 10. exp cos 9. DAM Fig. 9. DAM structure
2: CNN (Jaeyoung Kim et al.: Experimental Comparison of CNN-based Steganalysis Methods with Structural Differences) (a) (b) (c) (d) (e) 10. CNN. (a) [2], (b) [3], (c) [4], (d) [5], (e) [6] Fig. 10. Network structures of CNN-based steganalysis methods used in performance analysis. (a) Method of [2], (b) method of [3], (c) method of [4], (d) method of [5], (e) method of [6]
(JBE Vol. 24, No. 2, March 2019) 2. CNN BOSSBase 1.01 [23] 10,000 4 40,000. 40,000 S-UNIWARD WOW 40,000. [28] Windows C++ Matlab. payload 0.2, 0.4. 1 1 60,000 20,000. (tensorflow) [29] S- UNIWARD WOW 1, 2 3. 1 2 S-UNIWARD WOW. 11 20,000 32/32 / 1000. 1. S-UNIWARD Table 1. Classification results for S-UNIWARD fixed key random key paper iteration payload=0.2 payload=0.4 payload=0.2 payload=0.4 [2] 200,000 0.67258 0.75602 0.66748 0.79158 [3] 200,000 0.59453 0.75992 0.59631 0.74195 [4] 100,000 0.77894 0.86453 0.68609 0.80656 [5] 100,000 0.66656 0.79383 0.67247 0.78625 [6] 50,000 0.55388 0.73620 100,000 0.60098 0.76719 0.64994 0.77238 2. WOW Table 2. Classification results for WOW fixed key random key paper iteration payload=0.2 payload=0.4 payload=0.2 payload=0.4 [2] 200,000 0.63214 0.76269 0.69930 0.79702 [3] 200,000 0.61322 0.70811 0.70672 0.78714 [4] 100,000 0.66308 0.81608 0.74703 0.82398 [5] 100,000 0.62330 0.77574 0.70111 0.79234 [6] 50,000 0.53030 0.61514 100,000 0.65652 0.73864 0.68791 0.65486 3. CNN Table 3. Hyperparameters of CNNs used in experiments paper initializer optimizer convolution fully-connected optimizer moment learning rate [2] gaussian xavier Momentum 0.9 0.001 [3] gaussian xavier Momentum 0.9 0.001 [4] gaussian gaussian AdaDelta 0.95 0.4 [5] xavier xavier Momentum 0.95 0.01 [6] gaussian gaussian Momentum 0.9 0.001
2: CNN (Jaeyoung Kim et al.: Experimental Comparison of CNN-based Steganalysis Methods with Structural Differences) 1 S-UNIWRD. [2] [3]. payload [2] 0.67258, 0.66748. [4] payload 0.86453. CNN,. [5] [4], TLU [2] ABS, BN CNN. [6] DAM 3 CNN payload=0.2. 2 WOW. S-UNIWARD. [2] S-UNIWARD. [3] payload=0.2 0.10. [4] payload=0.2 0.66308 S-UNIWARD 0.13. [5] 0.04. [6] payload=0.4 0.12. [2] [3] HPF CNN. 1 HPF [2] 3 [3] 3 HPF [3]. 10 [2], [3] 5 8, 16, 32, 64, 128. [3] 3. 1 3 CNN. [4] CNN 4 TLU BN. 30 SRM 30, 30, 30, 32, 32, 32, 16, 16. [5] [2] ABS, BN, TLU. [4] [5] 30 SRM [4]. [4] [5]. [6], SRM Gabor 3 DAM 1 HPF [2]. 3, 3.. CNN S- UNIWARD WOW, payload. [7] [2-6] 5 CNN. BN 1 HPF, 3 HPF, 30 SRM, SRM Gabor. CNN 1 3 HPF, SRM,, SRM Gabor..
(JBE Vol. 24, No. 2, March 2019) 5 [2-6] CNN [4] CNN BN ABS CNN AdaDelta. [6] 3 1 [2]... [2] ABS, BN 5. CNN [4] CNN. CNN.. [4] [5] 30 SRM 30, 30, 30, 32, 32, 32, 16, 16 30, 30, 32, 64, 128. CNN CNN. (References) [1] L. Pibre, J. Pasquet, D. Ienco and M. Chaumont, Deep learning is a good steganalysis tool when embedding key is reused for different images, even if there is a cover source-mismatch, Society for Imaging Science and Technology, pp.1-11, 2016. [2] G. Xu and H. Wu, Structural design of convolutional neural networks for steganalysis, IEEE Signal Processing Letters, Vol.23, No.5, pp.708-712, 2016. [3] Y. Yuan, W. Lu, B. Feng and J. Weng, Steganalysis with CNN using multi-channels filtered residuals, ICCCS 2017, pp.110-120, 2017. [4] J. Ye, J. Ni and Y. Yi, Deep learning hierarchical representations for image steganalysis, IEEE Transactions on Information Forensics and Security, Vol.12, No.11, pp.2545-2557, 2017. [5] M. Yedroudj, F. Comby and M. Chaumont, Yedroudj-Net: an efficient CNN for spatial steganalysis, ICASSP 2018, pp.15-20, 2018. [6] B. Li, W. Wei, A. Ferreira and S. Tan, ReST-Net: diverse activation modules and parallel subnets-based CNN for spatial image steganalysis, IEEE Signal Processing Letters, Vol.25, No.5, pp.650-654, 2018. [7] J. Yang, K. Liu, X. Kang, E. Wong and Y. Shi, Steganalysis based on awareness of selection-channel and deep learning, IWDW 2017, pp.263-272, 2017. [8] D. Neeta and K. Snehal, Implementation of LSB steganography and it s evaluation for various bits, 1st International Conference on Digital Information Management, pp.173-178, 2006. [9] D. Wu and W. Tsai, A steganographic method for images by pixel-value differencing, Pattern Recognition Letters, Vol.24, pp.1613-1626, 2003. [10] K. Chang, C. Chang, P. S. Huang and T. Tu, A novel image steganographic method using tri-way pixel-value differencing, Journal of Multimedia, Vol.3, No.2, pp.37-44, 2008. [11] C. Balasubramanian, S. Selvakumar and S. Geetha, High payload image steganography with reduced distortion using octonary pixel pairing scheme, Multimed Tools Appl., Vol.73, pp.2223-2245, 2014. [12] G. Gul and F. Kurugollu, A new methodology in steganalysis: breaking highly undetectable steganography (HUGO), International Workshop on Information Hiding, pp.71-84, 2011. [13] V. Holub and J. Fridrich, Designing steganographic distortion using directional filters, International Workshop on Information Forensics and Security, 2012. [14] V. Holub, J. Fridrich and T. Denemark, Universal distortion function for stegangography in an arbitrary domain, EURASIP Journal of Information Security, 2014. [15] G. Cancelli, G. Doerr, I. J. Cox and M. Barni, Detection of ±1 LSB steganography based on the amplitude of histogram local extrema, ICIP, pp.1288-1291, 2008. [16] T. Pevny, P. Bas and J. Fridrich, Steganalysis by subtractive pixel adja cency matrix, IEEE Transactions on Information Forensics and Security, Vol.5, no.2, pp.215-224, June 2010. [17] J. Fridrich and J. Kodovsky, Rich models for steganalysis of digital im ages, IEEE Transactions on Information Forensics and Security, Vol. 7, No.3, pp.868-882, June 2012. [18] R. Haeb-Umbach and H. Ney, Linear discriminant analysis for improved large vocabulary continuous speech recognition, IEEE International Conference on Acoustics, Speech, and Signal Processing, San Francisco, CA, USA, pp.13-16, 1992.
2: CNN (Jaeyoung Kim et al.: Experimental Comparison of CNN-based Steganalysis Methods with Structural Differences) [19] Tin Kam Ho, Random decision forests, Proceedings of 3rd International Conference on Document Analysis and Recognition, Montreal, Quebec, Canada, Vol.1, pp.278-282, 1995. [20] Y. Lecun, L. Bottou, Y. Bengio and P. Haffner, Gradient-based learning applied to document recognition, in Proceedings of the IEEE, Vol. 86, No.11, pp.2278-2324, Nov. 1998. [21] A. Krizhevsky, I. Sutskever and G. E. Hinton, ImageNet classification with deep convolutional neural networks, NIPS, 2012. [22] C. Szegedy et al., Going deeper with convolutions, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, pp.1-9, 2015. [23] P. Bas, T. Filler and T. Pevny, Break our steganographic system the ins and outs of organizing BOSS, Information Hiding 13th International Conference, Vol.6958, pp.59-70, 2011. [24] S. Ioffe and C. Szegedy, Batch normalization: acceleration deep network training by reducing internal covariate shift, 2015, https://arxiv.org/abs/1502.03167. [25] K. He, X. Zhang, S. Ren and J. Sun, Spatial pyramid pooling in deep convolutional networks for visual recognition, ECCV, pp.346-361, 2014. [26] C. Cortes and V. Vapnik, Support-vector networks, Machine Learning, Vol.20, pp.273-297, 1995. [27] X. Song, F. Liu, C. Yang, X. Luo and Y. Zhang, Steganalysis of adaptive JPEG steganography using 2D Gabor filters, in Proc. 3rd ACM Inf. Hiding Multimedia Secur. Workshop, pp.1523, 2015. [28] DDE LAB download, http://dde.binghamton.edu/download/ [29] Tensorflow, https://www.tensorflow.org/ [30] K. Simonyan and A. Zisserman, Very deep convolutional networks for large-scale image recognition, ICLR, 2015. [31] The cifar-10 dataset, https://www.cs.toronto.edu/~kriz/cifar.html [32] ILSVRC, http://image-net.org [33] J. Sanchez and F. Perronnin, High-dimensional signature compression for large-scale image classification, CVPR 2011, Colorado Springs, CO, USA, pp.1665-1672, 2011. [34] S. Hochreiter, Y. Bengio, P. Frasconi and J. Schmidhuber, Gradient flow in recurrent nets: the difficulty of learning long-term dependencies, in A Field Guide to Dynamical Recurrent Networks, IEEE, 2001, doi: 10.1109/9780470544037. [35] S. Kong and M. Takatsuka, Hexpo: A vanishing-proof activation function, 2017 International Joint Conference on Neural Networks (IJCNN), Anchorage, AK, pp.2562-2567, 2017. [36] J. Ye, J. Ni and Y. Yi, Deep learning hierachical representations for image steganalysis, IEEE Transactions on Information Forensics and Security, Vol.12, No.11, pp.2545-2557, 2017. [37] J. Kim and H. Park, Image steganography using layered pixel-value differencing, Jounal of Broadcasting Engineering, Vol.22, No.3, 2017. [38] J. Kim and H. Park, A statistical approach for improving the embedding capacity of block matching based image steganography, Journal of Broadcast Engineering, Vol.22, No.5, 2017. [39] J. Kim, H. Park, J. Park, Experimental verification of the versatility of SPAM-based image steganalysis, Journal of Broadcast Engineering, Vol.23, No.4, pp.525-535, 2018. - 2012 : - 2016 ~ : - ORCID : http://orcid.org/0000-0003-2348-3626 - :,, - 2000 : - 2002 : - 2007 : - 2008 ~ 2011 : NHK - 2012 ~ : - ORCID : http://orcid.org/0000-0002-6968-4565 - :,, 3 /
(JBE Vol. 24, No. 2, March 2019) - 1987 : - 1989 : - 1995 : - 1992 ~ 1994 : NHK - 1995 ~ 1996 : - 1996 ~ 1999 : ATR - 1999 ~ : - ORCID : http://orcid.org/0000-0003-1000-4067 - :,, 3,