(JBE Vol. 24, No. 2, March 2019) (Regular Paper) 24 2, (JBE Vol. 24, No. 2, March 2019) ISSN

Transcription

1 (Regular Paper) 24 2, (JBE Vol. 24, No. 2, March 2019) ISSN (Online) ISSN (Print) HEVC a), a), b), b), c), c),, a) Performance Analysis of Super-Resolution based Video Coding for HEVC Sehwan Ki a), Dae-Eun Kim a), Ki Nam Jun b), Seung Ho Baek b), Jeung Won Choi c), Dong Hyun Kim c), and Munchurl Kim a)..,.,.., HEVC. HEVC. Abstract Since the resolutions of videos increase rapidly, there are continuing needs for effective video compression methods despite an increase in the transmission bandwidth. In order to satisfy such a demand, a reconstructive video coding (RVC) method by using a super resolution has been proposed. Since RVC reduces the resolution of the input video, when frames are compressed to the same size, the number of bits per pixel increases, thereby reducing coding artifacts caused by video coding. However, RVC method using super resolution is not effective in all target bitrates. Comparing the size of the loss generated while downsizing the resolution and the size of the loss caused by the video compression, only when the size of loss generated in the video compression is larger, RVC method can perform the improved compression performance compared to direct video coding. In particular, since HEVC has considerably higher compression performance than the previous standard video codec, it can be experimentally confirmed that the compression distortions become larger than the distortions of downsizing the resolution only in the very low-bitrate conditions. In this paper, we applied RVC based HEVC in various video types and measured the target bitrates that RVC method can be effectively applied. Keyword : Super-Resolution, Scalable Video Coding, High Efficiency Video Coding Copyright 2019 Korean Institute of Broadcast and Media Engineers. All rights reserved. This is an Open-Access article distributed under the terms of the Creative Commons BY-NC-ND ( which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited and not altered.

2 6 : HEVC (Sehwan Ki et al.: Performance Analysis of Super-Resolution based Video Coding for HEVC)... ISO/IEC Moving Picture Experts Group (MPEG) ITU-T Video Coding Experts Group (VCEG) Joint Collabora- tive Team on Video Coding (JCT-VC) High Efficiency Video Coding [1] , Versatile Video Coding (VVC) MPEG VCEG Joint Video Exploration Team (JVET) [2].,, [3-9]. [3-6], [7-9]., [10-14].,., a) (The School of Electrical Engineering, Korea Advanced Institute of Science and Technology) b) LIG (LIG Nex1) c) (Agency for Defense Development) Corresponding Author : (Munchurl Kim) mkimee@kaist.ac.kr Tel: ORCID: " " (UC170016ED). Manuscript received December 21, 2018; Revised February 28, 2019; Accepted February 28, HEVC, / / [10-14],, /2, 1/4.,,.. 1/2, 2., (frame rate up-conversion) (bit-depth enhancement) 10..

3 1. Fig 1. A block diagram of a reconstructive video coding..,.,.,.., 2. 2 (a) 1,,. 2...,. (a) 2. Fig 2. The performance graph of super-resolution based reconstructed video coding framework (b)

4 6 : HEVC (Sehwan Ki et al.: Performance Analysis of Super-Resolution based Video Coding for HEVC).,., 2 (a) A. A. A. A., AVC/H.264 HEVC/H.265 A. A. Bicubic..,. 2 (b),., A B.,... HEVC/H.265 Bicubic , Leaky ReLU. (residual learning),, skip. [12]. 1.,. 2 A bicubic 1/2., 1/2, ( ). QP (Quantization Parameter). QP 40, 43, 46, 49, 51 5 QP. QP., QP 40, 43 QP 46, 49, 51. CIF( ). L1,

5 3. Fig 3. Structure of up-sampling network based on a convolutional neural network (Adam optimizer) [14] , HEVC, 1/2 HEVC bicubic. QP, PSNR BQSquare, Basketball Pass, BlowingBubbles, RaceHorses. I- 32 GOP 8. HEVC HM ,, PSNR. QP 6. PSNR. 5. 5, 1/2 Bicubic.. 5-(a). Bicubic (a) BQSqaure (b) BasketballPass (c) BlowingBubbles (d) RaceHorses 4. Fig 4. Test sequences for performance test of reconstructed video coding framework

6 6 : HEVC (Sehwan Ki et al.: Performance Analysis of Super-Resolution based Video Coding for HEVC) 1. Table 1. Coding performance of the conventional HEVC and the reconstructed video coding framework Sequences BQSquare BasketballPass BlowingBubbles RaceHorses QP Downscale = 1 Downscale = 2 (Bicubic) Downscale = 2 (CNN) kbitrate PSNR(dB) kbitrate PSNR(dB) kbitrate PSNR(dB) Fig 5. The performance graph of reconstructed video coding framework for each test sequence

7 PSNR. BQSquare.. 5-(b), (c), (d) PSNR. 5-(b) 50kbps. QP 46.. QP db.,., Bicubic PSNR.,. 5-(c), 30kbps. 5-(d) 35kbps PSNR.. QP 46,...., HEVC... (References) [1] G. J. Sullivan, J.-R. Ohm, W.-J. Han, T. Wiegand, "Overview of the High Efficiency Video Coding (HEVC) standard", IEEE Trans. Circuits Syst. Video Technol., vol. 22, pp , Dec [2] B. Bross, Working Draft 1 of Versatile Video Coding, document JVET-J1001, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, Apr [3] W.-S. Park, M. Kim, "CNN-based In-loop Filtering for Coding Efficiency Improvement," Proceeding of IEEE Image Video and Multidimensional Signal Processing (IVMSP) workshop, Bordeaux, France, pp. 1-5, [4] N. Yan, D. Liu, H. Li, F. Wu, A convolutional neural network approach for half-pel interpolation in video coding, Proceeding of International Symposium on Circuits and Systems, Baltimore, Maryland, pp. 1-4, [5] D. Liu, H. Ma, Z. Xiong, F. Wu, CNN-based DCT-like transform for image compression, Proceeding of International Conference on Multimedia Modeling, Bangkok, Thailand, pp , [6] Z. Liu, X. Yu, Y. Gao, S. Chen, X. Ji, D. Wang, CU partition mode decision for HEVC hardwired intra encoder using convolution neural network, IEEE Trans. Image Processing, vol. 25, no. 11, pp , Nov [7] E. Agustsson, F. Mentzer, M. Tschannen, L. Cavigelli, R. Timofte, L. Benini, L. V. Gool, Soft-to-hard vector quantization for end-to-end learning compressible representations, Proceeding of Advances in Neural Information Processing Systems, Long beach, California, pp.

8 6 : HEVC (Sehwan Ki et al.: Performance Analysis of Super-Resolution based Video Coding for HEVC) , [8] J. Ballé, V. Laparra, E. P. Simoncelli, End-to-end optimized image compression, Proceeding of International Conference on Learning Representations, Toulon, France, [9] C.-Y. Wu, N. Singhal, P. Krähenbühl, Video compression through image interpolation, Proceeding of European Conference on Computer Vision, Munich, Germany, [10] D. Barreto, L. D. Alvarez, R. Molina, A. K. Katsaggelos, and G. M. Callicó, Region-based super-resolution for compression, Multidimensional Systems and Signal Processing, vol. 18, no. 2-3, pp , Sept [11] V.-A. Nguyen, Y.-P. Tan, and W. Lin, Adaptive Downsampling/ Upsampling for Better Video Compression at Low Bit Rate, Proceeding of IEEE ISCS, Seattle, WA, USA, pp , [12] M. Shen, P. Xue, and C. Wang Down-sampling Based Video Coding Using Super-Resolution Technique, IEEE Trans. CSVT, vol. 21, no. 6, pp , June [13] Y. Dar, and A. M. Bruckstein, (Apr. 2014). Improving low bit-rate video coding using spatio-temporal down-scaling, [Online]. Available: [14] H. Chen, X. He, M. Ma, L. Qing, and Q. Teng, Low bit rates image compression via adative block downsampling and super resolution, Journal of Electronic Imaging, vol. 25, no. 1, pp :1-10, Jan : : ~ : - ORCID : - :, : : ~ : - ORCID : hjttp://orcid.org/ : HDR, : : ~ : LIG - ORCID : - :,,,,

9 : : ~ : ~ : LIG - ORCID : - :,,, : : ( ) : ~ : ~ : - ORCID : - :,,,, : : ~ : 2 - ORCID : hjttp://orcid.org/ :,, : : University of Florida, Dept. of Electrical and Computer Engineering, : University of Florida, Dept. of Electrical and Computer Engineering, ~ :, ~ : / ~ : / - ORCID : hjttp://orcid.org/ : Perceptual Video Coding, SDR/HDR Image/Video Quality Assessment and Modeling, Super-Resolution, Image/Video Analysis and Understanding, Pattern Recognition, Machine Learning