(JBE Vol. 24, No. 1, January 2019) (Regular Paper) 24 1, 2019 1 (JBE Vol. 24, No. 1, January 2019) https://doi.org/10.5909/jbe.2019.24.1.142 ISSN 2287-9137 (Online) ISSN 1226-7953 (Print) a), a), b), a) Method of Video Stitching based on Minimal Error Seam Jeonho Kang a), Junsik Kim a), Sang-IL Kim b), and Kyuheon Kim a)...,. Abstract There is growing interest in ultra-high-resolution content that gives a more realistic sense of presence than existing broadcast content. However, in order to provide ultra-high-resolution contents in existing broadcast services, there are limitations in view angle and resolution of the image acquisition device. In order to solve this problem, many researches on stitching, which is an image synthesis method using a plurality of input devices, have been conducted. In this paper, we propose method of dynamic object based video stitching using minimal error seam in order to overcome the temporal invariance degradation of moving objects in the stitching process of horizontally oriented videos. Keyword : Video Stitching, Minimal Error Seam, Blending, Edge Detection a) (Department of Electronic Engineering, Kyung Hee University) b) (School of Media Seoul Institute of the Arts) Corresponding Author : (kyuheon Kim) E-mail: kyuheonkim@khu.ac.kr Tel: +82-31-201-3810 ORCID: http://orcid.org/0000-0003-1553-936x This research has been sponsored by the Korea Electric Power Corporation under grant number R18XA02. Manuscript received November 9,2018; Revised December 11, 2018; Accepted December 11, 2018.. 2012 HD(High Definition)., (3840*2160) UHD(Ultra High Definition) 50 TV [1]. Copyright 2016 Korean Institute of Broadcast and Media Engineers. All rights reserved. This is an Open-Access article distributed under the terms of the Creative Commons BY-NC-ND (http://creativecommons.org/licenses/by-nc-nd/3.0) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited and not altered.
3 : (Jeonho Kang et al.: Method of Video Stitching based on Minimal Error Seam) [2]. [3]. [4][5][6]. [7]. [8].,..,., [9].,. 2, 3, 4, 5.. Related work... [10]. 1. Feature Matching Pixel. 1. Fig. 1. Example of minimal error seam extracting
(JBE Vol. 24, No. 1, January 2019)... [11]. 1. 1 2.,. Pixel Error. Pixel.. (1).,,. i x pixel, j y.. e(i,j) Error. Error.. (2). min (2). S(i,j) e(i,j) y 3.., e(i,j) i 1., i 0 j,, i 1.., Error. Error Error., MES(j). min MES(j) Y.,. 2 system. 3.. The proposed stitching method based on Minimal error seam 2 Stitching. 3 stitching system 2.,..
3 : (Jeonho Kang et al.: Method of Video Stitching based on Minimal Error Seam) 2. Fig. 2. Stitching systems based on minimal error seam. HSV Edge Detection, Horizontal Variance Weight calculator. 1 HSV, 2 Ob- ject Detection, 3 Horizontal Variance Weight calculator. 1. HSV 2 1 pixel. pixel Hue, Saturation, Value. 2 (1) i,j Error, HSV (4) i,j Hue, Saturation, Value Error. 2. Object Detection 2 Object Detection. 3. Fig. 3. Problem of minimal error seam
(JBE Vol. 24, No. 1, January 2019) Object Detection 3. 4 23 5 23. object 23. Object detection 6. Object detection module, 4. 23 Fig. 4. 23th stitching region of left frame 5. 23 Fig. 5. 23th stitching region of right frame 6. Fig. 6. Object detection module 7. Fig. 7. Result of edge detection 8. Fig. 8. Result of edge detection with Gaussian filter
3 : (Jeonho Kang et al.: Method of Video Stitching based on Minimal Error Seam),, dilate & erode.. 7. 8. object dilate erode. object. 3. Horizontal Variance Weight calculator 3. 9 10. 11 92 12 93 9. 92 Fig. 9. 92th frame stitching result 10. 93 Fig. 10. 93th frame stitching result 11. 92 Fig. 11. 92th minimal error seam 12. 93 Fig. 12. 93th minimal error seam
(JBE Vol. 24, No. 1, January 2019)., x Horizontal variance weight(hvw), (5). Pr i X pixel, j Y pixel. j I error i. HVW 13. 13 Error, MES. MES HVW, MES MES. 3 MES, Error Hue, Saturation, Value 3 MES HVW MES. 4.. Experimental Result. OS Microsoft Windows 7, Video Stitcher Visual Studio 2013. Video Stitcher OpenCV 3.1.0.. 4 3. 1 HSV, 2 Object De- tection, 3 Horizontal Variance Weight calculator, 4 13. Horizontal Variance Weight Fig. 13. Method of generating Horizontal Variance Weight
3 : (Jeonho Kang et al.: Method of Video Stitching based on Minimal Error Seam). 1. HSV 4 1 3 1 HSV. 14 HSV. 14 HSV., HSV Object.,., HSV,. 2. Object Detection 4 2 3 2 Object Detection. 6 14. HSV Fig. 14. Existing minimal error seam Technology and HSV-based error seam Technology Stitching Results 15. Fig. 15. Result of labeling
(JBE Vol. 24, No. 1, January 2019) 15. 15,. Dilate Erode threshold Edge. 15 Dilate Erode. Dilate Erode,. 16. 3. Horizontal Variance Weight calculator 4 3 3 3 Horizontal Variance Weight. 17 20, 18 21.. MES edge detection. horizontal variance weight (5) MES MES. HVW 19 16. Fig. 16. Result of edge detection 17. 20 Fig. 17. Result of stitching (20th frame) 18. 21 Fig. 18. Result of stitching (21th frame)
3 : (Jeonho Kang et al.: Method of Video Stitching based on Minimal Error Seam) 19. HVW 20 Fig. 19. Result of stitching with HVW(20th frame) 20. HVW 21 Fig. 20. Result of stitching with HVW(21th frame) 20 20 21 MES. 4 1 MES HSV, 2 MES, 3 HVW MES.. Result,. MES HSV 3,, MES HVW.. (References) [1] Y. Sohn, M. Cho, and J. Paik, Design of MMT-based Broadcasting System for UHD Video Streaming over Heterogeneous Networks, JBE Vol. 20, No. 1, January 2015 [2] R. Szeliski and H.-Y. Shum, Creating full view panoramic image mosaics and environment maps, in Proc. 4th Annu. Conf. Comput. Graph. Interact. Techn., pp. 251 258, 1997. [3] M. S. Billah and H. Ahn, Objective Quality Assessment Method for Stitched Images, (JBE Vol. 23, No. 2, March 2018 [4] Y.D. Lowe, Distinctive image features from scale-invariant keypoints, International Journal of Computer Vision, vol. 60, no. 2, pp. 91-110, Nov. 2004. [5] B. He, G. Zhao, and Q. Liu, Panoramic video stitching in multicamera surveillance system, in Proc. 25th Int. Conf. Image Vis. Comput. New Zealand (IVCNZ), pp. 1 6, Nov. 2010. [6] W. Xu and J. Mulligan, Panoramic video stitching from commodity HDTV cameras, Multimedia Syst., vol. 19, no. 5, pp. 407 426, Oct. 2013. [7] W. Jiang and J. Gu, Video stitching with spatial-temporal content-preserving warping, in Proc. of IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 42-48, Jun. 2015. [8] Y. Xiong and K. Pulli, Fast panorama stitching for high-quality panoramic images on mobile phones, IEEE Trans. on Consumer Electronics, vol. 56, no. 2, pp. 298-306, May. 2010. [9] S. Shin, S. Yoon, H. Jung, J. Kim, K. Kim Video Blending based on Minimal Error Seam,. Korea Broadcast Media Engineering Conference, pp. 301-304, 2018. [10] Telecommunications Technology Association, Technical Report for High Quality Panorama Video, 2014. [11] Dr. Mayada F. Abdul-HalimT. Makela, N. A. IbraheemDental UTILIZING GENETIC ALGORITHMS FOR 2D TEXTURE SYNTHESIS, 3pg, 2015.
5 1 2 방송공학회논문지 제24권 제1호, 2019년 1월 (JBE Vol. 24, No. 1, January 2019) 저자소개 강전호 - 년 2월 : 경희대학교 전자전파공학과 공학사 년 2월 : 경희대학교 전자전파공학과 공학석사 년 3월 ~ 2015년 2월 : 주식회사 넥스트리밍 연구원 년 3월 ~ 현재 : 경희대학교 전자공학과 박사과정 : http://orcid.org/0000-0003-4405-3879 주관심분야 : 영상처리, 멀티미디어 시스템, 디지털 대화형방송 2010 2012 2012 2015 ORCID 김준식 년 경희대학교 전자전파공학과 공학사 년 현재 경희대학교 전자공학과 석사과정 주관심분야 영상처리 멀티미디어 시스템 디지털 대화형 방송 - 2017 : - 2017 ~ : - ORCID : https://orcid.org/0000-0002-0287-9640 :,, 김상일 - 년 2월 : 경희대학교 공과대학 전자공학과 학사 년 2월 : 광운대학교 정보콘텐츠대학원 3D콘텐츠 석사 년 7월 : 광운대학교 대학원 홀로그래피3D 박사 년 3월 ~ 현재 : 서울예술대학교 영상학부 교수 : https://orcid.org/0000-0003-4242-5405 주관심분야 : 4K, 8K UHD. 3DVR 1981 2014 2017 2016 ORCID 김규헌 - 년 2월 : 한양대학교 전자공학과 공학사 년 9월 : 영국 University of Newcastle upon Tyne 전기전자공학과 공학석사 년 7월 : 영국 University of Newcastle upon Tyne 전기전자공학과 공학박사 년 ~ 1997년 : 영국 University of Sheffield, Research Fellow 년 ~ 2006년 : 한국전자통신연구원 대화형미디어연구팀장 년 ~ 현재 : 경희대학교 전자정보대학 교수 : http://orcid.org/0000-0003-1553-936x 주관심분야 : 영상처리, 멀티미디어 통신, 디지털 대화형 방송 1989 1992 1996 1996 1997 2006 ORCID