(JBE Vol. 24, No. 1, January 2019) (Regular Paper) 24 1, (JBE Vol. 24, No. 1, January 2019) ISSN 2287

Transcription

1 (JBE Vol. 24, No. 1, January 2019) (Regular Paper) 24 1, (JBE Vol. 24, No. 1, January 2019) ISSN (Online) ISSN (Print) a), a), b), a) Method of Video Stitching based on Minimal Error Seam Jeonho Kang a), Junsik Kim a), Sang-IL Kim b), and Kyuheon Kim a)...,. Abstract There is growing interest in ultra-high-resolution content that gives a more realistic sense of presence than existing broadcast content. However, in order to provide ultra-high-resolution contents in existing broadcast services, there are limitations in view angle and resolution of the image acquisition device. In order to solve this problem, many researches on stitching, which is an image synthesis method using a plurality of input devices, have been conducted. In this paper, we propose method of dynamic object based video stitching using minimal error seam in order to overcome the temporal invariance degradation of moving objects in the stitching process of horizontally oriented videos. Keyword : Video Stitching, Minimal Error Seam, Blending, Edge Detection a) (Department of Electronic Engineering, Kyung Hee University) b) (School of Media Seoul Institute of the Arts) Corresponding Author : (kyuheon Kim) kyuheonkim@khu.ac.kr Tel: ORCID: This research has been sponsored by the Korea Electric Power Corporation under grant number R18XA02. Manuscript received November 9,2018; Revised December 11, 2018; Accepted December 11, HD(High Definition)., (3840*2160) UHD(Ultra High Definition) 50 TV [1]. Copyright 2016 Korean Institute of Broadcast and Media Engineers. All rights reserved. This is an Open-Access article distributed under the terms of the Creative Commons BY-NC-ND ( which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited and not altered.

2 3 : (Jeonho Kang et al.: Method of Video Stitching based on Minimal Error Seam) [2]. [3]. [4][5][6]. [7]. [8].,..,., [9].,. 2, 3, 4, 5.. Related work... [10]. 1. Feature Matching Pixel. 1. Fig. 1. Example of minimal error seam extracting

3 (JBE Vol. 24, No. 1, January 2019)... [11] ,. Pixel Error. Pixel.. (1).,,. i x pixel, j y.. e(i,j) Error. Error.. (2). min (2). S(i,j) e(i,j) y 3.., e(i,j) i 1., i 0 j,, i 1.., Error. Error Error., MES(j). min MES(j) Y.,. 2 system. 3.. The proposed stitching method based on Minimal error seam 2 Stitching. 3 stitching system 2.,..

4 3 : (Jeonho Kang et al.: Method of Video Stitching based on Minimal Error Seam) 2. Fig. 2. Stitching systems based on minimal error seam. HSV Edge Detection, Horizontal Variance Weight calculator. 1 HSV, 2 Ob- ject Detection, 3 Horizontal Variance Weight calculator. 1. HSV 2 1 pixel. pixel Hue, Saturation, Value. 2 (1) i,j Error, HSV (4) i,j Hue, Saturation, Value Error. 2. Object Detection 2 Object Detection. 3. Fig. 3. Problem of minimal error seam

5 (JBE Vol. 24, No. 1, January 2019) Object Detection object 23. Object detection 6. Object detection module, Fig th stitching region of left frame Fig th stitching region of right frame 6. Fig. 6. Object detection module 7. Fig. 7. Result of edge detection 8. Fig. 8. Result of edge detection with Gaussian filter

6 3 : (Jeonho Kang et al.: Method of Video Stitching based on Minimal Error Seam),, dilate & erode object dilate erode. object. 3. Horizontal Variance Weight calculator Fig th frame stitching result Fig th frame stitching result Fig th minimal error seam Fig th minimal error seam

7 (JBE Vol. 24, No. 1, January 2019)., x Horizontal variance weight(hvw), (5). Pr i X pixel, j Y pixel. j I error i. HVW Error, MES. MES HVW, MES MES. 3 MES, Error Hue, Saturation, Value 3 MES HVW MES. 4.. Experimental Result. OS Microsoft Windows 7, Video Stitcher Visual Studio Video Stitcher OpenCV HSV, 2 Object De- tection, 3 Horizontal Variance Weight calculator, Horizontal Variance Weight Fig. 13. Method of generating Horizontal Variance Weight

8 3 : (Jeonho Kang et al.: Method of Video Stitching based on Minimal Error Seam). 1. HSV HSV. 14 HSV. 14 HSV., HSV Object.,., HSV,. 2. Object Detection Object Detection HSV Fig. 14. Existing minimal error seam Technology and HSV-based error seam Technology Stitching Results 15. Fig. 15. Result of labeling

9 (JBE Vol. 24, No. 1, January 2019) ,. Dilate Erode threshold Edge. 15 Dilate Erode. Dilate Erode, Horizontal Variance Weight calculator Horizontal Variance Weight , MES edge detection. horizontal variance weight (5) MES MES. HVW Fig. 16. Result of edge detection Fig. 17. Result of stitching (20th frame) Fig. 18. Result of stitching (21th frame)

10 3 : (Jeonho Kang et al.: Method of Video Stitching based on Minimal Error Seam) 19. HVW 20 Fig. 19. Result of stitching with HVW(20th frame) 20. HVW 21 Fig. 20. Result of stitching with HVW(21th frame) MES. 4 1 MES HSV, 2 MES, 3 HVW MES.. Result,. MES HSV 3,, MES HVW.. (References) [1] Y. Sohn, M. Cho, and J. Paik, Design of MMT-based Broadcasting System for UHD Video Streaming over Heterogeneous Networks, JBE Vol. 20, No. 1, January 2015 [2] R. Szeliski and H.-Y. Shum, Creating full view panoramic image mosaics and environment maps, in Proc. 4th Annu. Conf. Comput. Graph. Interact. Techn., pp , [3] M. S. Billah and H. Ahn, Objective Quality Assessment Method for Stitched Images, (JBE Vol. 23, No. 2, March 2018 [4] Y.D. Lowe, Distinctive image features from scale-invariant keypoints, International Journal of Computer Vision, vol. 60, no. 2, pp , Nov [5] B. He, G. Zhao, and Q. Liu, Panoramic video stitching in multicamera surveillance system, in Proc. 25th Int. Conf. Image Vis. Comput. New Zealand (IVCNZ), pp. 1 6, Nov [6] W. Xu and J. Mulligan, Panoramic video stitching from commodity HDTV cameras, Multimedia Syst., vol. 19, no. 5, pp , Oct [7] W. Jiang and J. Gu, Video stitching with spatial-temporal content-preserving warping, in Proc. of IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp , Jun [8] Y. Xiong and K. Pulli, Fast panorama stitching for high-quality panoramic images on mobile phones, IEEE Trans. on Consumer Electronics, vol. 56, no. 2, pp , May [9] S. Shin, S. Yoon, H. Jung, J. Kim, K. Kim Video Blending based on Minimal Error Seam,. Korea Broadcast Media Engineering Conference, pp , [10] Telecommunications Technology Association, Technical Report for High Quality Panorama Video, [11] Dr. Mayada F. Abdul-HalimT. Makela, N. A. IbraheemDental UTILIZING GENETIC ALGORITHMS FOR 2D TEXTURE SYNTHESIS, 3pg, 2015.

11 5 1 2 방송공학회논문지 제24권 제1호, 2019년 1월 (JBE Vol. 24, No. 1, January 2019) 저자소개 강전호 - 년 2월 : 경희대학교 전자전파공학과 공학사 년 2월 : 경희대학교 전자전파공학과 공학석사 년 3월 ~ 2015년 2월 : 주식회사 넥스트리밍 연구원 년 3월 ~ 현재 : 경희대학교 전자공학과 박사과정 : 주관심분야 : 영상처리, 멀티미디어 시스템, 디지털 대화형방송 ORCID 김준식 년 경희대학교 전자전파공학과 공학사 년 현재 경희대학교 전자공학과 석사과정 주관심분야 영상처리 멀티미디어 시스템 디지털 대화형 방송 : ~ : - ORCID : :,, 김상일 - 년 2월 : 경희대학교 공과대학 전자공학과 학사 년 2월 : 광운대학교 정보콘텐츠대학원 3D콘텐츠 석사 년 7월 : 광운대학교 대학원 홀로그래피3D 박사 년 3월 ~ 현재 : 서울예술대학교 영상학부 교수 : 주관심분야 : 4K, 8K UHD. 3DVR ORCID 김규헌 - 년 2월 : 한양대학교 전자공학과 공학사 년 9월 : 영국 University of Newcastle upon Tyne 전기전자공학과 공학석사 년 7월 : 영국 University of Newcastle upon Tyne 전기전자공학과 공학박사 년 ~ 1997년 : 영국 University of Sheffield, Research Fellow 년 ~ 2006년 : 한국전자통신연구원 대화형미디어연구팀장 년 ~ 현재 : 경희대학교 전자정보대학 교수 : 주관심분야 : 영상처리, 멀티미디어 통신, 디지털 대화형 방송 ORCID