(JBE Vol. 20, No. 6, November 2015) (Regular Paper) 20 6, 2015 11 (JBE Vol. 20, No. 6, November 2015) http://dx.doi.org/10.5909/jbe.2015.20.6.880 ISSN 2287-9137 (Online) ISSN 1226-7953 (Print) a), a) Frame Rate Up-Conversion Considering The Direction and Magnitude of Identical Motion Vectors Jonggeun Park a) and Jechang Jeong a) EBME(Extended Bilateral Motion Estimation). EBME BME(Bilateral Motion Estimation) x, y, EBME. MVS(Motion Vector Smoothing). EBME PSNR(Peak Signal to Noise Ratio). Abstract In this paper, frame rate up conversion (FRUC) algorithm considering the direction and magnitude of identical motion vectors is proposed. extended bilateral motion estimation (EBME) has higher complexity than bilateral motion estimation (BME). By using average magnitude of motion vector with x and y direction respectively, dynamic frame and static frame are decided. We reduce complexity to decide EBME. also, After we compare the direction and magnitude of identical motion vectors, We reduce complexity to decide motion vector smoothing(mvs). Experimental results show that this proposed algorithm has fast computation and better peak singnal to noise ratio(psnr) results compared with EBME. Keyword : Frame rate up conversion, Extended bilateral motion estimation, Motion estimation, Identical motion vectors, Motion vector smoothing a) (Hanyang University Department of Electronic and Computer Engineering) Corresponding Author : (Jechang Jeong) E-mail: jjeong@hanyang.ac.kr Tel: +82-2-2220-4369 ORCID: http://orcid.org/0000-0002-3759-3116. 2015. Manuscript received August 3, 2015; Revised September 24, 2015; Accepted October 19, 2015.
1 : (Jonggeun Park et al.: Frame Rate Up-Conversion Considering The Direction and Magnitude of Identical Motion Vectors). UHD(Ultra High Definition),.. LCD (Liquid Crystal Display) (Motion Blur Effect), [1]. (decoder) FRUC(Frame Rate Up-Conversion) [2]-[4]. FRUC.. MC- FRUC(Motion-Compensated FRUC) (Motion Estimation) (Motion Vector) MCI(Motion- Compensated Interpolation). FRUC (Block Matching Algorithm) (Unilateral Motion Estimation). (hole) (overlap). BME(Bilateral Motion Estimation) [5]. EBME(Extended Bilateral Motion Estimation) full search, [3]. MC-FRUC FRUC [6][7]. EBME BME, BME, MVS(Motion Vector Smoothing) median. EBME x, y, EBME. EBME MVS..,.,.. 1. BME(Bilateral Motion Estimation) 1. Fig. 1. Bilateral Motion Estimation (Bilateral Motion Estimation) (Interpolated frame) [5]. 1 (1), (2) SBAD(Sum of Bilateral Absolute Differen-
(JBE Vol. 20, No. 6, November 2015) ces). min arg (1), (2) (, ),,.,. block Overlapped block. 16 16 8 8 SBAD. 3. MVS(Motion Vector Smoothing). (3), 3 (Correct motion vector) median [3]., 3. 2. EBME(Extended Bilateral Motion Estimation) (Extended Bilateral Motion Estimation) (grid),. 2 Original, Original block grid Overlapped block Overlapped. Original 3. MVS Fig. 3. MVS process 2. EBME Fig. 2. Example of the motion vector field obtained by the EBME process
1 : (Jonggeun Park et al.: Frame Rate Up-Conversion Considering The Direction and Magnitude of Identical Motion Vectors) flag flag=1, flag=0. (3),.. 4, x, y, EBME., MVS. OBMC(Overlapped Block Motion Compensation) [3]. 1. x, y Previous frame, Current frame Bilateral Motion BME Estimation Yes Xmag>T1 ymag>t1 No Yes Extended Bilateral Motion Estimation Identical Direction No Yes Identical Magnitude M<T2 No Motion Vector Smoothing Overlapped Block Motion Compensation Motion Compensated Interpolation Intermediate frame 4. Fig. 4. Flowchart of the proposed scheme
(JBE Vol. 20, No. 6, November 2015). -), (-, +),. if or (4), (5) BME EBME, EBME BME. (4), (5), x, y. DF, SF. 6. Fig. 6. Direction of the motion vectors (6), (7) M 3 3 flag=0 MVS.,, x, y. 2. 5, 3 3, MVS. if. 5. Fig. 5. Example of the identical motion vectors,. 3 3 6 (+, +), (+, -), (-, CIF(352 288) 9 (Bus, Coastguard, Container, Flower, Football, Foreman, Mobile, Stefan, Tempete). 7, PSNR(Peak Signal to Noise Ratio). OBMC, MCI (Motion Compensated Interpolation) [3].
박종근 외 1인 : 동일한 움직임 벡터들의 방향과 크기를 고려한 프레임율 증가기법 (Jonggeun Park et al.: Frame Rate Up-Conversion Considering The Direction and Magnitude of Identical Motion Vectors) 885 실험조건은 기본 블록 크기를 16 16으로 하였고, 탐색 알고리듬에서 OBMC는 기본 블록 크기의 2배인 윈도우사 범위는 ± 로 하였으며, =0.6, =0.3로 설정하였으며, 이즈를 주변 블록들과 중첩시켜 블록화 현상을 줄여주기 Visual Studio 2012로 구현하였다. 때문에 정확한 주변 블록들을 참고 하지 않으면, 영상의 화 질이 저화 된다. 제안한 알고리듬에서는 정적프레임으로 판단하여 OBMC를 생략함으로써 EBME 알고리듬 보다 달 력 숫자들이 명확하게 보임을 알 수 있다. 표 1에서 Mobile영상에서 다른 영상보다 PSNR향상 높 아 EBME 알고리듬 보다 제안한 알고리듬이 평균 0.011dB PSNR이 증가함을 볼 수 있고, 표 2에서 제안하는 알고리듬 의 수행속도가 전체적으로 EBME 알고리듬 보다 시간이 단축되었다. Container에서는 정적프레임을 많이 포함하여 그림 7. 짝수 보상프레임 생성 과정 수행시간이 많이 감소하였다. 카메라의 Global Motion을 Fig. 7. Creation process of the interpolated even frame 고려한다면, 움직임 벡터의 크기가 같더라도 일정한 값을 EBME 알고리듬과 제안된 알고리듬의 주관적 화질에서 는 그림 8과 같이 별 다른 차이점이 없다. 그림 9의 EBME (a) Original 가질 수 있어 EBME 적용 하지 않아도 되어 연산량을 줄 일수 있을 것으로 예상된다. (b) EBME (c) Proposed 그림 8. Container영상에 대한 주관적 화질 비교 Fig. 8. Subjective result compared with previous algorithm on the Container image (a) Original 그림 9. Mobile영상에 대한 주관적 화질 비교 (b) EBME Fig. 9. Subjective result compared with previous algorithm on the Mobile image (c) Proposed
(JBE Vol. 20, No. 6, November 2015) 1. PSNR Table 1. PSNR Comparison Sequence EBME(dB) Proposed(dB) Difference Bus 25.979 25.981 0.002 Coastguard 32.683 32.671-0.012 Container 39.074 39.360 0.286 Flower 30.125 30.126 0.001 Football 23.575 23.496-0.079 Foreman 36.552 36.425-0.127 Mobile 27.290 27.464 0.174 Stefan 21.763 21.757-0.006 Tempete 29.115 28.972-0.143 x, y., EBME,, BME.,, MVS. EBME 0.011dB PSNR, EBME 13.26%. Average 29.573 29.584 0.011 2. Table 2. Computation Times Sequence EBME(sec) Proposed(sec) Relative Rate(%) Bus 36.217 36.164 0.15 Coastguard 70.365 69.917 0.64 Container 73.086 38.866 46.82 Flower 61.349 61.134 0.35 Football 35.679 34.16 4.26 Foreman 68.566 58.905 14.09 Mobile 76.619 68.377 10.76 Stefan 69.230 66.932 3.32 Tempete 62.133 45.449 26.85 Average 61.472 53.323 13.26. EBME (References) [1] S. H. Chan, T.X.Wu and T.Q.Nguyen, "Comparison of two frame conversion schemes for reducing LCD motion blurs," IEEE Signal Processing Letters, vol. 17, no. 9, pp. 782-786, Sept. 2010. [2] S. H. Lee, O. Kwon, and R. H. Park, "Weighted-adaptive motion compensated frame rate up-conversion," IEEE Trans. Consumer Electronics, vol. CE-49, no. 3, pp. 485-492, Aug. 2003. [3] S. J. Kang, K. R. Cho, and Y. H. Kim, Motion Compensated Frame Rate Up-Conversion Using Extended Bilateral Motion Estimation, IEEE Trans. Consumer Electronics, vol. 53, no. 4, pp. 1759-1767, Nov. 2007. [4] D. J. Park, and J. C. Jeong, "Adaptive Extended Bilateral Mtion Estimation Considering Block Type and Frame Motion Activity," JBE, vol. 18, no. 3, May 2013. [5] B. D. Choi, J. W. Han, C. S. Kim, and S. J. Ko, Motion-Compensated Frame Interpolation Using Bilateral Motion Estimation and Adaptive Overlapped Block Motion Compensation," IEEE Trans. Circuits and Systems for Video Technology, vol. 17, no. 4, pp. 407-416, Apr. 2007. [6] S. J. Kang, S. Yoo, and Y. H. Kim, Dual Motion Estimation for Frame Rate Up-Conversion, IEEE Trans. Circuits and Systems for Video Technology, vol. 20, no. 12, pp. 1909-1914, Dec. 2010. [7] U. S. Kim and M. H. Sunwoo, "New Frame Rate Up-Conversion Algorithms with Low Computational Complexity," IEEE Trans. Circuits and Systems for Video Technology, vol. 24, no. 3, pp. 384-393, Mar. 2014.
1 : (Jonggeun Park et al.: Frame Rate Up-Conversion Considering The Direction and Magnitude of Identical Motion Vectors) - 2014 2 : - 2014 3 ~ : - ORCID : http://orcid.org/0000-0002-2475-3555 - :, - 1980 2 : - 1982 2 : KAIST - 1990 : - 1980 ~ 1986 : KBS ( ) - 1990 ~ 1991 : ( ) - 1991 ~ 1995 : (MPEG, HDTV, ) - 1995 ~ : ( ) - 1998 11 : - 1990 12 : - 2007 : IEEE Chester Sall Award - 2008 : ETRI Journal Paper Award - 2011 5 : 46 - ORCID : http://orcid.org/0000-0002-3759-3116 - :,, 3DTV