(JBE Vol. 21, No. 5, September 2016) (Regular Paper) 21 5, 2016 9 (JBE Vol. 21, No. 5, September 2016) http://dx.doi.org/10.5909/jbe.2016.21.5.782 ISSN 2287-9137 (Online) ISSN 1226-7953 (Print) a), b), a) Realtime Human Object Segmentation Using Image and Skeleton Characteristics Minjoon Kim a), Zucheul Lee b), and Wonha Kim a) segmentation. segmentation. color consistency focus segmentation. human skeleton. segmentation mobile. Abstract The object segmentation algorithm from the background could be used for object recognition and tracking, and many applications. To segment objects, this paper proposes a method that refer to several initial frames with real-time processing at fixed camera. First we suggest the probability model to segment object and background and we enhance the performance of algorithm analyzing the color consistency and focus characteristic of camera for several initial frames. We compensate the segmentation result by using human skeleton characteristic among extracted objects. Last the proposed method has the applicability for various mobile application as we minimize computing complexity for real-time video processing. Keyword : segmentation, camera, human, composite a) (Kyung Hee University) b) KT (KT Fusion Technology Institute) Corresponding Author : (Wonha Kim) E-mail: wonha@khu.ac.kr Tel: +82-31-201-2030 ORCID: http://orcid.org/0000-0002-4797-3526 (NRF-2015R1D1A1A01059722). Manuscript received June 3, 2016; Revised Setember 20, 2016; Accepted Setember 20, 2016.. Segmentation. /. intensity Segmentation
2: (Minjoon Kim et al.: Realtime Human Object Segmentation Using Image and Skeleton Characteristics). [1],[2],[3] Mixture of Gaussian [9],[10] Bayesian decision, Kernel density Segmentation. intensity. [4],[5],[6],[7],[8] texture feature. feature. [11].. [12],[13],[14],[15],[16] stereo... segmentation. color consistency focus intensity, texture segmentation., under/over. block segmentation, texture. human face skeleton segmentation.. segmentation. human skeleton.,. 1. 1. Fig 1. Overall Flowchart. color/focus segmentation 1,. 1 color consistency focus segmentation.. 3( Intensity,, Texture)
(JBE Vol. 21, No. 5, September 2016). 1 Intensity 2, 3 cascade. 1. Intensity Texture intensity. 30 intensity intensity intensity. (1) block intensity block intensity. k block intensity. intensity block intensity block. (3) Foreground, Background, Fuzzy region 2 intensity. intensity.. 2, y. 2 intensity (, ) Foreground, Background, Fuzzy region. ( ) Foreground 2, 3. Foreground 3 (Foreground, Background, Fuzzy) under/over estimation. k block Foreground Background. 3 Fuzzy region. (2). 2. Fig 2. Region depend on Adaptive Threshold 2. 1 Intensity Fore- ground 2. color RGB 3 (4) 3. block block color. (5) cos
2: (Minjoon Kim et al.: Realtime Human Object Segmentation Using Image and Skeleton Characteristics). (5) (6) block. 3.,, block (Average pixel value),,. n block.. color consistency. (9). 3. Texture (a) (b) Foreground 3 Texture. Texture intensity intensity. intensity block block block texture (10). 3. (a) (b) Fig 3. Comparison of Color Difference (a) Angle between Vectors (b) Subtraction between Unit Vector. block (i,j) block. texture.
(JBE Vol. 21, No. 5, September 2016) block texture. texture (12). intensity block. auto focus. 3(Intensity,, texture) 4. Intensity,, texture 4.(b) under/over estimation. under/over estimation.. Human face skeleton Face [17] Skeleton [18]. 1. Face Human Intensity,, texture human. Human classification and regression tree analysis(cart) face [17]. human (a) 4. (a) (b) Fig 4. Segmentation Result (a) Original Image (b) Probability Segmentation Image (b) labeling human human. 5 human (a) (b) (c) 5. Face (a) (b) Face (c) Fig 5. Compensation Using Face Detection (a) Probability Segmentation Image (b) Face Detection Image (c) Compensation Result Image
2: (Minjoon Kim et al.: Realtime Human Object Segmentation Using Image and Skeleton Characteristics). Human face human. 2. Skeleton 6 human. human skeleton.. skeleton human Dilation. skeleton line dilation window skeleton dilation. 7 human skeleton human. 7.(d) 7.(a) human 7.(d). [19] Alpha Blending Alpha., block. 6. Fig 6. Probability Segmentation Image Morpholo- gical [18] human skeleton. skeleton human skeleton human. Face face dilation window skeleton line. 8 texture feature LBP(Local Binary Pattern) Histogram [5] DCT(Discrete Cosine Transform). LBP 1 8 LBP. 1 (a) (b) (c) (d) 7. Skeleton (a) (b) Skeleton (c) (d) Fig 7. Compensation Using Skeleton (a) Probability Segmentation Image (b) Skeleton Image (c) Compensation Region Image (d) Compensation Result Image
(JBE Vol. 21, No. 5, September 2016) (a) (b) (c) (d) (e) 8. (a) (b)ground Truth (c)dct (d)lbp (e) Fig 8. Comparison of Segmentation Performance (a) Original Image (b) Ground Truth (c) DCT Method (d) LBP Method (e) Proposed Method 1. Table 1. Comparison of Segmentation Pixel Number Method Image No. Ground Truth DCT LBP Propose G.T-DCT G.T.-LBP G.T.-Propose Image (1) 72883 79424 78400 76038 6541 5517 3155 Image (2) 98742 97472 94528 97634 1270 4214 1108 Image (3) 56986 59264 55360 57413 2278 1626 427 Image (4) 89459 111616 97984 95981 22157 8525 6522 Image (5) 115370 118464 110784 117080 3094 4586 1710 Ground Truth. DCT, LBP. ( 1280x720, 921600)
김민준 외 2인: 영상 특성과 스켈레톤 분석을 이용한 실시간 인간 객체 추출 (Minjoon Kim et al.: Realtime Human Object Segmentation Using Image and Skeleton Characteristics) 표와 같이 제안한 방법에서 Ground Truth와 화소 개수의 것을 관찰할 수 있다. 제안하는 방 법의 성능이 더 뛰어난 이유는 그림 8을 통해 오검출/미검 차이가 상대적으로 작은 출 부분에 대한 보정 때문임을 알 수 있다. DCT방법과 LBP방법 모두 새로 유입된 물체 이외에 대한 과 human내부에 배경으로 분류된 한다. 험 실 에서 사용한 block의 정되었다. block의 오검출 영역 미검출 영역 또한 존재 크기는 8x8이며 경험적으로 설 Alpha 값으로 지정하여 Blending 처리하는 다. 언급하였듯이 새로운 배경에 자연스러운 합성을 위 해서는 영상 객체 추출값에 단순 평균 필터를 적용한 값을 앞서 다. 알고리즘은 OpenCV 기반으로 구현 하였으며 Intel Core 컴퓨터 환경에서 HD(1280x720)영 상에 대하여 약 28fps의 합성속도를 가지며 이는 일반적인 실시간 프레임속도라 할 수 있다. i7-4770 cpu 3.4GHz의 Ⅴ. 결 론 제안하는 알고리즘은 intensity와 색, 그리고 texture의 변 화를 측정하고 카메라의 color consistency와 focus 특성을 분석하여 배경으로부터 객체를 실시간으로 추출하는 방법 으로 추출된 객체 영역에 human skeleton 특성을 적용하여 최종 객체 추출 결과를 향상시켰다. 또한 intensity, 색 그리 (a) (c) 것이 효율적이 다. 그림 9는 현재 프레임을 새로운 배경에 합성한 영상이 크기가 작아지면 global texture 분석의 효과가 감소하며 복잡도가 증가하여 실시간 처리가 불가능 하다. 반대로 block의 크기가 커지면 객체 추출 결과에 step 효과가 발생하여 blending 처리 시 자연스러운 결과를 얻기 힘들다. 위와 같은 사항을 고려하여 경험적으로 결정하였 789 그림 9. 영상 분류의 응용 예시 (a) 원본 영상 (b) 분류 영상 (c) Alpha 영상 (d) 합성 영상 (b) (d) Fig 9. Application Example of Segmentation (a) Original image (b) Segmentation Result (c) Alpha Image (d) Composite Image
(JBE Vol. 21, No. 5, September 2016) texture. Alpha /. human skeleton human.. block texture human skeleton human face.,.. (References) [1] T. Yu, C. Zhang, M. Cohen, Y. Rui and Y. Wu, "Monocular Video Foreground/Background Segmentation by Tracking Spatial-Color Gaussian Mixture Models,". In IEEE Workshop on Motion and Video Computing,pages 55, 2007. [2] J.J. Verbeek, N. Vlassis and B. Kr ose, "Efficient Greedy Learning of Gaussian Mixture Models," Published in Neural Computation 15(2), pages 469-485, 2003. [3] Cheol-Jun Jeong, Tae-Ki Ahn, Jong-Hwa Park, and Goo-Man Park, Real Time Abandoned and Removed Objects Detection System, JBE, VOL. 16, NO. 3, 462-470, May 2011. [4] J. Yao and J.M. Odobez, "Multi-Layer Background Subtraction Based on Color and Texture," IN IEEE The CVPR Visual Surveillance Workshop (CVPR-VS), MINNEAPOLIS, June 2007. [5] X. Sun and F. Chang, "Background Model Combining Gauss Model with Local Binary Pattern Feature," Journal of Convergence Information Technology(JCIT) Volume 7, Number 17, Sep 2012. [6] M.Heikkila and M.Pietikainen, "A Texture-Based Method for Modeling the Background and Detecting Moving Objects," IEEE Transactions on Pattern Analysis and Machine Intelligence, VOL. 28, NO. 4, April 2006. [7] A. Shimada and R.I Taniguchi, "Hybrid Background Model using Spatial-Temporal LBP," DOI: 10.1109/AVSS.2009.12 Conference: Sixth IEEE International Conference on Advanced Video and Signal Based Surveillance, AVSS 2009, 2-4 September 2009. [8] B. Vishnyakov, V. Gorbatsevich, S. Sidyakin, Y. Vizilter, I. Malin and A. Egorov, "Fast Moving Objects detection Using ilbp Background Model," The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XL-3, 2014. [9] L. Jeisung and P. Mignon, "An Adaptive Background Subtraction Method Based on Kernel Density Estimation," Sensors journal, 2012. [10] L. Li, W. Huang, I. Y.H. Gu, and Q. Tian, "Foreground Object Detection from Videos Containing Complex Background," In Proceedings of the Eleventh ACM International Conference on Multimedia, Berkeley, CA, USA, 28 November 2003. [11] R. Rodriguez-Gomez, E. J. Fernandez-Sanchez, J. Diaz and E. Ros, "FPGA Implementation for Real-Time Background Subtraction Based on Horprasert Model," Sensors journal, 2012. [12] Soon-kak Kwon and Yoohyun Park, "Picture Quality Control Method for Region of Interest by UsingDepth Information", JBE, VOL. 17, NO. 4, 676-683, July 2012. [13] V. Kolmogorov, A. Criminisi, A. Blake, G. Cross and C. Rother, "Bilayer segmentation of binocular stereo video," In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, San Diego, CA, USA, 2025 pp. 407414. June 2005. [14] V.Ganapathi, C. Plagemann, D. Koller and S. Thrun, "Real Time Motion Capture Using a Single Time-Of-Flight Camera," In Proceedings of the 2010 IEEE Conference on Computer Vision and Pattern Recognition, San Francisco, CA, USA, 1318 pp. 755762. June 2010. [15] E. J. Fernandez-Sanchez and J.Diaz and E.Ros, "Background Subtraction Based on Color and Depth Using Active Sensors," Sensors journal, 2013. [16] Saeed Mahmoudpour and Manbae Kim, "Detecting Foreground Objects Under Sudden Illumination Change Using Double Background Models", JBE, VOL. 21, NO. 2, 268-271, March 2016. [17] R. Lienhart, A. Kuranov, and V. Pisarevsky, Empirical Analysis of Detection Cascades of Boosted Classifiers for Rapid Object Detection, Proc. DAGM 25th Pattern Recognition Symp., pp. 297-304, 2003. [18] R. C. Gonzalez, R. E. Woods and S. L. Eddins, Digital Image Processing Using MATLAB, Gatesmark Publishing, 2009. [19] T. porter and T.Duff. "Compositing Digital Images," Computer Graphics Volume 18, Number 3 July 1984.
2: (Minjoon Kim et al.: Realtime Human Object Segmentation Using Image and Skeleton Characteristics) - 2014 8 : - 2014 9 ~ : - ORCID : http://orcid.org/0000-0002-6743-7958 - : : /, - 1997 2 : - 2003 2 : - 2014 2 : University of California, San Diego - 1997 1 ~ : KT(Korea Telecom) - : /, 1, - 1985 2 : - 1988 5 : University of Wisconsin-Madison - 1996 1 ~ 7 : () Motorola - 1997 5 : University of Wisconsin-Madison - 1997 8 ~ 2000 2 : () Los Alamos National Lab. - 2000 3 ~ 2003 8 : - 2009 8 ~ 2010 8 : University of California San Diego (UCSD) - 2003 9 ~ : - ORCID : http://orcid.org/0000-0002-4797-3526 - : /,,