(JBE Vol. 23, No. 6, November 2018) (Special Paper) 23 6, (JBE Vol. 23, No. 6, November 2018) ISSN 2

Transcription

1 (JBE Vol. 23, No. 6, November 2018) (Special Paper) 23 6, (JBE Vol. 23, No. 6, November 2018) ISSN (Online) ISSN (Print) Generative Adversarial Network a), a) Night-to-Day Road Image Translation with Generative Adversarial Network for Driver Safety Enhancement Namhyun Ahn a) and Suk-Ju Kang a) (ADAS). ADAS Abstract Advanced driver assistance system(adas) is a major technique in the intelligent vehicle field. The techniques for ADAS can be separated in two classes, i.e., methods that directly control the movement of vehicle and that indirectly provide convenience to driver. In this paper, we propose a novel system that gives a visual assistance to driver by translating a night road image to a day road image. We use the black box images capturing the front road view of vehicle as inputs. The black box images are cropped into three parts and simultaneously translated into day images by the proposed image translation module. Then, the translated images are recollected to original size. The experimental result shows that the proposed method generates realistic images and outperforms the conventional algorithms. Keyword : Image translation, Image enhancement, Deep learning, Cycle consistency, Generative adversarial network a) (Dept. of Electronic Engineering, Sogang University) Corresponding Author : (Suk-Ju Kang) sjkang@sogang.ac.kr Tel: ORCID: This research was supported by the MSIT(Ministry of Science and ICT), Korea, under the ITRC(Information Technology Research Center) support program(iitp ) supervised by the IITP(Institute for Information & communications Technology Promotion, and by the Korea Institute of Energy Technology Evaluation and Planning (KETEP) and the Ministry of Trade, Industry & Energy (MOTIE) of the Republic of Korea (No ) Manuscript received September 4, 2018; Revised October 10, 2018; Accepted October 31, Copyright 2016 Korean Institute of Broadcast and Media Engineers. All rights reserved. This is an Open-Access article distributed under the terms of the Creative Commons BY-NC-ND ( which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited and not altered.

2 1 : Generative Adversarial Network (Namhyun Ahn et al.: Night-to-Day Road Image Translation with Generative Adversarial Network for Driver Safety Enhancement)., (ADAS). ADAS.. [1]., [2].. ADAS. [1].,.. [3]., [4] ,. patch.. Convolutional neural network(cnn). CNN Generative Adversarial Network Fig. 1. Concept of the night-to-day road image translation with generative adversarial network(gan) for driver safety enhancement

3 (JBE Vol. 23, No. 6, November 2018). patch [14]. Full-HD. patch. Patch.. 2. (Image translation) (Image translation).,. non-parametric texture model [5] Hertmann et al. Image Analogies [6]. CNN parametric method. CNN CycleGAN [7]. CycleGAN (Unpai- red image data)... CycleGAN. 2 CycleGAN. Cycle- GAN Generator AB, Generator BA, Discriminator A, Discriminator B, 4 (CNN). Generator AB A B, Generator BA B A. Discriminator A Generator BA (Image BA) A (Image A). Discriminator B Generator AB (Image AB) B (Image B). A B Discriminator A A 2. (CycleGAN). Generator AB, Generator BA, Discriminator A, Discriminator B,,, ( ). Fig. 2. The overall structure of training system for the proposed image translation module(cyclegan). It consists of 4 network; Generator AB, Generator BA, Discriminator A, Discriminator B, and trained by 3 loss functions;,, ( )

4 1 : Generative Adversarial Network (Namhyun Ahn et al.: Night-to-Day Road Image Translation with Generative Adversarial Network for Driver Safety Enhancement) 1, Generator BA 0. Discriminator B B 1, Generator AB 0. (1) adversarial loss [8]. Generator AB, Generator BA, Discriminator A, Discriminator B. A B,. Discriminator A, Discriminator B. Discriminator. Generator AB Generator BA Discriminator. Discriminator. Generator Discriminator., Generator AB Generator BA cycle consistency loss [7]. Cycle consistency loss (2). Generator AB Generator BA Generator., Generator BA Generator AB Generator., Generator AB, Generator BA. 3. A, B CycleGAN log log log log 3.. ( A) ( B). Fig. 3. Example images in collected data. Left two columns: night road images (domain A), right two columns: day road images (domain B)

5 (JBE Vol. 23, No. 6, November 2018) 4...,,,. Fig. 4. Qualitative results of different methods for night-to-day road image translation. Each row shows the output images of compared algorithms for each input. From left to right: input, histogram equalization, gamma correction, proposed system.. Generator AB 1. 2,000, random crop horizontal flip. Adam optimization, batch size 1 30 epoch. -2 cross entropy adversarial loss least square adversarial loss [9].. 1. [10] [11]. 4.,..,

6 1 : Generative Adversarial Network (Namhyun Ahn et al.: Night-to-Day Road Image Translation with Generative Adversarial Network for Driver Safety Enhancement) 1. NIQMC. NIQMC. 4 NIQMC.,,, Table 1 NIQMC values of different methods for night-to-day road image translation. NIQMC is no-reference quality metric of contrast-distorted images. Each row shows NIQMC values of images for each rows in Fig 4. From right to left: input, histogram equalization, gamma correction, proposed system. Image Input Gamma. Histogram. Proposed. Image Image Image Image Image ,.. 2. Peak Signal-to- Noise Ratio(PSNR) Structural Similarity(SSIM) [12]., (Ground truth).. Noreference Image Quality Metric for Contrast distortion (NIQMC) [15] (b) ~ (d) (a). 5 class Inception Top-5 class. (a), (b), (c), (d). Fig. 5. Misclassification results of each algorithm for same image. (b) ~ (d) show the translated images of (a) by each image translation method. The 5 classes at the right side of the pictures represents Top-5 classes. (a) Input, (b) Gamma correction, (c) Histogram equalization, (d) Proposed system.

7 (JBE Vol. 23, No. 6, November 2018). 4 NIQMC., NIQMC., NIQMC., NIQMC.,. Inception network [16]. Inception network image classification. Inceptionscore [13] Inception network. Inception network Top-5 class( class 5 class). 5 Inception network Top-5 class., Top-5 class fire engine class., car mirror, alp, limousine., network class... CycleGAN.,., Inception.. (References) [1] A. Paul, R. Chauhan, R. Srivastava, and M. Baruah, Advanced Driver Assistance Systems, SAE Technical Paper , 2016, doi: / [2] Z. Sun, G. Bebis and R. Miller, "On-road vehicle detection using optical sensors: a review," Proceedings. The 7th International IEEE Conference on Intelligent Transportation Systems (IEEE Cat. No.04TH8749), Washington, WA, USA, pp , 2004, doi: /ITSC [3] S. Plainis and IJ. Murray, Reaction times as an index of visual conspicuity when driving at night, Ophthalmic Physiol Opt., Vol.22, No.5, pp , Sep, [4] T. Hacibekir, S. Karaman, E. Kural, E.S. Ozturk, M. Demirci and B. Aksun Guvenc, Adaptive Headlight System Design Using Hardware-In-The-Loop Simulation, 2006 IEEE Conference on Computer Aided Control System Design, 2006 IEEE International Conference on Control Applications, 2006 IEEE International Symposium on Intelligent Control, Munich, Germany, pp , October, 2006, doi: /cacsd-cca-isic [5] A. A. Efros and T. K. Leung. Texture synthesis by non-parametric sampling, In ICCV, [6] A. Hertzmann, C. E. Jacobs, N. Oliver, B. Curless, and D. H. Salesin. Image analogies. In SIGGRAPH, pp ACM, [7] J. Zhu, T. Park, P. Isola, and A.A. Efros, Unpaired image-to-image translation using cycle-consistent adversarial networks, In ICCV, [8] I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio. Generative adversarial nets. In NIPS, [9] X. Mao, Q. Li, H. Xie, R. Y. Lau, Z. Wang, and S. P. Smolley. Least squares generative adversarial networks, In CVPR, 2017.

8 1 : Generative Adversarial Network (Namhyun Ahn et al.: Night-to-Day Road Image Translation with Generative Adversarial Network for Driver Safety Enhancement) [10] S. D. Chen, and A. R. Ramli, Contrast enhancement using recursive mean-separate histogram equalization for scalable brightness preservation, IEEE Transactions on Consumer Electronics, Vol. 49, No. 4, pp , [11] W. Kubinger, M. Vincze and M. Ayromlou, "The role of gamma correction in colour image processing," 9th European Signal Processing Conference (EUSIPCO 1998), Rhodes, pp. 1-4, [12] A. Hore and D. Ziou, "Image Quality Metrics: PSNR vs. SSIM," th International Conference on Pattern Recognition, Istanbul, pp , 2010, doi: /ICPR [13] T. Salimans, I. Goodfellow, W. Zaremba, V. Cheung, A. Radford, and X. Chen, Improved techniques for training gans, In Advances in Neural Information Processing Systems, pp , [14] Zhang, Y., Li, K., Li, K., Wang, L., Zhong, B., Fu, Y.: Image super-resolution using very deep residual channel attention networks. In ECCV [15] Ke Gu, Jun Zhou, Jun-Fei Qiao, Guangtao Zhai, Weisi Lin, Alan Conrad Bovik, "No-reference quality assessment of screen content pictures," IEEE Transactions on Image Processing (T-IP), vol. 26, no. 8, pp , Aug [16] C. Szegedy, V. Vanhoucke, S. Ioffe, J. Shlens, and Z. Wojna. Rethinking the inception architecture for computer vision. In CVPR, : ( ) : - ORCID : - :,, : : ( ) ~ 2012 : LG Display ~ 2015 : ~ : - ORCID : - :,,