영상및자연어처리분야인공지능기술동향및전망 주재걸교수 고려대학교정보대학컴퓨터학과
주재걸교수 1 사진 고려대컴퓨터학과 2 3 4 2
3 연구실구성원 9 박사및석박통합과정 22 석사과정 15+ 학부연구생수행과제한국연구재단정보통신기술진흥센터한국산업기술평가관리원한국전력공사삼성리서치마이크로소프트리서치네이버웹툰 SK 텔레콤엔씨소프트자문기관신한금융투자 LG CNS 네이버 Clova AI 삼성 SDS
Deep Learning Deep learning refers to artificial neural networks that are composed of many layers. Fei-Fei Li & Andrej Karpathy & Justin Johnson Lecture 3-11 Jan 2016 Neural Network and Backpropagation Lecture 1-6 2016 / 11/ 4
Deep Learning
https://www.ibm.com/blogs/systems/deep-learning-performance-breakthrough/ 6
Data: Large datasets 딥러닝의성공요인 Hardware: GPU acceleration Algorithm: Advanced techniques (e.g., batch norm, ADAM, attention) 7
기계학습의기본세팅 총 100 명의환자 (data item) 를적당한수의학습데이터및 ( 예측용도의 ) 테스트데이터로분리예로, 학습데이터로서 70 명환자들의피쳐정보와그들의 target label 을기계학습모델의인풋으로줌. 모델은내부적으로어떤피쳐가 target label 을예측하는데중요한지를판단하여, 주어진모든피쳐를조합하여 target label 을예측함. 예상수명을예측하고자할경우, 0.7* 키 -0.1* 몸무게 +0.0001* 혈액형 => 예측수명에가장근사적인값을주는식이를 30 의테스트데이터를대상으로얼마나예상수명을잘맞추는지를테스트함. 이데이터는학습에사용하지않았으므로, 보지못한 (unseen) 데이터에대해얼마나모델이잘예측하는지로모델성능을평가함. 8
Human Brain and Neural Network Fei-Fei Li & Andrej Karpathy & Justin Johnson Lecture 3-11 Jan 2016 Neural Network and Backpropagation Lecture 1 32 2016 / 11/ 4
Difference of Deep Learning from Existing Machine Learning 10 Img src: Goodfellow 2016
Deep Learning Why is deep learning a growing trend? - few feature engineering - state-of-the-art performance
Deep Learning Applications 영상인식 : 현재사람보다뛰어난성능을보임 자연어처리 : 기계번역, 대화시스템 음성인식 : text-to-speech, 화자인식, 노이즈캔슬링 게임인공지능 : 바둑, 스타크래프트 의료 : 질병자동진단 법률 : 판결예측 12 금융 : 주식예측, 자산관리및투자
Game AI
Machine Translation
Face Detection & Recognition
Object Detection & Recognition
Image Captioning
Image Captioning
Style Transfer
Style Transfer
Style Transfer
인공지능연구의최신동향 데이터의생성및변환모델 Generative adversarial networks, variational autoencoder 벡터표현형을통한이종데이터의통합및변환영상 <-> 자연어 <-> 소리 / 음성 <-> 기타정형데이터 비지도및자가지도학습 (self-supervised learning) Computer vision: colorization, jigsaw puzzle Language model: BERT, GPT2 22 실제활용될때의이슈들인공지능모델의해석가능성인공지능모델의취약점및보안사용자인터페이스
Generative Model 인식태스크를넘어서서데이터생성의영역으로.. 23
Generative Model https://www.slideshare.net/carpedm20/pycon-korea-2016( 24 지적대화를위한깊고넓은딥러닝, 김태훈 )
Generative Model https://www.slideshare.net/carpedm20/pycon-korea-2016( 25 지적대화를위한깊고넓은딥러닝, 김태훈 )
Generative Model https://www.slideshare.net/carpedm20/pycon-korea-2016( 26 지적대화를위한깊고넓은딥러닝, 김태훈 )
Generative Model https://www.slideshare.net/carpedm20/pycon-korea-2016( 27 지적대화를위한깊고넓은딥러닝, 김태훈 )
Generative Model https://www.slideshare.net/carpedm20/pycon-korea-2016( 28 지적대화를위한깊고넓은딥러닝, 김태훈 )
Generative Model https://www.slideshare.net/carpedm20/pycon-korea-2016( 29 지적대화를위한깊고넓은딥러닝, 김태훈 )
Generative Adversarial Networks 가장대표적인생성모델 Generator ( 생성자 ) 및 Discriminator ( 식별자 ) 로구성되어서로가적대적인학습을진행하여각각의성능을극대화함식별자를통해 loss function 자체도스스로학습할수있음 30
31 StyleGAN: Style-Based Generator Architecture for GAN [CVPR 19]
Image-to-Image Translation pix2pix (Isola et al., 2017) CycleGAN (Zhu et al., 2017) Photo to Emoji DTN (Taigman et al., 2017) MUNIT (Huang et al., 2018) 32
CVPR 18 StarGAN: Generative Adversarial Networks for Multi-Domain Image Translation 33
ECCV 18 Coloring with Words: Guiding Image Colorization Through Text-based Palette Generation 34
ECCV 18 Coloring with Words: Guiding Image Colorization Through Text-based Palette Generation 35
36 ECCV 18
CVPR 19 Coloring With Limited Data: Few-Shot Colorization via Memory Augmented Networks 37
ECCV 18 Coloring with Words: Guiding Image Colorization Through Text-based Palette Generation 38
PaintsChainer: Deep Learning-based Manga Colorization https://paintschainer.preferred.tech/ 39
Everybody Dance Now https://youtu.be/pcbtzh41ris?t=50 40
자연어이해및생성연구동향 상대적으로컴퓨터비젼분야에비해발전이더딤딥러닝모델이나방법론의진보라기보다엔지니어링적이고휴리스틱한연구가상대적으로많은부분을차지 언어는인간이가진생각과지능적사고의표현방식 인공지능이인간의언어를이해하고적절히생성할수있다는것은인간의지능을정복한것임 자가지도학습및대규모데이터를사용하여이해및생성능력이점점고도화되고있음 41
Seq2seq Model 시퀀스를입력으로받아서, 시퀀스를출력으로생성많은자연어처리태스크의기본모델로활용됨 : 챗봇, 기계번역, 질의응답 Sutskever et al. 2014 42 Sequence to Sequence Learning with Neural Networks Encode source into fixed length vector, use it as initial recurrent state for target decoder model
43 Reading Comprehension-based Question Answering
EMNLP 18 Deep and Wide Reader: Effective Memory Augmenting Method for Question Answering Utilizes self-attention and memory controller of differentiable neural computer for question answering 44
Word Embedding 딥러닝기반자연어처리모델의시작점 Word2vec, GloVe, ELMo, CoVe 45
Transformer based on Self-Attention 최근딥러닝기반자연어처리모델의기본구조로사용됨 46
Attention Model 어텐션모델은다양한딥러닝모델에서활발히활용됨예 : image captioning 47
Machine translation 다른언어들간의어순을학습함관사등의필요없는단어는건너뜀 Attention Model 48
BERT 최근다양한자연어처리태스크의성능을크게끌어올림 Masked language modeling 태스크를통해학습대규모데이터및대규모모델을사용함 49
GPT2 Transformer 를기반으로한모델을사용하여, 대규모의양질의 reddit 텍스트데이터및다수의고성능 GPU 를사용하여성능을극대화함 50
51 GPT2
RetainVis: Interactive Visual RNNs on Electronic Medical Records IEEE VIS 18 인공지능을사용하는사람의측면에서인터페이스도중요 52 https://vimeo.com/272587219
Our approach CHI 19 AILA: Attentive Interactive Labeling Assistant for Document Classification 레이블링도사람의입장에서효율적으로해야함 53
향후전망 기술발전속도는점점더뎌지고있음 모델은점점사용하기쉬워지고, 오픈소스분위기로인해, 기술에대한진입장벽은점점낮아질것임기술자체는공공재가되어가고있음 결국확보된데이터의종류나그양과질측면에서승부처가나뉠것임구글, 페이스북, 마이크로소프트등의메이저회사들이독식할가능성이높음경쟁이더치열해지고, 양극화가심화됨 ( 부자는더부자로.. 가난한사람은더가난해짐 ) 54 문제를발굴하는능력과, 기술의이해를바탕으로어떤식으로해당문제를 formulate 할것인지가중요도메인지식또한여전히중요
인공지능을공부하려면? 55 온라인강의가잘돼있어서, 오늘부터라도공부할수있음필수코스선형대수 https://ocw.mit.edu/courses/mathematics/18-06-linear-algebraspring-2010/ 머신러닝 https://www.coursera.org/learn/machine-learning 딥러닝 http://cs231n.stanford.edu/ 페이스북그룹 https://www.facebook.com/groups/aikoreaopen https://www.facebook.com/groups/tensorflowkr 기타정보 https://blog.naver.com/joyfull1
인공지능을공부하려면? 공부는최대한깊게머신러닝 / 딥러닝은알고보면거의다수학임본인스스로잘알고있는지.. 모르고지나친건없는지끊임없이고민해야함외우지않아도직관적으로깊이이해하는것이중요문제를스스로발굴할줄아는능력이중요머신러닝의장점은데이터만있으면어디든적용할수있음남이하라고준공부, 시험, 숙제만으로는부족함따라서, 남이보지못하는새로운문제를볼줄아는능력이중요좋아하고잘하는찾고, 남들보다더열심히오픈소스등정보는누구나원하면얻을수있음경쟁이더치열해지고, 양극화가심화됨 ( 부자는더부자로.. 가난한사람은더가난해짐 ) 성공하려면옛날사람보다더열심히해야함그러려면좋아하거나잘하는걸찾아서선택 / 집중해야함 56
Collaborators IBM TJ Watson Research, New York Univ. Univ. Maryland, Georgia Tech, HKUST, Tsinghua Univ., Virginia Tech Recent Publications [CVPR 19] Coloring with Limited Data: Few-Shot Colorization via Memory-Augmented Networks [CVPR 19] Image-to-Image Translation via Group-wise Deep Whitening and Coloring [CHI 19] AILA: Attentive Interactive Labeling Assistant for Document Classification [AAAI 19] Paraphrase Diversification as Guided Style Transfer [EMNLP 18] MemoReader: Large-Scale Reading Comprehension through Neural Memory Controller [ECCV 18] Coloring with Words: Guiding Image Colorization Through Text-based Palette Generation [VIS 18] RetainVis: Visual Analytics with Interpretable and Interactive RNNs on Electronic Medical Records [CVPR 18] StarGAN: Unified GANs for Multi-Domain Image-to-Image Translation [IJCAI 18] MEGAN: Mixture of Experts of GANs for Image Generation via Categorical Reparameterization [CG&A 18] Explainable, Interactive Deep Learning [EuroVis 18] PixelSNE: Pixel-Aligned Stochastic Neighbor Embedding for Efficient 2D Visualization with Screen Precision [WWW 18] Short-Text Topic Modeling via Non-negative Matrix Factorization Enriched with Local Word-Context Correlations [CHI 18] ExTopicTile: Tile-Based Spatio-Temporal Event Analytics on Social Media [VIS 17] ConceptVector: Text Visual Analytics via Interactive Lexicon Building using Word Embedding [ICDM 17] STExNMF: Spatio-Temporally Exclusive Topic Discovery for Anomalous Event Detection [KAIS 17] Localized User-Driven Topic Discovery via Boosted Ensemble of Nonnegative Matrix Factorization [JMIR 17] Toward Predicting Social Support Needs in Online Health Social Networks [IJCAI 17a] End-to-End Prediction of Buffer Overruns from Raw Source Code via Neural Memory Networks [IJCAI 17b] Toward Predicting Social Support Needs in Online Health Social Networks [TKDD 17] VisIRR: Interactive Visual Information Retrieval and Recommendation for Large-scale Documents [AAAI 17] PIVE: Per-Iteration Visualization Environment for Real-Time Interactions with Dimension Reduction and Clustering [TVCG 17a] TopicLens: Efficient Multi-Level Visual Topic Exploration of Large-Scale Document Collections 57 Thank you! [TVCG 17b] Axisketcher: Interactive Nonlinear Axis Mapping of Visualizations through User Drawings [ICDM 16] L-EnsNMF: Boosted Local Topic Discovery via Ensemble of Nonnegative Matrix Factorization