Visual Search At SK-Planet sk-planet Machine Intelligence Lab. 나상일
1. 개발배경 2. 첫접근방법 3. 개선된방법 A. Visual recognition technology B. Guided search C. Retrieval system
개발배경
개발배경 상품검색을좀더쉽게 Key-word 트렌치코트버튺벨트
개발배경 개발목표 - 영상을 Query 로상품을검색
본내용에앞서
사전지식 Deep 더자세핚 learning 설명은 Tech Planet 정의 2015 Deep Learning 기반모바일싞용카드번호자동인식기참고최찬규 / Deep 매니저 Artificial, SK planet Neural Network + Machine Learning http://readme.skplanet.com/?p=11744 Artificial Neural Network 사람의싞경망을모방핚컴퓨터 학습을핛수있음 최고성능의 Machine Learning 알고리즘 특징 학습을위해많은데이터가필요함 학습시갂도오래걸림 GPU로병렬처리가능 종류 DBN, RNN, CNN,
접근방법 첫시작
Visual Search 개발 Deep learning 기술적용 티셔츠, 가방, 싞발등카테고리예측 카테고리예측 카테고리 : 티셔츠 관심영역추출 Grab-cut + Saliency HoG+SVM 를통한검증 특징추출 Pattern feature Color feature
Visual Search 개발 카테고리예측 동의어처리 정확한카테고리? Convolution Relu activation Subsample pooling Fully connected layer Dropout Softmax covolution kernel Deng, Jia, et al. "Hedging your bets: Optimizing accuracy-specificity tradeoffs in large scale visual recognition." Computer Vision and Pattern Recognition (CVPR), 2012 IEEE Conference on. IEEE, 2012.
Visual Search 개발 관심영역검출 영역검증 Dense HoG + SVM
Visual Search 개발 관심영역검출결과
Visual Search 개발 검색결과 질의영상 결과
개선
Visual Search 의개선 기존 system 평가 잘됐나? 잘됐다. - 단, Single Object에대해 - 단순한배경에대해 - 진짜잘됐나? 실제 DB 는 - Multi-Object - 야외촬영 유사상품? 유사상품?
Visual Search 의개선 개선방향 - 여러상품이있어도모두검색할수있도록 Detection 기술적용 - 야외에서촬영한것도검색이잘되도록 Deep learning 을적용한특징추출 - 누구나유사하다느낄수있도록 속성기반의인식
Detection
Visual recognition technology RPN 을 CNN 이후에적용 RoI Pooling layer 를이용각 Region Proposal 에대해 classification 수행 Ren et al, Faster R-CNN: Towards Real- Time Object Detection with Region Proposal Networks, NIPS 2015
Feature
Visual recognition technology 특징추출 - 딥러닝을이용핚특징추출 Deep feature
Visual recognition technology Deep feature 실험 - 유사상품에대핚 Ground truth를만드는것은매우어려움 검색결과가명확핚공개 DB를대상으로가능성평가 - UkBench Dataset 대상검색실험 map 0.929 Deep-feature is more suitable for a sort of semantic similarity retrieval application.
Visual recognition technology Feature Extractor Extract Feature Szegedy, Christian, et al. "Going deeper with co nvolutions." Proceedings of the IEEE Confe rence on Computer Vision and Pattern Recogni tion. 2015.
Visual recognition technology Feature Extractor Residual learning He, Kaiming, et al. "Deep residual l earning for image recognition." arxiv preprint arxiv:1512.03385 (2015).
Visual recognition technology Feature Extractor X 2 grid size: 8 X 1 X 4 grid size: 17 X 1 grid size: 35 X 3
Visual recognition technology Feature Extractor
Attribute
Visual recognition technology Attribute Classifier 어떤것이비슷핚것인가? 동일핚속성이있는의류를검색하자!
Visual recognition technology Fashion-attribute dataset Maximum 100 man-months Almost 1 year More than 90 attributes (thousands of unique fashion-styles by their combination) includes ROIs for a fashion item (For detection) Almost 1 million images
Visual recognition technology Fashion Attribute Dataset GREAT CATEGORY (3CLASSES) FASHION CATEGORY (19CLASSES) GENDER (2CLASSES) SILHOUETTE (14CLASSES) COLLAR (18CLASSES) SLEEVE LENGTH (6CLASSES) Bottom T-shirts Male Normal Shirt Long Multi-label classification task Top Pants Female A-line Turtle A half Others Bags H-line Round Sleeveless
Visual recognition technology Attribution Classification Multi-label classification 은 conditional probability 로표현가능 RNN 으로접근가능!
Visual recognition technology Attribution Classification Vinyals, Oriol, et al. "Show and tell: A neural image caption generator." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2015.
Visual recognition technology Attribution Classification Feed-forward Neural Networks get way deeper by using the residual shortcut path Recurrent Neural Networks get way of preserving long-term dependency in their hidden context by using the residual short cut path
Visual recognition technology Attribution Classification Result
Visual recognition technology Attribution Classification Result
Visual recognition technology Attribution Classification Result
Visual recognition technology Attribution Classification Result
Visual recognition technology Attribution Classification Result
Guided Search
Visual recognition technology Guided fashion product recognition Problem - 판매자의 tag 정보는정제되어있지않다! Tag: brend-new/women's shirts, blouse 검색되기를원하는상품은? - 사용자 / 판매자가검색되기원하는상품을선택하도록하자 - 판매의경우입력된 Tag 정보를이용
Visual recognition technology Natural Language Processing Parsing and Morphological Analysis Post-process Synonym dictionary and Stopword dictionary Output women, cardigan, knit, round-neck women, blouse
검색시스템
Visual recognition technology 검색시스템
Visual recognition technology 검색결과
Visual recognition technology 검색결과 User guide search Guide: jacket
Visual recognition technology 검색결과 User guide search Guide: shoes
Visual recognition technology 검색결과 User guide search Guide: bag
Visual recognition technology Demo Video
분산검색시스템적용
검색시스템개선 Problem - 검색대상이많은데 - DB 관리도되야하는데 Solution - System Software 개발팀의분산처리솔루션적용
검색시스템개선 분산처리 Redis 기반의분산인메모리기반병렬처리기술적용 - 효율적인 DB 관리 - 검색속도개선
For More details, checkout our paper! http://arxiv.org/abs/1609.07859
sang.il.na79@gmail.com