오피니언검색연구동향 Research Trends in Opinion Retrieval 2013 년 6 월 26 일 고려대학교자연어처리연구실이승욱 (swlee@nlp.korea.ac.kr)
Contents 1. 서론 2. 오피니언마이닝 & 오피니언검색 3. 오피니언검색연구동향 4. 결론 5. Appendix: 데이터셋과성능평가, 참고문헌 2
서론 (1) 소셜미디어의증가 3
서론 (2) 개인중심의미디어의급증 블로그, 포럼, 게시판, 소셜네트워크서비스 (SNS) 다양한주제에대해자유롭게개인의의견이나느낌을표현 의견 (Opinion) A belief about matters commonly considered to be subjective, i.e., it is based on that which is less than absolutely certain, and is the result of emotion or interpretation of facts This may refer to unsubstantiated ( 근거없는 ) information, in contrast to knowledge and fact-based belief - Wikipedia 4
서론 (3) 서비스업체, 제조업체, 관공서등다양한조직에서제품, 서비스, 정책의품질을개선하기위해사람들의평가를반영하고자노력 온라인환경에축적된대량의주관적피드백 (subjective feedback) 은품질향상과마케팅전략을수립하는데중요한잠재적정보원으로활용가능 기존의사람들이작성한설문조사는시간적, 인적자원이지나치게소모 일반사용자역시의사결정을위해다른이들이가진생각을참조 제품구매에앞서온라인리뷰문서참고 대다수온라인쇼핑몰은평가및상품평을제공 오피니언분석은산학에서중요한이슈로떠오름 5
6
Contents 1. 서론 2. 오피니언마이닝 & 오피니언검색 3. 오피니언검색연구동향 4. 결론 5. Appendix: 데이터셋과성능평가, 참고문헌 7
오피니언마이닝 Opinion Mining/Sentiment Analysis The application of natural language processing, computational linguistics, and text analytics to identify and extract subjective information in source materials - Wikipedia Opinion Word Dictionary Construction Opinion Summarization Polarity Classification Opinion Mining Opinion Extraction Automatic Feature Extraction Opinion Retrieval 8
오피니언검색 (1) Document Collection Information (Ad-hoc) Retrieval Relevant Sentiment Analysis Subjective Opinion Retrieval 9
오피니언검색 (2) Query: iphone 5 Document Collection Information (Ad-hoc) Retrieval Sentiment Analysis iphone 5 s spec Relevant Subjective Nikon D4 is amazing Battery life of iphone 5 The spaghetti was awful The history of computer science Opinion Retrieval iphone 5 is the best iphone 5 is not innovative any more 10
오피니언검색 (3) Query: iphone 5 11
오피니언검색 (4) Query: Barack Obama 12
오피니언검색 (5) 오피니언마이닝에있어서전처리역할 오피니언검색은특정주제에관련된의견을포함하는문서를순위화 오피니언마이닝은일반적으로사람, 제품등과같은특정한주제 (target-specific) 를대상으로수행 문서마다연관성과의견성을측정가능하다면우선순위를두어마이닝수행가능 비연관문서나의견을포함하지않은문서를여과하여마이닝시스템의효율성증대가능 13
오피니언검색 (6) 가장쉬운방법? 기존검색시스템의질의어 (query) 에의견어 (opinion words) 를추가하여검색 예 ) Windows 8 Window 8 amazing awkward defects flaws 한계점 질의어와의견어의단어가중치 (term weighting) 을위한방법의부재 Topic drift 현상 제한된수의의견어사용 낮은효율성 14
오피니언검색 (6) 가장쉬운방법? 기존검색시스템의질의어 (query) 에의견어 (opinion words) 를추가하여검색 예 ) Windows 8 Window 8 amazing awkward defects flaws 한계점 질의어와의견어의단어가중치 (term weighting) 을위한방법의부재 Topic drift 현상 제한된수의의견어사용 낮은효율성 15
Contents 1. 서론 2. 오피니언마이닝 & 오피니언검색 3. 오피니언검색연구동향 4. 결론 5. Appendix: 데이터셋과성능평가, 참고문헌 16
Text REtrieval Conference (TREC) 출처 : Information Retrieval: A Health and Biomedical Perspective, Third Edition, William Hersh, M.D. 17
2 단계 (two-phase) 접근법 (1) 문서의적합성 (topical relevance) 과의견성 (subjectivity) 을각각계산한후선형적 (linear) 으로결합 가장많이사용된단순한방법론 BM25 나언어모델과같은기존정보검색모델을그대로이용하여적합성을계산 의견성계산을위한방법론연구에집중 이론적근거의부재 18
2 단계 (two-phase) 접근법 (2) Query: iphone 5 Opinion word D 1 Apple iphone Steve Jobs amazing iphone.. D 2 non-innovative iphone nice Samsung Galaxy Note 2 Topic: 2/100 = 0.02 Topic: 1/100 = 0.01 Opinion: 1/100 = 0.01 Opinion: 2/100 = 0.02 Final: λ 0.02 + (1- λ ) 0.01 Final: λ 0.01 + (1- λ ) 0.02 19
통합적접근법 2 단계접근법이높은효율성과검색성능을보임에따라최근이론적근거를내세운다양한연구들이소개됨 Vector Space 모델에기반한연구 [Vechtomova, 2007] 문서내출현한의견어들에해당되는가중치를부여 질의확장 (query expansion) 기법을활용한연구 [Huang and Croft, 2009] 언어모델프레임워크에서질의에종속적 / 비종속적인의견어를질의어의확장된단어로서간주 생성모델에기반한연구 [Zhang and Ye, 08], [Lee et al, 11] 적합성과의견성을동시에고려 20
주요순위화요소 (1) 의견어 (opinion word) 대다수의연구는미리구축한의견어사전을활용 의견어들의문서내출현빈도는대다수의연구에서사용 [Kovacevic and Huang 2008], [Clark et al, 2006], [Hanna et al., 2007], 의견어들의개별중요성을계산하는데있어다양한기법들이소개됨 기계학습이용 [Joshi et al., 2006] Diversity 척도를이용 [Hanna et al., 2007] 학습집합을이용한확률로추정 [Huang et al., 2007] 품사정보를활용 [Wiebe et al., 2004] [Yang et al., 2006] 웹을활용 [Oard et al., 2006], [Turney and Littman 2003] WordNet 이나 SentiWordNet 을사용 [Kim and Hovy, 2005]. [Bermingham et al., 2008] [Na et al., 09] 21
주요순위화요소 (2) 근접성 (Proximity) Bag-of-Words 가정을넘어서서두단어의출현위치정보는검색순위화에있어도움된다고알려짐 오피니언검색에서역시질의단어와의견어들의위치정보는중요하게다루어짐 질의어와인접한의견어들이보다주제에적합한의견을표현하고있을가능성이높음 질의어부근에출현한의견어들의점수만을고려 [Yang et al, 2006] Vector space 모델에서질의어와인접한의견어들만고려 [Vechtomova, 2007] 질의에기정의된크기의 window ( 예, 10 단어 ) 내에출현한의견어만고려 [Zhou et at., 2007] 다양한종류의근접성기반커널함수 (proximity-based kernel functions) 를비교 [Gerani et al., 2010] 22
Contents 1. 서론 2. 오피니언마이닝 & 오피니언검색 3. 오피니언검색연구동향 4. 결론 5. Appendix: 데이터셋과성능평가, 참고문헌 23
결론 정보검색과오피니언마이닝기술을접목한오피니언검색은다양한요소를고려하며발전 의견어출현정보, 개별의견어중요도, 질의어와의근접성, 정보검색모델과의결합방법등 향후연구주제 언어학적정보 (linguistic clue) 를보다적극적으로활용 구문분석, 의미분석, 개체명인식, 대용어인식 (anaphora resolution) 노이즈제거기법 스팸탐지, HTML 태그처리, Blog 템플릿제거 SNS, micro blogs 등의도메인확장 트위터, 페이스북등의보다짧고철자오류가많이포함된텍스트를처리하기위한방법론고안 24
감사합니다. 25
Contents 1. 서론 2. 오피니언마이닝 & 오피니언검색 3. 오피니언검색연구동향 4. 결론 5. Appendix: 데이터셋과성능평가, 참고문헌 26
데이터셋 평가콜렉션 TREC Blogs06 3.2M permalinks ( 문서 ) 질의 2006, 2007, 2008 년에공개된 150 개의 topic 의견어사전 General Inquirer (3,600 단어 ) SentiWordNet 27
Topic 예제 28
Title List Topic-06 Topic-07 Topic-08 March of the Penguins, larry summers, state of the union, Ann Coulter, abramoff bush, macbook pro, jon stewart, super bowl ads, letting india into the club, arrested development, mardi gras, blackberry, netflix, colbert report, basque, Whole Foods, cheney hunting, joint strike fighter, muhammad cartoon, barry bonds, cindy sheehan, brokeback mountain, bruce bartlett, coretta scott king, american idol, life on mars, sonic food industry, jihad, hybrid car, natalie portman, Fox News Report, seahawks, heineken, Qualcomm, shimano, west wing, World Trade Organization, audi, scientology, olympics, intel, Jim Moran, zyrtec, board chess, Oprah, global warming, ariel sharon, Business Intelligence Resources, cholesterol, mcdonalds jstor, lactose gas, Steve jobs, alterman, king funeral, davos, brrreeeport, carrie underwood, Barilla, Aperto Networks, SCI FI CHANNEL, nasa, sag awards, northernvoice, allianz, dice com, snopes, varanasi, pfizer, andrew coyne, Christianity Today, howard stern, challenger, mark driscoll, mashup camp, hawthorne heights, oscar fashion, big love, brand manager, ikea, fort mcmurray, goobuntu, winter olympics, cointreau, mozart, grammys, LexisNexis, plug awards, Beggin Strips, Lance Armstrong, teri hatcher, lawful access, censure, Opera Software OR Opera Browser OR Opera Mobile OR Opera Mini, bolivia, tivo, sasha cohen, sorbonne, ford bell, Hitachi Data Systems Carmax, Wikipedia primary source, Jiffy Lube, Starbucks, Windows Vista, Mark Warner for President, women in Saudi Arabia, UN Commission on Human Rights, Frank Gehry architecture, Picasa, Chipotle Restaurant, Ed Norton, Iceland European Union, tax break for hybrid automobiles, Whole Foods wind energy, Papa John's Pizza, Mahmoud Ahmadinejad, MythBusters, China one child law, intelligent design, Sheep and Wool Festival, Subway Sandwiches, Yojimbo, Zillow, Nancy Grace, flag burning, NAFTA, Oregon Death with Dignity Act, Morgan Freeman, System of a Down, Sew Fast Sew Easy, I Walk the Line, World Bank, Ruth Rendell, Mayo Clinic, Project Runway, New York Philharmonic Orchestra, israeli government, The Geek Squad, TomTom, federal shield law, David Irving, A Million Little Pieces, talk show hosts, Women on Numb3rs, universal health care, Trader Joe's, Sopranos, YouTube, George Clooney 29
정답집합 각 Topic 마다적합하고의견을포함한정보를문서단위로단계적으로평가 풀링 (Pooling) 방법으로구축 다양한참가팀이제출한결과물에서상위랭크된문서들을합하여정답집합구축 Topical Retrieval Opinion Retrieval 30
평가척도 정보검색의다양한척도를사용하여평가를수행 Precision at 10, mean average precision (MAP) 검색결과를두가지측면 (topical retrieval, opinion retrieval) 에서각각평가 따라서네가지척도를사용 Topic MAP, Opinion MAP Topic P@10, Opinion P@10 31
Baseline 언어모델 (query likelihood model) 을이용하여정보검색과오피니언검색의성능측정 평탄화파라메터인 μ 에따라각각의 topic 집합마다유사한경향성을보임 정보검색과오피니언검색은강한 corelation 을가짐 μ 32
Baseline Correlation 33
TREC 참가팀의성능 Query expansion, phrasal indexing, passage-based search, 34
Reference (1) Vechtomova, O. (2007). Using subjective adjectives in opinion retrieval from blogs. In TREC 2007: Proceedings of The Sixteenth Text REtrieval Conference, Gaithersburg, Maryland, USA. Huang and Croft (2009), A unified relevance model for opinion retrieval, Proceedings of the 18th ACM conference on Information and knowledge management, 2009 Zhang, M. and X. Ye (2008), A generation model to unify topic relevance and lexicon-based sentiment for opinion retrieval. In SIGIR 08: Proceedings of the 31st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, New York, NY, USA, pp. 411 418. ACM Lee, S.-W, Lee, J.-T., Song, Y.-I., Han, K.-S., Rim, H.-C (2011), A New Generative Opinion Retrieval Model Integrating Multiple Ranking Factors, Journal of Intelligent Information Systems (JIIS), May. 2011 Kovacevic, M. and X. Huang (2008). York University at TREC 2008: Blog Track. In TREC 2008: Proceedings of the Sixteenth Text REtrieval Conference, Gaithersburg, Maryland, USA. Clark, M., U. C. Beresi, S. Watt, and D. Harper (2006). RGU at the TREC blog track. In TREC 2006: Proceedings of the Fifteenth Text REtrieval Conference, Gaithersburg, Maryland, USA. Hannah, D., C. Macdonald, J. Peng, B. He, and I. Ounis (2007). University of glasgow at TREC 2007: Experiments in blog and enterprise tracks with terrier. In TREC 2007: Proceedings of The Sixteenth Text REtrieval Conference, Gaithersburg, Maryland, USA. Joshi, H., C. Bayrak, and X. Xu (2006). UALR at TREC: Blog track. In TREC 2006: Proceedings of the Fifteenth Text REtrieval Conference, Gaithersburg, Maryland, USA. Hoang, L., S.-W. Lee, G.-W. Hong, J.-Y. Lee, and H.-C. Rim (2008). A Hybrid Method for Opinion finding Task (KUNLP at TREC 2008 Blog Track). In TREC 2008: Proceedings of the Sixteenth Text REtrieval Conference, Gaithersburg, Maryland, USA. 35
Reference (2) Wiebe, J., T. Wilson, R. Bruce, M. Bell, and M. Martin (2004). Learning subjective language. Computational linguistics 30(3), 277 308. Yang, K., N. Yu, A. Valerio, and H. Zhang (2006). WIDIT in TREC-2006 Blog Track. In TREC 2006: Proceedings of the Fifteenth Text REtrieval Conference, Gaithersburg, Maryland, USA. Oard, D., T. Elsayed, J. Wang, Y. Wu, P. Zhang, E. Abels, J. Lin, and D. Soergel (2006). TREC-2006 at Maryland: Blog, Enterprise, Legal and QA Tracks. In TREC 2006: Proceedings of the Fifteenth Text REtrieval Conference, Gaithersburg, Maryland, USA. Turney, P. and M. Littman (2003). Measuring praise and criticism: inference of semantic orientation from association. In ACM Transactions on Information Systems, Volume 21, pp. 315 346. Kim, S.-M. and E. H. Hovy (2005). Automatic detection of opinion bearing words and sentences. In IJCNLP-05: Companion Volume to the Proceedings of the Second International Joint Conference on Natural Language Processing Bermingham, A., A. F. Smeaton, J. Foster, and D. Hogan (2008). DCU at the TREC 2008 Blog Track. In TREC 2008: Proceedings of the Sixteenth Text REtrieval Conference, Gaithersburg, Maryland, USA. Na, S.-H. and H. T. Ng (2009). A 2-poisson model for probabilistic coreference of named entities for improved text retrieval. In Proceedings of the 32nd international ACM SIGIR conference on Research and development in information retrieval, SIGIR 09, New York, NY, USA, pp. 275 282. ACM. Yang, K., N. Yu, A. Valerio, and H. Zhang (2006). WIDIT in TREC-2006 Blog Track. In TREC 2006: Proceedings of the Fifteenth Text REtrieval Conference, Gaithersburg, Maryland, USA. Zhou, G., H. Joshi, and C. Bayrak (2007). Topic categorization for relevancy and opinion detection. In TREC 2007: Proceedings of The Sixteenth Text REtrieval Conference, Gaithersburg, Maryland, USA. Gerani, S., M. J. Carman, and F. Crestani (2010). Proximity-based opinion retrieval. In Proceedings of the 33rd international ACM SIGIR conference on Research and development in information retrieval, SIGIR 10, New York, NY, USA, pp. 403 410. ACM. 36