2 2 3 3 2 5 2 5 22 6 23 8 3 Bayesian network 3 Bayesian robabilistic method 32 8 33 20 34 2 35 25 i
4 27 4 network toology 27 42 32 43 35 5 39 5 39 52 39 6 42 43 ii
morhological analysis word sense disambiguation morheme
word sense disambiguation word sense disambiguation 2 aanese to Korean machine translation word sense disambiguation collocation [ 96] [ 98] collocation attern 2
machine learning natural language rocessing corus morhological analysis word sense disambiguation [Gale 92] [Brown 9] 2 arallel corus 50 context 3 : m : n 40% 3
machine learning Bayesian network corus 2 3 Bayesian network 4 5 6 4
2 2 2 direct transfer ivot [ 94] arsing idiom 2 3 noun noun article verb unctuation 22 5
2 noun noun[ ] article[ ] verb unctuation 23 3 22 2 6
2 7 kara 2 + 3 4 5 3
2 kara 5 50% context kara 60% Bayesian network formalization 23 8
2 Bayesian network formalization kara + kara gendou chichi kangaeru kuru kuru comlement tegami + ga + V N N2 2 discrete value comlement + 9
2 V N N2 2 0
3 Bayesian network 3 Bayesian network Bayesian network Heckerman 95 deendency causal relationshi causality robabilistic semantics rior knowledge data overfitting 3 Bayesian robabilistic method Bayesian robabilistic method N N heads tails hysical robability N
3 Bayesian network N uncertainty N Θ θ θ arameter Θ uncertainty robability density function θ ξ ξ X l l l N D X x X N x N rior robability distribution θ ξ x N D ξ D Θ θ ξ D θ ξ θ D ξ = D ξ D ξ = D θ ξ θ ξ dθ 2 D θ ξ Θ 2
3 Bayesian network D θ θ h t θ ξ θ θ θ D ξ = 3 D ξ h t D θ ξ θ D ξ rior robability distribution osterior robability distribution h t binomial samling sufficient statistics Θ N X N + = heads D ξ = X N + = = heads θ ξ θ D ξ dθ θ θ D ξ dθ E θ D ξ θ 4 Θ rior robability distribution beta distribution θ ξ Beta Γ α α h α t = θ αh αt Γ α h Γ αt θ θ 5 α h α t hyerarameter α α h α t Γ Γ x xγ x Γ 3
3 Bayesian network osterior robability distribution Heckerman 95 Γ α + N θ D ξ = Γ α + h Γ α h t θ + t θ = Beta θ α + h α α h + h α t + t + h t t 6 θ α h θbeta θ α α dθ = 7 α h t N α h + h X N + = heads Dξ = 8 α + N rior robability distribution θ ξ imagined future data equivalent samles Heckerman 95 beta rior robability θ ξ = 04Beta20 + 04Beta20 + 02Beta22 9 04 4
3 Bayesian network 02 hidden variable binomial distribution x s ξ = f x s 0 fxs s likelihood function X Gaussian hysical robability distribution µ v x s ξ = 2πv / 2 e 2 x µ / 2v s = {µ v} rior Bayes theorem osterior 5
3 Bayesian network D s ξ s ξ s D ξ = 2 D ξ s arameter S xn D ξ xn + s ξ s D ξ + = ds 3 closed form binomial multinomial normal Gamma Poisson multinomial samling X discrete r x x likelihood function k X = x s ξ = s k = r k 4 s s s arameter s s i binomial samling hysical robability D X x X N x N 6
3 Bayesian network sufficient statisticsn N r N i D x i multinomial samling conjugate rior robability Dirichlet distribution s ξ = Dir s α α r Γ α r Γ α k = k k = r s αk k 5 α α α r α k k r osterior robability distribution s D ξ = Dir s α + N α + N r r 6 imagined future data equivalent samles Dirichlet distribution conjugate rior robability distribution D k αk + N k X N + = x D ξ = sk Dir s α + N α r + N r ds = 7 α + N quantity marginal likelihood evidence D ξ 7
3 Bayesian network Γ α D ξ = Γ α + N r k N k k = Γ αk Γ α + 8 ξ ξ 32 Bayesian network joint robability distribution X X X n X conditional indeendence assertion S 2 local robability distribution P S directed acyclic grah S X X i Pa i S X i S S X joint robability distribution 8
3 Bayesian network n x = Pa i= x i i 9 P Π x S P hysical robability Bayesian robability rior knowledge Heckerman f a s g j 9
3 Bayesian network 20 33 Bayesian network X joint robability distribution f = = ' ' f j g s a f j g s a f j g s a j g s a f j g s a f 20 discrete variable = = ' ' ' ' ' ' ' ' f f s a f j f g f s a f j f g f s a f j f g s a f s a f j f g s a f j g s a f 2 2 a s a s f
3 Bayesian network conditional indeendence assertion Howard Shachter Lauritzen ensen conditional indeendence assertion NP-hard Cooer aroximate inference NP-hard undirected cycle toology roblem domain 34 Bayesian network local robability distribution X hysical joint robability distribution h n x z S = x a z S s i= i i i h 22 2
3 Bayesian network z i x i a i z i S h arameter z s z z n S h hysical joint robability distribution S hyothesis random samle D x x N D x l l case z s uncertainty Z s rior robability density function z s S h D z s D S h z i x i a i z i S h local distribution function robabilistic classification regression conditional indeendence assertion unrestricted multinomial distribution linear regression with Gaussian noise generalized linear regression unrestricted multinomial distribution X i X r i discrete value Pa i configuration multinomial distribution 22
3 Bayesian network x i k i a z Pa i = z ijk j i a z S i qi i ri k= 2 h i qi j= = z q = ijk > 0 X Pa i i r Pa i i 23 arameter z ij = zij2 zijr i 24 unrestricted Pa i osterior robability distribution z s D S h closed form D missing data D comlete z ij h i z S = z s n q i= j= ij S h 25 [Siegelhalter and Lauritzen 90] arameter indeendence 23
3 Bayesian network h i z D S = z s n q i= j= ij D S h 26 z ij z ij rior robability distribution Dirz ij α ij α ijri osterior distribution h ij ij ij ij z D S = Dir z α + N α + N ijr i ijri 27 case configuration x N+ D S h z s x h D S E h D S z z s N + = n i= ijk 28 D n n h h h N + D S = zijk z s D S dz s = zijk zij D S i= i= x dz ij 29 7 24
3 Bayesian network x h N + D S = N i= α α ijk ij + N + N ijk ij 30 35 2 4 V N N2 2 5 5 causal relationshi rior knowledge corus random samle local robability distribution udate N discrete variable 25
3 Bayesian network inference 26
4 4 4 network toology causal relationshi fitness criterion searching method 5 2 27
4 N: : N2: 2: N2 V: V 2 2 N N2 2 28
4 2 2 6 ga karadeni madewo discrete variable 2 5 4 2 29
4 00 7 900 4000 semantic class natural language rocessing semantic Wordnet [ 96] [ 82] [ 98] [ 98] 22 2 3 gakkou kaisha P 30
4 yoru toki T hohoemi ai AC koe kanji AF kangae rigai AR en metoru AU juuji sikaku AT hou kisoku AI katana enitsu CI idou setsyoku AP itami nemuri AB ame kaze AN gakumei densai ASP kami mokuzai CMT kuruma jikatetsu CT terebi razio CMA tegami denwa ACM satyou souri CS okane taiya CC me mimi kutsi CH eigou kankokugou AL hazimari AS ketsumatsu kureru G iu hanasu S iku kuru D kiku ukeru R sinsetsuda B daisetsuda REL kinzuru L kazaru C siru kangaeru K kaeru kiru TR sinu neru HC 3
4 nakasareru yorokobu kanasii 2 PA F V 3 N N2 22 cycle unrestricted multinomial distribution 3 42 V 2 N N2 corus incomlete data comlete data 3 32
4 V hazimaru TR N zi T 2 N2 incomlete case - Monte-Carlo Gibbs samling Gibbs samling X = {X X n } joint robability distribution x Gibbs samler x fx X assign 2 X i unassign n 3 X i fx 4 2 3 fx fx E x fx Gibbs samler irreducible X configuration configuration 33
4 rior robability distribution equivalent samle size 5 equivalent samles [Heckerman 95] Gibbs samling corus incomlete data D = {y y N } y l incomlete case D c D X il h h x' il Dc \ xil S x' il Dc \ xil S = 3 h x" D \ x S x" il il c il D c \ x il x il X il 8 h D S = n qi r Γ αij i Γ αijk + Nijk i= j= Γ αij + Nij k = Γ αijk 32 i j 34
4 configuration k comlete data D c 25 27 osterior robability density z s D c S h 000 43 4 kara de V 2 N2 N V suteru D wo N gomi CC Nmado CC V N V iu S 35
4 N tachiba CS N N N tokyo P made N kyoto P N Ngatsu T 4 2 N2 V N 36
4 37 2 2 N V V N V N V N V N V N V N V N N V argmax argmax argmax argmax argmax argmax = = = = = = 33 2 3 2 N = = = = = = V N N N N N N N 2 2 2 2 2 2 2 argmax argmax argmax argmax argmax argmax 34 4
4 = argmax N = argmax N 35 context 38
5 5 5 7000 rerocessing case V N 2 N2 kara de 723 2367 52 4 kara de 80% 3 % kara 44 28 889% + 73 56 902% 44 39 886% 27 89 87% 45 27 876% 723 639 884% 39
5 de 996 907 9% 203 089 905% 68 34 798% 2367 230 899% 3 kara 723 44 73 44 27 45 28 56 39 89 27 884% de 996 203 68 907 089 34 889% 4 kara de 300% 508% 598% 703% 884% 899% 4 kara de 40
5 4
6 6 Bayesian network corus context corus arsing 00% word sense disambiguation 42
[ 94] 994 [ 97] 24 284-296 997 98 270-280 998 96 Wordnet 996 [ 98] 998 96 996 [ 95] 995 [ 98] 998 [ 9] Technical Reort No 900 99 [ 82] : 982 [ 90] : 990 [Allen 95] Allen Natural Language Understanding Benjamin Cummings 995 [Brown 9] Brown P F Pietra S A D Pietra V D and Mercer R L Word-sense disambiguation using statistical method In Proceedings of Association for Comutational Linguistics ACL 264-270 99 [Charniak 9] Charniak E Bayesian Networks without Tears AI Magazine Vol 2 No 4 50-63 99 [Charniak 94] Charniak E Statistical Language Learning Cambridge MA:MIT Press 994 [Cherkassky 98] Cherkassky V and Mulier F LEARNING FROM DATA Concets Theory and Methods ohn Wiley & Sons 998 [Cooer 90] Cooer G Comutational comlexity of robabilistic inference using Bayesian belief networks Research note Artificial Intelligence Vol 42 393-405 990 [Gale 92] Gale W A Church K W and Yarowsky D A A method for disambiguating word senses in a large corus Comuters and the Humanities 992 43
[Heckerman 95] Heckerman D A Tutorial on Learning With Bayesian Networks Technical Reort MSR-TR-95-06 Microsoft Research Redmond WA 995 [Howard 8] Howard R and Matheson Influence diagrams In Howard R and Matheson editors Readings on the Princiles and Alications of Decision Analysis Vol 2 72-762 Strategic Decisions Grous Menlo Park CA 98 [ensen 90] ensen F Lauritzen S and Olsen K Bayesian udating in recursive grahical models by local comutations Comutational Statistics Quarterly Vol 4 269-282 990 [ensen 96] ensen F An Introduction to Bayesian Networks Sringer 996 [Lauritzen 88] Lauritzen S and Siegelhalter D Local comutation with robabilities on grahical structures and their alication to exert systems Royal Statistical Society B Vol 50 57-224 988 [Mitchell 97] Mitchell T M MACHINE LEARNING McGraw-Hill 997 [Pearl 87] Pearl Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference San Mateo CA:Morgan Kaufmann 987 [Shachter 88] Shachter R Probabilistic inference and influence diagrams Oerations Research Vol 36 589-604 988 [Siegelhalter and Lauritzen 90] Siegelhalter D and Lauritzen S Sequential udating of conditional robabilities on directed grahical structure Networks Vol 20 579-605990 44