인공지능이어떻게핵전쟁의 위험에영향을 미칠수있습니까? EDWARD GEIST ANDREW J. LOHN 전망 시의적절한정책문제에관한전문가의통찰 C O R P O R A T I O N
인쇄및전자방식배포권리가제한됨 본문서및이에포함된상표 ( 들 ) 은법률에의해보호됩니다. 이러한 RAND 의지적재산권표시는비상업적용도로만제공됩니다. 본발행물의온라인무단게시는금지됩니다. 본문서는변경되지않고온전하게사용하는한개인적인용도로만복제가허용됩니다. 본연구문서의일부또는전부를상업적용도를위해다른형식으로복제하거나재사용하려면 RAND 의승인을받아야합니다. 재인쇄및링크권한에대한자세한내용은 www.rand.org/pubs/permissions.html 을참조하십시오. RAND 의발행물은연구를의뢰한고객및후원자의의견을반드시반영하지는않습니다. R 은등록상표입니다. 이발행물의자세한정보는 www.rand.org/t/pe296 을참조하십시오. Copyright 2018 RAND Corporation
오.. (AI). 1 RAND Corporation AI 2040 AI,. AI,.,.,. AI,.. AI 25. AI,. " ", AI.,.. ( ). AI,. AI - 1
-., AI. AI. 3 AI,,,,. AI. AI ( ). AI 2040.. AI, AI., AI.. AI 핵발사장치와직접연결되지않을경우, AI 는단계적확대문제에대해인간에게여전히조언을제공할수있습니다.., ( ),.,. 핵균형앞에놓인중요한변화들의암시 2015 11,,. " ". TV " Status-6". (3 ) Status-6 (Sutyagin, 2016)., 100km/hr 2
. AI. 2 Status-6 AI,, AI. 3. AI. AI 2040.. (Doomsday Drone). 4 2040? Status-6 Status-6 의구성요소 Project 09852 SSN 자체추진수중차량의핵잠수함운반대 Project 09851 SSN 자체추진수중차량 센서 탄두 유도시스템 원자로격실 구동장치 작전종심 = <1,000m, 속도 = <185km/hr, 범위 = <10,000km, 직경 = 1.6m 거대한어뢰처럼생겼고소형원자로로구동되는 해양다목적시스템 Status-6 는바다속의거의모든것보다더빠를수있는속 도와범위로적의방어를무력화할것입니다. 3
.,. AI. 2017 5 6, RAND Corporation AI AI 3. AI., AI. 워크샵의방법론과설명 25 AI, RAND 2017 5 6. AI. 워크샵 1 2017 5 1 RAND. 16 AI RAND. AI AI. 이처럼낮선전략적배치는미국과소련연방이불편한평화를유지했던때보다덜안정적이라고입증될수있습니다... 1. (New START) 2030 2. 3. 4., AI C4ISR(,,,,, ). AI. 4
AI... 워크샵 2 2017 5 25 19. 7 AI, 5, 3 AI, 4. AI, AI AI,. 3. AI, ( ). AI (Generative Adversarial Networks, ) 2.,. AI. AI, AI.., AI. AI, AI. AI, AI (, )., AI. AI. (GPU).. 5
어떤하위그룹은미래의 AI 시스템이반드시무기통제체제가될수있고, 인간의투입없이도준수를모니터링하고위반을판정할수있다고도발적인주장을했습니다. 워크샵 3 2017 6 9 RAND. 8, AI 5 15. AI AI. 2. RAND,,.,., AI. AI., ( ) AI. AI.,. (ICBM), "3 ". AI. AI. AI,., AI.. AI. AI의잠재적영향을평가하기위한이론적, 역사적배경, (MAD),. MAD 6
(Buchan et al., 2003).,., Ronald Reagan, (Garthoff, 1987; Geist, 2012). MAD. MAD, NATO. MAD,.,, (Long, 2008).,.,, (Slocombe, 1981). ( ).. ( ) ( ) (Cimbala, 2002)., (Long, 2008, p. 9)., 2... Denis Healey, 5%, 95% (Healey, 1989). 핵전략목표범주 측면 정의 강요 억제력 적국이하려는것을단념시킴 강압 적국이하려하지않는것을하도록강요 확신 안보보장이신뢰할만하다고동맹국을설득 안심 도발행위를삼가면공격받지않을거라고적국을설득 7
. (Schelling, 1966).. 5.. 2 (Cimbala, 2002, p. 66)., 1960 (Cimbala, 2002, p. 98).,.,.,. (Cimbala, 2002, p. 110).,.., AI MAD. AI.,,,...,,., AI, (ISR).,. AI. AI, AI. ISR AI (ATR)., AI 전략적안정성은적이도발적인행동에나설만한커다란동기가결여되어있을때존재합니다. 8
. AI,. Paul Bracken, AI (Bracken, 2017). 냉전시대의 AI Marvin Minsky AI (Minsky, 1968, p. v). AI 1950,. AI. 1950 1980,. 20. AI ( ).,,, AI, (DNNs) (Goodfellow, Bengio, and Courville, 2016). DNNs AI,,. Alphabet DeepMind AlphaGo AI DNNs. AI 60.,,? AI 50,. AI,. Claude Shannon 1950 (Programming a Computer for Playing Chess) (Shannon, 1950, p. 256). 1950 AI (Simon and Newell, 1958; Newell, Shaw, and Simon, 1959).. 1950, Herman Kahn (doomsday machines) (Kahn, 1960, pp. 145 154). Kahn 9
, (Colossus, 1970), (WarGames, 1983), (Terminator, 1984)., AI..,... 6 ICBM. (Dead Hand) (Perimetr) (Hoffman, 2009).,, AI. 7 AI. 1980 (SAPE) AI ICBM. SAPE B-2. SAPE (Roland and Shiman, 2002, p. 305; Long and Green, 2012). AI와새로운지정학적질서 20 AI,.,. ISR, ATR.. ICBM,,, ICBM. ( )...,. 10
. Alfred T. Mahan,. (Mahan, 1912, p. 105).,..,. AI ICBM (Holmes, 2016)..... AI.,. 8.,., Status-6 AI 2,. AI., RS-28 (Sarmat) 2 (MIRV) ICBM. MIRV ICBM. 21 ICBM. ICBM. 핵전략의중요한난제는적국이한국가의안보보복전력을선제타격위협이나인류파멸의흉기로해석하고, 그에따라반응할수있다는것입니다. 11
점점다극화되어가는핵환경은 AI 의잠재적전략적영향도악화시키고있습니다.., (Mozyr ). AI. 6, 5.. 9..,,.,.., AI.,. 가능한 AI 미래에대한공개적인전문가의견수렴,. AI의발전예측 AI 4...,. AI, 13.. 슈퍼인텔리전스. Nick Bostrom(2014),. Bostrom AI.. 12
,.,. AI,. 제한적인발생,.,., AI,. AI,. 지속적으로증가하는발전 AI.,,,. AI. 전문가의견으로본대체 AI 미래 가능한 AI 의미래상태 전문가의견의범주 인공지능의겨울 제한적인발생또는지속적으로증가하는발전 슈퍼인텔리전스 자기만족주의자들 가능함 AI 의발전에도불충분한데이터와너무복잡한문제들 존재할가능성은낮지만인간보다는더안전할듯함 불안조성자들 가능성낮음 거의작동하지않는알고리즘은적을경보할수있고, 사용할경우실패할수있음 궁극적으로피할수없고, 의도적이든우발적이든인류를파괴할수있음 파괴주의자들 중립 AI 가비극적으로실패하도록만들어질수있음, 또는 AI 의실패유도가안정화확신을제공할수있음 파괴와인간통제에모두저항하는슈퍼인텔리전스 13
AI. 2040 20, AI 1990., 20.,. AI 정체기 AI. AI., AI. AI. AI,., (Moore) (Mack, 2011). AI,. 핵안보에예상되는영향,. AI.. 자기만족주의자들 AI.. AI, AI,. AI, AI., 2040 AI. AI 14
또한 AI 정체기 는현재의능력으로부터상당한발전이일어난후에발생할수있고, 이전두가지시나리오와유사한많은핵안보난제들을만들어낼수있습니다..,. 불안조성자들 AI..., AI. AI. 2,. 파괴주의자들 AI.. AI,..,. AI,, 2,. AI,. 구체적인사례 : 이동식미사일발사장치추적 AI. RAND. 15
확실한 2차타격에대한영향., ( ).. ICBM. AI ISR,. ICBM. AI,. ATR,.. 어려운기술적도전 RAND, 매우근거리에서발사될경우재래식무기라도실행가능한옵션이될수있으며따라서선제대항타격의신뢰성을크게증가시킬수있습니다.. 9 AI, 2040.,.,.,. 17 0~5km. (km) "., 475-kT W88 1 Trident II 3 10, 100-kT W76 1 5., ( 30-kT CM 200-kT CM) (CM). ( ), 16
표적타격에필요한최소무기수 5.0 4.0 1 필요한무기수 2 W88 3 4 5 (km) 3.0 200-kT CM MMIII 6 10 MMIII 2.0 30-kT CM W76 11 20 1.0 ATACM >20 0.0 0 JDAM 15 30 ( ) 이그림은유효반경이 0~5km 인무기로이동식표적을파괴하는데필요한다양한유형의탄두수를보여줍니다. 거대한 살상반경 은 직경 (km) 으로측정되지만, 탄도미사일에의해운반되는여러개의열원자탄두는미사일발사장치를파괴하는데커다란확신을주기위 해필요합니다. ATACM = 육군전술미사일시스템, JDAM = 합동정밀직격탄, kt = 킬로톤 ; MMIII = 대륙간탄도탄 III.,. AI ICBM,. ICBM, AI.,,. 17
,.,.. 구체적인사례 믿음직한조언자 AI 2, AI,,., AI.,. AI ( ). 의사결정에있어 AI의가능한역할 AI. Alphabet DeepMind AlphaGo AI (Etherington, 2017).,.. DeepMind, AI 는급격히발전하면서점점복잡한업무에서인간을능가하는성과를보이고있습니다. (Starcraft) AI (Woyke and Kim, 2017)., 2040 AI., AI..,.,, AI 2040. AI.. 18
적대적행위로인한제한적효과 AI., 1945,..,,,.,. AI,.,,. 해킹 AI, AI.,.,. AI,. 교육데이터공격 AI.. (Anderson et al., 2017; Biggio, Nelson, and Laskov, 2012; Kearns and Li, 1992),..., (Kegelmeyer et al., 2015).. 입력조작 AI. AI.., AI (Karpathy, 2015)..,., 19
, AI.,, AI (Papernot, McDaniel, and Goodfellow, 2016).,. 핵안보에대한제한된효과의영향, AI,., AI. AI,., AI. AI.,,,.... AI 의발전은거침없어보입니다. 기업과정부들은공격적인사용과방어적인사용을모두포함하여계속확대되는응용범위에앞다퉈 AI 를채용하고있습니다. 일부분가능한 AI 의안정성향상효과,. AI,. AI. AI. AI. AI,.. 높은오류발생을넘어서는시기 AI...,, 20
,. AI,. 전략적안정성을위한잠재적협력., 1983 (Hoffman, 2009, pp. 6 11).,. AI....,,.,.,.,.,. AI.,..., ( AI ). AI,.,. 철저한투명성, AI... AI. AI. AI. (Kerckhoff, 1883). AI, AI. 21
결론 AI, 2040. AI. AI AI. AI. AI. AI. AI., AI,., AI.,.,,.. 22
참고 1,, AI.,,, AI,. 2 Status-6 (AI). Tuchkov(2016) Status-6 (Rossiiskii proekt Status-6 meniaet sootnosheniia iadernykh sil v mire) (2016). Status-6. 3, AI,. 4,. Ronald Reagan (Strategic Defense Initiative) Oznobishev, Potapov, and Skokov(2008). Vladimir Putin., 2012 - (Putin, 2012). 5 2010 (Nuclear Posture Review). (U.S. Department of Defense, 2010). 6,.. 7 (Perimetr) (AI). Timoshenko(2015) Valagin(2014). 8 Akhmerov, Akhmerov, and Valeev(2016). 9 Brien Alkire and Jim Powers RAND. 23
참고문헌 Akhmerov, D. E., E. N. Akhmerov, and M. G. Valeev, Uiazvimost kontseptsii neiadernogo razoruzheniia strategicheskikh iaderniyi sil Rossii [ The Dubiousness of the Concept of a Non-Nuclear Disarming Strike Against Russia s Strategic Nuclear Forces ], Vestnik akademii voennykh nauk, Vol. 54, No. 1, 2016, pp. 37 41. Anderson, H. S., A. Kharkar, B. Filar, and P. Roth, Evading Machine Learning Malware Detection, blackhat.com, July 2017. As of August 15, 2017: https://www.blackhat.com/docs/us-17/thursday/us-17-anderson-bot-vs-bot- Evading-Machine-Learning-Malware-Detection-wp.pdf Biggio, B., B. Nelson, and P. Laskov, Poisoning Attacks Against Support Vector Machines, Proceedings of the 29th International Conference on Machine Learning, July 2012, pp. 1467 1474. As of August 15, 2017: https://arxiv.org/pdf/1206.6389.pdf Bostrom, Nick, Superintelligence: Paths, Dangers, Strategies, Oxford: Oxford University Press, 2014. Bracken, P., The Intersection of Cyber and Nuclear War, The Strategy Bridge, blog post, January 17, 2017. As of August 15, 2017: https://thestrategybridge.org/the-bridge/2017/1/17/the-intersection-of-cyberand-nuclear-war Buchan, G., D. Matonick, C. Shipbaugh, and R. Mesic, Future Roles of U.S. Nuclear Forces: Implications for U.S. Strategy, Santa Monica, Calif.: RAND Corporation, MR-1231-AF, 2003. As of March 8, 2018: https://www.rand.org/pubs/monograph_reports/mr1231.html Cimbala, S. J., The Dead Volcano: The Background and Effects of Nuclear War Complacency, Westport, Conn.: Praeger, 2002. Etherington, D., Google s AlphaGo AI Beats the World s Best Human Go Player, TechCrunch, May 23, 2017. As of August 15, 2017: https://techcrunch.com/2017/05/23/googles-alphago-ai-beats-the-worldsbest-human-go-player/ Garthoff, R. L., Refocusing the SDI Debate, Bulletin of the Atomic Scientists, Vol. 43, No. 7, September 1987. Geist, E., Was There a Real Mineshaft Gap? Bomb Shelters in the USSR, 1945 62, Journal of Cold War Studies, Vol. 14, No. 2, Spring 2012, pp. 3 28. Goodfellow, I., Y. Bengio, and A. Courville, Deep Learning, Cambridge, Mass.: MIT Press, 2016. Healey, D., The Time of My Life, London: Michael Joseph, 1989, p. 243. Hoffman, D. E., The Dead Hand, New York: Doubleday, 2009. Holmes, J. R., Sea Changes: The Future of Nuclear Deterrence, Bulletin of the Atomic Scientists, Vol. 72, No. 4, 2016, pp. 228 233. Kahn, H., On Thermonuclear War, Princeton, N.J.: Princeton University Press, 1960. Karpathy, A., Breaking Linear Classifiers on ImageNet, Andrej Karpathy blog, March 30, 2015. As of August 15, 2017: http://karpathy.github.io/2015/03/30/breaking-convnets/ Kearns, M., and M. Li, Learning in the Presence of Malicious Errors, SIAM Journal on Computing, Vol. 22, No. 4, March 1992, pp. 807 837. As of August 15, 2017: https://doi.org/10.1137/0222052 Kegelmeyer, P., T. M. Shead, J. Crussell, K. Rodhouse, D. Robinson, C. Johnson, D. Zage, W. Davis, J. Wendt, J. Doak, T. Cayton, R. Colbaugh, K. Glass, B. Jones, and J. Shelburg, Counter Adversarial Data Analytics, Albuquerque, N.M.: Sandia National Laboratories, SAND2015-3711, May 2015. As of August 15, 2017: http://www.sandia.gov/~wpk/pubs/publications/cada-full-uur.pdf Kerckhoff, A., La Cryptographie Militaire, Journal des Sciences Militaires, January 1883. Long, A., Deterrence from Cold War to Long War: Lessons from Six Decades of RAND Research, Santa Monica, Calif.: RAND Corporation, MG-636-OSD/AF, 2008. As of March 8, 2018: https://www.rand.org/pubs/monographs/mg636.html Long, A., and B. R. Green, Stalking the Secure Second Strike: Intelligence, Counterforce, and Nuclear Strategy, Journal of Strategic Studies, Vol. 38, Nos. 1 2, August 2012, pp. 38 76. Mack, C. A., Fifty Years of Moore s Law, IEEE Transaction on Semiconductor Manufacturing, Vol. 24, No. 2, May 2011, pp. 202 207. As of August 15, 2017: https://doi.org/10.1109/tsm.2010.2096437 Mahan, A. T., Armaments and Arbitration: Or, The Place of Force in the International Relations of States, New York: Harper & Brothers, 1912. Minsky, M., Semantic Information Processing, Cambridge, Mass.: MIT Press, 1968. Newell, A., J. C. Shaw, and H. A. Simon, Report on a General Problem-Solving Program, Santa Monica, Calif., RAND Corporation, Report P-1584, revised February 9, 1959. 24
Oznobishev, S. K., V. Ia. Potapov, and V. V. Skokov, Kak gotovilisia asymmetrichnyi otvet na Strategicheskuiu oboronnyiu initsiativu R.Reigana. Velikhov, Kokoshin i drugie [How the Asymmetric Response to R. Reagan s Strategic Defense Initiative Was Prepared], Moscow: Legand, 2008. Papernot, N., P. McDaniel, and I. Goodfellow, Transferability in Machine Learning: from Phenomena to Black-Box Attacks Using Adversarial Samples, arxiv, May 24, 2016. As of August 15, 2017: https://arxiv.org/abs/1605.07277 Putin, V., Byt sil nymi: garantii natsional noi bezopasnosti dlia Rossii [ Being Strong Is the Guarantee of National Security for Russia ], Rossiiskaia gazeta, February 20, 2012. As of December 5, 2017: https://rg.ru/2012/02/20/putin-armiya.html/ Roland, A., and P. Shiman, Strategic Computing: DARPA and the Quest for Machine Intelligence, 1983 1993, Cambridge, Mass.: MIT Press, 2002. Rossiiskii Proekt Status-6 Meniaet Sootnosheniia Iadernykh Sil v Mire [ The Russian Status-6 Project Is Changing the World s Nuclear Balance of Forces ], Russkaia Politika, November 14, 2016. As of December 4, 2016: http://ruspolitica.ru/post/ rossiyskiy-proekt-status-6-menyaet- sootnoshenie- yadernyih-sil-v-mire/ Schelling, T., Arms and Influence, New Haven, Conn.: Yale University Press, 1966. Shannon, C. E. Programming a Computer for Playing Chess, Philosophical Magazine, Vol. 41, No. 7, 1950, pp. 256 275. Slocombe, W., The Countervailing Strategy, International Security, Vol. 5, No. 4, Spring 1981, pp. 18 27. Sutyagin, I., Russia s Underwater Doomsday Drone : Science Fiction, but Real Danger, Bulletin of the Atomic Scientists, Vol. 72, No. 4, June 2016, pp. 243 246. Timoshenko, M., Mertvaia ruka na strazhe perimetra Rossii [ The Dead Hand Guarding Russia s Periphery ], Telekanal Zvezda, February 18, 2015. As of August 15, 2017: http://tvzvezda.ru/news/krasnaya_zvezda/content/201502181414-gskc.htm Tuchkov, V., Status-6: Oruzhie Vosmezdia, Vognavshee Pentagon v Stupor [Status-6: The Retaliatory Weapon That Drove the Pentagon into a Stupor], Svobodnaia Pressa, December 11, 2016. As of December 4, 2016: http://svpressa.ru/war21/article/162378/ U.S. Department of Defense, Nuclear Posture Review Report, Washington, D.C., April 2010. Valagin, A., Garantirovannoe vozmezdie: Kak rabotaet rossiiskaia sistema Perimetr [ Assured Retaliation: How the Russian Perimetr System Works ], Rossiiskaia gazeta, January 22, 2014. As of August 15, 2017: https://rg.ru/2014/01/22/perimetr-site.html Woyke, E., and Y. Kim, Starcraft Pros Are Ready to Battle AI, MIT Technology Review, May 19, 2017. As of August 15, 2017: https://www.technologyreview.com/s/607888/starcraft-pros-are-ready-to-battle-ai/ Simon, H. A., and A. Newell, Heuristic Problem Solving: The Next Advance in Operations Research, Operations Research, Vol. 6, No. 1, January February 1958, pp. 1 10. 25
이전망에관하여 RAND Andrew Parasiliti, RAND Ventures. Angela O Mahoney Bill Welser. Sonni Efron, Doug Irving, Greg Baumann, Hosay Yaqub., (Chatham House Rule). 안보 2040,,, 2040 RAND Ventures. RAND. 지구위험및안보를위한 RAND 센터 (CGRS) RAND Corporation. RAND,,, RAND.,,. RAND www.rand.org/ international/cgrs. RAND Ventures RAND. RAND. RAND Ventures.. RAND,,. RAND. 저자소개 Edward Geist RAND. (CISAC) MacArthur Nuclear Security RAND Stanton Nuclear Security Edward 2013 5. Andrew J. Lohn RAND Corporation.,. Lohn (UCSC). 26