스플렁크개요 회사 제품 고객 글로벌 HQs: 샌프랜시스코 (AMER) 런던 (EMEA) 홍콩 (APAC) 직원수 1,800+ 명 연매출약 8,000 억원 (YoY +49%) 나스닥상장 : SPLK 무료버전에서시작, 대규모분산확장 스플렁크제품 : Splunk Enterprise Splunk Cloud Hunk Splunk Light Splunk MINT Premium Apps 고객사 ( 전체 ): 11,000+ 고객사 ( 한국 ): 350+ 국가기준 : 100 개국 + 중소기업, 대기업그룹계열사 포춘 100 대기업 : 80+ 최대라이센스 : 1.3+ PB/ 일
Key Priorities in Manufacturing Expense control Cybersecurity Globalization Product innovation Regulatory requirements Internet of Things Operational efficiency Quality and safety Copyright 2015 Splunk Inc. Massive Volumes of Machine Data
머신에서 발생하는 빅데이터 Volume Velocity Variety Variability GPS, RFID, 웹 서버, 하이퍼바이저, 이메일, 메세징, 클릭스트림, 모바일, 전화, IVR, 데이터베이스, 서버, 보안기기, 데스크탑, 센서, 컴퓨터 통신, 스토리지,
SCADA Tag Data CMMS Observation Based Safety System Industrial Data Is Machine Data 05/27/2014T10:24:17GMT applicationid="safetyobs" eventtype="safety" assetid="cv1002384-1045" employeeid="114635" jobsite="plec-2014-gc" observationid="184568-451124-256" observation="control Valve handle extracted to manual position. No lockout/tagout or other tag visible. Process is running." observationcriticality="5" imageid="plec-2014-gc-184568-451124-256" imageuri="https://mybucket.s3.amazonaws.com/plec-2014-gc-184568-451124-256.png" 1543541, workorder, bsic, 78544, pipefitting, CV1002384, "install manual bleed bypass", 04/13/2014, 05/21/2014, 25663, complete 05/22/2014 03:17:31 Tag="CV1002384.ValvePos" Value= 50" Quality= Good 05/22/2014 03:17:46 Tag="CV1002384.ValveCmd" Value= 100" Quality= Good 05/22/2014 03:19:22 Tag="CV1002384.ValveCmd" Value= 100" Quality= Good 05/22/2014 03:19:27 Tag="CV1002384.ValvePos" Value= 50" Quality= Bad
SCADA Tag Data CMMS Observation Based Safety System Connect the Dots Make New Discoveries 05/27/2014T10:24:17GMT applicationid="safetyobs" eventtype="safety" assetid="cv1002384-1045" employeeid="114635" jobsite="plec-2014-gc" observationid="184568-451124-256" observation="control Valve handle extracted to manual position. No lockout/tagout or other tag visible. Process is running." observationcriticality="5" imageid="plec-2014-gc-184568-451124-256" imageuri="https://mybucket.s3.amazonaws.com/plec-2014-gc-184568-451124-256.png" 1543541, workorder, bsic, 78544, pipefitting, CV1002384, "install manual bleed bypass", 04/13/2014, 05/21/2014, 25663, complete MTBF Technician Asset ID Asset ID Eval 05/22/2014 03:17:31 Tag="CV1002384.ValvePos" Value= 50" Quality= Good 05/22/2014 03:17:46 Tag="CV1002384.ValveCmd" Value= 100" Quality= Good 05/22/2014 03:19:22 Tag="CV1002384.ValveCmd" Value= 100" Quality= Good Alert 05/22/2014 03:19:27 Tag="CV1002384.ValvePos" Value= 50" Quality= Bad Asset ID Completed
산업용데이터분석의어려움 Getting Insights Is Not Easy Legacy systems limit high-velocity data collection Analytics constrained by inflexible tools Data silos prevent insights across the organization Lack of platform capabilities restricts customization 7
스플렁크란? = 머신데이터플랫폼 머신데이터를아무런제약없이수집 > 저장 > 분석 > 시각화할수있는실시간분산플랫폼 머신데이터 (Machine Data) 제약없음 (No Limits) 엔드투엔드 (End-to-End) 실시간및분산 (Real-time) 플랫폼 (Platform) 서버 /NW 로그 각종설비데이터 애플리케이션로그 기타모든텍스트형태의데이터 비정형 / 정형데이터 데이터포맷무관 데이터용량무관 데이터속도무관 제약없이수용 별도의외부솔루션불필요 복잡한코딩및 SI 개발이필요없음 데이터의생성부터가치획득까지모두 모든데이터실시간처리, 즉시결과확인 분산저장, 분산검색 성능및용량의선형적확장 커스터마이징용이 외부시스템과손쉬운연동 앱을통한기능확장 개발프레임워크
Early vs. Late Structure Binding Early Structure Binding 전통적인분석방법 SELECT customers.* FROM customers WHERE customers.customer_id NOT IN(SELECT customer_id FROM orders WHERE year(orders.order_date) = 2004) Structure Schema created at design time Queries understood at design time for maximum performance Data Homogeneous must fit into tables or be converted to fit into tables Must exactly match constraints
Late Structure Binding: Schema On The Fly 데이터 정규화 필요 없는 실시간 분석 Auto Detected Fields and Values Structure Data Schema-less Created at search time Queries/searches can be ad-hoc Heterogeneous can come from any textual source Constantly changing No conversion required, no constraints
빠른데이터인사이트확보 Early Structure Binding Late Structure Binding Decide question to ask Design the schema Normalize data + write DB insertion code Create SQL & feed into analytics tool Days Weeks Months Destructive Write Semantic Events Collect Create Searches, Reports & Graphs Minutes Non-Destructive
정형데이터로산업용센서데이터연관분석 ICS Tag Data Asset ID 9/8/15 4:41:48.055 PM 2015-09-08 23:41:48.055 +0000 Tag="Windfarm_10.Turbine_10.Wind_Direction" Value="132.959152" AssetID= K23441gF4224 Quality="good" demo=windfarm host = 127.0.0.1 source = tcp:9997 sourcetype = opc 9/8/15 4:41:48.055 PM 2015-09-08 23:41:48.055 +0000 Tag="Windfarm_10.Turbine_10.Temperature" Value="19.3928394" Quality="good" demo=windfarm host = 10.7.102.1 source = tcp:9997 sourcetype = opc 9/8/15 4:41:48.055 PM 2015-09-08 23:41:48.055 +0000 Tag="Windfarm_10.Turbine_10.Stator_Oil_Temperature" Value="85.4567337" Quality="good" demo=windfarm host = 127.0.0.1 source = tcp:9997 sourcetype = opc Tag Value Tag Quality Host 9/8/15 Tag Asset ID Technician Date Serviced Part Number Lot Number 50446 9/7/15 1224-56-A B00747 Workorder, Asset Databases Asset ID Location Site 7 Location Latitude Longitude Site ID Address Line 1 Site 7 39.11515 84.45651 A345 409 Park St.
확장성있는분산저장 / 처리구조
다양한 App 생태계를사용한빠른구축 스플렁크고객은다양한장비나유즈케이스에대한기설정된앱생태계를활용하여빠르게쉽게요구사항을구현할수있습니다. http://apps.splunk.com 1000+ 이상의다양한앱생태계구축 Weather BigFix Sendmail PDF Report Server F5 Radio Sta ons WebSphere XenDesktop NetScaler Mul cast MS Exchange FISMA Ruby on Rails Google Maps Whois lookup PCI Compliance Puppet Conf. Mgt Python Mail NetFlow Audible Alerts Stock Quote FISMA Monitoring Twi er Windows Nagios Unix and Linux Sourcefire Splunk Monitoring SNORT FireEye Malware POST/GET Rqsts Citrix NetScaler Security Javamail BlueCoat ProxySG Solera DeepSee IMAP YouTube Encrypt/Decrypt Enterprise Security AS/400 - iseries Transac on Profiling Security SCOM TCP/UDP Sending IronPort IronPort WSA WSA RSS Input JMS receiver Geo Loca on VMware Fin. Inf. exchange Splunk Mobile
파트너에코시스템 SDKs Advanced Analytics and ML IoT and ICS Security UI Custom User Interfaces Ingest and Platforms Services and Delivery 16
스플렁크포트폴리오 스플렁크프리미엄솔루션 강력한앱 / 애드온생태계 운영인텔리전스 (OI) 를위한머신데이터플랫폼 포워더 Syslog/TCP 모바일 IoT 단말기 네트워크패킷데이터 하둡 관계형 DB 메인프레임데이터
100여개국 의 11,000이상의 고객 보유 Fortune 100 회사중 80개 회사가 스플렁크를 사용 Cloud and Online Services Education Energy and Utilities Financial Services and Insurance Government Healthcare Manufacturing Media Retail Technology Telecommunications Travel and Leisure
기술과사업전반에걸친가치전달 애플리케이션딜리버리 IT 운영 보안, 규정준수및사기방지 비즈니스분석 업종데이터사물인터넷 Application Delivery IT Operations Security, Compliance and Fraud Business Analytics Industrial Data and Internet of Things 개발자플랫폼 (REST API, SDKs)
국내사례 - 통합관제대시보드 K 사 S 사 H 사
Measuring Results in Real Time Printer Manufacturer Splunk Use Cases Collect final printer configuration device data Monitor testing and configuration of printing devices Gain insights into product performance and production 2015 Splunk Inc. All rights reserved.
국내사례 : A 반도체제조운영공정실시간관제 / 경보시스템구축 장애대응속도향상으로장비가동율증가및반도체수율향상 물류로그 Work/Rule 로그 Appl. 서버로그 랑데뷰로그 300 개이상의프로세스패턴정의 / 감사 비정상무한루프 스탭별딜레이시간 비정상순차스텝 추가발견비정상프로세스감시 정상 비정상 장애대응속도향상 생산성향상 실시간통합품질공정관리로의확산 공정프로세스들의정상 / 비정상구분화 적용대상확대중 ( 라인내서버 전서버및 PC) 기존에파악하지못한비정상프로세스들의발견 머신러닝기반의상관분석의자동화해당프로세스에대한감지룰생성 장애선대응체제구축 단위작업별품질측정모니터링 장애탐지시간감소 (10~30여분 1~2분 ) 실시간통합품질공정관리구현
A 전자사례 : 실시간제조공정관제확대 S 사는 MOS 실시간공정모니터링의성공을바탕으로실시간통합품질공정관리를목표로스플렁크의활용도를확대하고있음 랑데뷰로그각종설정파일변경물류로그 Rule 로그 Work 로그 EES 어플리케이션로그 YMS 어플리케이션로그공정 PC 성능로그 반도체전공정의실시간관제및알람 각공정세부레시피별분석 장애대응시간감소및가동률증가 장애선대응체제구축 장애에대한인과관계분석장애에영향을미치는설정파일변경, 센서데이터연관성분석 장애재발방지및모니터링체계개선 변수별품질영향도분석 (TO-BE) 각종센서측정값들과결과품질의상관분석
기존기술대비차별점 원시데이터수집을위한어댑터개발이나정제를위한별도의추출작업없이원시데이터에서즉각적인분석 손쉬운데이터수집및무한한확장성, 검색기반의손쉬운분석환경을이용하여효과적으로개선
A 전자 : Why Splunk? S 사구축 PoC 결과경쟁사대비스플렁크는구현편의성, 가격, 구축후신규요구사항대응및유지보수등에서우월함이입증 PoC 시나리오운영시스템의예비로그서버에서 daily 300GB 이상발생하는랑데뷰로그를실시간수집, 필드추출. 각공정레시피단계별소요시간분석대시보드및알람구성. PoC 평가기준 - 수집 / 저장 / 쿼리성능 - 데이터정합성 - 대용량데이터관리기능요건충족 - 노드장애시클러스터안정성 - 분석쿼리및대시보드구현유연성 - 요구사항별예상소요 M/M 시스템구성 POC 결과 본프로젝트예상 M/M 본프로젝트예상기간 /TCO 경쟁사 데이터수집시스템 ( 개발 ) CEP(Complex Event Processing) 데이터베이스 / 하둡 시각화툴 주어진시간내에전체시나리오구현불가판단 컨설턴트 6 명 개발자 12 명 12 달 5~10X 스플렁크 스플렁크 ( 단일플랫폼 ) 모든요건구현완료 컨설턴트 2명 개발자 2명 3 달 1X
SKT 사례 : 스플렁크를이용한통합품질관리시스템전사 IT 인프라에대한통합실시간모니터링플랫폼구축으로 IT 자산품질고도화
Data-Driven Refreshment Aggregate machine data from freestyle machines Vending machine performance and diagnostics Insights into customer interactions and decisions Reduced Downtime and Increased Consumer Satisfaction
28
Medical Devices Driving Better Patient Insights Device Manufactured Tracking Medical Device Supply Chain to Drive Critical Insights Shipped to Physician Prescribed to patient Returned to irhythm Patient Behavior Prescription Patterns Supply Chain Analytics
Operational Analytics for Connected Medical Devices Comply with Meaningful Use provisions of HITECH Act 5000+ devices across 165 hospitals 19,000 patients monitored daily Enabled change in nurse behavior: 27,000 nurses using the devices $3 million in cost savings for new batteries
Understanding Customer Behavior Content browsed, purchased and watched. All tracked by time and MAC address + Customer profile and MAC address / device assignments Customer behavior analytics
Nexon 게임사용자분석 실시간사용자분석 컨텐츠이용현황 실시간재화 / 거래분석 비정상거래탐지 어뷰징예측패턴탐지
요약
머신데이터를운영인텔리전스로전환 실시간 Complex Event Processing Collect, store and query data in real time 1 운영인텔리전스플랫폼 Store and summarize data in native distributed computing environment 2 Analysis /Integrations Analyze, visualize and extend data to services 3
Use Cases for Manufacturing OPERATIONAL EFFICIENCY TROUBLESHOOTING & PREVENTIVE MAINTENANCE SECURITY, COMPLIANCE & SAFETY BUSINESS ANALYTICS Heterogeneous Data: Sensors, Devices, SCADA/ICS/DCS Data, Application Logs, Wire Data, RDBMS-Backed Safety, Execution, Maintenance and Management Systems
다양한부서에서활용할수있는빅데이터플렛폼
하둡연계성 ( 고도화안 ) 1) 대량데이터이동없는분석 : 스플렁크인덱스데이터를 HDFS에저장하고데이터셋을하둡의분석툴로직접연계. 2) 하둡의데이터분석을스플렁크의검색문과 UI 동일하게사용 : Map/Reduce 작업을사용한 raw data 정제작업필요없이손쉽고빠르게 하둡에 staging된 Raw Data에대한직접분석가능 3) Closed Loop Analysis: IBM BigSQL의분석결과를 Splunk에서참조하여데이터의미보강 실시간 Splunk 데이터 + 하둡대용량데이터통합검색 하둡데이터에대한가장빠르고손쉬운분석 + 시각화 배치결과를스플렁크에서사용하여데이터의미보강 BI Tool WARM Hadoop Clusters Splunk Archive Reader for Hadoop 과거성 WARM,COLD 버킷 HDFS 아카이빙으로 TCO 감축자동저장 COLD FROZEN 실시간성데이터처리 ( 보안탐지, 장애모니터링등 ) 과거성데이터 IBM BigSQL 스플렁크데이터포맷개방으로하둡분석툴들과쉽게연계 대규모배치성빅데이터처리 ( 마케팅분석, 사용자분석등 ) 39
모두에게머신데이터에대한접근성과사용성그리고가치를제공 40
감사합니다. Contact : 한국밸런스김형덕영업대표 Mobile : 010-7138-8889 Email : hdkim@valence.co.kr