digital_transformation_software_engineering

Size: px

Start display at page:

Download "digital_transformation_software_engineering"

민중 방
5 years ago
Views:

1 Digital Transformation and Software engineering Andrew Yongjoon Kong

Government Broad Cast Agency Adjunct Prof.

Professor for Bigdata Member of National

Kakaocorp, Cloud Part Lead Talks cloud native

Scalable Loadbalancer with VM orchestrator

2 Who am I Andrew. Yongjoon kong Cloud Technical Advisory for Government Broad Cast Agency Adjunct Prof. Ajou Univ Korea Data Base Agency Acting Professor for Bigdata Member of National Information Agency Bigdata Advisory committee Kakaocorp, Cloud Part Lead Talks cloud native platform 9RUM, kakao Scalable Loadbalancer with VM orchestrator (2017, netdev, korea) Embrace clouds (2017, openstack days, korea) Full route based network with linux (2016, netdev, Tokyo) SDN without SDN (2015, openstack, Vancouber) Supervised, Korean edition Korean Edition.

3 Before starting Open Google Search for Andrew Yongjoon Kong Open Daum Search for 공용준

4 Before starting FYI KAKAO KT SK C&C STX Engine 피스페이스 마이다스IT 이파워게이트

5 Can you remember? What is Engineering? What is definition of Like When is your heydays? Now I want to add two terms What is laborer? What is 인문학?

6 What is your DREAM? Dream! Maybe, your Dream now is getting a (Good) Job.

7 There s trend

8 There s transforming toward Digital

9 Cloud changes the elements of style:

10 Cloud changes the IT operations : Source (spring boot) source code à +test code à +monitoring code Packaging javac à maven install à maven install dockerfile:push Deploying cp jar/war à maven deploy à maven fabric8:deploy Running java cp à maven spring-boot:run à docker run -p 8080:8080 -t springio/gs-spring-boot-docker Operating ( health check. restart if failed) java à maven (is it possible?) à (in cloud) You go sleep, I ll take care

11 Transformation sample

12 After you write code, Platform will take cares IT operations. Management Plane Centralized CMDB Centralized Measuring System Centralized Deploying System your code DataCenter Contol/Data plane

13 BTW What is good job

14 Getting Jobs, Myths You submit a resume, you get job once! For me, I wrote more than 200 resumes and did 50 interviews to get a job. Getting Job is equivalent to being competent Not exactly. It s rather situational Job seeker is a sinner Job seeker vs Job provider First is only beginning more than 40 year is in front of you.

15 Getting Jobs, Practical way. Do not care about applications. stick to the fundamentals! People expect not that much from novice. Focus on basic theory and practice. Do not pretend to know everything. Saying I don t know exactly when you don t know. It ruins nothing Find the company which can help you to grow ( not to consume )

16 Getting Jobs, Practical way(2).

17 Tools Local Development Environment Version Control Development Tools Artifact Management Automated Test

18 if (kakao): IaaS (service name KRANE) Inhouse Auth. keystone Authentication KRANE Service Topology IMS DNS AEZORO BareMetal TBD VM LB DB

19 if (kakao): KRANE(The Tech.) Data Center Level Scalable Network Neutron Floating IP Compute node1 neutron-l3-agent Switch Namespace linux bridge IP: /32 IPTable DNAT connected Veth pair empty empty Gateway Host dest /32 to Route dest to eth Compute Node Router Routing Table Default GW eth1 Host Route dest /32 to dest to connected eth0 eth deactivate VIP global name space LB vm neutron-dhcp-agent neutronlinuxbridge-agent dhclient vm agent LB vm

20 if (kakao): MaaS (service name KEMI) IMS 서비스정보 KEMI Stats 메트릭데이터 ETL* 시계열 DB 전사 HBASE 메트릭정보수집 KRANE 일반시스템 KEMI Alert 로깅정보수집 임계치기반알림 키워드빈도기반알림 전사 Elastic Search 서비스 KEMI Log 로그데이터 ETL* 실시간 DB 장기 DB 전사하둡 Secure DB ETL: Extract Transform Loading

21 if (kakao): KEMI(The Tech.)

22 if (kakao): KEMI(The Tech.) Lambda Architecture RealTime KEMI Log Tailer (CLI) App Log KEMI Log Aggregator KEMI DIKE Container Log every 5m every 5~15m Batch

23 if (kakao): MaaS (The Key.) Monitoring Abstraction curl -XPOST -d '{ "start": , "end": , "queries": [{ "metric": "-", "aggregator": "max", "downsample": "15m-max", "filters": [{ "filter": "data.load.avg_5m", "groupby": true, "tagk": "metric", "type": "literal_or" }, { "filter": "dkos-marathon", "groupby": true, "tagk": "host", "type": "literal_or" }] }]' Connection with IMS(CMDB) by Events SAURON API Retrieval Standardization RBAC API limiting

if (kakao): CaaS (service name DKOS) 사내 LDAP keystone 사용자정보 KRANE 서비스정보 dns 정보 IMS 메트릭정보수집 설정정보 KEMI Stats 설정정보 메트릭데이터 ETL* 시계열DB 전사 HBASE AEZORO KEMI Alert 임계치기반알림 키워드빈도기반알림 물리머신 가상머신 데이터베이스 컨테이너

24 if (kakao): CaaS (service name DKOS) 사내 LDAP keystone 사용자정보 KRANE 서비스정보 dns 정보 IMS 메트릭정보수집 설정정보 KEMI Stats 설정정보 메트릭데이터 ETL* 시계열DB 전사 HBASE AEZORO KEMI Alert 임계치기반알림 키워드빈도기반알림 물리머신 가상머신 데이터베이스 컨테이너 로드밸런서 public cloud 일반서비스 로깅정보수집 KEMI Log 로그데이터 ETL* 실시간 DB 장기 DB Secure DB 전사 Elastic Search 전사하둡 인증, 컴퓨팅 API 메트릭로깅 API Container Orchestratio n Container Scheduling Componen t Platform 메타케이지 DKOS D2Hub 사내 Github

25 if (kakao): DKOS(The Tech.) Autoscale KEMI Alert Setup DKOS Autoscale Group Setup

26 if (kakao): DKOS(The Tech.) Autoscale Result

설정정보 메트릭데이터 ETL* 시계열 DB 전사 HBASE AEZORO KEMI Alert 임계치기반알림키워드빈도기반알림 물리머신 가상머신 컨테이너 인증, 컴퓨팅 API 데이터베이스

27 if (kakao): DKOS(The key.) DKOS is connected to everything 사내 LDAP keystone 사용자정보 KRANE 서비스정보 dns 정보 IMS 메트릭정보수집 설정정보 KEMI Stats 설정정보 메트릭데이터 ETL* 시계열 DB 전사 HBASE AEZORO KEMI Alert 임계치기반알림키워드빈도기반알림 물리머신 가상머신 컨테이너 인증, 컴퓨팅 API 데이터베이스 로드밸런서 컨테이너메트릭 컨테이너로깅 public cloud 일반서비스 로깅정보수집 KEMI Log 로그데이터 ETL* 실시간 DB 장기 DB Secure DB 전사 Elastic Search 전사하둡 컨테이너오케스트레이션 컨테이너스케줄링 DKOS 머신러닝플랫폼 알람, 오토스케일조건설정 D2Hub 메타케이지 사내 Github

28 if(kakao) Product & Service in Kakao Cloud Category Target Product Description Connected internal service Authentication Compute Project&Role based Authentication KEYSTONE 프로젝트 / 롤기반사용자인증및권한관리내부 LDAP 와연동 Virtual machine KRANE 가상머신제공가상볼륨, 가상머신기반 DB/LB 제공 Autoscale 기능준비중 Container DKOS 컨테이너 orchestrator(marathon) 제공. 컨테이너용 Loadbalancer 제공컨테이너기반 Machinelearningframework 제공 (Spark, GPU) IMS, *AEZORO, *Meta-kage 연동 *Tenth 연동 KRANE, KEMI, AEZORO 와연동 Management Monitoring KEMI Stats 전사메트릭데이터 (cpu, memory, disk) & custom (application, component) 데이터수집및분석 Logging KEMI Log 전사로그데이터 (syslog, app, container) 로그수집 / 저장 / 분석 Alert KEMI Alert KEMI 데이터기반알람 ( 톡및 url 전송 ) *IMS 와연동 sauron 연계 IMS 와연동공용하둡, Hbase 연동 Developer tools Package Repository Container Registry / Builder repo.kakao.com OS 패키지, Python/Ruby 패키지제공 D2Hub 컨테이너용이미지제작및등록 Keystone 연동 DNSas a service AEZORO 가상머신 / 컨테이너기반 DNS 자동등록 / 삭제내부 DNS 연동 *AEZORO( 사내 DNS as a Service API) *IMS(Infra Management System) *Meta-kage(swift[ 오픈스택스토리지 API] 용 kage) *Ceph-Tenth: Ceph API 호환 Tenth

kakao cloud is heading for DevOps ( Self service) IT

Automated Orchestration API Declarative Scheduling

29 kakao cloud is heading for DevOps ( Self service) IT Services Service Repo. Service topology Notification setup Initial Setup Event / Alert Cloud API Platform Abstract Resource API Automated Orchestration API Declarative Scheduling Analytics Change Data Centralized Measuring API Automated Deploying API DataCenter Contol/Data plane

30 if (kakao): 9rum (The key) Integrated SSO. KRANE DKOS KEMI SSO IMS (CMDB) ERP Authorization QUOTA RBAC Keystone DB Project authentication Inhouse auth (Hello-MIS)

31 if ( kakao ) : some numbers Virtual to Physical ratio (%) Containerslaveratio overcloud (%) IndexingData (TB/Day) 80 0 Expense $MM/Yr (compared to public cloud, excluding traffic/storage) License Expense

32 Prediction Practice makes your prediction perpect!

2018_ifkakao_cloud_andrew_final

2018_ifkakao_cloud_andrew_final () 공용준 9. )( ( ) ) K P I ADII EI A FIN 7 B C F ) CLIK IK I K G M KI ) DN M 5KI )DIN C 1IK, M L )A ) MC A 5KI LLIK IK CA G KI MCI F 0 IKG MCI )A CA M ) 1 E I IK FIN 5 KM 2 7 FEL 6 F F 2I F K PCMB IK B LMK