IBM Business Intelligence Solution Seminar 2005 Choose the Right Data Integration Solution ; Best Practices on EII/EAI/ETL IBM DB2 Technical Sales BI

Similar documents
ETL_project_best_practice1.ppt

untitled

세션 3 (오이식).ppt

Model Investor MANDO Portal Site People Customer BIS Supplier C R M PLM ERP MES HRIS S C M KMS Web -Based

Service-Oriented Architecture Copyright Tmax Soft 2005

I. - II. DW ETT Best Practice

歯부장

ecorp-프로젝트제안서작성실무(양식3)

DW 개요.PDF

Portal_9iAS.ppt [읽기 전용]

슬라이드 1

CRM Fair 2004

PCServerMgmt7

Oracle Apps Day_SEM

Microsoft PowerPoint - Smart CRM v4.0_TM 소개_ pptx

Backup Exec

PowerPoint 프레젠테이션

Intra_DW_Ch4.PDF

Intro to Servlet, EJB, JSP, WS

スライド タイトルなし

02이승민선생_오라클.PDF

세션 2-2(허태경).ppt

Session3. 한국마이크로소프트(전사적 데이터 통합 컨퍼런스).ppt

歯목차45호.PDF

untitled

Analyst Briefing

untitled

untitled

김기남_ATDC2016_160620_[키노트].key

Microsoft PowerPoint - 3.공영DBM_최동욱_본부장-중소기업의_실용주의_CRM

SchoolNet튜토리얼.PDF

Copyright 2012, Oracle and/or its affiliates. All rights reserved.,.,,,,,,,,,,,,.,...,. U.S. GOVERNMENT END USERS. Oracle programs, including any oper

歯CRM개괄_허순영.PDF

리포트_03.PDF

PowerPoint 프레젠테이션

歯튜토리얼-이헌중.PDF

E-BI Day Presentation


ORANGE FOR ORACLE V4.0 INSTALLATION GUIDE (Online Upgrade) ORANGE CONFIGURATION ADMIN O

Innovation: CEO In 2006, 2 in 3 CEOs said they would have to bring about fundamental change to their business in the next 2 years to implement their s

thesis

vm-웨어-01장

당사의 명칭은 "주식회사 다우기술"로 표기하며 영문으로는 "Daou Tech Inc." 로 표기합니다. 또한, 약식으로는 "(주)다우기술"로 표기합니다. 나. 설립일자 및 존속기간 당사는 1986년 1월 9일 설립되었으며, 1997년 8월 27일 유가증권시장에 상장되

The Self-Managing Database : Automatic Health Monitoring and Alerting

PowerPoint 프레젠테이션

歯이시홍).PDF

15_3oracle

¨ìÃÊÁ¡2

슬라이드 제목 없음

<C0CCBCBCBFB52DC1A4B4EBBFF82DBCAEBBE7B3EDB9AE2D D382E687770>

untitled

Chap7.PDF

1217 WebTrafMon II

Microsoft Word - 조병호

SW¹é¼Ł-³¯°³Æ÷ÇÔÇ¥Áö2013

Oracle9i Real Application Clusters

untitled

1.장인석-ITIL 소개.ppt

Simplify your Job Automatic Storage Management DB TSC

Oracle Database 10g: Self-Managing Database DB TSC

차세대 시스템 개발과 스마트 캠퍼스 구축의 시대! 2014년 현재 대학 정보화 화두는 차세대, 스마트 캠퍼스, 개인정보보호 입니다. 대학 정보화 동향 1990년대 후반부터 2000년대 초반 붐처럼 일었던 학사행정 시스템 구축의 시기를 지나 2000년대 중 후반 부터는

Gartner Day

歯김한석.PDF

Interstage5 SOAP서비스 설정 가이드

<4D F736F F F696E74202D20BDC7BDC3B0A320B5A5C0CCC5CD20C5EBC7D520B1E2BCFA20BCD2B0B F31>

No Slide Title

PowerPoint


Cache_cny.ppt [읽기 전용]

오늘날의 기업들은 24시간 365일 멈추지 않고 돌아간다. 그리고 이러한 기업들을 위해서 업무와 관련 된 중요한 문서들은 언제 어디서라도 항상 접근하여 활용이 가능해야 한다. 끊임없이 변화하는 기업들 의 경쟁 속에서 기업내의 중요 문서의 효율적인 관리와 활용 방안은 이

이제는 쓸모없는 질문들 1. 스마트폰 열기가 과연 계속될까? 2. 언제 스마트폰이 일반 휴대폰을 앞지를까? (2010년 10%, 2012년 33% 예상) 3. 삼성의 스마트폰 OS 바다는 과연 성공할 수 있을까? 지금부터 기업들이 관심 가져야 할 질문들 1. 스마트폰은

Integ

PowerPoint 프레젠테이션

歯두산3.PDF

Title of the presentation This is the subtitle

Session XX-XX: Name

PowerPoint 프레젠테이션

J2EE & Web Services iSeminar

Agenda Stage Stage 7.5

MS-SQL SERVER 대비 기능

thesis

PowerPoint Presentation

F1-1(수정).ppt

2Q SWG Teleweb Business Plan & 1Q Recovery Plan April 2, 2003

Microsoft PowerPoint - SVPSVI for LGNSYS_ ppt

정보기술응용학회 발표

\(주\)투비소프트 사업계획서

歯sql_tuning2

IPAK 윤리강령 나는 _ 한국IT전문가협회 회원으로서 긍지와 보람을 느끼며 정보시스템 활용하 자. 나는 _동료, 단체 및 국가 나아가 인류사회에 대하여 철저한 책임 의식을 가진 다. 나는 _ 활용자에 대하여 그 편익을 증진시키는데 최선을 다한다. 나는 _ 동료에 대해

dbms_snu.PDF

untitled

歯통신41호.PDF

chapter4

<49534F C0CEC1F520BBE7C8C4BDC9BBE720C4C1BCB3C6C320B9D D20BDC3BDBAC5DB20B0EDB5B5C8AD20C1A6BEC8BFE4C3BBBCAD2E687770>

VOL /2 Technical SmartPlant Materials - Document Management SmartPlant Materials에서 기본적인 Document를 관리하고자 할 때 필요한 세팅, 파일 업로드 방법 그리고 Path Type인 Ph

Voice Portal using Oracle 9i AS Wireless

초보자를 위한 ADO 21일 완성

Joseph Hwang, IBM Rational Software

PowerPoint 프레젠테이션

AGENDA 모바일 산업의 환경변화 모바일 클라우드 서비스의 등장 모바일 클라우드 서비스 융합사례

Transcription:

Choose the Right Data Integration Solution ; Best Practices on EII/EAI/ETL IBM DB2 Technical Sales BI Team (byrhee@kr.ibm.com) 2005 IBM Corporation

Agenda I. II. ETL, EII, EAI III. ETL, EII, EAI Best Practice IV. V. 2

; Integration People Processes Information Wired / Wireless Devices application asset 24x7 access / real-time legacy packaged data source data type application / DW DM IT 40% 85% 40% 30~50%... 3

Horizontal Integration is the New Challenge Bridging the gap between business transformation and IT Customer Connections Internal Systems Supplier Networks 4

Integration Challenges repository merge format semantic layer Integration Governance Model,, data flow 5

vs. 3 Layer Business Layer Business rules Data Layer ; update Presentation Layer ; web, PDAs, etc. Application level EII (data layer level) Consolidated database level workflow action 6

Design Develop data mart Visible Costs 10% of costs Evolve Operate Change business rules Upgrade to new version Migrate from development to production Investigate bottlenecks Reuse in another project Change OS & DB Upgrade source systems Invisible Costs 90% of costs. 7

; OLAP ( / ) Executive Area Ad-hoc Query Require Specific Area / / / Recursive Summary Area Subject Area Raw Data Area L5 L4 L3, L2 L1 / 2,3 / 1 / 3 / / 1 / / 8

vs. : Current Data dynamic join / : Local performance / / (ETL) / Result Set (Replication) (near) real time 9

Agenda I. II. ETL, EII, EAI III. ETL, EII, EAI Best Practice IV. V. 10

EAI App1 App2 App3 MQIn Formatting & Mapping RouteToLabel MQOut EII ETL Data Warehouse Data Mart Data Warehouse Data Mart Data Mart 11

ETL ETL Extract( ), Transform( ), Load( ),,,,. ETL tool Extract / Transform / Load : parallel concurrent impact analysis 12

DataStage Architecture DB2 UDB Informix Oracle Microsoft SQL Server Sybase UniVerse UniData Sequential Mainframe QSAM VSAM DB2 IMS IDMS Adabas Datacom Others Any SQL Native API ODBC Hash File Sequential or FTP Change Data Capture Mainframe Data Access Windows Client DataStage Manager Designer Director Administrator Repository Server Windows NT/2000 Server ODBC Sequential or FTP Bulk Loaders Sun Solaris, HP-UX, IBM AIX, Compaq ; Parallel Extender Meta Data Exchange Native API Hash File Meta data Oracle 7 & 8 OCI Informix CLI Sybase OC UniVerse UniData Staged Data Meta data Repository Data Warehouse Load Utilities Bulk Loaders DB2 UDB Informix RedBrick Oracle BCP (Sybase & Microsoft) Sybase IQ 13

DataStage ETL DB Import - DBMS - - XML(Web) ETL GUI Drag & Drop 400 GUI,,,, 14

EII,. Marketing e-commerce Fulfillment Integrated Information Enterprise Information Integration / ; read/write, 15

WebSphere Information Integrator WS Information Integrator DB2 Catalog wrapper, server, nickname Server Nickname Remote functions Nickname Nickname Nickname Server Server Wrapper Wrapper : (Oracle, Sybase, MS SQL..) Server : Nickname : II : Global catalog User data MQT 16

WebSphere Information Integrator ; How it works WS II DB2 Nickname DB2 catalog Federated Query DB2 cost-based optimizer Optimizer query WS II. Cost-based optimizer Wrapper pushdown query Client library native client library. WS Information Integrator DB2 cost-based optimizer Local Execution Plan + Remote SQL Wrapper Client library Nickname Nickname Table 17

Federation (aka Enterprise Information Integration) SQL SQL Content Mainframe databases Mainframe files Relational databases XML Web services Packaged applications Web Other Collaboration Systems Content Repositories and Imaging Systems Workflow systems 18

EAI EAI (OS, DBMS ) (ERP, SCP, DW, CRM, Legacy ),, (Interface & Integration) Interface.,. Enterprise Application Integration 19

EAI EAI Adapter ; / Middleware ; ; DB, File ERP, CRM Middleware(MQ) ; / (Integration Broker ) ;, Integration Broker(WMQI) ; (Transformation) ; (Routing) HUB Broker Rules Format DB DB/File Information Adapter Adapter Integrator DBMS DBMS DBMS SAM FileSAM File File WBI WMQ 20

WebSphere MQ A B, (Queue) ( ) / (Queue), (Queue). (Queue) FIFO (First In First Out) FIFO (First In First Out),,, API (C/C++, Java, COBOL ) MOM (Message-Oriented Middleware) 78%, 87% / (Synchronous/Asynchronous Processing) ; A B B (Triggering) B.. (Assured Data Delivery) ; " " (Commercial Environment). 21

Agenda I. II. ETL, EII, EAI III. ETL, EII, EAI Best Practice IV. V. 22

Strength ETL vs. EII vs. EAI Strengths and Challenges ETL EII Extract, Transform, Load Middleware DW DM, Websphere MQ Websphere Information Integrator DataStage structured data structured data unstructured Batch data( ) focus real-time data read/write, metadata API real-time near real-time remote source global access event,,, GUI view workflow impact analysis metadata, metadata Data Grid EAI 23

Challenge ETL vs. EII vs. EAI Strengths and Challenges ETL EII ; SQL - key match data type mismatch 10 staging ; resource ; ~ out-of-sync DW network metadata ; metadata import/export Semantic integrity network EAI ETL tool 24

ETL vs. EII vs. EAI Data Flow Data ETL source to target Batch Job Daily - Monthly Best Medium Low /Metadata ETL Job SQL ESQL, view, DB database object catalog metadata FTP direct database connection EII Query time - Query (SQL) managed Real-time Direct database connection Transaction triggered Transaction managed (Near) Real-time Messaging Very large Medium Small ~ ~ 10 pipe SQL broker semantic transformation EAI 25

ETL vs. EII vs. EAI Event Monitoring Very Limited Versioning Workflow Control ETL Full support EII, Job flow None Job, error exception handling Limited trigger Limited support custom build EAI Best event Limited support custom build rule 26

ETL Best Practices ETL tool ETL I/O bound staging step storage I/O lookup data file ETL data mart data mart locking Key to running many concurrent processes in parallel Query, Load, Backup 27

EII Best Practices Marketinge-CommerceFulfillment Integrated Information ad-hoc WII query WII query DB2 Query Patroller remote source operation WII permanent basis WII virtual warehouse, ad-hoc remote federated query Remote data target access remote federated server remote 28

EAI Best Practices Point-to-point Hub broker workflow trace 29

EII vs. EAI vs. ETL When to use ETL When to use EII select repository connect EDW SQL When to use EAI join query Combination is normally used 30

Agenda I. II. ETL, EII, EAI III. ETL, EII, EAI Best Practice IV. V. 31

Extending the Data Warehouse Data Warehouse WS Information Integrator BI Nonrelational data Real-time data remote data(, ) DB2 ; remote data query remote data ; optimization access path target 32

POS DW Store A App WebSphere MQ DataStage Store B App WebSphere MQ Store C App WebSphere MQ ) MQStage DataStage Job. MQ DB2. 33

DataStage can leverage WS II ETL job Federation lookup join unstructured data fetch SOA external Web Services DataStage extract clean/transform lookup load DB2 II (Federation) II (Federation) source1 source2 source3 sourcea sourceb 34

WII MQ CICS/VSAM BI Portal - EDW E-Commerce Mainfram WPS WBI Server DB2 SQL via JDBC WBI JDBC Adapter DB2 SQL via JDBC WS II ODBC Wrapper MS-SQL Wrapper Oracle Wrapper SQL via ODBC WS II Classic Federation MS-SQL Oracle WBI JDBC Adapter Oracle Call Center EDW DB2 35

Agenda I. II. ETL, EII, EAI III. ETL, EII, EAI Best Practice IV. V. 36

Session Summary ; EAI App1 App2 App3 MQIn Formatting & Mapping RouteToLabel MQOut EII ETL Data Warehouse Data Mart Data Warehouse Data Mart Data Mart 37

Session Summary ; Quiz Batch Job ETL - DataStage right time EII WebSphere Information Integrator real time EAI WebSphere MQ 38