Microsoft PowerPoint - Sysplex.ppt

Size: px

Start display at page:

Download "Microsoft PowerPoint - Sysplex.ppt"

은비 목
6 years ago
Views:

1 The Best Reliable Partner for High Availability * 2008 상반기효과적인시스템관리를위한기술세미나 - COEX, Seoul, Korea P-Sysplex performance Updates 고태호과장 (koth@kr.ibm.com) Advisory IT Specialist MTS, IBM Global Technology Services Korea April 8, IBM Corporation

2 Agenda 1. Performance on Sysplex components 1) Coupling Facility CF Service time and Delays System-Managed Duplexing CF Structure performance CFCC Level 15 Updates 2) XCF 2. Performance on Sysplex configurations 2/ 51

3 1. Performance on Sysplex components 1) Coupling Facility CF Service time and Delays System-Managed Duplexing CF Structure performance CFCC Level 15 Updates 2) XCF 2. Performance on Sysplex configurations 3/ 51

4 성능측면에서의 Sysplex 구성요소 아키텍쳐 Structures (Lock, List, Cache) Commands Coupling Technology CF 하드웨어 Coupling Facility CPs (Shared,Dedicated) Links High speed links Sender 와 Receiver link 소프트웨어 CFCC z/os와서브시스템 XCF, XES 4/ 51

5 Tuner s View of the Coupling Facility 3가지형태의 structure LIST (and Serialized List) Lock Cache Each exploiter has a different implementation Beware of general concepts like good or bad Use the performance data provided by the exploiter Make a general review of workloads and rates Are the CF accesses equivalent? Are the service times equivalent? Are there any indicators of delay of type links or subchannels? What are the CPU indicators? 5/ 51

6 Elements of CF model Link/Path : CEC와 CF를연결하는channel Link buffers : link내에존재하는 buffer Compatibility mode : link당 2개의 buffer Peer mode : link당 7개의 buffer Software subchannel z/os에서 CF request의전송시사용되는 buffer Link buffer와 1:1 관계 6/ 51

7 CF Request type Synchronous immediate Lock type access Delay time과 Service time이 CPU busy time에포함됨 Synchronous non-immediate Cache type access (transfers of data < 4k) Service time만 CPU busy time에포함됨 (if not converted) Async request로변환될수있음 Asynchronous Cache type access (transfers of data > 4k) 주로 IOP/SAP에의해처리됨 Subchannel busy일경우에는requeued, path busy일경우에는redriven 7/ 51

8 CF Request flow Sync/Async Sync Request APPL SQL DB2 IRLM XES H/W CF Processor Sysplex cost CPU busy Active Loop CFCC CF s CPU cost Async Request OS s CPU time APPL SQL DB2 IRLM XES H/W CF Processor CFCC OS s CPU time Elapse time In SQL ops. CPU Release HSA CF s CPU cost 8/ 51

9 CF 측정요소 COUPLING FACILITY NAME = CF SUBCHANNEL ACTIVITY # REQ REQUESTS DELAYED REQUESTS SYSTEM TOTAL -- CF LINKS -- PTH # -SERVICE TIME(MIC)- # % OF AVG TIME(MIC) NAME AVG/SEC TYPE GEN USE BUSY REQ AVG STD_DEV REQ REQ /DEL STD_DEV /ALL b XXX 3914K CBP SYNC 2903K LIST/CACHE 69K a SUBCH ASYNC LOCK CHANGED INCLUDED IN ASYNC TOTAL 71K 1.8 UNSUCC c d Request delay time : Subchannel Queuing time or Spinning time on Subchannel Service time : Link Queuing time + Link transfer time + CF CP queuing time + CF CP processing time + Time for MVS to recognize that the command has completed 9/ 51

10 CF Request 지연요소 Sync immediate Subchannel busy Sync immediate Path busy Sync non-immediate Subchannel busy Sync non-immediate Path busy Async Subchannel busy Async Path busy SW Subchannel Link Buffer CF CF CF CF CF CF Software subchannel busy 1 Sync immediate : spin/wait (HOST CPU cost 증가 ) 3 Sync non-immediate : Sync -> Async 5 Async : deferred (requeue) Link buffer busy ( 두개이상의 LAPR 가 CF link 를공유하는경우발생 ) Sync immediate : spin/wait Sync non-immediate : Sync -> Async Async : redriven (#of Req 증가 ) 10 / 51

11 PATH Busy 상태의처리 z/os V1.2 이전에는, CF 로연결되는 path 가 busy 상태일때는, sync req 는즉시 retry 되었고, async req 는 redriven 되었음 z/os V1.2 이후에는, 모든 req 들은 path busy 상태에서즉시 retry 됨 PATH BUSY retry 시간은 CF service time 에포함 PATH BUSY 는전체 REQ 의 10% 이하를유지하는것이바람직 Subchannel Tuning (OW54796) Path busy 가빈번히발생하여일정한 Threshold 값을넘기게되면, XES 가일시적으로 subchannel 을 offline 시킴 Path busy 상황을 Subchannel busy 상황으로유도 Path busy상태를 10~20% 이하로낮추려는시도 제한된 link 환경에서 link 의효과적인사용을극대화시키기위함 11 / 51

12 Heuristic Sync/Async 변환알고리즘 Long running SYNC CF request 는 sender 측의 CPU 를더많이사용함 z/os V1.2 이전에는 XES 가 LIST/CACHE SYNC request 를정해진 rule 에따라 Async 로변환하였음 Lock type 의 sync request 는예외 Request 형태 : long-running type 의 CF commands Sender 와 receiver 의프로세서형태 : Low-weighted CF partition with shared CPUs Data 의양 : 4KB 이상의 data 의전송시 z/os V1.2 이상에서는 Heuristic conversion 알고리즘적용 BEST 는아니지만 NICE or GOOD 을찾아가는알고리즘 Sync 처리의 cost 가 HOST CPU capacity 측면에큰영향을미치는것을방지하기위해 sync- >async ( 일정한 threshold 값을넘기는경우 ) 모든 CF operation 은 sync->async 의대상 구성에따라 threshold 값은달라짐 12 / 51

13 Heuristic 변환알고리즘의효과 Request rate 이많아지면, heuristic 알고리즘이개입 Heuristic 알고리즘이개입되면, response time 은증가하고 CPU cost 는일정수준으로유지됨 13 / 51

14 CF Service Time Service time 에영향을미치는요소들 Hardware speed of sender Hardware implementation of sender CFCC level Number of CF link adapter Type of link adapters Speed of the links Distance of links Request type and content (sync/async) Distribution of the CF request arrivals CF speed and number of CPs CF CPs dedicated vs. shared Availability of IOP/SAPs for async request What is the best possible response to expect Costs R64 to C to 1xx 3xx to 3xx 109 to 2xx Sync Async 가정 1) 가장빠른 CF link 사용시 2) HOST 와 CF 사이의거리를 0 으로가정 3) Well-tuned Sysplex 14 / 51

15 1. Performance on Sysplex components 1) Coupling Facility CF Service time and Delays System-Managed Duplexing CF Structure performance CFCC Level 15 Updates 2) XCF 2. Performance on Sysplex configurations 15 / 51

16 System-managed CF structure Duplexing CF exploiter에서받은 update request는각각의cf에대한두개의request로분기됨 CF to CF link상에서 signal 교환을통하여 CF의 sync상태를유지함 각각의 CF의 response는하나의response로 merge되어 exploiter에전달됨 Sync/Async type에관계없이모든 request에해당 16 / 51

17 Duplex Request 의 Response time 17 / 51

RMF Duplexed DB2 Lock PR WT : The amount of time that the system was holding one subchannel while waiting to get the other subchannel, to launch the duplexed

18 RMF Duplexed DB2 Lock PR WT : The amount of time that the system was holding one subchannel while waiting to get the other subchannel, to launch the duplexed operation. PR CMP : One of the two duplexed operations has completed, but the completed subchannel remains unavailable for use until the other operation completes 18 / 51

19 Costs of Duplexing z/os CPU 사용율 두개의request/response를일치시키는데드는비용 Additional pathlength is required to overlap the sending of the requests Sync request인경우, 두개의request가모두끝나기를기다려야함 CF CPU 사용율 Secondary structure를가지고있는cf의추가적인사용 CF-to-CF operation에사용되는시간 CF link 사용율 Secondary structure update를위한link의사용 CF-to-CF link의사용 CF-to-CF operation으로인해각 request의 response time이증가하고, response time의증가는 subchannel utilization을상승시킴 Costs Storage z/os CPU CF Links (time) CF CPU User Managed (DB2 GBPs) 2x 2x 2x 2x System Managed (LOCK) 2x 4x 5x 8x System Managed (LIST) 2x 3x 4x 6x 19 / 51

20 1. Performance on Sysplex components 1) Coupling Facility CF Service time and Delays System-Managed Duplexing CF Structure performance CFCC Level 15 Updates 2) XCF 2. Performance on Sysplex configurations 20 / 51

21 Coupling Facility Structures - Cache Cache structures Read/write access가가능한고속의 buffer Cached data의 coherency를보장하기위해 buffer invalidation mechanism 사용 형태 Directory only : 누가어떤 data를사용하고있는가에대해서만tracking, CF내에 data는없음 (IMS OSAM, IMS VSAM) Store-through : data가 CF와 DASD에동시에기록됨 (IMS OSAM, VSAM/RLS) Store-in : 변경된 data는 CF에쓰여지고, DASD에는일정시간후에쓰여짐 (IMS Fast Path, DB2) CPC 1 Cache Structure CONNECTION A Local Cache Vector Directory ENTRY 1 Data Entry 1 ADJ Data 1 CPC 2 CONNECTION B ENTRY m Data Entry m ADJ Data m Local Cache Vector ENTRY n Data Entry n ADJ Data n 21 / 51

22 Coupling Facility Structures - Cache 22 / 51

23 Coupling Facility Structures - List List Structures Shared work queue, shared control information LIFO/FIFO, serialized/non-serialized XCF, System logger, WebSphere MQ, DB2 SCA, JES2 Checkpoint CPC 1 CONNECTION A CONNECTION B CPC 2 CONNECTION C Serialized List Structure Lock Table m List 0 Header List 1 Header List n Header List Entry Controls Data Entry ADJ Data List Entry Controls Data Entry ADJ Data List Entry Controls ADJ Data 23 / 51

24 Coupling Facility Structures - List 24 / 51

25 Coupling Facility Structures - Lock Lock Structures 공유된 data의 serialized access를보장하기위해 shared/exclusive locking 기능제공 GRS, data sharing lock manager (IRLM, VSAM/RLS) Lock contention발생하는경우, XES에의한lock negotiation Lock table과 Record data entry로구성 CPC 1 CPC 2 Lock Structure Lock Entry x nn Excl nn Connector token Shr yyyyyyyy Resource TOM.PS1.729 TIM.PS FRANK.PS HARRY.PS dbid.psid.page# 25 / 51

26 Coupling Facility Structures - Lock 26 / 51

27 1. Performance on Sysplex components 1) Coupling Facility CF Service time and Delays System-Managed Duplexing CF Structure performance CFCC Level 15 Updates 2) XCF 2. Performance on Sysplex configurations 27 / 51

28 CFCC Level 15 Updates D CF output 과 RMF reports 에 CF 구성에관한보다자세한정보제공 RMF pp reports와 Monitor III에 structure별 CF processor 사용율제공 Concurrent CF task의수가48에서 112개로증가 Structure size의증가 System-Managed Duplexing의성능향상 CF-to-CF의교환signal 감소 28 / 51

29 CFCC Level 15 Updates 29 / 51

30 CFCC Level 15 Updates 30 / 51

31 CFCC Level 15 Updates 31 / 51

32 CFCC Level 15 Updates 32 / 51

33 CFCC Level 15 Updates Concurrent CF task 의수가 48 개에서 114 개로증가로인해 structure size 도증가하게됨 CF 내의각 structure 의일정부분은 control information 을저장하는데할당됨 더많은 task 의수는더많은 control information 을필요로함 각 structure 별로고정된양만큼의 structure size 가증가하게됨 14MB 10MB 1000 Elements/ 500 Entries 10MB 6MB 1000 Elements/ 500 Entries 6MB 1000 Elements/ 500 Entries 2MB Control Information Control Information Control Information CF level 14 CF level 15 CF level / 51

34 CFCC Level 15 Updates 34 / 51

35 CFCC Level 15 Updates 35 / 51

36 1. Performance on Sysplex components 1) Coupling Facility CF Service time and Delays System-Managed Duplexing CF Structure performance CFCC Level 15 Updates 2) XCF 2. Performance on Sysplex configurations 36 / 51

37 XCF 의성능 Transport class definition Buffer definitions Signaling paths 37 / 51

38 XCF - Overview COUPLE SYSPLEX(&SYSPLEX.) PCOUPLE(SYS1.XCFCDS.PRI) ACOUPLE(SYS1.XCFCDS.ALT) INTERVAL(85) CLEANUP(15) MAXMSG(2000) RETRY(10) CLASSDEF CLASS(DEFAULTS) CLASSLEN(956) GROUP(UNDESIG) CLASSDEF CLASS(SMALL) CLASSLEN(4028) GROUP(UNDESIG) CASSDEF CLASS(LARGE) CLASSLEN(16316) GROUP(UNDESIG) PATHOUT STRNAME(IXCSTR1) CLASS(DEFAULT) PATHOUT STRNAME(IXCSTR2) CLASS(SMALL) PATHOUT STRNAME(IXCSTR3) CLASS(LARGE) PATHOUT STRNAME(IXCSTR4) CLASS(DEFAULT) PATHIN STRNAME(IXCSTR1,IXCSTR2,IXCSTR3,IXCSTR4) 38 / 51

39 XCF Message Buffers Message Buffer 의크기 XCF는고정된 internal buffer크기를가지고있음 : 1K, 4K, 8K,,64K 실제 message 크기에가장맞는transport class를선택 Transport class에정의된것들보다큰 message는가장큰message buffer를 expand시킨후에전달 Message buffer를 expand/contract 시도를위해서는변경을위한 XCF internal signal이만들어져서교환되어야함으로 overhead가발생함 => 이를줄이기위해서 GROUP(UNDESIG) 권장 D XCF,CD,CLASS=ALL command로실제각transport class를사용하고있는 message들의실제 SIZE를 display할수있음 XCF USAGE BY SYSTEM REMOTE SYSTEMS OUTBOUND FROM XXXX BUFFER ALL TO TRANSPORT BUFFER REQ % % % % PATHS REQ SYSTEM CLASS LENGTH OUT SML FIT BIG OVR UNAVAIL REJECT XXXY DEFAULT , LONG36K 36,796 1, LONG62K 62, LONG8K 8, / 51

40 XCF Message Buffers D XCF,ClassDef,CLASS=ALL 40 / 51

41 XCF Message Buffers Message Buffer Pools MAXMSG 파라미터를이용하여 message buffer pool 에할당될수있는 virtual storage 의최대크기를설정 MAXMSG 파라미터는 COUPLE definition 에, CLASSDEF, PATHIN, PATHOUT 에정의할수있음 XCF buffer 가부족한경우에는심각한성능저하현상이발생할수있음 => buffer 의부족으로 reject 되는 message 가존재하는지반드시확인해야함 MAXMSG 의 default 값은 SETXCF command 를사용하여동적으로변경가능 41 / 51

Member의수에관계없이정의가능 Application의 SRB time이증가 CTC를통한연결 Inbound/Outbound

42 XCF Signaling Paths Signaling path의종류 CF List Structure를통한연결 하나의 path가 inbound/outbound로사용가능 Mail box 방식 가장빠른 data transfer rate을가짐 모든크기의 message 전송에적합 Member의수에관계없이정의가능 Application의 SRB time이증가 CTC를통한연결 Inbound/Outbound path에대한별도의정의 Point to point 연결 작은크기의 message 전송에적합 Member의수가늘어날경우정의하기가까다로움 SAP Utilization 증가 42 / 51

43 XCF Signaling Paths Path 의수 No path 상황 XCF 는다른 transport class 를이용하여 request 를전달함 Path definition 상의문제나, physical path 의문제일가능성이큼 Path 가부족한상황 CF structure : PATHOUT link 의수가모자라면 AVG Q LENGTH 와 BUSY count 가증가함 CTC uni-directional, bi-directional 이모두가능하나성능측면을고려하면 uni-directional 을권장함 Uni-directional 구성에서는최소 4 개의 pysical CTC link 가필요함 CTC link 에서는 AVG Q LENGTH 가의미가없으며, RMF Channel activity report 를참조해야함 Channel utilization 70% 이하권장 43 / 51

44 XCF Signaling Paths Message Transfer Time Signaling path의용량이충분한지를알아볼수있는지표 2 millisecond 이하로유지 MXFER TIME의단위는 microsecond D XCF,PI,STRNM=IXCSTR1,STATUS=WORKING STRNAME REMOTE PATHIN UNUSED LAST MXFER PATHIN SYSTEM STATUS PATHS RETRY MAXMSG RECVD TIME IXCSTR1 WORKING WBA1 WORKING WBA2 WORKING WBA3 WORKING WBI1 WORKING / 51

45 XCF Tuning 목표 Transfer time을줄여라 Mean transfer time MXFER TIME in RMF Traffic을줄이기위해서는, Locking false contention을줄여라 ENQs를줄이기위해GRS star 구성 Link 사용의최적화 CTC 보다는 CF-based link사용 Transport Class buffer size Buffer가크면storage 낭비 Buffer가작으면expand를위한cost 발생 45 / 51

46 1. Performance on Sysplex components 1) Coupling Facility CF Service time and Delays System-Managed Duplexing CF Structure performance CFCC Level 15 Updates 2) XCF 2. Performance on Sysplex configurations 46 / 51

47 CF partition 구성 제 2 장 : Performance on Sysplex configurations Standalone CF Dedicated processor(s) Best for production Shared processor(s) Test 혹은 non-datasharing production Dynamic CF dispatching Competition from other partitions ICF (Internal Coupling Facility) Dedicated or shared processor(s) Optional shared processors(s) expansion Availability considerations Addressed by CF duplexing 47 / 51

48 Shared CF CP 구성 제 2 장 : Performance on Sysplex configurations PR/SM Dedicated CP Shared CP OS #2 CF #2 OS #1 프로세서 A Sync Async Async Sync Async 프로세서 B CF #1 CF #3 CF #4 Dedicated CP Shared CP PR/SM Shared CP를사용하는 CF로가는Sync req는 PR/SM에의해Async로변환된다. RMF의 CHNGED에반영되지않는다. 48 / 51

CF Coupling Link Performance 제 2 장 : Performance on Sysplex configurations Compatibility mode (=c*) Connect G6 and earlier Link 당 2 개의 buffer Peer mode zseries/z9 CPCs간의연결만가능 Link당 7개의 buffer

49 CF Coupling Link Performance 제 2 장 : Performance on Sysplex configurations Compatibility mode (=c*) Connect G6 and earlier Link 당 2 개의 buffer Peer mode zseries/z9 CPCs간의연결만가능 Link당 7개의 buffer Effective data rate 200MBs/s (HCD type=cfp) 500MBs/s (HCD type=cbp) 1500MBs/s (HCD type=cbp) xx (HCD type=icp) xx (HCD type=icp) 하나의 Link가 sender/receiver 역할 49 / 51

50 Coupling Technology vs. Host Processor Speed 제 2 장 : Performance on Sysplex configurations CF HOST z890 z9-bc z9-ec z10-ec z890 ISC 13% 15% 16% 21% ICB/IC 9% 10% 10% 13% z990 ISC 13% 14% 14% 19% ICB/IC 9% 9% 9% 13% z9-bc ISC 12% 13% 14% 19% CIB % ICB/IC 8% 9% 10% 12% z9-ec ISC 12% 13% 14% 18% CIB % ICB/IC 8% 8% 9% 11% z10-ec ISC 11% 12% 13% 17% CIB % ICB/IC 7% 7% 8% 10% * 9 CF ops/mips 기준 Data Sharing에소요되는HOST CPU측 cost (HOST Effect라고부름 ) CF의 processor 속도가 HOST processor 속도보다현저히느릴경우, HOST측에가해지는 cost 증가 z/os V1.2 이상에서는 heuristic algorithm에의해host Effect를 15% 정도이하로유지 CF ops/mips 계산방식 (sum of total req/sec of two CFs) (total MIPS of attached systems) 50 / 51

51 Q&A 감사합니다. 51 / 51

solution map_....

solution map_.... SOLUTION BROCHURE RELIABLE STORAGE SOLUTIONS ETERNUS FOR RELIABILITY AND AVAILABILITY PROTECT YOUR DATA AND SUPPORT BUSINESS FLEXIBILITY WITH FUJITSU STORAGE SOLUTIONS kr.fujitsu.com INDEX 1. Storage System