DataCenter of the Future 2010 클라우드스토리지네트워크의혁명 FCoE 박세훈과장 (sehpark@cisco.com) 시스코시스템즈코리아
목차 혁명적인데이터센터 IO 통합의시작 Fiber Channel vs. Ethernet + TCP/IP IO 통합을위한기반기술진화와그의미통합 IO와클라우드스토리지네트워크
I/O 통합이란? Processor Memory Processor Memory I/O I/O I/O I/O Subsystem Storage IPC LAN Storage IPC LAN IPC: Inter-Process Communication
서버의 I/O 통합 (Converged Network Adapters) NICs, HBAs, HCAs 대신에적은수의 CNAs 블레이드서버를위한제한된인터페이스수 FC HBA SAN 트래픽 FC HBA SAN 트래픽 NIC NIC Enet 트래픽 Enet 트래픽 CNA CNA 모든트래픽 10G 통합 NIC Enet 트래픽 HCA HPC 트래픽 HCA HPC 트래픽
케이블과 I/O 의통합
Ethernet 과 Fiber Channel 의특성 안정성있는파이버채널 유동적인이더넷
Buffer to Buffer 크레딧 Fibre Channel 흐름제어 B2B 크레딧은 FC 전송이프레임손실없는것을보증함 크레딧의개수는프레임이하나나갈때마다감소함 16 15 가용한 BB_Credits 16 15 가용한 BB_Credits FC 디스크로향하는 FC 프레임 FC 프레임전송확인 8
파이버채널포트만살면통신할수있을까요? FLOGIs/PLOGIs 와 Fabric Zoning FC Fabric Target FC FCID 10.00.01 Initiator FCID pwwn 10:00:00:00:c9:76:fd:31 FLOGI HBA F_Port N_Port E_Port FCID 11.00.01 pwwn 50:06:01:61:3c:e0:1a:f6 zone name EXAMPLE_Zone fcid 0x10.00.01 [pwwn 10:00:00:00:c9:76:fd:31] [initiator] fcid 0x11.00.01 [pwwn 50:06:01:61:3c:e0:1a:f6] [target]
FCID FLOGI PLOGI Zoning Buffer credit Name Service World Wide Name 유사하지만, 너무나도다른두세계 Ethernet + TCP/IP MAC address SPAN-TREE IP address Window Fiber Channel
Ethernet 이 Lossless 가될수있을까? 전파는? Yes, Ethernet PAUSE 프레임과함께해결가능 Ethernet Link STOP PAUSE Queue Full Switch A IEEE 802.3 Annex 31B 에서정의됨 Switch B The PAUSE operation 은특정시간동안데이터프레임의전송을제한하기위해사용됨 Ethernet PAUSE 는 Ethernet 을무손실패브릭으로변환
Ethernet 의 PAUSE 프레임과파이버채널의 B2B_Credit 비교 A R_RDY 파이버채널 B2B_Credit Threshold B Ethernet 의 PAUSE 프레임 A PAUSE Frame 프레임을전송을다시중단전송시작 B 12
Priority Flow Control in Action IEEE 802.1Qbb (PFC) Standard track in IEEE 802.1Qbb 16 3 1 12 Bits IEEE 802.1Q Tag EtherType = IEEE 802.1Q Priority CFI VLAN ID Transmit Queues Ethernet Link Receive Queues One One Two Two Three Four Three Four 여덟개의 Priorities Five Five 트래픽 Six STOP PAUSE Six 간섭제거 Seven Seven Eight Eight Switch A Switch B a.k.a. PPP (Per Priority Pause)
Bandwidth Management 링크대역폭할당 IEEE 802.1Qaz (ETS) 의예 Offered Traffic 10 GE Link Realized Traffic Utilization 3 Gbs 3 Gbs 2 Gbs (30%) HPC Traffic (30%) (20%) 3 Gbs 4 Gbs 6 Gbs (30%) LAN Traffic (40%) (50%) 3 Gbs 3 Gbs 3 Gbs (30%) Storage Traffic (30%) (30%) T1 T2 T3 HPC Traffic Priority Class High 20% Guaranteed Bandwidth LAN Traffic Priority Class Medium 50% Guaranteed Bandwidth Storage Traffic Priority Class Medium-High 30% Default Bandwidth T1 T2 T3
Data Center Bridging Exchange Protocol (DCBX) LLDP 기반 (Link Level Discovery Protocol) DCBX: Data Center Bridging exchange Data Center Ethernet Links DCBX Data Center Ethernet Links with Partial Enhancements DCBX Legacy Ethernet Links Legacy Ethernet Network Data Center Ethernet Cloud Hop-by-hop negotiation for: PFC, BM, CM, Link down
FCoE: FC over Ethernet FCoE 는 Ethernet 상에서 FC 스토리지 I/O 트래픽을통합 FC 트래픽은다른트래픽들과함께 Ethernet 링크에서제어됨 lossless Ethernet 패브릭이필요 Fiber Channel 관점에서는, Ethernet 이라고불리우는새로운케이블타입위에서 FC 연결을한것으로보임 Ethernet 관점에서는, 또하나의전송하기위한상위레벨프로토콜로인식 Ethernet Fibre Channel Traffic 1, 10 Gbps
운영체제상의 View 와 CNA 표준드라이버 동일한관리 운영체제의인식은 : Dual-port, 10 Gigabit Ethernet adapter Dual-port, 4 Gbps Fiber Channel HBAs CNA 10GbEE 10GbEE Link Fiber Channel Ethernet PCIe Fiber Channel Drivers Ethernet Drivers
FCoE 주소체계와포워딩 FCoE 는 : No Gateway Model 검증된 FC 를그대로사용 대규모 FC 망구축에유리 FC-MAP FC-ID 24 Bits 24 Bits (ex 02-12-34) 7.8.9 MAC Address FC-MAP (ex 02-12-34) 48 Bits FC-ID 7.8.9 FC Storage FCID 7.1.1 FC Fabric FC Domain 7 FC Fabric FC Domain 3 MAC A Ethernet Fabric FC Domain 1 MAC B Ethernet Fabric FCID 1.1.1 MAC C D_ID = FC-ID (1.1.1) S_ID = FC-ID (7.1.1) FC Frame D_ID = FC-ID (1.1.1) S_ID = FC-ID (7.1.1) FC Frame Dest. = MAC B Srce. = MAC A D_ID = FC-ID (1.1.1) S_ID = FC-ID (7.1.1) Dest. = MAC C Srce. = MAC B D_ID = FC-ID (1.1.1) S_ID = FC-ID (7.1.1) FCoE Frame
통합 IO 스위치 Nexus 5000 FCoE Ready!!! Nexus 5010 20 Fixed Ports 10G/FCoE/IEEE DCB 1 Expansion Module Slot up to 6 X 8G line rate FC ports up to 8 X 4G line rate FC ports Redundant Fans & Power Supplies (450 W) ~ 500 Gbps Aggregate Bandwidth up to 48 Gbps line rate FC BW 데이터센터스위칭 Nexus 5000 series Unified IO - FCoE Virtual Networking Simple Management Low Latency Ethernet Loop Free L2 Network Nexus 5020 40 Fixed Ports 10G/FCoE/IEEE DCB 2 Expansion Module Slots up to 12 X 8G line rate FC ports up to 16 X 4G line rate FC ports Redundant Fans & Power Supplies (450W) ~ 1 TB Aggregate Bandwidth up to 96 Gbps line rate FC BW Ethernet Ethernet + Fibre Channel Fibre Channel 8G Fibre Channel 6 Ports 10G/FCoE/IEEE DCB 4 Ports 10G/FCoE/IEEE DCB 4 Ports 1/2/4G Fibre Channel 8 Ports 1/2/4G Fibre Channel 6 Ports 2/4/8G Fibre Channel NX-OS, DCNM, CiscoWorks and Fabric Manager
통합 IO Nexus 와 UCS 시리즈 Nexus 4000 블레이드스위치 6개외부10 GE 포트 14개내부10GE 포트 4G FC 8 포트 UCS 통합컴퓨팅시스템기본 10G FCoE 지원 Nexus 7018 16 개모듈 + 2개슈퍼바이저슬롯 5개패브릭모듈최대 512 개 10G 포트최대 768개 10/100/1000 포트 Nexus 5010 1 확장슬롯최대 26개 10GE 포트최대 8개 4G FC 포트최대 6개 8G FC 포트 Nexus 5020 1 확장슬롯최대 52개 10GE 포트최대 16개 4G FC 포트최대 12개 8G FC 포트 Nexus 7010 8 개모듈 + 2개슈퍼바이저슬롯 5개패브릭모듈최대 256 개 10G 포트최대 384개 10/100/1000 포트
클라우드스토리지네트워크는 케이블이슈를해결해야함 데이터센터내시설최적화는효율적인환경구축의핵심! 1 2 3 4 케이블링의복잡도증가케이블링의비용증가냉각공조흐름의방해운영및관리적인측면의부담 서버가상화는 FCoE 가핵심경쟁력효과적인자원활용, 케이블링및전반적인구축비용의절감
클라우드스토리지네트워크는 고성능요구를해결해야함 Nexus 5000 과 2 세대 Converged Network Adapters 성능! Throughput Qlogic QLE 8152 1,200 Throughput Emulex OneConnect UCNA Throughput (MB/s) 1,200 1,000 800 600 400 200 0 2K 4K 8K 16K 32K 64K 8G FC Reads 8G FC Writes 10G FCoE Reads 10G FCoE Writes Throughput (MB/s) 1,000 800 600 400 200 0 2K 4K 8K 16K 32K 64K 8G FC Writes 8G FC Reads 10G FCoE Writes 10G FCoE Reads Block Size (KB) Block Size (KB) 대용량파일전송과고속트랜잭션워크로드에높은성능발휘! Database, Back-up, Data Warehousing, Video, Graphics, Animation
클라우드스토리지네트워크는 비용절감요구를해결해야함 [Intel Case Study Nexus 5000 활용사례 ] Standard Top Of Rack 10GE Top Of Rack Qty Cost Qty Cost Server 20 20 Quad Card 1 $ 400 0 Dual Port HBA 1 $ 1,500 0 Cable 5/6/7 6 $ 180 1 $ 30 Fiber Cable 2 $ 120 0 SFP + Copper 0 2 $ 250 GE Switch Port 6 $ 2,640 1 $ 440 FC Switch Port 2 $ 2,400 0 Dual Port CNA 0 1 $ 1,500 Nexus 5000 Port 0 2 $ 4,000 Host 당비용 $ 7,240 $ 6,220 Rack 당비용 $ 144,800 $ 124,400
통합 IO 블루프린트 = 심플한네트워크 SAN A SAN B FCoE 적용된네트워크 25
복잡한케이블, Cisco 가심플하게만들어드립니다. 27