QoS Parameters to Network Performance Metrics Mapping for SLA Monitoring

QoS Parameters to Network Performance Metrics Mapping for SLA Monitoring

QoS Parameters to Network Performance Metrics Mapping for SLA Monitoring by Hyo-Jin Lee Division of Electrical and Computer Engineering (Computer Science and Engineering) POSTECH A thesis submitted to the faculty of POSTECH in partial fulfillment of the requirements for the degree of Master of Science in the Division of Electrical and Computer Engineering (Computer Science and Engineering) Pohang, Korea December 13, 2002 Approved by Major Advisor

ABSTRACT Service Level Agreement (SLA) is formal negotiated agreement between a service provider and a customer Today, the importance of SLA is increasing continuously on the telecommunication market In the SLA contract, QoS parameters that specify the quality level of service that the service provider will guarantee are included SLA monitoring, one of functionalities of Service Level Management (SLM), involves monitoring the performance status of the offered service and provide relevant information to the SLM system In order for the SLM system to verify whether the specified QoS parameters are being met, SLA monitoring system must gather performance data from the underlying network performance monitoring system and map such data to the QoS parameters In this thesis, we propose a formal mapping mechanism between QoS parameters in SLA and the network performance metrics in the network access service And we propose a general SLA monitoring system architecture that can be used to monitor service levels for various services offered by service providers Finally, we introduce POS-SLMS (POSTECH-Service Level Monitoring System) implemented in the POSTECH gigabit Ethernet backbone network

I 1 II 4 21 SLA 4 22 SLA 6 23 SLA 10 24 SLA 12 25 13 251 13 252 15 253 SNMP agent 17 III QOS 19 31 QOS 19 32 22 IV QOS 25 41 NSPISP QOS NPM 25 42 QOS NPM 30 V SLA 32 51 32 52 34 VI SLA 37 VII SLA 39 71 SLA 39 72 SLA 39 73 POS-SLMS (POSTECH SERVICE LEVEL MONITORING SYSTEM) 41 7311 POS-SLMS 41 7312 POS-SLMS 43 7313 POS-SLMS SLA 46 VIII 49 51

1 SLA 4 2 SLA 10 3 NSPISP QOS 21 4 NSPISP NPM 23 5 QOS NPM 26 6 30 7 31 8 SLA 40 9 42 10 POS-SLMS 43 11 POS-SLMS 47

1 ISP SLA8 2 WORLDCOM SLA9 3 14 4 16 5 17 6 QOS 19 7 QOS 20 8 CAIDA IPPM WORKING GROUP 22 9 QOS (1)27 10 QOS (2)28 11 QOS (3) 29 12 QOS 37 13 POS-SLMS (1)44 14 POS-SLMS (2)45 15 POS-SLMSSLA 47

I (Network Service Provider, NSP), (Internet Service Provider, ISP), (Application Service Provider, ASP),,,,, SLA SLAService Level Agreement (Quality of Service, QoS) [2, 3] SLA SLA SLA SLA

SLA QoS SLA SLA (SLA Management)TM Forum[1] SLA [3] SLA SLA (SLA Planning & Development), SLA (SLA Negotiation) (SLA Order Handling), (SLA Service Configuration) SLA QoS (SLA Monitoring), (SLA Reporting) (SLA Rating & Discounting) SLA SLA SLA SLA QoS QoS NSPISP QoS SLA SLA SLA

, QoS SLA SLA SLA NSPISP SLA QoS 2 SLA SLA SLA 3 NSPISP SLA QoS (Network Performance Metric, NPM) 4 QoS NPM 5 6 SLA QoS 7 SLA SLA SLA POS-SLMS (POSTECH-Service Level Monitoring System) 8

II SLA SLA SLA SLA SLA 21 SLA SLAService Level Agreement 1 SLA TM ForumSLA Management Handbook[3] Source: SLA Management Handbook TMForumGB917

(NSP), (ISP), (ASP),,,, SLA SLA SLA SLA SLA (SLA Template) SLA SLA SLA

22 SLA SLA SLA 20028 SLA SLA 2002 81 10 / / SLA 1Mbps, 500Kbps SLA

,, PC 30 10 60% 1 30% 1 AS 24 3 3 1 15, SLA, KT SLA 1 SLA (QoS) WorldCom[41], AT&T[42], Sprint[43], Cable & Wireless[44] ISP NSP SLA 2

WorldCom Private Label DSL (Access) SLA QoS 3

Service Name: Private Label DSL (Access) SLA QoS Parameter Network Availability Network Latency Data Delivery Mean Time To Restore Guaranteed Degree Measuring Periods Measuring Points Credits Avg 997% Avg 30ms Avg 95% A month From DSLAM to WorldCom s ATM switch 10% of the MRC 4-hour intervals for 5 consecutive days From WAN interface of end user (CPE) to WorldCom s ATM switch 10% of the MRC Measuring Method 4-hour intervals for 5 consecutive days From WAN interface of end user (CPE) to WorldCom s ATM switch 10% of the MRC 24hrs-Premium 48hrs-Basic A month From WAN interface of end user (CPE) to WorldCom s ATM switch 10% of the MRC Network Availability Network Latency Data Delivery Mean Time To Restore ( Total Number of DSL Circuit * Total minutes in Month - Total Minutes DSL Circuit Outages in Month )/ Circuits * Total minutes in month The time it takes for 64 byte packet to traverse from the WAN interface of a customer s DSL CPE First measurement is the NMC-to-CPE network delay by PING test Second measurement is the NMC-to-MSC network delay by PING test (NMC-to-CPE Delay - NMC-to-MSC Delay) /2, NMC: Network Management Center, MSC: Metro Service Center A minimum of 100 packets per measurement to ensure statistical validity The average data delivery is then calculated as the mathematical average of all measurements WorldCom is committed to restoring DLS service within certain periods of time based on the severity of the problem in addition to whether single or multiple clients are affected

23 SLA TM Forum[1], SLA Telecom Operation Map(TOM)[4] TM Forum SLA management Handbook[3]SLA TOM Model SLA SLA SLA 2 7 [66] Business Level Management Customer Customer Interface Management Service Level Management SLA Creation SLA Assessment SLA Negotiation SLA Reporting SLA Provisioning SLA Maintenance SLA Monitoring Network Level Management SLA Creation: SLA

SLA SLA Negotiation: SLA QoS SLA, SLA SLA Provisioning: MPLS Diffserv IP SLA Provisioning SLA Engineering SLA Monitoring: QoS SLA Monitoring SLA SLA Maintenance: SLA Provisioning SLA Reporting: SLA Reporting

SLA Assessment: SLA 24 SLA 7 SLA SLA (Service Level management Solution, SLS) CiscoWorks2000[5] SLS Service Assurance Agent (SAA) Management Engine 1110SAA SNMP Service Level ManagerSLA Management Engine1110 XMLSLA CiscoWorks2000 agent SNMP agent agent HP Openview NetMetrix[6] SLS AprismaSLA SPECTRUM[7]BMC Software PATROL[8] Compuware EcoSCOPE[9] SLA

(KRNIC)[10] SLA [11] SLA [12, 13, 14, 15] 25 (Active Monitoring)[16, 17] (Passive Monitoring)[17, 18] SNMP Agent[30, 31, 32] 251 (Active monitoring) NPM GPS[19], NTP(Network Time Protocol) [18, 20] Surveyor[19], RIPE NCC Test Traffic[5, 20], PingER [21], NLANR AMP[22], Skitter[23] 3

Surveyor RIPE PingER AMP Skitter Measureme nt 1-way delay & loss 1-way delay & loss 2-way ping 2-way ping Traceroute like Time Sync GPS GPS NTP NTP NTP Location US, Asia, Europe US, Europe, New Zealand Cover all six continents US, New- Zealand (NZ) America, Asia, NZ, Europe Measure Frequency Based on request Every 05 sec Ping every 30 minutes Traceroute every min Hourly Data Availability Upon request Upon request Available online Available online Upon request Sponsors NSF, DARPA CSG, Advanced DOE NSF, NLANR, Internet 2 NSF, DARPA, CAIDA SurveyorIPPM,, RIPE NCC Surveyor IPPM Surveyor 05

PingERPing End-to-end Reporting IEPM[45] 30 Ping PingER (end-to-end) NLANR AMP(Active Monitoring Programming)[22]Skitter[23] 3 252 (Passive monitoring),, Ntop[24], Cisco NetFlow FlowScan [25], CAIDA CoralReef[26], Waikato WAND[27], NLANR

PMA[28], SprintIP-Mon[29], POSTECHWebTrafMon [18] NG-MON[67] 4 (Throughput) FlowScan NetFlow (raw traffic) WAND Input Ntop FlowScan CoralReef Raw Traffic NetFlow NetFlow Raw Traffic Output Throughput Throughput Throughput Sprint IPMon Raw Traffic Packet Trace NG-MON (POSTECH) Raw Traffic Throughput Link Speed <<100 Mbps <<155 Mbps <<622 Mbps 10 Gbps 10 Gbps Solution Software Software Hardware+ Software Hardware+ Software Software Sampling No Yes(device) Configurable No Configurable Analysis On-line On-line On-line Off-line On-line

[68] 253 SNMP agent SNMP SNMP agent

RMON[30], Internet2[31] MAWI[32] SNMP agent 5 3 SLA

III QoS NSPISP SLA QoS NPM (Network Performance Metric) 31 QoS QoS QoS [3] QoS, SLA TM ForumSLA Management Handbook[3] QoS IP Layer QoS IP packet loss ratio, IP packet transfer delay, IP packet delay variation, Availability, Throughput, Utilization ATM Layer CER (cell error ratio), CD (cell delay), CLR (cell loss ratio), CTD (cell transfer Delay), CDV (cell delay variation) FR Layer xdsl Layer FTD (Frame Transfer Delay), Availability, FLR (Frame Loss Ratio) System parameters of Bit rates (up and downstream), Reach, Crosstalk, Radiation (Technology Specific)

ATM, Frame Relay, xdsl, IP QoS 6 Call connectivity, Attenuation, Noise, Crosstalk, echo, Distortion, Delay, Propagation delay Image quality, Character error rate Noise, Crosstalk, Stereo channel interface Call completion rate (Service Specific) QoS (Availability) (Reliability) 7 (Mean Time To Repair, MTTR) (Mean Time to Restore Service, MTRS) (Mean Time Between Failure, MTBF) SLA QoS SLA Management Handbook SLA QoS QoS

TM Forum NSPISP SLA QoS 3 Availability Delivery QoS Parameters Latency Bandwidth MTBF MTRS Availability Availability TM Forum QoS Delivery Latency Bandwidth MTBF MTRS MTTRMTRS

32 (Network Performance Metric, NPM) SLAQoS QoS 8 CAIDA Metric Working Group[33]IETF[55]IP-Performance Metrics (IPPM) Working Group[34] NPM CAIDA Metric Working Group NPMIPPM NPM LatencyIPPM Round Trip Delay[38] Connectivity[35], One-Way Delay[36]One-Way Packet Loss[37]

IPPM NSPISP QoS NPM 4 NPMs Availability Loss Delay Connectivity Functionality One way loss RT loss One way delay RTT delay Delay variance Utilization Capacity Bandwidth Throughput NPMAvailabilityQoS Availability QoS Availability NPM Availability (Connectivity) (Functionality) LossDelay IPPM Utilization

IV QoS 3 NSPISP QoS NPM QoS SLA SLA QoS NPMQoS, NPM SLA agent SLA agent Application agent, System agent, Traffic agent, Network agent [12] QoS NPM (Measurement mapping) NPM QoS SLA (Evaluation mapping) SLA 41 NSPISP QoS NPM QoS 5NSPISP QoS NPM NPM 5 MTBFMTRS

NPM NPM MTBF MTRS QoS Parameters Availability Delivery Latency Bandwidth MTBF MTRS NPMs Availability Loss Delay Utilization Connectivity Functionality One way loss RT loss One way delay RTT Delay Delay variance Capacity Bandwidth Throughput QoS NPM (Measurement mapping) (Evaluation mapping) QoS NPM NPM QoS SLA NPM QoS SLA

number of lost packet 100 total number of test packet 100 (%)

(m sec) packet RTT test of number total RTT _

9, 10, 11 QoS SLA SLA SLA (Evaluation Function) QoS

42 QoS NPM QoS NPM SLA Q q1 q2 q3 q4 QoS Parameter m(x) 2 N {n1,n2} {n2, n3} {n4} {n5,n6} NPM n(x) E e1 e2 e3 e4 e5 Evaluation Function 6 Q, N, E Q QoS NNPM E 6 m(x), n(x) m(x) (Measurement mapping) n(x) (Evaluation mapping) QoS NPM Q N N 2 (Powerset of N) NPM QoS 6

7 N NPM (N1,N2,N3,N4) H Monitoring Method (A,P,S) I Monitoring Point J Monitoring Period K Agent Type (aa, na, ta, sa) QoS NPM (Monitoring Method) / /SNMP agent (Monitoring point) (Monitoring Period) agent (Agent Type)

V SLA QoS NPM NPM 51 Internet2[31] 200 Internet2 Measurement Working Group [46] Internet2 QoS Internet2University Corporation for Advanced Internet Development (UCAID) National Internet Measurement Infrastructure (NIMI)[47] NIMI NIMI Probe NIMI NIMI National Science Foundation (NSF)[48] Defense Advanced Research Projects Agency (DARPA)[49]

National Laboratory for Applied Network Research (NLANR)[50]NFS NLANR vbns[51] NLANR Measurement and Network Analysis Group[52] National Analysis Infrastructure (NAI)[53] PMA[28] AMP[22]SNMP BGP Cooperative Association for Internet Data Analysis (CAIDA)[54] CAIDA CAIDA skitter[23] Forward IP Path, Round Trip Time, Persistent Routing Changes CAIDA IETF[55]IPPM IP IPPM (connectivity), (one-way delay and loss), (round-trip delay and loss), (delay variation), (loss pattern), (packet reordering), bulk transport capacity, link bandwidth capacity IPPM

Surveyor[19]Advanced Network and Services[56] (delay) (loss) Jitter 42 Surveyor box PC box Global Positioning System (GPS) receiver GPS Surveyor IPPM RIPE Network Coordination Center (RIPE NCC)[20], 52 52 1 ICMP 2 TCPUDP 3 4, ICMP ICMP echo request reply RTT (packet loss) (reachability) [57, 58, 59, 60, 61]

ICMP Ping Traceroute ICMP ICMPDOS (denial of service), ICMP ISPNSP ICMP (point-to-point) bandwidth Bandwidth ping (BING)[57], AMP[22] on-demand throughput, pchar/pathchar[58, 59] bandwidth, NIMI (one-way delay) Treno (Traceroute RENO)[60]Poip (poission ping)[61] ICMP (one-way delay) (time synchronization) GPS, TCPUDP [62, 63] ICMP TCP 3-tier UDP TCP, ICMPDOS ICMP TCP 51 ICMPTCP UDP

TCPUDP Netperf[62] TCP UDP BSD CPU (CPU-Utilization) Bulk Transfer Performance / Sting[63] TCP TCP ACK, [64, 65] Throughput Nettimer[65] bandwidth throughput, IP Measurement Protocol (IPMP)[69] NLANR IPMP Timestamp (path recoder) (one-way delay), 4 NPM 6, SLA

VI SLA SLA QoS 3, SLA QoS 12 QoS 12 ATM, Frame *100(%) _ 100 *100(%) ) _ ( ) _ ( 100 _ timestamp) - timestamp ( _ *100(%) ) _ ( ) _ ( 100 _ timestamp - timestamp ) (

Relay QoS ATM NSP (Delivery) QoS ATM ATM (cell) ATM CLR(Cell Loss Ratio) CLR ATM (%) = N I Cells N ECells *100 N Cells I N I N E, 12

VII SLA 4 QoS NPM SLA POS-SLMS (POSTECH-Service Level Monitoring System) 71 SLA SLA, SLA SLA Provisioning [39, 70] SLA QoS NPM, SLA IP- VPN xdsl agent SLA, SLA 72 SLA

71 SLA SLA SLA 8 Customer Business System SLA Maintenance System Related SLM Components Reporting System Billing System Service Planning and Development Problem Manager Data Store Reporting Manager Parameter Mapper Customer SLA Performance Analyzer Map DB PM Collector PM Collector PM Collector Customer SLA module SLA Agent SLA Agent Monitoring module SLA Agent Parameter Mapper module SLA SLA Customer SLA Module SLA Parameter Mapper ModuleSLA QoS NPM QoS Monitoring ModuleSLA SLA AgentNPM traffic agent, system agent, network agent, application agent

Agent PM collector Performance Analyzer PM collector Customer SLA DBSLA Map DB SLA Performance Analyzer Data Store Problem Manager SLA Reporting Manager SLA SLA SLA SLA SLA, SLA SLA SLM SLA SLA ISP NSP SLA, SLA 73 POS-SLMS (POSTECH-Service Level Monitoring System) 72 SLA POS-SLMS (POSTECH Service Level Monitoring System) 7311 POS-SLMS POS-SLMS ISP SLA

9 (Boranet) (Kornet) ISP Metro Ethernet Service Cisco 7513 POS-SLMS 10 SLA Linux Redhat73 OS10/100 One-way delayone-way loss DPNM IP PC QoS NPM

SLA DPNM NPM CPHP INTERNET 14122318022 POS-SMS Test Machine 1Gbps Optical Link CISCO 7513 SLA Agent -Packet Generator (ICMP) PM Collector Performance Analyzer Presenter L4 Switch Customer SLA DB Map DB Data Store DB PIRL GBS Gigabit Ethernet Backbone Network RSM RSM Parameter Mapper RSM RSM RSM RSM RSM SLA Agent - Packet Generator (TCP/UDP) RSM 14122382146 POS-SMS PIRL Test Machine 7312 POS-SLMS 13, 14 POS-SLMS QoS NPM

*100(%) _ 100 *100(%) ) _ ( ) _ ( 100 *100(%) ) _ ( ) _ ( 100

_ timestamp - timestamp ) ( _

POS-SLMS (Availability), (Delivery) (Latency) (Bandwidth) NPMOne-way delay NTP[71] GPS NTP NTPms POS-SLMS RTT1ms NTP One-way delay GPS GPS 7313 POS-SLMSSLA POS-SLMS SLA SLA SLA ISP SLA POS-SLMS 15 SLA

11POS-SLMS POS-SLMS

QoS SLA SLA, QoS SLA Costomer SLA moduleparameter Mapper module 72 Configuration Reporting and Billing SLA

VIII SLA SLA SLA SLA NSP ISP SLA NSPISP SLA QoS NPM, SLA NPM NSPISP SLA QoS CAIDAIPPM NPM SLA NPM QoS NPM QoS SLA NSPISP SLA

ISP SLA ISP SLA SLA

[1] TM Forum, http://wwwtmforumorg/ [2] TM Forum, "Performance Reporting Concepts and Definitions," TMF701 v20, Nov, 2001 [3] TM Forum, "Service Level Agreement Management Handbook," GB917 v15, Jun, 2001 [4] TM Forum, "Telecom Operation Map," GB910, v21, Mar, 2000 [5] Cisco, CiscoWorks2000, http://wwwciscocom/warp/public/44/jump/ciscoworksshtml/ [6] HP, Openview NetMetrix, http://wwwopenviewhpcom/ [7] Aprisma, SPECTRUM, http://wwwaprismacom/products/servicelevelshtml/ [8] BMC Software, PATROL, http://wwwbmccom/patrol/ [9] Compuware inc, EcoSCOPE, http://wwwecoscopenl/uk/indexhtml/ [10] KRNIC, http://trafficnicorkr/indexhtml/ [11], http://speedncaorkr/main/ [12] Lundy L, Pradeep R, "On the migration from enterprise management to integrated service level management," IEEE Networks, volume16, issue1, Jan, 2002, pp 8-14 [13] Appleby, K, Fakhouri, S, Fong, L, Goldszmidt, G, Kalantar, M, Krishnakumar, S, Pazel, DP, Pershing, J, Rochwerger, B, "Oceano - SLA based management of a computing utility," Proc of IM 2001, Seattle, USA, May, 2001, pp 855-868 [14] CISCO, "Successful Implementation Strategies for Service Level Management," CISCO white paper, 2000 [15] P Bhoj, et al, "SLA management in federated environments," Proc of IM 99, Boston, MA, USA, May, 1999, pp 293-308

[16] Maximo Alves, Luigi Corsello, Daniel Karrenberg, Cagdas Ogut, "New Measurement with the RIPE NCC Test Traffic Measurement Setup," RIPE- NCC-New Project Group, Proc of PAM 2002 Workshop, Colorado, USA, Mar, 2002 [17] Thomas Lindh, "A New Approach to Performance Monitoring in IP Networks-Combining Active and Passive Methods," Proc of PAM 2002 Workshop, Colorado, USA, Mar, 2002 [18] S H Hong, J Y Kim, B R Cho, J W Hong, "Distributed Network Traffic Monitoring and Analysis using Load Balancing Technology," Proc of APNOMS 2001, Sydney, Australia, Sep, 2001, pp 172-183 [19] Surveyor, http://wwwadvancedorg/csg-ippm/ [20] RIPE NCC Test Traffic Measurements, http://wwwripenet/testtraffic/indexhtml/ [21] PingER, http://www-iepmslacstanfordedu/pinger/ [22] NLANR AMP, http://ampnlanrnet/amp/ [23] Skitter, http://wwwcaidaorg/tools/measurement/skitter/ [24] Ntop, http://wwwntoporg/ [25] FlowScan, http://wwwcaidaorg/tools/utilities/flowscan/indexxml/ [26] CoralReef, http://analacaidaorg/coralreef/demos/cerfnet/link/ [27] WAND, http://wandcswaikatoacnz/wand/wits/ [28] NLANR PMA, http:// moatnlanrnet/pma/ [29] Sprint ATL, "IP Monitoring Project, http://wwwsprintlabscom/department/ip-interworking/monitor/ [30] S Waldbusser, "Remote Network Monitoring Management Information Base," IETF RFC1757, Feb, 1995 [31] Internet2 (I2), http://hydrauitsiuedu/~abilene/traffic/ [32] MAWI, http://wwwwideadjp/wg/mawi/ [33] CAIDA, "Network Measurement FAQ," CAIDA Metrics Working Group,

Jan, 2002, http://wwwcaidaorg/outreach/metricswg/faqxml/ [34] IPPM, http://wwwadvancedorg/ippm/ [35] JMahdavi, V Paxson, "IPPM Metrics for Measuring Connectivity," IETF RFC2678, Sep, 1999 [36] G Almes, S Kalidindi, M Zekauskas, "A One-way Delay Metric for IPPM," IETF RFC2679, Sep, 1999 [37] G Almes, S Kalidindi, M Zekauskas, "A One-way Packet Loss Metric for IPPM," IETF RFC2680, Sep, 1999 [38] G Almes, S Kalidindi, M Zekauskas, "A Round-trip Delay Metric for IPPM," IETF RFC2681, Sep, 1999 [39] Taesang Choi, Seunghyun Yoon, Hyungsuk Chung, Changhoon Kim, Jungsook Park, Bungjun Lee, Taesoo Chung, "Wise: Traffic Engineering Server for A Large-scale MPLS-based IP Network," Proc of NOMS 2002, Florence, Italy, Apr, 2002, pp 251-264 [40] S Blake, D Black, M Carlson, E Davies, Z Wang, W Weiss, "An Architecture for Differentiated Services," IETF RFC2475, Dec, 1998 [41] WorldCom, http://wwwworldcomcom/terms/service_level_guarantee/ [42] Sprint, http://wwwsprintworldwidecom/solutions_slahtml/ [43] Cable & Wireless, http://wwwslacwnet/ [44] AT&T, http://wwwipservicesattcom/ [45] IEPM, http://www-iepmslacstanfordedu/ [46] Internet2 Measurement Working Group, http://wwwinternet2edu/measurement/ [47] National Internet Measurement Infrastructure (NIMI), http://ncnenlanrnet/nimi/ [48] National Science Foundation (NFS), http://wwwnsfgov/ [49] Defense Advanced Research Projects Agency (DARPA), http://wwwdarpamil/ [50] National Laboratory for Applied Network Research (NLANR),

http://wwwnlanrnet/ [51] Very high performance Bandwidth Network Service (vbns), http://wwwvbnsnet/ [52] NLANR Measurement and Network Analysis Group, http://moatnlanrnet/ [53] NLANR Network Analysis Infrastructure, http://moatnlanrnet/infrastructurehtml/ [54] Cooperative Association for Internet Data Analysis (CAIDA), http://wwwcaidaorg/ [55] IETF, http://wwwietforg/ [56] Advanced Network and Services, http://wwwadvancedorg/surveyor/ [57] Bandwidth ping (BING), http://wwwcnamfr/reseau/binghtml/ [58] Pchar, http://wwwemployeesorg/~bmah/software/pchar/ [59] Pathchar, http://wwwcaidaorg/tools/utilities/others/pathchar/ [60] Treno, http://wwwpscedu/networking/treno_infohtml/ [61] Poip, http://wwwpscedu/networking/papers/nimihtml/ [62] Netperf, http://wwwnetperforg/netperf/netperfpagehtml/ [63] S Savage, "Sting: A tcp-based network measurement tool," USENIX Symposium on Internet Technologies and Systems, Boulder, CO, USA, Oct, 1999, pp 71-79 [64] Thomas Lindh, A New Approach to Performance Monitoring in IP Networks-Combining Active and Passive Methods,Proc of PAM 2002 Workshop, Colorado, USA, Mar, 2002 [65] Kevin Lai, Mary Baker, Nettimer: A Tool for Measuring Bottleneck Link Bandwidth,Proc of PAM 2001 Workshop, Amsterdam, Netherlands, Apr, 2001 [66] H J Lee, M S Kim, J W Hong and G H Lee, Mapping Between QoS Parameters and Network Performance Metrics for SLA Monitoring,Proc of APNOMS 2002, Jeju, Korea, Sep, 2002, pp 97-108

[67] Se-Hee Han, Myung-Sup Kim, Hong-Taek Ju and James W Hong, "The Architecture of NG-MON: A Passive Network Monitoring System," Proc of DSOM 2002, Montreal, Canada, Oct, 2002, pp 16-27 [68] Tanja Zseby, "Deployment of Sampling Methods for SLA Validation with Non-Intrusive Measurements," Proc of PAM 2002 Workshop, Colorado, USA, Mar, 2002 [69] IPMP, http://wattnlanrnet/amp/ipmp/ [70] A Feldman, et al, "NetScope: Traffic Engineering for IP Networks," IEEE Network, Vol14, No 2, Mar, 2000, pp11-19 [71] NTP, http://wwweecisudeledu/%7emills/bibliohtml/

: : 1978 827 : : 919-3 1997 2001 : ( ) 2001 2003 : ( ) ( )

Conference Papers Hyo-Jin Lee, Myung-Sup Kim and James W Hong, "Mapping between QoS Parameters and Network Performance Metrics for SLA monitoring", Proc of APNOMS 2002, Jeju, Korea, September 25-27, 2002, pp 97-108 H T Ju, M J Choi, S H Han, Y J Oh, J H Yoon, H J Lee and J W Hong, "An Embedded Web Server Architecture for XML-Based Network Management", Proc of the IEEE/IFIP Network Operations and Management Symposium (NOMS 2002), Florence, Italy, April 2002, pp 5-18 Projects A Study on Network Traffic Monitoring and Analysis Methods for SLA (SLA ), DPNM Project, 2002 Element and Network management Using Internet and EWS Technologies ( ), DPNM Project, 2001