IBM Software Group 테스팅프로세스전반에걸친테스트자동화 2008 IBM Corporation
목 차 Managing Test Early & Rational QM Portfolio Rational Test Management Model Test Planning Test Authoring Test Execution Test Reporting
Software Development Lifecycle Analyst Model, simulate, assemble, and monitor processes Architect Visually model applications and data Developer Rapidly construct, transform, integrate and generate code Tester Design, create, and execute tests Deployment Manager Provision, configure, tune and troubleshoot applications Project Manager Follow a common process Manage and measure projects and portfolios Manage requirements Manage change and assets Manage quality Executive Align investments with business objectives Analyze and monitor project portfolios
Rational Portfolio Analyst Rational RequisitePro Rational Software Modeler Architect Rational Software Architect Developer Rational Application Developer Rational PurifyPlus / AppScan Tester Rational Functional Tester Plus Rational Performance Tester Deployment Manager Tivoli ITCAM Tivoli Configuration Manager Project Manager Rational Unified Process / Rational Method Composer Rational ClearCase / ClearQuest / BuildForge Rational Asset Manager Executive Rational Portfolio Manager
The test market is changing Test is evolving into quality management Process management Project management Software delivery Organizational Influence Current Industry Focus Manage Quality Early Where industry is going Ensure Business Flexibility Enhance Organizational Efficiencies Governance Resource availability Process Maturity Manual Testing Automated Testing Automated Software Quality Quality Management Business-Driven Quality Management
Usual Quality Activities 일반적으로품질과관련된업무는개발자가코딩을끝내는시점에시작되며, 대부분품질과관련된노력은이이후에집중된다. Analyst Architect Developer Tester Operation Define Design Code Test Debug Functional Test Load Test Deploy Monitor 품질과관련된첫업무시작시점 대부분의노력을투입하는시점 품질은다양한측면에서요구사항에맞는최종산출물을의미 대부분의경우주어진요구사항에맞추어품질에대한검증이시작되며, 잘못분석된요구사항에대한검증은이루어지지않음 품질은 QA 팀에서관리하는것으로인식하는경우가많으며이방법으로는대부분 deploy 직전에문제를발견
Cost of Defects vs. Where Found 발견 발생 결함이발견된단계및수정에소요되는비용 Inception Elaboration Construction Transition Inception $25 ~ $100 Elaboration $100 ~ $500 $50 ~ $250 Construction $500 ~ $1K $250 ~ $1.5K $75 ~ $500 Transition $8K ~ $10K $1.5K ~ $5K $500 ~ $3K? 프로젝트후반으로갈수록발견된결함을해결하는데더많은비용및노력소요 Source : Profiles of Level 5 CMMI Organizations by Donald J. Reifer
Manage Quality Early 초기부터검증시작 자동화도구 개발, 테스트팀및운영팀간의협업 품질과관련된업무가시작되어야하는시점 일반적으로품질과관련된업무가시작되는시점 C O S T Iterative Process Business applications 변화하는고객의요구에빠르게대응 전략적인우위를점할수있음 애플리케이션의신뢰도와가용성확보 Early test to 고객의요구만족 의도된대로 소프트웨어가 작동하는지검증 실환경도입후애플리케이션이제대로동작하는지확인
Rational QM Tools in Test Lifecycle Rational Functional Tester Plus Architect Use Case Test Script Requirements Test Case Test Log Analyst Rational RequisitePro Tester Rational ClearQuest Defect End User Analyst Architect Developer Tester
Pains we are facing 테스트케이스등테스트관련산출물을문서로관리하는데따르는비효율성 테스트결과외에이와관련된결함및요구사항에대한정보파악이어려움에따른통합솔루션의부재 반복적인테스트수행에소요되는노력및이에따른테스트커버리지의저하 다양한 metric을통한테스트프로젝트진행상황파악필요 QA 팀원과다른프로젝트팀원들간의의사소통의결여 개발팀및 QA 팀이사용하는여러소프트웨어벤더사도구들의혼재로인한 Learning curve 및관리비용증가
Test Management with ClearQuest Eclipse Based Test Management Web clients & native windows clients also available Key Functional Areas Test Planning - 테스트계획, 테스트케이스등의산출물생성및관리 Test Execution - 테스트케이스및연관자동 / 수동스크립트실행 Test Reporting - 테스트진행상황파악을위한각종차트및리포트제공
ClearQuest TestManager Benefits 하나의 View에서전체테스트공정관리및분석 - 단순히테스트와관련된결과를분석하는것보다원인이되는요구사항부터테스트및파생된결함을통합하여분석 Eclipse 기반의테스트관리도구 - 팀원간의보다효과적인협업방법제공 환경에맞는테스트 / 결함관리시스템구축용이 - 조직마다요구되는복잡한환경에대응가능
Lifecycle Traceability Requirements Tests Defects What can be analyzed 요구사항과연관된테스트결과 테스트결과와연관된결함 결함을발생시킨테스트와이의근거가되는요구사항등 다양한질의, 차트, 보고서작성이가능 요구사항과테스트, 결함에이르는전체테스트공정에서추적성을제공하여다양한결과도출및분석을통해테스트생산성향상가능
Common Platform 서로다른업무를수행사지만공통된 IDE 를사용함으로써팀원간의효과적인데이터및기술공유가능 Rational 도구나 Eclipse 를사용하는사용자는추가의도구를사용할필요없이 CQTM 플러그인을통해기존개발환경에서작업가능 변경요청에대해보다신속한대응을통한품질개선효과기대 Eclipse 기반의테스트 / 결함관리 분석업무 설계업무 개발업무 검증업무 Analyst Architect Developer Tester 요구사항 모델 소스코드 테스트결과
Configure and Enforce Your Process ClearQuest는요구되는환경에맞춰적용 - ClearQuest Designer를이용하여애플리케이션의다양한동작과 UI를 4GL IDE 도구의방법과유사하게개발가능 - 다양한종류의변경요청레코드타입을생성할수있으며, 각타입별로고유한필드, 폼및프로세스를만들어낼수있음 - 확장성이높은 API를통해외부시스템과의연계가능 - 지속적으로변화되는환경에대응
Rational Test Management Model Planning Reporting Authoring Execution
Rational Test Management Model 테스트계획단계의산출물 Test Planning: 테스트케이스정의및분류 Test Plan: 전체적인테스트목표및방법등을기술하는문서로테스트케이스를분류하는컨테이너역할 Test Cases : 테스트대상시스템내에서검증되어야하는내용을나타내는문서 Configured Test Cases: 테스트가수행되는환경및 iteration 을명시하는 Test Case 의구체적인문서로수행될수있는단위 Test Suites: 일련의 Configured Test Case 묶음으로테스트를수행하고자하는시나리오구성
ClearQuest 의산출물관리 테스트플랜 테스트케이스 Configured Test Case 테스트계획단계의산출물계층적관리 - 테스트플랜과테스트케이스의구조표현 - 산출물의 duplication을통한재사용가능 Test Suite Iteration Configuration
Test Planning using ClearQuest Rational RequisitePro Requirement Iteration Configuration Test Plan Test Case Configured Test Case 테스트플랜 / 테스트케이스생성 - 요구사항연계 Configured test case 생성 - Iteration - Configuration
테스트산출물양식 원하는포맷작성 - ClearQuest Designer를통해코딩없이요구되는양식작성 - 테스트플랜, 테스트케이스등의레코드타입에따른양식작성
Test Management Model Test Authoring: 테스트계획단계에서계획된테스트케이스를수행하기위한문서혹은자동화도구스크립트를생성하는과정 Test Execution: 테스트를실행하고테스트결과를산출하는과정 Test Reporting: 테스트관리관점에서의결과및진행상황등을제공하는과정
Test Authoring 도구 자동및수동테스트지원 Rational Functional Tester Record 방식으로스크립트생성 자동화테스트스크립트실행으로애플리케이션테스트반복실행을통한회귀테스트수행 Rational Manual Tester 모든테스트가자동으로수행될수는없음 자세하고따라하기쉬운수동테스트스크립트생성 테스트스크립트재사용및테스트리포트생성으로수동테스트효율향상
Rational Functional Tester Record & Playback 방식 자동테스트스크립트생성및실행 Object 인식기반의자동테스트수행 Eclipse 및 MS VS.Net 환경제공 Java or VB.Net 스크립트 다양한검증포인트제공으로애플리케이션의작동상태확인 애플리케이션변경시회귀테스트에대응하는기능제공 Object Map, ScriptAssure TM Data Driven 테스트지원으로동일한테스트에대해다양한입출력데이터테스트수행 다양한기법적용및테스트커버리지증대 테스트결과보고 Fail 시검증포인트, 스크립트등의원인보고
Rational Manual Tester 테스트수행자가이해하기쉬운문서작성 이미지및입력 / 비교데이터포함 외부 Word나 Excel 문서 import 가능 테스트스크립트재사용 중앙의재사용공간에저장하여스크립트변경에효율적대응 On Screen 테스트수행 수행할테스트스텝표시및스텝별테스트결과입력 자동데이터입력및텍스트값비교검증 자동리포트생성 입력된테스트결과에따른보고서생성
Test Authoring using Rational Functional Tester Plus Test Plan Requirement Iteration Configuration Test Case Configured Test Case Test Suite Rational Functional Tester Plus Test Script 자동화혹은수동스크립트
Test Script Association 테스트케이스를실행할수있는자동테스트스크립트혹은매뉴얼테스트스크립트로구현 테스트단위의결과보다전체적인테스트케이스관점의보고서도출 Configured Test Case Test Script
Test Execution from CQTM 자동스크립트실행혹은매뉴얼테스트수행 스크립트연계로실행된 Configured Test Case 에결과저장 Configured Test Case Test Script Test Result : Pass
Test Execution Steps Configured Test Case 단위의단독실행및 Suite 단위의그룹실행 Test Suite Suite Log Configured Test Case #1 Test Script 1 Test Log Configured Test Case #2 Test Script 2 Test Log Defect
실행결과및결함등록 Configured Test Case 단위혹은 Test Suite 단위의실행결과로테스트케이스별성공 / 실패여부보고 테스트케이스별자세한로그확인가능 결함발견시테스트결과에서직접결함등록가능 결과와관련된정보가자동으로포함
Test & Change Management Process Test Lifecycle 구현 Test Script 수행 Requirements 분석 Test Case 수행 Test Log Change & Defect Tracking Process Submitted Assigned Opened Resolved Validated Closed
Test Reporting and Metrics 쿼리, 차트및리포트 기본제공 사용자정의 Examples Status 계획된테스트 vs 구현된 (authored) 테스트 특정회차에서성공 vs 실패테스트 결함이제출되지않은실패한테스트 테스트케이스가작성되지않은요구사항 담당자별테스트등 Trend over time 테스트케이스수 성공 vs 실패한테스트의수
Demo
Thank You