Software Verification 1 st Testing System Test Team 5 201410373 고예은 201411266 김수현 201411268 김아름 1
Index 1. Specification Review... 3 1.1. Stage 1000 Planning... 3 1.2. Stage 2030 Analysis... 3 1.3. Stage 2040 Design... 4 1.4. Stage 2050/2060 Implementation & Unit Test... 6 2. Category-partition Testing... 8 2.1. Test Cases... 8 2.2. Testing Result... 10 3. Pairwise Testing... 12 3.1. Testing Result... 12 4. Brute Force Testing... 13 4.1. Testing Result... 13 4.2. Failed Case Report... 13 5. Overall... 14 5.1. System Test Result... 14 2
1. Specification Review 1.1. Stage 1000 Planning 1001. Define Draft Plan 4. Non Functional Requirements - 금융거래프로그램이므로보안이잘되어야한다. -> 이후문서에서보안에대한명세존재하지않음 1003. Define Requirements 1. Functional Requirements - 성공한거래의송금금액 / 입금계좌 / 이름을보여줌 -> 이후단계에서거래후잔고를보여주는명세가존재하지않음 ( 실제구현된코드에서는자기계좌의남은잔고만보여줌 ) - ATM 잔고가일정수준 (50 만원 ) 이하가되면잔고부족메시지를출력 -> 이후단계에서사라짐 - 1001 단계에서는 manage_atm 부분이서술되어있지않음문서전체에서 withdraw 에대한해석을통일하고있지않음 ( 출금 / 인출 ) 1.2. Stage 2030 Analysis 2031. Define Essential Use Cases 2010 까지는존재하는 수수료 와 명세서출력 에대한서술이모두사라짐 1. Withdraw - 출금화면을출력한다 & 출금금액을입력한다 표현모호 - ATM의잔고와출금금액을비교한다 는내용추가해야함 2. Deposit - 계좌잔액을표시한다 는내용추가해야함 3. Remittance - 계좌잔액을표시한다 는내용추가해야함 - Exceptional courses of Events: 자기자신의계좌에는송금할수없다 가추가되어야함 3
5. Random_Jackpot - User 가돈을인출했을때 표현이모호함 -> User 가올바른 password 를입력했을때 - pre-requisites: 사용자가인출중이어야한다. - 당첨되는순서를랜덤하게선택한다. -> Random 범위설명필요 - ATM 의잔고에서 5 만원을감소시킨다 는내용추가해야함 6. manage_atm - 갱신후관리자의입출금선택화면으로돌아간다 는내용추가되어야함 - Exceptional Courses of Events: ATM 잔고의최대값에대한내용이추가되어야함 2037. Define State Diagram - WaitForAmount 와 Menu 사이에잔고를보여주는상태가존재 2038. Refine System Test Case - 1009 에서정의한 System test case 와항목및내용이완전히달라짐 1.3. Stage 2040 Design 2041. Design Real Use Cases 2031 바탕으로전체적인수정필요 1. Withdraw - TotalMoney 뿐만아니라지폐권에따른각잔고도감소시킨다는내용이추가되어야함 4
2. Deposit - TotalMoney 뿐만아니라지폐권에따른각잔고도감소시킨다는내용이추가되어야함 6. manage_atm - TotalMoney 뿐만아니라지폐권에따른각잔고도증가 / 감소시킨다는내용이추가되어야함 2042. Define Reports UI, and Storyboards - 돌아가기 버튼에대한명세가전혀없었으나 UI 단계에서등장 - 금액을입력할때지폐권을나누어서입력한다는명세가없었음 2044. Define Interaction Diagrams - 실제로 GUI_Interface 라는클래스는존재하지않음 -> 대신 System_Control & MainFrame 존재 - Timer 클래스사용하지않는 diagrams 에서는생략필요 5
1.4. Stage 2050/2060 Implementation & Unit Test 2051. Implement Class & Method Definitions Class Definitions System_Control - 명세에정의되지않은메소드가소스코드상에존재 (getatmcount, get_inputmoney, inputrid, get_isjackpot, set_usercount) 1. System_Control - 생성자메소드에대한명세누락 2. input_menu - 명세와소스코드의 input 값자료형불일치 - Abstract Operation: 사용자가입력한기능 이라는표현모호분명한 attribute 로수정필요 3. input_id - 명세와소스코드의 input, output 값자료형불일치 4. input_amount - 명세와소스코드의 input, output 값자료형불일치 - Purpose, Abstract Operation 에서는 inputamount attribute 가존재하나소스코드상에서는존재하지않음. 5. input_pw - 명세와소스코드의 input, output 값자료형불일치 - Purpose, Abstract Operation 에서는 inputpw attribute 가존재하나소스코드상에서는존재하지않음. 6. updatebalance - 명세와소스코드의 output 값자료형불일치 - Abstract Operation 에서는 Account.Balance 가존재하나소스코드상에서는존재하지않음. 7. input_mid - 명세와소스코드의 input, output 값자료형불일치 - Abstract Operation 에서는 inputid 가존재하나소스코드상에서는존재하지않음. 6
8. input_pom - 명세와소스코드의 input 값자료형불일치 - Abstract Operation 에는 inpupw 가존재하나소스코드상에는존재하지않음. - 오타 : 입력력 > 입력, inpupw > inputpw 9. input_mpw - 명세와소스코드의 input, output 값자료형불일치 Class Definitions Manager 1. Manager - 생성자메소드에대한명세누락 2. Manager.checkID - 소스코드상에서는 checkid - Abstract Operation 오타 : inputmid > inputid, ManagerIID > ManagerID 3. Manager.CheckPW - 소스코드상에서는 checkpw - Abstract Operation 오타 : inputmpw > inputpw 2055. Write Unit Test Code - Unit Test 를거치지않은함수들존재 2063. System Testing - System Testing 결과차이존재 (2-1, 3-1, 5-1, 6-1, 7-1: Pass -> Fail) -> 숫자가아닌다른 character 를입력하거나 Integer 범위밖에있을때는오류메시지를출력하지않는다. - 8-1 plus 를입력하였을때 표현매우모호 - 8-2 minus 를입력하였을때 표현매우모호 - test case 의내용자체가굉장히광범위하게작성됨 7
2. Category-partition Testing 2.1. Test Cases - TSL 작성 (Error Constraints, Single Constraints, Property Constraints 적용 ) - User -> Test Case 개수 32 개 8
- Manager -> Test Case 개수 32 개 9
2.2. Testing Result Test Suite Test Case Latest Exec Result User atm-5:test Case 1 Passed User atm-6:test Case 2 Failed User atm-7:test Case 3 Failed User atm-8:test Case 4 Failed User atm-9:test Case 5 Failed User atm-10:test Case 6 Failed User atm-11:test Case 7 Passed User atm-12:test Case 8 Failed User atm-13:test Case 9 Failed User atm-14:test Case 10 Failed User atm-15:test Case 11 Passed User atm-16:test Case 12 Passed User atm-17:test Case 13 Passed User atm-18:test Case 14 Failed User atm-19:test Case 15 Passed User atm-20:test Case 16 Passed User atm-21:test Case 17 Failed User atm-22:test Case 18 Failed User atm-23:test Case 19 Failed User atm-24:test Case 20 Failed User atm-25:test Case 21 Failed User atm-26:test Case 22 Failed User atm-27:test Case 23 Failed User atm-28:test Case 24 Failed User atm-29:test Case 25 Failed User atm-30:test Case 26 Failed User atm-31:test Case 27 Failed User atm-32:test Case 28 Failed User atm-33:test Case 29 Failed User atm-34:test Case 30 Failed User atm-39:test Case 31 Failed User atm-40:test Case 32 Passed Manager atm-41:test Case 1 Passed Manager atm-42:test Case 2 Failed Manager atm-43:test Case 3 Failed Manager atm-44:test Case 4 Failed Manager atm-45:test Case 5 Failed Manager atm-46:test Case 6 Failed Manager atm-47:test Case 7 Passed Manager atm-48:test Case 8 Failed Manager atm-49:test Case 9 Passed Manager atm-50:test Case 10 Failed 10
Manager atm-51:test Case 11 Failed Manager atm-52:test Case 12 Failed Manager atm-53:test Case 13 Failed Manager atm-54:test Case 14 Failed Manager atm-55:test Case 15 Failed Manager atm-56:test Case 16 Passed Manager atm-57:test Case 17 Failed Manager atm-58:test Case 19 Passed Manager atm-59:test Case 20 Failed Manager atm-60:test Case 21 Failed Manager atm-61:test Case 22 Passed Manager atm-62:test Case 23 Passed Manager atm-63:test Case 24 Failed Manager atm-64:test Case 25 Passed Manager atm-65:test Case 26 Passed Manager atm-66:test Case 27 Failed Manager atm-67:test Case 28 Passed Manager atm-68:test Case 29 Passed Manager atm-69:test Case 30 Failed Manager atm-70:test Case 31 Passed Manager atm-71:test Case 32 Passed Manager atm-72:test Case 33 Failed 21/64 = 32% Pass 11
3. Pairwise Testing 3.1. Testing Result Test Suite Test Case Latest Exec Result User atm-73:[pairwise]test Case1 Passed User atm-74:[pairwise]test Case2 Failed User atm-75:[pairwise]test Case3 Failed User atm-76:[pairwise]test Case4 Passed User atm-77:[pairwise]test Case5 Failed User atm-78:[pairwise]test Case6 Failed User atm-79:[pairwise]test Case7 Failed User atm-80:[pairwise]test Case8 Failed User atm-81:[pairwise]test Case9 Failed User atm-82:[pairwise]test Case10 Failed User atm-83:[pairwise]test Case11 Failed User atm-84:[pairwise]test Case12 Failed User atm-85:[pairwise]test Case13 Failed User atm-86:[pairwise]test Case14 Failed Manager atm-87:[pairwise]test Case1 Failed Manager atm-88:[pairwise]test Case2 Passed Manager atm-89:[pairwise]test Case3 Failed Manager atm-90:[pairwise]test Case4 Passed Manager atm-91:[pairwise]test Case5 Failed Manager atm-92:[pairwise]test Case6 Failed Manager atm-93:[pairwise]test Case7 Failed Manager atm-94:[pairwise]test Case8 Failed Manager atm-95:[pairwise]test Case9 Failed Manager atm-96:[pairwise]test Case10 Failed Manager atm-97:[pairwise]test Case11 Failed Manager atm-98:[pairwise]test Case12 Failed 4/26 = 15% Pass 12
4. Brute Force Testing 4.1. Testing Result Test Suite Test Case Result Manager 관리자프로세스에서 5 번반복하여입 / 출금할수있다. Failed Manager 관리자프로세스에서 10 번반복하여입 / 출금할수있다. Failed User 프로그램초기실행시난수가제대로생성된다. Passed User 출금 10 번이내에 Jackpot 이터지는지확인한다. Passed User Jackpot 이터지면해당 User 에게 5 만원을추가인출하며 ATM 의잔고에서 5 만원이빠져나간다. Failed User 1 일 1 명만 Jackpot 이터진다. Failed User 출금시에만 Jackpot 이터진다. Failed User 하나의통장으로여러번출금할시 Jackpot 당첨이여러번되지않게한다. Failed User 모든입력에대한기기의반응은 1 초이내로이루어진다. Failed User 거래는 1 분이내에이루어져야한다. Passed User 사용자가보기편한화면을제공해야한다. Passed User 세번이상연속적으로출금할수있다. Passed User 세번이상연속적으로입금할수있다. Failed User 세번이상연속적으로송금할수있다. Failed User 세번이상연속적으로조회할수있다. Passed User User User 출금시금액입력값에 minus 정수를입력하면다음단계로넘어가지않거나, 오류메시지가출력된다. 입금시금액입력값에 minus 정수를입력하면다음단계로넘어가지않거나, 오류메시지가출력된다. 송금시금액입력값에 minus 정수를입력하면다음단계로넘어가지않거나, 오류메시지가출력된다. Failed Failed Failed Manager ATM 잔고가일정수준 (50 만원 ) 이하가되면잔고부족메시지를출력한다. Failed 6/19 = 31% Pass 4.2. Failed Case Report Test Suite Test Case Manager 잔고가어느일정범위이상커지면 " 잔고확인하세요 " 라는오류메시지가출력된다. ( 잔고최대값명세에존재하지않음 ) User Jackpot 이터져도 ATM 의잔고에는변화가생기지않는다. User 초기실행후 Jackpot 이한번터진후에도다시터진다. User 송금시사용자의비밀번호를입력하고나서도 Jackpot 이터지는경우존재 User integer 이내의정수값이아니라면, 1 초이내로반응이없다 ( 반응이아예없다 ) User 같은계좌로출금을 5 회이상반복했을때, 연속으로 2 번잭팟이터지는경우가있다 User 잔고가어느일정범위이상커지면 " 비밀번호확인하세요 " 라는오류메시지가출력된다. User 3 번이상송금하려고하면알맞은비밀번호여도오류메세지가나온다 User 출금시 minus 정수값을금액으로입력하면해당정수의절대값만큼계좌의잔액이증가한다. 예외처리필요 User 입금시 minus 정수값을금액으로입력하면해당정수의절대값만큼계좌의잔액이감소한다. 예외처리필요 User 송금시 minus 정수값을금액으로입력하면해당정수의절대값만큼계좌의잔액이증가한다. 예외처리필요 Manager ATM 잔고가 50 만원이하가되어도아무메시지도출력되지않는다. 13
5. Overall 5.1. System Test Result - Category Partition Test 21/64 = 32% Pass - Pairwise Test 4/26 = 15% Pass - Brute Force Test 6/19 = 31% Pass 5.2. Summary - 문서들의단계마다일치하지않는부분들이있다. - 실제구현된소스코드와문서간차이점이존재한다. - 초반설계한 Functional Requirements 중구현하지않은부분이있다. - 보안등의 Non Functional Requirements에대한명세가 Stage 1000에서만존재, 구현상으로도 minus 입력이가능한점등의소홀함이보인다. - Keyboard 입력에대한예외처리가아예되어있지않다. - Specification이대체로명확하지않고모호함. 추가로작성해야할것들이많다. - 실생활에활용되는 ATM이라는점을고려했을때기능이너무부실하다. 14