以下內容大部分來自 ron-patton-software-testing
PART V Working with Test Documentation
Chapter 16 Planning Your Test Effort
1. What’s the purpose of a test plan?測試計劃的目的
To paraphrase the ANSI/IEEE definition, the purpose of a test plan is to define the scope,approach, resources, and schedule of the testing activities and to identify the items being tested, the features to be tested, the testing tasks to be performed, the personnel responsible for each task, and the risks associated with the plan.
In short, to tell and get agreement from the rest of the project team exactly how the heck the test team intends to test the software.
2. Why is it the process of creating the plan that matters, not the plan itself?爲什麼制定計劃的過程很重要，而不是計劃本身呢？
Because all the issues and questions defined in a test plan either impact or are influenced by other project functional groups or team members. Getting everyone to understand and agree to the contents of the plan is what matters. Privately creating a paper document and putting it on a shelf is not just a waste of time, but also jeopardizes the project.
3. Why is defining the software’s quality and reliability goals an important part of test planning?
Because left to their own, everyone will have different ideas of what quality and reliability mean to them. Since they’re all different, they all can’t be achieved.
4. What are entrance and exit criteria?
These requirements must be met to move from one testing phase to another. A phase can’t be left until its exit criteria are met. A new phase can’t be entered until its entrance criteria are met.
5. Name a few typical testing resources that should be considered when test planning.
People, equipment, offices and labs, software, outsourcing companies, and miscellaneous supplies.
6. True or False: A schedule should be made to meet absolute dates so that there’s no question when a testing task or phase is to start and when it’s to end.
False. Because testing depends so much on other aspects of the project (for example, you can’t test something until it’s coded), a test schedule is best made relative to the delivery dates.
Chapter 17 Writing and Tracking Test Cases
1. What are the four reasons for test case planning?
Organization, repeatability, tracking, and proof of testing.
2. What is ad hoc testing? 臨時測試
Ad hoc testing is testing without a plan. It’s easy and fun but it’s not organized, it’s not repeatable, it can’t be tracked, and when it’s over, there’s no proof that it was ever done.
3. What’s the purpose of a test design specification?
The purpose of the test design spec is to organize and describe the testing that needs to be performed on a specific feature. It outlines the features to be tested and the approach to be used. It identifies the test cases, but doesn’t specify them, and what the pass/fail criteria is.
測試設計規範的目的是組織和描述需要對特定功能執行的測試。 它概述了要測試的功能和使用的方法。 它標識測試用例，但不指定它們，以及通過/失敗的標準是什麼
4. What is a test case specification?
This document defines the actual input values used for testing and the expected outputs.
It also lists any special environmental needs or procedure requirements and any interdependencies among test cases.
5. Other than a traditional document, what means can you use to present your test cases?
Tables, matrices, lists, graphical diagrams—whatever means most efficiently presents the test cases to you, other testers, and other members of your product team.
6. What’s the purpose of a test procedure specification?
The purpose of the test procedure spec is to identify all the steps required to perform the test cases, including how to set up, start, run, and shut down the test. It also explains what to do in case the test doesn’t go as planned.
7. At what level of detail should test procedures be written?
That’s a question without a specific answer. It greatly depends on who will be using the procedures. Too little detail makes the test procedures ambiguous and variable. Too much detail can bog down(停滯) the test process. The level of detail should be set by the industry, the company, the project, and the test team.
Chapter 18 Reporting What You Find
1. Cite a few reasons that a bug might not be fixed.
There’s not enough time in the schedule,
it’s not a bug,
it’s too risky,
it’s not worth it,
and the bug wasn’t reported properly.
2. What basic principles can you apply to your bug reports to give them the best chance of getting the bug fixed?
Log them as soon as possible.
Effectively describe the bug, making sure it’s minimal,singular, obvious and general, and reproducible.
Be nonjudgmental in your approach.
Follow the report through its life cycle.
儘快記錄它們。
有效地描述錯誤，並確保其最小，單一，明顯和普遍且可重現。
在您的方法中要有判斷力。
跟蹤報告的整個生命週期。
3. Describe a few techniques for isolating and reproducing a bug.
Record what you do and review it carefully.
Use white-box test techniques to look for race conditions, boundary conditions, memory leaks, and other similar problems.
See if the bug is state related, such as initial state or later state dependent. Consider resource dependencies and even hardware problems as the source of the bug.
4. Suppose that you’re running tests on the Windows Calculator and find that 1+1=2, 2+2=5, 3+3=6, 4+4=9, 5+5=10, and 6+6=13. Write a bug title and bug description that effectively describes this problem.
Title: Adding a pair of even numbers gives an answer that’s one too much.
Description:
Test Case: Simple addition
Setup Steps: Start Version 1.0 of Calculator
Repro Steps: Try adding pairs of even number such as 2+2, 4+4, and 10+10. Also try adding pairs of odd numbers such as 3+3, 5+5, and 13+13 and pairs of mixed odd and even numbers such as 1+2, 5+6, and 12+13.
Expected Result: Correct answer for all pairs—2+2=4, 4+4=8…
Actual Result: For pairs of even numbers, the answer is one too high—2+2=5, 4+4=9, 10+10=21, and so on.
Other Info: This wasn’t tried exhaustively, but the bug occurred in many instances from 2+2 to 65536. The bug doesn’t seem to occur with odd numbers or mixed pairs.
5. What severity and priority would you give to a misspelling in a company’s logo on the software’s start-up screen?
Probably Severity 3 (minor次要 problem), Priority 2 (must fix before release).
6. What are the three basic states of a software bug’s life cycle and the two common additional states?
Open, Resolved, and Closed are the basic states.
Review and Deferred延期 are two possible other states.
7. List a few reasons that a database bug-tracking system is so much more useful than a paper-based system.
You can see at a glance what a bug’s life cycle has been—even if it has been complex.
The current status of a bug can be instantly known.
Bugs can’t be lost or neglected as easily.
Chapter 19 Measuring Your Success
1. If you were using metrics from the bug-tracking database to measure your progress or success at testing, why would just counting the number of bugs you find per day or computing your average find rate be an insufficient measure?
（如果您使用錯誤跟蹤數據庫中的指標來衡量測試的進度或成功程度，爲什麼僅計算每天發現的錯誤數量或計算平均發現率是不夠的？）
It doesn’t tell the entire story.
You could be testing the most complex area of the software.
Your area could have been written by the most experienced programmer. It could have been written by the least experienced programmer.
The code you’re testing may have already been tested or may be brand new.
2. Given your answer to question 1, list a few additional software metrics that could be used to measure more accurately and precisely your personal progress or success at testing.
Average number of bugs found per day.
Total bugs found so far.
Ratio of fixed bugs to all bugs found.
Ratio of Severity 1 or Priority 1 bugs to all bugs found.
Average time from the Resolved state to the Closed state.
3. What would a database query look like (any format you want) that would extract all the resolved bugs assigned to Terry for the Calc-U-Lot v3.0 project?
Product EQUALS Calc-U-Lot AND
Version EQUALS 3.0 AND
Status EQUALS Resolved AND
Assign TO EQUALS Terry
4. If the bug-find rate for a project was decreasing like the one shown in Figure 19.8 and everyone was excited that the project was getting close to releasing, what might be a couple reasons why this wouldn’t be true, that the numbers were lying?
- It’s possible that the software was released to testing in phases and not all the software was tested yet—it might only be leveling off for the current phase.
- The testers might be busy regressing and closing bugs and not looking for new ones. It could have been a very warm and sunny week or the testers might be out on vacation.