自動化測試的重要概念介紹
The Benefits of Automation and Tools
Regression testing 迴歸測試
目的：check that the bugs you found in previous test runs were
indeed fixed and that no new bugs were introduced
This process of rerunning your tests is known as regression testing.
The principal attributes of tools and automation are
Speed
Think about how long it would take you to manually try a few thousand test cases for the Windows Calculator. You might average a test case every 5 seconds or so.
Automation might be able to run 10, 100, even 1000 times that fast.
Efficiency
While you’re busy running test cases, you can’t be doing anything else. If you have a test tool that reduces the time it takes for you to run your tests, you have more time for test planning and thinking up new tests.
Accuracy and Precision. After trying a few hundred cases, your attention span will
wane and you’ll start to make mistakes. A test tool will perform the same test and check the results perfectly, each and every time.
Relentlessness. Test tools and automation never tire or give up. They’re like that
battery-operated bunny of the TV commercials—they can keep going and going and.…
Reminder
Software test tools aren’t a substitute for software testers—they just help software testers perform their jobs better.
Test Tools
分類：non-invasive and invasive
Non-invasive
If a tool is used only to monitor and examine the software without modifying it, it’s considered non-invasive.
Invasive
If, however, the tool modifies the program code or manipulates the operating environment in any way, it’s invasive.
There are varying degrees of invasiveness and testers usually try to use tools that are as non-invasive as possible to reduce the possibility that their tools are affecting the test results.
Viewers and Monitors
A viewer or monitor test tool allows you to see details of the software’s operation that you wouldn’t normally be able to see.
A code coverage analyzer is an example of a viewing tool.
Most code coverage analyzers are invasive tools because they need to be compiled and linked into the program to access the information they provide.
The code debuggers that come with most compilers are also considered viewers because they allow programmers or white-box testers to view internal variable values and program states.
Drivers
Drivers are tools used to control and operate the software being tested.
One of the simplest examples of a driver is a batch file, a simple list of programs or commands that are executed sequentially.
Stubs 樁
Stubs are essentially the opposite of drivers in that they don’t control or operate the software being tested; they instead receive or respond to data that the software sends.
When to use
Stubs are frequently used when software needs to communicate with external devices.
Stub vs Emulator
emulator used to describe a device that’s a plug-in replacement for the real device.
The difference between an emulator and a stub is that the stub also provides a means for a tester to view and interpret the data sent to it.
Stress and Load Tools
Interference Injectors and Noise Generators
Analysis Tools
a best-of-the-rest group
Screen capture and comparison software
• Debugger
• Binary-hex calculator 二進制轉十六進制
• Stopwatch
Software Test Automation
Macro Recording and Playback 宏的錄製和回放
什麼是宏
可編程的宏 Programmed Macros
Fully Programmable Automated Testing Tools
Verification
Verification is the last big hurdle to overcome with automated software testing. Once you have
that, you can take nearly any test case and create automation that will make trying that case
either much easier or completely automatic.
Test Varification vs Tset Oracle
Tset Oracle: 準則
Random Testing: Monkeys and Gorillas
The term test monkey comes from the idea that if you had a million monkeys typing on a million keyboards for a million years, statistically, they might eventually write a Shakespearean play, Curious George, or some other great work. “測試猴子”這個詞來源於這樣一個想法：如果你有一百萬只猴子在一百萬年的鍵盤上打字，從統計上看，他們最終可能會寫一部莎士比亞的戲劇，“好奇” 喬治，或者其他偉大的作品。
Dumb/Semi-Smart/Smart Monkey
Realities of Using Test Tools and Automation
- The software changes.
- There’s no substitute for the human eye and intuition.
- Verification is hard to do.
- It’s easy to rely on automation too much.
- Don’t spend so much time working on tools and automation that you fail to test the software.
- If you’re writing macros, developing a tool, or programming a monkey, you’re doing development work.
- Some tools are invasive and can cause the software being tested to improperly fail.