Notes on: The Fundamentals of Testing (Ch1)
Software testing is costly in human effort
The goal of testing is to be practical and cost effective
Errors are where major system failures usually being. These initial failures can come in the form of an incorrect thought or a wrong assumption. When an error is written down it becomes a defect of a 'fault'
A faulty component that is built into a system becomes a failure in the system
Testing is to avoid and rectify
Software failures can lead to
a. Loss of money
b. Loss of time
c. Loss of business reputation
d. Injury or death when there is high risk (i.e. aircraft carrying passengers)
Exhaustive testing of complex systems is not possible
The more risk, the more and better testing should be performed
The software development process must handle competing demands for resources
Resources triangle:
Testing is one component of quality assurance. Testing seeks to ensure that the system that enters service without defects that lead to serious failures
Notes continued:
Prioritisation: achieving acceptable results from a finite and limited amount of testing.
a. Prioritize the most important tests first
b. Most important functions as defined by the users
c. This will also address the most significant risk
Testing sets criteria of when its safe to stop testing, and how much level of defects can be tolerated in a delivered product
Testing is an activity that reduces risk and improves quality
Testing can be systematic (planned)
Testing can be methodical (a process)
Testing is an activity performed on the product (not just code)
Testing has a variety of techniques
More on Testing:
Testing explores the component or system with the main aim of finding and reporting defects
Testing does not include the correction of defects, these are passed on to the developers
Testers can also debug
Root cause - tries to answer why the problem came about in the first place
Quality assurance - ensures that the testing process is undertaken correctly
Quality control - checks the quality of something
Static testing - can involve reviews
Dynamic testing - exercises the program under test with some data
Testing Process:
Test set up > test execution > record test results
A good test finds a defect if there is one present
Testing shows the presence of defects, not the absence of defects
Exhausting testing is impossible
The earlier a problem (defect) is found, the less it costs to fix
Defects Cluster together:
System complexity
The effects of change on change
Development staff experience
Development staff inexperience
Pesticide paradox - running the same tests will not continue to find new defects
Test planning:
Answers what is going to be tested, how it will be achieved, and who will do them
This activity produces plans and schedules
Test monitoring and control:
Monitoring - what has been achieved and what is expected to be done at this point and time
Control - taking necessary action to meet the original or revised objectives as given in the test plan
Produces test reports which compares actual progress against the plan
Test analysis - what to test
Identifies testable features
Defines and identifies associated test conditions along with risk and risk levels
Test design - how to test
create test cases
prioritize test cases and sets of test cases
identify test data
designing test environment if necessary
Test Implementation
Do we have everything in place to run the test?
Creating and prioritizing test procedures
Creating automated test script
Test Execution
Running tests
Recording the identification and version of what is being tested
Comparing actual and expected results
Reporting defects based on failures
Recording results ( pass, fail, blocked)
Produces the status of individual tests
Test completion - test summary report (i.e. Test Rail summary report)
Traceability between test basis and test work products
Enables the evaluation of test coverage
Makes tests auditable
Relating testing to stakeholders
Improving understandability of test progress reports
** Defect reports need to be raised against the software and not the individuals