Smoke testing is the initial testing process to check whether the software under test is ready/stable for further testing. The term ‘Smoke Testing’ is came from the hardware testing; in the hardware testing initial pass is done to check if it did not catch the fire or smoked in the initial switch on. In this testing few test cases are created initially. These test cases are executed to identify crucial functionalities or to check critical functionalities of the program is working fine without bothering details. The objective is not to perform exhaustive testing, the tester need check the navigation’s & adding simple things, tester needs to ask simple questions “Can tester able to access software application?”, “Does user navigates from one window to other?”, “Check that the GUI is responsive” etc. Smoke testing is effectively applied to build testing where in code that is changed is validated before checking in that change into product’s configuration library. In other terms, here smoke testing is restricted only to test and validate against changed code but not for entire product that is built from configuration library that includes even changed code. Like this, smoke tests are designed to confirm that changes in the code are implemented and work as expected and also, change does not destabilize an entire build. Every file is compiled, linked, and combined into an executable program every day, and the program is then put through a “smoke test,” a relatively simple check to see whether the product “smokes” when it runs.
Advantages of Smoke Testing:
- It helps to find issues introduced in integration of modules.
- It helps to find issues in the early phase of testing.
- It helps to get confidence to tester that fixes in the previous builds not breaking major features (off course, only features exercised by smoke testing).
- It minimizes integration risk.
- It reduces the risk of low quality.
- It supports easier defect diagnosis.
- It improves morale.
- Check for broken builds.
- Compile all files, libraries, and other components successfully.
- Link all files, libraries, and other components successfully.
- Not contain any showstopper bugs that prevent the program from being launched or that make it hazardous to operate.
Smoke test daily. The smoke test should exercise the entire system from end to end. It does not have to be exhaustive, but it should be capable of exposing major problems. The smoke test should be thorough enough that if the build passes, you can assume that it is stable enough to be tested more thoroughly.
The smoke test must evolve as the system evolves. At first, the smoke test will probably test something simple, such as whether the system can say, “Hello, World.” As the system develops, the smoke test will become more thorough. The first test might take a matter of seconds to run; as the system grows, the smoke test can grow to 30 minutes, an hour, or more.
Effective testing means better test coverage and more number of reported errors. This demands more number of test suites. However it is difficult to measure accuracy of test suites. The method of Mutation Testing was introduced to measure the accuracy of test suites.
In mutation testing, we perform following activities:
Step 1: We consider a perfect program and its corresponding perfect test suite. This means, we select test suite that covers all possible and a program that passes this test suite.
Step 2: We change the code of the program. This activity is called mutating. The resulting program with this change is referred to as mutant.
Step 3: Execute test suite on mutant.
Step 4: Observe the result. If results of the program are affected by the change and test suite detects the change, then the mutant is called a ‘killed mutant’. If the results of the program do not change and the test suite does not detect the mutation, then the mutant is called an ‘equivalent mutant’.
Step 5: Continue creating more mutants and run tests using test suites.
Step 6: Count Total Number of Mutants; Total Number of Killed Mutants
Step 7: Calculate Ratio of Total Number of Killed Mutants to Total Number of Mutants. The value arrived at provides information about accuracy of the test suite
This means impromptu testing using test cases that have been developed with no support of project related documents. These techniques require creative thinking and many a times use knowledge gleaned in the past on programmers or product. Ad hoc testing is less comprehensive, often inventive requiring lesser time, effort and documentation.
Primary and Secondary Qualities
Requirements specifications is a ”representational” perceptual model that is built based on inner “ideas”, “impressions’ or “sense data” of an observer (requirements study team member) and his inferences. As such what is real and what is represented always differ. We can see that there will be differences between what is expressed and what is represented. If we find a way to directly establish a link between the observer’s inner world and external object, we can have better representation of requirements. However, major hurdle to achieve this is unreliability of our perceptions.
To address such an unreliability and thereby to reduce the gap between perceptual model of our inner ideas and outer objects , we need to understand that there are two types of qualities, namely, Primary Quality (Absolute Quality) and Secondary Quality (Relative Quality).
The color of the User Interface is not a property of the screen itself but a product of the interaction of various factors, including certain physical attributes of the screen such as power supply, resolution, the peculiarities of our own sensory system; and the environmental conditions prevailing at the time of the observation. All these properties do not belong to the screen as such but are extrinsic. Such properties are said to be “Secondary Qualities”. These qualities vary based on time and conditions and as such they define relative quality.
At the same time, a screen has certain true properties which are intrinsic, such as its size and shape, which do not depend on the conditions under which the screen is observed or on the existence of the viewer. These are defined as “Primary Qualities” of screen. Primary qualities also help us in explaining and also, developing an experience of the secondary qualities. Unlike secondary qualities, our ideas that we develop in our mind on primary qualities closely resemble the the physical object itself. Thus, primary qualities of physical objects define absolute quality.
We can extend these concepts of primary and secondary qualities to requirements specification. While capturing requirements always think on primary qualities of clients wants and needs. If you are able to identify such primary qualities, then you can have concrete requirements that are beyond skepticism. Such requirements which can be represented using primary qualities are implementable and measurable. For example, requirements like accuracy of numbers, length of any text field, number of permissible users, number of transactions that need to be supported by the system etc are primary qualities.
On the other hand, secondary qualities of requirements can not be concrete and as such they are the basis for skepticism. Secondary qualities can not be implemented to perfection and also, can not be measured. For example, requirements like system shall be user friendly, system shall have recover ability feature, user interfaces shall be pleasing etc are secondary qualities.
Thus, while arriving at requirements specification if we focus on primary qualities then our requirements will be concrete. Else requirements will be representation full of skepticism.