Term Paper on "Software Testing"

>The objective of this work is to provide an overview of software testing as well as the need for testing and how testing fits into the software development lifecycle. The research component will be how to formulate a software testing strategy prior to deploying and release a software product.

Introduction

Software testing is a critical aspect of software product development. In fact testing of software consumes approximately 50% of the total time spent in development of software. Software testing holds an inevitable place in the product lifecycle.

I. Software Testing Overview

According to a work of Carnegie Mellon University, Jiantao Pan (1999) in the document entitled: "Software Testing" the testing of software is an activity aimed at the evaluation of "an attribute or capability of a program or system and determining that it meets its required results." Software testing is not only critical for quality issues related to software but as well, "software testing still remains an art..." (Pan, 1999) the reason that is true is due to the limited comprehension held by most of the principles of software. Testing, according to Pan (1999) is much more than a simple debug process but as well testing is for the purpose of: "...quality assurance, verification and validation, or reliability estimation." (Pan, 1999)

Two major areas of software testing are:

1) correctness testing; and 2) reliability testing. (Pan, 1999)

Pan states that software testing in actuality represents a "trade-off between budget, time and quality." (Pan, 1999) Defects that are found in software are generally caused by errors in design instead of errors in the manufacturing process. Pan (1999) states that software:

Does not suffer from corrosion or wear-and-tear;

Generally does not change until upgrades are released or it becomes obsolete;

Bugs are often found existing in practically any "software module with moderate size" and this is stated to be due to the "complexity of the software [being] generally intractable" added to the fact that human beings possess ability only in limited amounts to "manage complexity." (Pan, 1999)

Complete testing is ideal however, it has not been feasible to testing completely and it is further complicated by the "dynamic nature of programs." (Pan, 1999) Software could be ensured better without the complexity however, as pointed out in the work of Pan: "Society seems to be unwilling to limit complexity because we all want that extra bell, whistle and feature interaction. Thus, our users always push us to the complexity barrier and how close we can approach that barrier is largely determined by the strength of the techniques we can wield against more complex and subtle bugs." (Beizer, 1990; as cited in Pan, 1999) While limitations certainly exist: "testing is an integral part of software development t. It is broadly deployed in the very phase in the software development cycle." (Pan, 1999) in fact, more than 1/2 of time spent in product development is spent in testing.

In the software product development initiative quality may be understood to mean: "...the conformance to the specified design requirement." (Pan, 1999)

Testing plays a key role in the 'Verification and Validation' phase of software development in that testing can: "...make claims based on interpretations of the testing results...[and] it is also possible to "compare the quality among different products under the same specification, based on results from the same test." (Pan, 1999) While quality cannot be tested in a direct manner, related factors may be tested in order to "...make quality visible." (Pan, 1999)

Quality contains three sets of factors which are those of:

1) functionality;

2) engineering; and 3) adaptability." (Pan, 19999)

The chart below illustrates the quality considerations, which are cited most often.

Typical Software Quality Factors

Functionality (exterior quality)

Engineering (interior quality)

Adaptability (future quality)

Correctness

Efficiency

Flexibility

Reliability

Testability

Reusability

Usability

Documentation

Maintainability

Integrity

Structure

Source: Pan (1999)

When tests are conducted for the express reason of validation then the "product works are named clean tests, or positive tests." (Pan, 1999) Pan points out the fact that software testing is still in its' infancy and that software testing is still "an art." (1999) While testing software is a costly pursuit, failure to test software could result in astronomical costs. Various and sundry methods and techniques of testing software exist and all of these serve "multiple purposes in different life cycle." (Pan, 1999)

II. Purpose of Software Testing Categories

Categorized according to purpose of testing software may be grouped into:

1) Correctness testing;

2) Performance testing;

3) Reliability testing; and 4) Security testing. (Pan, 1999)

III. Life-cycle Phase Testing Categories

When grouped according to the life-cycle phase testing of software may be categorized as:

1) Requirement phase testing;

2) Design phase testing;

3) Program phase testing;

4) Evaluating test results;

5) Component testing;

6) Integration testing; and 7) System testing." (Pan, 1999)

The following is a list with accompanying descriptions of each:

Correctness testing - the minimal requirement of testing software;

Black-box testing - a method of testing whereby derivation of data is from "the specified functional requirements without regard to the final program structure." (Pan, 1999) This is also known as 'functional testing' because the focus is 'functionality'. (Pan, 1999) This functionality is sought through the method of maximizes testing effectiveness yet with minimal costs.

Blackbox testing has also been referred to as a) "Data-driven, b) Input/output drive, or Requirements-based testing.

White-box testing' or also referred to as a "glass-box" testing since the "structure and flow of the software" undergoing testing is clearly visible. This type of testing is also referred to as:

a) glass-box testing;

b) logic-driven testing; or design-based testing." (Pan, 1999)

Performance-Testing' - tests for problems in design that causes degradation in the software. Performance testing includes:

a) Resource usage;

b) Throughput;

Stimulus-response time; and d) Queue lengths, which detail the "average of maximum number of tasks waiting to be serviced by selected resources."(Pan, 1999)

Reliability-testing' - just as it sounds referring to "the probability of failure-free operation of a system." (Pan, 1999)

Two types of reliability testing are:

a) Robustness testing - Robustness refers to the degree to which a component can correctly function while receiving exceptional inputs or while under conditions of a stressful environment. (Pan, 1999) and b) Stress-testing - Robustness refers to the degree to which a component can correctly function while receiving exceptional inputs or while under conditions of a stressful environment. (Pan, 1999) Stress testing is also referred to as "load testing" and is generally the method used in whole-system testing instead of only testing of software. Stress is inclusive of:

1) resource exhaustion;

2) burst of activities; and 3) sustained high loads."

Security testing looks for software flaws that open the door to malicious attacks. Pan (1999) states that "software quality, reliability, and security are tightly coupled." Software applications and services generally have security measures that have been integrated against intruders. Security testing of these software systems have the purpose of: (a) identification and removal of flaws in software that might open the way to violations of security; and (b) Validation of effectiveness of measures of security. 'Automation of testing' is used in cutting on time and cost. (Pan, 1999; paraphrased)

IV. Software Testing Tools

Testing as already stated is "a trade-off between budget, time and quality and the drivers of software testing is that of 'profit models.' A great supply of software testing tools are in existence including:

Mothara - an automated mutations testing tool-set which was a Purdue University development and is one in which the "tester can create and execute test cases, measure test adequacy, determine input-output correctness, locate and remove faults or bugs, and control and document the test." (Pan, 1999)

MuMega's Boundschecker and… READ MORE