Credit Card Type Credit Card Number
American Express 378282246310005
________________________________________________________________________________
American Express 371449635398431
________________________________________________________________________________
American Express Corporate 378734493671000
________________________________________________________________________________
Australian BankCard 5610591081018250
________________________________________________________________________________
Diners Club 30569309025904
________________________________________________________________________________
Diners Club 38520000023237
________________________________________________________________________________
Discover 6011111111111117
________________________________________________________________________________
Discover 6011000990139424
_________________________________________________________________________________
JCB 3530111333300000
_________________________________________________________________________________
JCB 3566002020360505
_________________________________________________________________________________
MasterCard 5555555555554444
_________________________________________________________________________________
MasterCard 5105105105105100
_________________________________________________________________________________
Visa 4111111111111111
_________________________________________________________________________________
Visa 4012888888881881
_________________________________________________________________________________
Visa 4222222222222
_________________________________________________________________________________
Note : Even though this number has a different character count than the other test numbers, it is the correct and functional number.
Processor-specific Cards
Dankort (PBS) 76009244561
________________________________________________________________________________
Dankort (PBS) 5019717010103742
________________________________________________________________________________
Switch/Solo (Paymentech) 6331101999990016
__________________________________________________________________________________
Here is a list of dummy credit card number that can be used while testing your applications involving credit card transactions:
Visa: 4111-1111-1111-1111
_________________________________________________________________________________
MasterCard: 5431-1111-1111-1111
__________________________________________________________________________________
Amex: 341-1111-1111-1111
__________________________________________________________________________________
Discover: 6011-6011-6011-6611
_________________________________________________________________________________
Credit Card Prefix Numbers:
Visa: 13 or 16 numbers starting with 4
______________________________________________________
MasterCard: 16 numbers starting with 5
______________________________________________________
Discover: 16 numbers starting with 6011
______________________________________________________
AMEX: 15 numbers starting with 34 or 37
Friday, November 28, 2008
Saturday, October 18, 2008
ACID Testing
ACID stands for :
Atomicity
Consistency
Isolation
Durability
For OLTP databases and when databases deal with mission-critical business transactions and critical information ACID features are much need for reliability & stability. For customers that demand high quality of databases to maintain confidentiality of their information, ACID features is what you need.
Atomicity:-
Atomicity describes database operations that are defined within a single transactions of nature “all or nothing”. If any single operation fails then the whole transaction fails. This ensures that the database is in a valid state at all times.
For instance, a transaction to transfer money from one bank account to another involves withdrawal & deposit to two different account. If deposit fails, withdrawal should also be reversed, else money will be lost. So these operations should be grouped into single atomic transaction to ensure data integrity. Databases provide atomicity in two ways. 1. Rollback feature, 2. Two phase commit protocol.
Consistency:-
Consistency of data is maintained by allowing only valid transactions that comply to preset database constraints. Some constraints are:
1. Unique constraint on a column - For eg: Only one SSN can be allowed per person record. if a duplicate SSN appears then that transaction should be rejected.
2. Rules: Some rules check if data is in specified format. Eg: Salary should always be numeric and alphabets should be rejected.
Many other features of database help ensure data consistency.
Isolation:-
This concurrent transactions don't overlap each other. Every transaction occurs before or after all other transactions. If huge number of concurrent transactions are expected, then a robust database system is needed which has guaranteed isolation. Databases use Locking mechanism to implement Isolation. The row, table or entire database can be locked when a particular transaction takes place. Although this ensures isolation it poses issues with response times.
Durability:-
A committed or completed transaction should persist regardless of failures of any kind. Eg : Hard disk crash, network unavailability should not affect the saved data in database. Durability is often achieved via
1. Database replication or
2. Load balanced servers where two servers simultaneously run on separate machines. Backups alone cannot provide "durability" of all transactions.
Atomicity
Consistency
Isolation
Durability
For OLTP databases and when databases deal with mission-critical business transactions and critical information ACID features are much need for reliability & stability. For customers that demand high quality of databases to maintain confidentiality of their information, ACID features is what you need.
Atomicity:-
Atomicity describes database operations that are defined within a single transactions of nature “all or nothing”. If any single operation fails then the whole transaction fails. This ensures that the database is in a valid state at all times.
For instance, a transaction to transfer money from one bank account to another involves withdrawal & deposit to two different account. If deposit fails, withdrawal should also be reversed, else money will be lost. So these operations should be grouped into single atomic transaction to ensure data integrity. Databases provide atomicity in two ways. 1. Rollback feature, 2. Two phase commit protocol.
Consistency:-
Consistency of data is maintained by allowing only valid transactions that comply to preset database constraints. Some constraints are:
1. Unique constraint on a column - For eg: Only one SSN can be allowed per person record. if a duplicate SSN appears then that transaction should be rejected.
2. Rules: Some rules check if data is in specified format. Eg: Salary should always be numeric and alphabets should be rejected.
Many other features of database help ensure data consistency.
Isolation:-
This concurrent transactions don't overlap each other. Every transaction occurs before or after all other transactions. If huge number of concurrent transactions are expected, then a robust database system is needed which has guaranteed isolation. Databases use Locking mechanism to implement Isolation. The row, table or entire database can be locked when a particular transaction takes place. Although this ensures isolation it poses issues with response times.
Durability:-
A committed or completed transaction should persist regardless of failures of any kind. Eg : Hard disk crash, network unavailability should not affect the saved data in database. Durability is often achieved via
1. Database replication or
2. Load balanced servers where two servers simultaneously run on separate machines. Backups alone cannot provide "durability" of all transactions.
Monday, September 22, 2008
Linear Code Sequence And Jump (LCSAJ)
LCSAJ means A Linear Code Sequence And Jump, consisting of the following three items (conventionally identified by line numbers in a source code listing): the start of the linear sequence of executable statements, the end of the linear sequence, and the target line to which control flow is transferred at the end of the linear sequence.
LCSAJ testing is a test case design technique for a component in which test cases are designed to execute LCSAJs.
An LCSAJ is defined as unbroken linear sequence of statements:
(a) Which begins at either the start of the program or a point to which the control flow may jump,
(b) Which ends at either the end of the program or a point from which the control flow may jump?
(c) And the point to which a jump is made following the sequence.
LCSAJ coverage = I/L
Where: I = Number of LCSAJs exercised at least once.
L = Total number of LCSAJs.
LCSAJs depends on the topology of a module's design and not just its semantics, they do not map onto code structures such as branches and loops. LCSAJs are not easily identifiable from design documentation. They can only be identified once code has already been written. LCSAJs are consequently not easily comprehensible.
Definitions:
LCSAJ: A Linear Code Sequence And Jump, consisting of the following three items (conventionally identified by line numbers in a source code listing): the start of the linear sequence of executable statements, the end of the linear sequence, and the target line to which control flow is transferred at the end of the linear sequence.
LCSAJ coverage: The percentage of LCSAJs of a component which are exercised by a test case suite.
LCSAJ testing: A test case design technique for a component in which test cases are designed to execute LCSAJs.
LCSAJ testing is a test case design technique for a component in which test cases are designed to execute LCSAJs.
An LCSAJ is defined as unbroken linear sequence of statements:
(a) Which begins at either the start of the program or a point to which the control flow may jump,
(b) Which ends at either the end of the program or a point from which the control flow may jump?
(c) And the point to which a jump is made following the sequence.
LCSAJ coverage = I/L
Where: I = Number of LCSAJs exercised at least once.
L = Total number of LCSAJs.
LCSAJs depends on the topology of a module's design and not just its semantics, they do not map onto code structures such as branches and loops. LCSAJs are not easily identifiable from design documentation. They can only be identified once code has already been written. LCSAJs are consequently not easily comprehensible.
Definitions:
LCSAJ: A Linear Code Sequence And Jump, consisting of the following three items (conventionally identified by line numbers in a source code listing): the start of the linear sequence of executable statements, the end of the linear sequence, and the target line to which control flow is transferred at the end of the linear sequence.
LCSAJ coverage: The percentage of LCSAJs of a component which are exercised by a test case suite.
LCSAJ testing: A test case design technique for a component in which test cases are designed to execute LCSAJs.
Thursday, September 18, 2008
Traceability Matrix
Definition
A traceability matrix is a table that correlates any two base lined documents that require a many to many relationship to determine the completeness of the relationship. It is often used with high-level requirements (sometimes known as marketing requirements) and detailed requirements of the software product to the matching parts of high-level design, detailed design, test plan, and test cases.
Common usage is to take the identifier for each of the items of one document and place them in the left column. The identifiers for the other document are placed across the top row. When an item in the left column is related to an item across the top, a mark is placed in the intersecting cell. The numbers of relationships are added up for each row and each column. This value indicates the mapping of the two items. Zero values indicate that no relationship exists. It must be determined if one must be made. Large values imply that the item is too complex and should be simplified.
To ease the creation of traceability matrices, it is advisable to add the relationships to the source documents for both backward traceability and forward traceability. In other words, when an item is changed in one base lined document, it's easy to see what needs to be changed in the other.
Description
A table that traces the requirements to the system deliverable component for that stage that responds to the requirement.
Size and Format
For each requirement, identify the component in the current stage that responds to the requirement. The requirement may be mapped to such items as a hardware component, an application unit, or a section of a design specification.
Requirements of Traceability Matrix
Traceability matrices can be established using a variety of tools including requirements management software, databases, spreadsheets, or even with tables or hyperlinks in a word processor.
A traceability matrix is created by associating requirements with the work products that satisfy them. Tests are associated with the requirements on which they are based and the product tested to meet the requirement.
Above is a simple traceability matrix structure. There can be more things included in a traceability matrix than shown. In traceability, the relationship of driver to satisfier can be one-to-one, one-to-many, many-to-one, or many-to-many.
Traceability requires unique identifiers for each requirement and product. Numbers for products are established in a configuration management (CM) plan.
Traceability ensures completeness, that all lower level requirements come from higher level requirements, and that all higher level requirements are allocated to lower level requirements. Traceability is also used to manage change and provides the basis for test planning.
Baseline Traceability Matrix
Description
A table that documents the requirements of the system for use in subsequent stages to confirm that all requirements have been met.
Size and Format
Document each requirement to be traced. The requirement may be mapped to such things as a hardware component, an application unit, or a section of a design specification.
Building a Traceability Matrix
Use a Traceability Matrix to:
• verify and validate system specifications
• ensure that all final deliverable documents are included in the system specification, such as process models and data models
• improve the quality of a system by identifying requirements that are not addressed by configuration items during design and code reviews and by identifying extra configuration items that are not required. Examples of configuration items are software modules and hardware devices.
• provide input to change requests and future project plans when missing requirements are identified.
• provide a guide for system and acceptance test plans of what needs to be tested.
Useful Traceability Matrices
Various traceability matrices may be utilized throughout the system life cycle. Useful ones include:
• Functional specification to requirements document: It shows that each requirement (obtained from a preliminary requirements statement provided by the customer or produced in the Concept Definition stage) has been covered in an appropriate section of the functional specification.
• Top level configuration item to functional specification: For example, a top level configuration item, Workstation, may be one of the configuration items that satisfies the function Input Order Information. On the matrix, each configuration item would be written down the left hand column and each function would be written across the top.
• Low level configuration item to top level configuration item: For example, the top level configuration item, Workstation, may contain the low level configuration items Monitor, CPU, keyboard, and network interface card.
• Design specification to functional specification verifies that each function has been covered in the design.
• System test plan to functional specification ensures you have identified a test case or test scenario for each process and each requirement in the functional specification.
Although the construction and maintenance of traceability matrices may be time-consuming, they are a quick reference during verification and validation tasks.
Sample Traceability Matrix
A traceability matrix is a report from the requirements database or repository. What information the report contains depends on your need. Information requirements determine the associated information that you store with the requirements. Requirements management tools capture associated information or provide the capability to add it.
The examples show forward and backward tracing between user and system requirements. User requirement identifiers begin with "U" and system requirements with "S." Tracing S12 to its source makes it clear this requirement is erroneous: it must be eliminated, rewritten, or the traceability corrected.
For requirements tracing and resulting reports to work, the requirements must be of good quality. Requirements of poor quality transfer work to subsequent phases of the SDLC, increasing cost and schedule and creating disputes with the customer.
A variety of reports are necessary to manage requirements. Reporting needs should be determined at the start of the effort and documented in the requirements management plan.
A traceability matrix is a table that correlates any two base lined documents that require a many to many relationship to determine the completeness of the relationship. It is often used with high-level requirements (sometimes known as marketing requirements) and detailed requirements of the software product to the matching parts of high-level design, detailed design, test plan, and test cases.
Common usage is to take the identifier for each of the items of one document and place them in the left column. The identifiers for the other document are placed across the top row. When an item in the left column is related to an item across the top, a mark is placed in the intersecting cell. The numbers of relationships are added up for each row and each column. This value indicates the mapping of the two items. Zero values indicate that no relationship exists. It must be determined if one must be made. Large values imply that the item is too complex and should be simplified.
To ease the creation of traceability matrices, it is advisable to add the relationships to the source documents for both backward traceability and forward traceability. In other words, when an item is changed in one base lined document, it's easy to see what needs to be changed in the other.
Description
A table that traces the requirements to the system deliverable component for that stage that responds to the requirement.
Size and Format
For each requirement, identify the component in the current stage that responds to the requirement. The requirement may be mapped to such items as a hardware component, an application unit, or a section of a design specification.
Requirements of Traceability Matrix
Traceability matrices can be established using a variety of tools including requirements management software, databases, spreadsheets, or even with tables or hyperlinks in a word processor.
A traceability matrix is created by associating requirements with the work products that satisfy them. Tests are associated with the requirements on which they are based and the product tested to meet the requirement.
Above is a simple traceability matrix structure. There can be more things included in a traceability matrix than shown. In traceability, the relationship of driver to satisfier can be one-to-one, one-to-many, many-to-one, or many-to-many.
Traceability requires unique identifiers for each requirement and product. Numbers for products are established in a configuration management (CM) plan.
Traceability ensures completeness, that all lower level requirements come from higher level requirements, and that all higher level requirements are allocated to lower level requirements. Traceability is also used to manage change and provides the basis for test planning.
Baseline Traceability Matrix
Description
A table that documents the requirements of the system for use in subsequent stages to confirm that all requirements have been met.
Size and Format
Document each requirement to be traced. The requirement may be mapped to such things as a hardware component, an application unit, or a section of a design specification.
Building a Traceability Matrix
Use a Traceability Matrix to:
• verify and validate system specifications
• ensure that all final deliverable documents are included in the system specification, such as process models and data models
• improve the quality of a system by identifying requirements that are not addressed by configuration items during design and code reviews and by identifying extra configuration items that are not required. Examples of configuration items are software modules and hardware devices.
• provide input to change requests and future project plans when missing requirements are identified.
• provide a guide for system and acceptance test plans of what needs to be tested.
Useful Traceability Matrices
Various traceability matrices may be utilized throughout the system life cycle. Useful ones include:
• Functional specification to requirements document: It shows that each requirement (obtained from a preliminary requirements statement provided by the customer or produced in the Concept Definition stage) has been covered in an appropriate section of the functional specification.
• Top level configuration item to functional specification: For example, a top level configuration item, Workstation, may be one of the configuration items that satisfies the function Input Order Information. On the matrix, each configuration item would be written down the left hand column and each function would be written across the top.
• Low level configuration item to top level configuration item: For example, the top level configuration item, Workstation, may contain the low level configuration items Monitor, CPU, keyboard, and network interface card.
• Design specification to functional specification verifies that each function has been covered in the design.
• System test plan to functional specification ensures you have identified a test case or test scenario for each process and each requirement in the functional specification.
Although the construction and maintenance of traceability matrices may be time-consuming, they are a quick reference during verification and validation tasks.
Sample Traceability Matrix
A traceability matrix is a report from the requirements database or repository. What information the report contains depends on your need. Information requirements determine the associated information that you store with the requirements. Requirements management tools capture associated information or provide the capability to add it.
The examples show forward and backward tracing between user and system requirements. User requirement identifiers begin with "U" and system requirements with "S." Tracing S12 to its source makes it clear this requirement is erroneous: it must be eliminated, rewritten, or the traceability corrected.
For requirements tracing and resulting reports to work, the requirements must be of good quality. Requirements of poor quality transfer work to subsequent phases of the SDLC, increasing cost and schedule and creating disputes with the customer.
A variety of reports are necessary to manage requirements. Reporting needs should be determined at the start of the effort and documented in the requirements management plan.
Wednesday, September 10, 2008
IEEE 829: Standard For Software Test Documentation
IEEE 829-1998, also known as the 829 Standard for Software Test Documentation, is an IEEE standard that specifies the form of a set of documents for use in eight defined stages of software testing, each stage potentially producing its own separate type of document. The standard specifies the format of these documents but does not stipulate whether they all must be produced, nor does it include any criteria regarding adequate content for these documents. These are a matter of judgments outside the purview of the standard. The documents are:
• Test Plan: a management planning document that shows:
• How the testing will be done (including SUT configurations).
• Who will do it
• What will be tested
• How long it will take (although this may vary, depending upon resource availability).
• What the test coverage will be, i.e. what quality level is required
• Test Design Specification: detailing test conditions and the expected results as well as test pass criteria.
• Test Case Specification: specifying the test data for use in running the test conditions identified in the Test Design Specification
• Test Procedure Specification: detailing how to run each test, including any set-up preconditions and the steps that need to be followed
• Test Item Transmittal Report: reporting on when tested software components have progressed from one stage of testing to the next
• Test Log: recording which tests cases were run, who ran them, in what order, and whether each test passed or failed
• Test Incident Report: detailing, for any test that failed, the actual versus expected result, and other information intended to throw light on why a test has failed. This document is deliberately named as an incident report, and not a fault report. The reason is that a discrepancy between expected and actual results can occur for a number of reasons other than a fault in the system. These include the expected results being wrong, the test being run wrongly, or inconsistency in the requirements meaning that more than one interpretation could be made. The report consists of all details of the incident such as actual and expected results, when it failed, and any supporting evidence that will help in its resolution. The report will also include, if possible, an assessment of the impact upon testing of an incident.
• Test Summary Report: A management report providing any important information uncovered by the tests accomplished, and including assessments of the quality of the testing effort, the quality of the software system under test, and statistics derived from Incident Reports. The report also records what testing was done and how long it took, in order to improve any future test planning. This final document is used to indicate whether the software system under test is fit for purpose according to whether or not it has met acceptance criteria defined by project stakeholders.
Relationship with other standards
Other standards that may be referred to when documenting according to IEEE 829 include:
• IEEE 1008, a standard for unit testing
• IEEE 1012, a standard for Software Verification and Validation
• IEEE 1028, a standard for software inspections
• IEEE 1044, a standard for the classification of software anomalies
• IEEE 1044-1, a guide to the classification of software anomalies
• IEEE 1233, a guide for developing system requirements specifications
• IEEE 730, a standard for software quality assurance plans
• IEEE 1061, a standard for software quality metrics and methodology
• IEEE 12207, a standard for software life cycle processes and life cycle data
• BSS 7925-1, a vocabulary of terms used in software testing
• BSS 7925-2, a standard for software component testing
Use of IEEE 829
The standard forms part of the training syllabus of the ISEB Foundation and Practitioner Certificates in Software Testing promoted by the British Computer Society. ISTQB, following the formation of its own syllabus based on ISEB's and Germany's ASQF syllabi, also adopted IEEE 829 as the reference standard for software testing documentation.
• Test Plan: a management planning document that shows:
• How the testing will be done (including SUT configurations).
• Who will do it
• What will be tested
• How long it will take (although this may vary, depending upon resource availability).
• What the test coverage will be, i.e. what quality level is required
• Test Design Specification: detailing test conditions and the expected results as well as test pass criteria.
• Test Case Specification: specifying the test data for use in running the test conditions identified in the Test Design Specification
• Test Procedure Specification: detailing how to run each test, including any set-up preconditions and the steps that need to be followed
• Test Item Transmittal Report: reporting on when tested software components have progressed from one stage of testing to the next
• Test Log: recording which tests cases were run, who ran them, in what order, and whether each test passed or failed
• Test Incident Report: detailing, for any test that failed, the actual versus expected result, and other information intended to throw light on why a test has failed. This document is deliberately named as an incident report, and not a fault report. The reason is that a discrepancy between expected and actual results can occur for a number of reasons other than a fault in the system. These include the expected results being wrong, the test being run wrongly, or inconsistency in the requirements meaning that more than one interpretation could be made. The report consists of all details of the incident such as actual and expected results, when it failed, and any supporting evidence that will help in its resolution. The report will also include, if possible, an assessment of the impact upon testing of an incident.
• Test Summary Report: A management report providing any important information uncovered by the tests accomplished, and including assessments of the quality of the testing effort, the quality of the software system under test, and statistics derived from Incident Reports. The report also records what testing was done and how long it took, in order to improve any future test planning. This final document is used to indicate whether the software system under test is fit for purpose according to whether or not it has met acceptance criteria defined by project stakeholders.
Relationship with other standards
Other standards that may be referred to when documenting according to IEEE 829 include:
• IEEE 1008, a standard for unit testing
• IEEE 1012, a standard for Software Verification and Validation
• IEEE 1028, a standard for software inspections
• IEEE 1044, a standard for the classification of software anomalies
• IEEE 1044-1, a guide to the classification of software anomalies
• IEEE 1233, a guide for developing system requirements specifications
• IEEE 730, a standard for software quality assurance plans
• IEEE 1061, a standard for software quality metrics and methodology
• IEEE 12207, a standard for software life cycle processes and life cycle data
• BSS 7925-1, a vocabulary of terms used in software testing
• BSS 7925-2, a standard for software component testing
Use of IEEE 829
The standard forms part of the training syllabus of the ISEB Foundation and Practitioner Certificates in Software Testing promoted by the British Computer Society. ISTQB, following the formation of its own syllabus based on ISEB's and Germany's ASQF syllabi, also adopted IEEE 829 as the reference standard for software testing documentation.
Monday, August 11, 2008
Bug Tracking
-Locating and repairing software bugs is an essential part of software development.
-Bugs can be detected and reported by engineers, testers, and end-users in all phases of the testing process.
-Information about bugs must be detailed and organized in order to schedule bug fixes and determine software release dates.
Bug Tracking involves two main stages: reporting and tracking.
1.Report Bugs
Once you execute the manual and automated tests in a cycle, you report the bugs (or defects) that you detected. The bugs are stored in a database so that you can manage them and analyze the status of your application.
When you report a bug, you record all the information necessary to reproduce and fix it. You also make sure that the QA and development personnel involved in fixing the bug are notified.
2.Track and Analyze Bugs
The lifecycle of a bug begins when it is reported and ends when it is fixed, verified, and closed.
-First you report New bugs to the database, and provide all necessary information to reproduce, fix, and follow up the bug.
-The Quality Assurance manager or Project manager periodically reviews all New bugs and decides which should be fixed. These bugs are given the status Open and are assigned to a member of the development team.
-Software developers fix the Open bugs and assign them the status Fixed.
-QA personnel test a new build of the application. If a bug does not reoccur, it is Closed. If a bug is detected again, it is reopened.
Communication is an essential part of bug tracking; all members of the development and quality assurance team must be well informed in order to insure that bugs information is up to date and that the most important problems are addressed.
The number of open or fixed bugs is a good indicator of the quality status of your application. You can use data analysis tools such as re-ports and graphs in interpret bug data.
-Bugs can be detected and reported by engineers, testers, and end-users in all phases of the testing process.
-Information about bugs must be detailed and organized in order to schedule bug fixes and determine software release dates.
Bug Tracking involves two main stages: reporting and tracking.
1.Report Bugs
Once you execute the manual and automated tests in a cycle, you report the bugs (or defects) that you detected. The bugs are stored in a database so that you can manage them and analyze the status of your application.
When you report a bug, you record all the information necessary to reproduce and fix it. You also make sure that the QA and development personnel involved in fixing the bug are notified.
2.Track and Analyze Bugs
The lifecycle of a bug begins when it is reported and ends when it is fixed, verified, and closed.
-First you report New bugs to the database, and provide all necessary information to reproduce, fix, and follow up the bug.
-The Quality Assurance manager or Project manager periodically reviews all New bugs and decides which should be fixed. These bugs are given the status Open and are assigned to a member of the development team.
-Software developers fix the Open bugs and assign them the status Fixed.
-QA personnel test a new build of the application. If a bug does not reoccur, it is Closed. If a bug is detected again, it is reopened.
Communication is an essential part of bug tracking; all members of the development and quality assurance team must be well informed in order to insure that bugs information is up to date and that the most important problems are addressed.
The number of open or fixed bugs is a good indicator of the quality status of your application. You can use data analysis tools such as re-ports and graphs in interpret bug data.
Change Request
1. Initiating a Change Request
A user or developer wants to suggest a modification that would improve an existing application, notices a problem with an application, or wants to recommend an enhancement. Any major or minor request is considered a problem with an application and will be entered as a change request.
2.Type of Change Request
Bug the application works incorrectly or provides incorrect information. (for example, a letter is allowed to be entered in a number field)
Change a modification of the existing application. (for example, sorting the files alphabetically by the second field rather than numerically by the first field makes them easier to find)
Enhancement new functionality or item added to the application. (for example, a new report, a new field, or a new button).
3. Priority for the request
Low the application works but this would make the function easier or more user friendly.
High the application works, but this is necessary to perform a job.
Critical the application does not work, job functions are impaired and there is no work around. This also applies to any Section 508 infraction.
A user or developer wants to suggest a modification that would improve an existing application, notices a problem with an application, or wants to recommend an enhancement. Any major or minor request is considered a problem with an application and will be entered as a change request.
2.Type of Change Request
Bug the application works incorrectly or provides incorrect information. (for example, a letter is allowed to be entered in a number field)
Change a modification of the existing application. (for example, sorting the files alphabetically by the second field rather than numerically by the first field makes them easier to find)
Enhancement new functionality or item added to the application. (for example, a new report, a new field, or a new button).
3. Priority for the request
Low the application works but this would make the function easier or more user friendly.
High the application works, but this is necessary to perform a job.
Critical the application does not work, job functions are impaired and there is no work around. This also applies to any Section 508 infraction.
Test Execution
Test Execution is the heart of the testing process. Each time your application changes, you will want to execute the relevant parts of your test plan in order to locate defects and assess quality.
1.Create Test Cycles
During this stage you decide the subset of tests from your test database you want to execute.
Usually you do not run all the tests at once. At different stages of the quality assurance process, you need to execute different tests in order to address specific goals. A related group of tests is called a test cycle, and can include both manual and automated tests
Example: You can create a cycle containing basic tests that run on each build of the application throughout development. You can run the cycle each time a new build is ready, to determine the application's stability before beginning more rigorous testing.
Example: You can create another set of tests for a particular module in your application. This test cycle includes tests that check that module in depth.
To decide which test cycles to build, refer to the testing goals you defined at the beginning of the process. Also consider issues such as the current state of the application and whether new functions have been added or modified.
Following are examples of some general categories of test cycles to consider:
• sanity cycle checks the entire system at a basic level (breadth, rather than depth) to see that it is functional and stable. This cycle should include basic-level tests containing mostly positive checks.
• normal cycle tests the system a little more in depth than the sanity cycle. This cycle can group medium-level tests, containing both positive and negative checks.
• advanced cycle tests both breadth and depth. This cycle can be run when more time is available for testing. The tests in the cycle cover the entire application (breadth), and also test advanced options in the application (depth).
• regression cycle tests maintenance builds. The goal of this type of cycle is to verify that a change to one part of the software did not break the rest of the application. A regression cycle includes sanity-level tests for testing the entire software, as well as in-depth tests for the specific area of the application that was modified.
2. Run Test Cycles (Automated & Manual Tests)
Once you have created cycles that cover your testing objectives, you begin executing the tests in the cycle. You perform manual tests using the test steps. Testing Tools executes automated tests for you. A test cycle is complete only when all tests-automatic and manual-have been run.
With Manual Test Execution you follow the instructions in the test steps of each test. You use the application, enter input, compare the application output with the expected output, and log the results. For each test step you assign either pass or fail status.
During Automated Test Execution you create a batch of tests and launch the entire batch at once. Testing Tools runs the tests one at a time. It then imports results, providing outcome summaries for each test.
3.Analyze Test Results
After every test run one analyze and validate the test results. And have to identify all the failed steps in the tests and to determine whether a bug has been detected, or if the expected result needs to be updated.
1.Create Test Cycles
During this stage you decide the subset of tests from your test database you want to execute.
Usually you do not run all the tests at once. At different stages of the quality assurance process, you need to execute different tests in order to address specific goals. A related group of tests is called a test cycle, and can include both manual and automated tests
Example: You can create a cycle containing basic tests that run on each build of the application throughout development. You can run the cycle each time a new build is ready, to determine the application's stability before beginning more rigorous testing.
Example: You can create another set of tests for a particular module in your application. This test cycle includes tests that check that module in depth.
To decide which test cycles to build, refer to the testing goals you defined at the beginning of the process. Also consider issues such as the current state of the application and whether new functions have been added or modified.
Following are examples of some general categories of test cycles to consider:
• sanity cycle checks the entire system at a basic level (breadth, rather than depth) to see that it is functional and stable. This cycle should include basic-level tests containing mostly positive checks.
• normal cycle tests the system a little more in depth than the sanity cycle. This cycle can group medium-level tests, containing both positive and negative checks.
• advanced cycle tests both breadth and depth. This cycle can be run when more time is available for testing. The tests in the cycle cover the entire application (breadth), and also test advanced options in the application (depth).
• regression cycle tests maintenance builds. The goal of this type of cycle is to verify that a change to one part of the software did not break the rest of the application. A regression cycle includes sanity-level tests for testing the entire software, as well as in-depth tests for the specific area of the application that was modified.
2. Run Test Cycles (Automated & Manual Tests)
Once you have created cycles that cover your testing objectives, you begin executing the tests in the cycle. You perform manual tests using the test steps. Testing Tools executes automated tests for you. A test cycle is complete only when all tests-automatic and manual-have been run.
With Manual Test Execution you follow the instructions in the test steps of each test. You use the application, enter input, compare the application output with the expected output, and log the results. For each test step you assign either pass or fail status.
During Automated Test Execution you create a batch of tests and launch the entire batch at once. Testing Tools runs the tests one at a time. It then imports results, providing outcome summaries for each test.
3.Analyze Test Results
After every test run one analyze and validate the test results. And have to identify all the failed steps in the tests and to determine whether a bug has been detected, or if the expected result needs to be updated.
Tuesday, July 15, 2008
Heuristics of Software Testing
Testability
Software testability is how easily, completely and conveniently a computer program can be tested.
Software engineers design a computer product, system or program keeping in mind the product testability. Good programmers are willing to do things that will help the testing process and a checklist of possible design points, features and so on can be useful in negotiating with them.
Here are the two main heuristics of software testing.
1. Visibility
2. Control
1.Visibility
Visibility is our ability to observe the states and outputs of the software under test. Features to improve the visibility are
• Access to Code
Developers must provide full access (source code, infrastructure, etc) to testers. The Code, change records and design documents should be provided to the testing team. The testing team should read and understand the code.
• Event logging
The events to log include User events, System milestones, Error handling and completed transactions. The logs may be stored in files, ring buffers in memory, and/or serial ports. Things to be logged include description of event, timestamp, subsystem, resource usage and severity of event. Logging should be adjusted by subsystem and type. Log file report internal errors, help in isolating defects, and give useful information about context, tests, customer usage and test coverage.
The more readable the Log Reports are, the easier it becomes to identify the defect cause and work towards corrective measures.
• Error detection mechanisms
Data integrity checking and System level error detection (e.g. Microsoft Appviewer) are useful here. In addition, Assertions and probes with the following features are really helpful
Code is added to detect internal errors.
Assertions abort on error.
Probes log errors.
Design by Contract theory---This technique requires that assertions be defined for functions. Preconditions apply to input and violations implicate calling functions while post-conditions apply to outputs and violations implicate called functions. This effectively solves the oracle problem for testing.
• Resource Monitoring
Memory usage should be monitored to find memory leaks. States of running methods, threads or processes should be watched (Profiling interfaces may be used for this.). In addition, the configuration values should be dumped. Resource monitoring is of particular concern in applications where the load on the application in real time is estimated to be considerable.
2.Control
Control refers to our ability to provide inputs and reach states in the software under test.
The features to improve controllability are:
• Test Points
Allow data to be inspected, inserted or modified at points in the software. It is especially useful for dataflow applications. In addition, a pipe and filters architecture provides many opportunities for test points.
• Custom User Interface controls
Custom UI controls often raise serious testability problems with GUI test drivers. Ensuring testability usually requires:
Adding methods to report necessary information
Customizing test tools to make use of these methods
Getting a tool expert to advise developers on testability and to build the required support.
Asking third party control vendors regarding support by test tools.
• Test Interfaces
Interfaces may be provided specifically for testing e.g. Excel and Xconq etc.
Existing interfaces may be able to support significant testing e.g. InstallSheild, AutoCAD, Tivoli, etc.
• Fault injection
Error seeding---instrumenting low level I/O code to simulate errors---makes it much easier to test error handling. It can be handled at both system and application level, Tivoli, etc.
• Installation and setup
Testers should be notified when installation has completed successfully. They should be able to verify installation, programmatically create sample records and run multiple clients, daemons or servers on a single machine.
A BROADER VIEW
Below are given a broader set of characteristics (usually known as James Bach heuristics) that lead to testable software.
Categories of Heuristics of software testing
•Operability
The better it works, the more efficiently it can be tested.
The system should have few bugs, no bugs should block the execution of tests and the product should evolve in functional stages (simultaneous development and testing).
•Observability
What we see is what we test.
Distinct output should be generated for each input
Current and past system states and variables should be visible during testing
All factors affecting the output should be visible.
Incorrect output should be easily identified.
Source code should be easily accessible.
Internal errors should be automatically detected (through self-testing mechanisms) and reported.
•Controllability
The better we control the software, the more the testing process can be automated and optimized.
Check that
All outputs can be generated and code can be executed through some combination of input.
Software and hardware states can be controlled directly by the test engineer.
Inputs and output formats are consistent and structured.
Test can be conveniently, specified, automated and reproduced.
•Decomposability
By controlling the scope of testing, we can quickly isolate problems and perform effective and efficient testing.
The software system should be built from independent modules which can be tested independently.
•Simplicity
The less there is to test, the more quickly we can test it.
The points to consider in this regard are functional (e.g. minimum set of features), structural (e.g. architecture is modularized) and code (e.g. a coding standard is adopted) simplicity.
•Stability
The fewer the changes, the fewer are the disruptions to testing.
The changes to software should be infrequent, controlled and not invalidating existing tests. The software should be able to recover well from failures.
•Understandability
The more information we will have, the smarter we will test.
The testers should be able to understand well the design, changes to the design and the dependencies between internal, external and shared components.
Technical documentation should be instantly accessible, accurate, well organized, specific and detailed.
•Suitability
The more we know about the intended use of the software, the better we can organize our testing to find important bugs.
The above heuristics can be used by a software engineer to develop a software configuration (i.e. program, data and documentation) that is convenient to test and verify.
Software testability is how easily, completely and conveniently a computer program can be tested.
Software engineers design a computer product, system or program keeping in mind the product testability. Good programmers are willing to do things that will help the testing process and a checklist of possible design points, features and so on can be useful in negotiating with them.
Here are the two main heuristics of software testing.
1. Visibility
2. Control
1.Visibility
Visibility is our ability to observe the states and outputs of the software under test. Features to improve the visibility are
• Access to Code
Developers must provide full access (source code, infrastructure, etc) to testers. The Code, change records and design documents should be provided to the testing team. The testing team should read and understand the code.
• Event logging
The events to log include User events, System milestones, Error handling and completed transactions. The logs may be stored in files, ring buffers in memory, and/or serial ports. Things to be logged include description of event, timestamp, subsystem, resource usage and severity of event. Logging should be adjusted by subsystem and type. Log file report internal errors, help in isolating defects, and give useful information about context, tests, customer usage and test coverage.
The more readable the Log Reports are, the easier it becomes to identify the defect cause and work towards corrective measures.
• Error detection mechanisms
Data integrity checking and System level error detection (e.g. Microsoft Appviewer) are useful here. In addition, Assertions and probes with the following features are really helpful
Code is added to detect internal errors.
Assertions abort on error.
Probes log errors.
Design by Contract theory---This technique requires that assertions be defined for functions. Preconditions apply to input and violations implicate calling functions while post-conditions apply to outputs and violations implicate called functions. This effectively solves the oracle problem for testing.
• Resource Monitoring
Memory usage should be monitored to find memory leaks. States of running methods, threads or processes should be watched (Profiling interfaces may be used for this.). In addition, the configuration values should be dumped. Resource monitoring is of particular concern in applications where the load on the application in real time is estimated to be considerable.
2.Control
Control refers to our ability to provide inputs and reach states in the software under test.
The features to improve controllability are:
• Test Points
Allow data to be inspected, inserted or modified at points in the software. It is especially useful for dataflow applications. In addition, a pipe and filters architecture provides many opportunities for test points.
• Custom User Interface controls
Custom UI controls often raise serious testability problems with GUI test drivers. Ensuring testability usually requires:
Adding methods to report necessary information
Customizing test tools to make use of these methods
Getting a tool expert to advise developers on testability and to build the required support.
Asking third party control vendors regarding support by test tools.
• Test Interfaces
Interfaces may be provided specifically for testing e.g. Excel and Xconq etc.
Existing interfaces may be able to support significant testing e.g. InstallSheild, AutoCAD, Tivoli, etc.
• Fault injection
Error seeding---instrumenting low level I/O code to simulate errors---makes it much easier to test error handling. It can be handled at both system and application level, Tivoli, etc.
• Installation and setup
Testers should be notified when installation has completed successfully. They should be able to verify installation, programmatically create sample records and run multiple clients, daemons or servers on a single machine.
A BROADER VIEW
Below are given a broader set of characteristics (usually known as James Bach heuristics) that lead to testable software.
Categories of Heuristics of software testing
•Operability
The better it works, the more efficiently it can be tested.
The system should have few bugs, no bugs should block the execution of tests and the product should evolve in functional stages (simultaneous development and testing).
•Observability
What we see is what we test.
Distinct output should be generated for each input
Current and past system states and variables should be visible during testing
All factors affecting the output should be visible.
Incorrect output should be easily identified.
Source code should be easily accessible.
Internal errors should be automatically detected (through self-testing mechanisms) and reported.
•Controllability
The better we control the software, the more the testing process can be automated and optimized.
Check that
All outputs can be generated and code can be executed through some combination of input.
Software and hardware states can be controlled directly by the test engineer.
Inputs and output formats are consistent and structured.
Test can be conveniently, specified, automated and reproduced.
•Decomposability
By controlling the scope of testing, we can quickly isolate problems and perform effective and efficient testing.
The software system should be built from independent modules which can be tested independently.
•Simplicity
The less there is to test, the more quickly we can test it.
The points to consider in this regard are functional (e.g. minimum set of features), structural (e.g. architecture is modularized) and code (e.g. a coding standard is adopted) simplicity.
•Stability
The fewer the changes, the fewer are the disruptions to testing.
The changes to software should be infrequent, controlled and not invalidating existing tests. The software should be able to recover well from failures.
•Understandability
The more information we will have, the smarter we will test.
The testers should be able to understand well the design, changes to the design and the dependencies between internal, external and shared components.
Technical documentation should be instantly accessible, accurate, well organized, specific and detailed.
•Suitability
The more we know about the intended use of the software, the better we can organize our testing to find important bugs.
The above heuristics can be used by a software engineer to develop a software configuration (i.e. program, data and documentation) that is convenient to test and verify.
Monday, July 14, 2008
Testing artifacts
Software testing process can produce several artifacts.
Test case
A test case is a software testing document, which consists of event, action, input, output, expected result, and actual result. Clinically defined a test case is an input and an expected result.[25] This can be as pragmatic as 'for condition x your derived result is y', whereas other test cases described in more detail the input scenario and what results might be expected. It can occasionally be a series of steps (but often steps are contained in a separate test procedure that can be exercised against multiple test cases, as a matter of economy) but with one expected result or expected outcome. The optional fields are a test case ID, test step or order of execution number, related requirement(s), depth, test category, author, and check boxes for whether the test is automatable and has been automated. Larger test cases may also contain prerequisite states or steps, and descriptions. A test case should also contain a place for the actual result. These steps can be stored in a word processor document, spreadsheet, database, or other common repository. In a database system, you may also be able to see past test results and who generated the results and the system configuration used to generate those results. These past results would usually be stored in a separate table.
Test script
The test script is the combination of a test case, test procedure, and test data. Initially the term was derived from the product of work created by automated regression test tools. Today, test scripts can be manual, automated, or a combination of both.
Test suite
The most common term for a collection of test cases is a test suite. The test suite often also contains more detailed instructions or goals for each collection of test cases. It definitely contains a section where the tester identifies the system configuration used during testing. A group of test cases may also contain prerequisite states or steps, and descriptions of the following tests.
Test plan
A test specification is called a test plan. The developers are well aware what test plans will be executed and this information is made available to the developers. This makes the developers more cautious when developing their code. This ensures that the developers code is not passed through any surprise test case or test plans.
Test harness
The software, tools, samples of data input and output, and configurations are all referred to collectively as a test harness.
In software testing, a test harness or automated test framework is a collection of software and test data configured to test a program unit by running it under varying conditions and monitor its behavior and outputs. It has two main parts: the test execution engine and the test script repository.
Test harnesses allow for the automation of tests. They can call functions with supplied parameters and print out and compare the results to the desired value. The test harness is a hook to the developed code, which can be tested using an automation framework.
A test harness should allow specific tests to run (this helps in optimising), orchestrate a runtime environment, and provide a capability to analyse results.
The typical objectives of a test harness are to:
* Automate the testing process.
* Execute test suites of test cases.
* Generate associated test reports.
A test harness typically provides the following benefits:
* Increased productivity due to automation of the testing process.
* Increased probability that regression testing will occur.
* Increased quality of software components and application.
Test case
A test case is a software testing document, which consists of event, action, input, output, expected result, and actual result. Clinically defined a test case is an input and an expected result.[25] This can be as pragmatic as 'for condition x your derived result is y', whereas other test cases described in more detail the input scenario and what results might be expected. It can occasionally be a series of steps (but often steps are contained in a separate test procedure that can be exercised against multiple test cases, as a matter of economy) but with one expected result or expected outcome. The optional fields are a test case ID, test step or order of execution number, related requirement(s), depth, test category, author, and check boxes for whether the test is automatable and has been automated. Larger test cases may also contain prerequisite states or steps, and descriptions. A test case should also contain a place for the actual result. These steps can be stored in a word processor document, spreadsheet, database, or other common repository. In a database system, you may also be able to see past test results and who generated the results and the system configuration used to generate those results. These past results would usually be stored in a separate table.
Test script
The test script is the combination of a test case, test procedure, and test data. Initially the term was derived from the product of work created by automated regression test tools. Today, test scripts can be manual, automated, or a combination of both.
Test suite
The most common term for a collection of test cases is a test suite. The test suite often also contains more detailed instructions or goals for each collection of test cases. It definitely contains a section where the tester identifies the system configuration used during testing. A group of test cases may also contain prerequisite states or steps, and descriptions of the following tests.
Test plan
A test specification is called a test plan. The developers are well aware what test plans will be executed and this information is made available to the developers. This makes the developers more cautious when developing their code. This ensures that the developers code is not passed through any surprise test case or test plans.
Test harness
The software, tools, samples of data input and output, and configurations are all referred to collectively as a test harness.
In software testing, a test harness or automated test framework is a collection of software and test data configured to test a program unit by running it under varying conditions and monitor its behavior and outputs. It has two main parts: the test execution engine and the test script repository.
Test harnesses allow for the automation of tests. They can call functions with supplied parameters and print out and compare the results to the desired value. The test harness is a hook to the developed code, which can be tested using an automation framework.
A test harness should allow specific tests to run (this helps in optimising), orchestrate a runtime environment, and provide a capability to analyse results.
The typical objectives of a test harness are to:
* Automate the testing process.
* Execute test suites of test cases.
* Generate associated test reports.
A test harness typically provides the following benefits:
* Increased productivity due to automation of the testing process.
* Increased probability that regression testing will occur.
* Increased quality of software components and application.
Subscribe to:
Posts (Atom)
