Smoke Testing: Software Testing done to ensure that whether the build can be accepted for through software testing or not. Basically, it is done to check the stability of the build received for software testing.
In computer programming and software testing, smoke testing is a preliminary to further testing, which should reveal simple failures severe enough to reject a prospective software release. In this case, the smoke is metaphorical.
Smoke testing is done by developers before the build is released to the testers, or by testers before accepting a build for further testing. Microsoft claims[1] that after code reviews, smoke testing is the most cost effective method for identifying and fixing defects in software.
n software engineering, a smoke test generally consists of a collection of tests that can be applied to a newly created or repaired computer program. Sometimes the tests are performed by the automated system that builds the final software. In this sense a smoke test is the process of validating code changes before the changes are checked into the larger product’s official source code collection or the main branch of source code.
In software testing, a smoke test is a collection of written tests that are performed on a system prior to being accepted for further testing. This is also known as a build verification test. This is a "shallow and wide" approach to the application. The tester "touches" all areas of the application without getting too deep, looking for answers to basic questions like, "Can I launch the test item at all?", "Does it open to a window?", "Do the buttons on the window do things?".
The purpose is to determine whether or not the application is so badly broken that testing functionality in a more detailed way is unnecessary. These written tests can either be performed manually or using an automated tool. When automated tools are used, the tests are often initiated by the same process that generates the build itself.
Sanity testing: After receiving a build with minor changes in the code or functionality, a subset of regression test cases are executed that to check whether it rectified the software bugs or issues and no other software bug is introduced by the changes. Sometimes, when multiple cycles of regression testing are executed, sanity testing of the software can be done at later cycles after through regression test cycles. If we are moving a build from staging / testing server to production server, sanity testing of the software application can be done to check that whether the build is sane enough to move to further at production server or not.
In computer science, a sanity test is a very brief run-through of the functionality of a computer program, system, calculation, or other analysis, to assure that the system or methodology works as expected, often prior to a more exhaustive round of testing.
In software development, the sanity test (a form of software testing which offers "quick, broad, and shallow testing"[1]) determines whether it is reasonable to proceed with further testing.
Software sanity tests are commonly conflated with smoke tests.A smoke test determines whether it is possible to continue testing, as opposed to whether it is reasonable.
A software smoke test determines whether the program launches and whether its interfaces are accessible and responsive (for example, the responsiveness of a web page or an input button). If the smoke test fails, it is impossible to conduct a sanity test. In contrast, the ideal sanity test exercises the smallest subset of application functions needed to determine whether the application logic is generally functional and correct (for example, an interest rate calculation for a financial application). If the sanity test fails, it is not reasonable to attempt more rigorous testing. Both sanity tests and smoke tests are ways to avoid wasting time and effort by quickly determining whether an application is too flawed to merit any rigorous testing. Many companies run sanity tests and unit tests on an automated build as part of their development process.
The Hello world program is often used as a sanity test for a development environment. If Hello World fails to compile or execute, the supporting environment likely has a configuration problem. If it works, the problem being diagnosed likely lies in the real application being diagnosed.
Difference between Smoke & Sanity Software Testing:
* Smoke testing is a wide approach where all areas of the software application are tested without getting into too deep. However, a sanity software testing is a narrow regression testing with a focus on one or a small set of areas of functionality of the software application.
* The test cases for smoke testing of the software can be either manual or automated. However, a sanity test is generally without test scripts or test cases.
* Smoke testing is done to ensure whether the main functions of the software application are working or not. During smoke testing of the software, we do not go into finer details. However, sanity testing is a cursory software testing type. It is done whenever a quick round of software testing can prove that the software application is functioning according to business / functional requirements.
* Smoke testing of the software application is done to check whether the build can be accepted for through software testing. Sanity testing of the software is to ensure whether the requirements are met or not.
Wednesday, March 30, 2011
Wednesday, January 12, 2011
The Most Significant Changes in HP Quality Center 10
The Most Significant Changes
The new QC Version 10 has seen a series of changes, of which the most significant ones are:
• Version control
• Baselining
• Integrated dashboard
• Shared libraries
• Cross Project Customization
Version Control
Finally, QC has been provided with a fully integrated version control. In past versions, one had to make do with third-party integration, which generally stumbled rather than ran. The version control, if desired, must first be activated for every individual project via the SiteAdmin. Once this is done, all entities falling under the version control become Version 1. QC entities include requirements, tests, test resources, and business components. If one wishes to add a test step to a test case for example, one is automatically requested to check this test case. As soon as there is more than one version of an entity, one can compare various versions against each other or retrieve an older version.
Summary: The whole thing is intuitive and easy to operate. But what’s the use of having eight versions of a requirement, when one doesn’t know which software release a version belongs to?
Baselining
Along with version control comes baselining, which is intended to answer the question above. Using the new “Management” module that replaces the “Releases” introduced in 9.x one gets to the baseline function via the “Libraries” tab. This enables one to obtain a summary of a complete testing release and retrieve it if necessary. In this way, a test set can be pinned to a baseline in the test lab. In other words, as of now, the manual copying of entire trees into the test plan module is a thing of the past. At last, test managers will be able to properly organize software that has multiple parallel releases (in production, current release, future release).
Summary: Setting up the baseline works very well. While creating it, a log keeps one updated on what is currently taking place. However, setting up baselines probably needs to be done during off-peak hours since it can take a while for larger QC projects.
Integrated Dashboard
Equally interesting for test managers is the new integrated dashboard that can be found on the left-hand navigation bar where the “Management” module is, too. The special feature of the new dashboard does not pertain to the graphics, which are not particularly appealing, but the “Cross-Project” functionality. It is now finally possible, when working on a QC project, to get an overview of all of one’s ongoing projects. These dashboards are freely configurable and can be designated to be personal favorites or publicly accessible. Special Excel reports also enable direct access to the database via SQL. The generated reports can then be graphically processed at the same time by means of VBScript.
Summary: The new dashboard module is quickly customized and achieves its purpose in ongoing projects. What is missing is a sensible way of printing content for a given project meeting, for example.
Shared Libraries
Libraries, which are located in the “Management” module, can be re-used and distributed with Version 10. A library represents a collection of entities in a QC project, including their relationships to each other. When dealing with many similar projects, it offers the advantage of not having to repeatedly create entities. Libraries can be imported from project A into project B, compared against each other, or even synchronized. A library also allows one to collect the same entities as in versioning. Defects are not included, but they can be shared with the new “HP Quality Center Synchronizer” manually among several QC projects. As mentioned, the advantages really only present themselves when one has many and/or large-scale projects. I suppose that is why this functionality is available only in the QC Premier Edition (also available are the Standard and Enterprise editions).
Summary: It remains to be seen whether this function will actually be used in real-world applications. In my opinion, it makes perfect sense to be able to take over pre-defined assets from another project so that one doesn’t have to keep re-inventing the wheel.
Cross Project Customization
And now here’s the last big change: Cross Project Customization. Many organizations have defined standards, such as a uniform defect status field or a standardized priority scale, for their software quality-related areas. However, these fields and lists were often changed or even deleted by QC project administrators. Some companies have even gone to great lengths in using their own programming to define a template that can be distributed to all QC projects, thereby establishing a uniform standard.
For all those who want to spare themselves this time and effort or do not wish to keep an in-house programmed interface going, there is a solution. Site Administrator now provides a way to link projects with a template. If the template is changed, the delta can then be passed on later at the right point in time. This function has been awaited not only by test managers who like to have the same configuration in all their projects, but especially also by the respective operators of QC installations, namely the system administrators. Cross Project Customization is also only available in the Premier Edition.
Summary: This change is awesome! Finally, testing organizations or Test Factory managers are able to implement a certain basic standard in their projects. Once this is accomplished, nothing can get in the way of standardized, across-the-board reporting.
Is an Upgrade Worth It?
The new Version 10 is an absolute milestone not only for tests managers. It also makes life easier for testers, Test Factory managers, as well as QC system administrators. The new functions have been anticipated for quite some time and have been implemented in the new product in a well-conceived manner.
The new QC Version 10 has seen a series of changes, of which the most significant ones are:
• Version control
• Baselining
• Integrated dashboard
• Shared libraries
• Cross Project Customization
Version Control
Finally, QC has been provided with a fully integrated version control. In past versions, one had to make do with third-party integration, which generally stumbled rather than ran. The version control, if desired, must first be activated for every individual project via the SiteAdmin. Once this is done, all entities falling under the version control become Version 1. QC entities include requirements, tests, test resources, and business components. If one wishes to add a test step to a test case for example, one is automatically requested to check this test case. As soon as there is more than one version of an entity, one can compare various versions against each other or retrieve an older version.
Summary: The whole thing is intuitive and easy to operate. But what’s the use of having eight versions of a requirement, when one doesn’t know which software release a version belongs to?
Baselining
Along with version control comes baselining, which is intended to answer the question above. Using the new “Management” module that replaces the “Releases” introduced in 9.x one gets to the baseline function via the “Libraries” tab. This enables one to obtain a summary of a complete testing release and retrieve it if necessary. In this way, a test set can be pinned to a baseline in the test lab. In other words, as of now, the manual copying of entire trees into the test plan module is a thing of the past. At last, test managers will be able to properly organize software that has multiple parallel releases (in production, current release, future release).
Summary: Setting up the baseline works very well. While creating it, a log keeps one updated on what is currently taking place. However, setting up baselines probably needs to be done during off-peak hours since it can take a while for larger QC projects.
Integrated Dashboard
Equally interesting for test managers is the new integrated dashboard that can be found on the left-hand navigation bar where the “Management” module is, too. The special feature of the new dashboard does not pertain to the graphics, which are not particularly appealing, but the “Cross-Project” functionality. It is now finally possible, when working on a QC project, to get an overview of all of one’s ongoing projects. These dashboards are freely configurable and can be designated to be personal favorites or publicly accessible. Special Excel reports also enable direct access to the database via SQL. The generated reports can then be graphically processed at the same time by means of VBScript.
Summary: The new dashboard module is quickly customized and achieves its purpose in ongoing projects. What is missing is a sensible way of printing content for a given project meeting, for example.
Shared Libraries
Libraries, which are located in the “Management” module, can be re-used and distributed with Version 10. A library represents a collection of entities in a QC project, including their relationships to each other. When dealing with many similar projects, it offers the advantage of not having to repeatedly create entities. Libraries can be imported from project A into project B, compared against each other, or even synchronized. A library also allows one to collect the same entities as in versioning. Defects are not included, but they can be shared with the new “HP Quality Center Synchronizer” manually among several QC projects. As mentioned, the advantages really only present themselves when one has many and/or large-scale projects. I suppose that is why this functionality is available only in the QC Premier Edition (also available are the Standard and Enterprise editions).
Summary: It remains to be seen whether this function will actually be used in real-world applications. In my opinion, it makes perfect sense to be able to take over pre-defined assets from another project so that one doesn’t have to keep re-inventing the wheel.
Cross Project Customization
And now here’s the last big change: Cross Project Customization. Many organizations have defined standards, such as a uniform defect status field or a standardized priority scale, for their software quality-related areas. However, these fields and lists were often changed or even deleted by QC project administrators. Some companies have even gone to great lengths in using their own programming to define a template that can be distributed to all QC projects, thereby establishing a uniform standard.
For all those who want to spare themselves this time and effort or do not wish to keep an in-house programmed interface going, there is a solution. Site Administrator now provides a way to link projects with a template. If the template is changed, the delta can then be passed on later at the right point in time. This function has been awaited not only by test managers who like to have the same configuration in all their projects, but especially also by the respective operators of QC installations, namely the system administrators. Cross Project Customization is also only available in the Premier Edition.
Summary: This change is awesome! Finally, testing organizations or Test Factory managers are able to implement a certain basic standard in their projects. Once this is accomplished, nothing can get in the way of standardized, across-the-board reporting.
Is an Upgrade Worth It?
The new Version 10 is an absolute milestone not only for tests managers. It also makes life easier for testers, Test Factory managers, as well as QC system administrators. The new functions have been anticipated for quite some time and have been implemented in the new product in a well-conceived manner.
Subscribe to:
Posts (Atom)
