User Interface Testing Checklist
1. USER INTERFACE
1.1 COLORS
1.1.1 Are hyperlink colors standard?
1.1.2 Are the field backgrounds the correct color?
1.1.3 Are the field prompts the correct color?
1.1.4 Are the screen and field colors adjusted correctly for non-editable mode?
1.1.5 Does the site use (approximately) standard link colors?
1.1.6 Are all the buttons are in standard format and size?
1.1.7 Is the general screen background the correct color?
1.1.8 Is the page background (color) distraction free?
1.2 CONTENT
1.2.1 All fonts to be the same1.
2.2 Are all the screen prompts specified in the correct screen font?
1.2.3 Does content remain if you need to go back to a previous page, or if you move forward to another new page?
1.2.4 Is all text properly aligned?
1.2.5 Is the text in all fields specified in the correct screen font?
1.2.6 Is all the heading are left aligned
1.2.7 Does the first letter of the second word appears in lowercase? Eg:
1.3 IMAGES
1.3.1 Are all graphics properly aligned?
1.3.2 Are graphics being used the most efficient use of file size?
1.3.3 Are graphics optimized for quick downloads?
1.3.4 Assure that command buttons are all of similar size and shape, and same font & font size.
1.3.5 Banner style & size & display exact same as existing windows
1.3.6 Does text wrap properly around pictures/graphics?
1.3.7 Is it visually consistent even without graphics?
1.4 INSTRUCTIONS
1.4.1 Is all the error message text spelt correctly on this screen?
1.4.2 Is all the micro-help text(i.e tool tip) spelt correctly on this screen?
1.4.3 Microhelp text(i.e tool tip) for every enabled field & button
1.4.4 Progress messages on load of tabbed(active screens) screens
1.5 NAVIGATION
1.5.1 Are all disabled fields avoided in the TAB sequence?
1.5.2 Are all read-only fields avoided in the TAB sequence?
1.5.3 Can all screens accessible via buttons on this screen be accessed correctly?
1.5.4 Does a scrollbar appear if required?
1.5.5 Does the Tab Order specified on the screen go in sequence from Top Left to bottom right? This is the default unless otherwise specified.
1.5.6 Is there a link to home on every single page?
1.5.7 On open of tab focus will be on first editable field
1.5.8 When an error message occurs does the focus return to the field in error when the user cancels it?
1.6 USABILITY
1.6.1 Are all the field prompts spelt correctly?
1.6.2 Are fonts too large or too small to read?
1.6.3 Are names in command button & option box names are not abbreviations.
1.6.4 Assure that option boxes, option buttons, and command buttons are logically grouped together in clearly demarcated areas “Group Box”
1.6.5 Can the typical user run the system without frustration?
1.6.6 Do pages print legibly without cutting off text?
1.6.7 Does the site convey a clear sense of its intended audience?
1.6.8 Does the site have a consistent, clearly recognizable “look-&-feel”?
1.6.9 Does User cab Login Member Area with both UserName/Email ID ?
1.6.9 Does the site look good on 640 x 480, 600×800 etc.?
1.6.10 Does the system provide or facilitate customer service? i.e. responsive, helpful, accurate?
1.6.11 Is all terminology understandable for all of the site’s intended users?
Performance & Security Testing Checklist
1. PERFORMANCE
1.1 LOAD
1.1.1 Many users requesting a certain page at the same time or using the site simultaneously
1.1.2 Increase the number of users and keep the data constant
1.1.3 Does the home page load quickly? within 8 seconds
1.1.4 Is load time appropriate to content, even on a slow dial-in connection?
1.1.5 Can the site sustain long periods of usage by multiple users?
1.1.6 Can the site sustain long periods of continuous usage by 1 user?
1.1.7 Is page loading performance acceptable over modems of different speeds?
1.1.8 Does the system meet its goals for response time, throughput, and availability?
1.1.9 Have you defined standards for response time (i.e. all screens should paint within 10 seconds)?
1.1.10 Does the system operate in the same way across different computer and network configurations, platforms and environments, with different mixes of other applications?
1.2 VOLUME
1.2.1 Increase the data by having constant users
1.2.2 Will the site allow for large orders without locking out inventory if the transaction is invalid?
1.2.3 Can the site sustain large transactions without crashing?
1.3 STRESS
1.3.1 Increase both number of users and the data
1.3.2 Performance of memory, CPU, file handling etc.
1.3.3 Error in software, hardware, memory errors (leakage, overwrite or pointers)
1.3.4 Is the application or certain features going to be used only during certain periods of time or will it be used continuously 24 hours a day 7 days a week? Test that the application is able to perform during those conditions. Will downtime be allowed or is that out of the question?
1.3.5 Verify that the application is able to meet the requirements and does not run out of memory or disk space.
1.4 SECURITY
1.4.1 Is confidentiality/user privacy protected?
1.4.2 Does the site prompt for user name and password?
1.4.3 Are there Digital Certificates, both at server and client?
1.4.4 Have you verified where encryption begins and ends?
1.4.5 Are concurrent log-ons permitted?
1.4.6 Does the application include time-outs due to inactivity?
1.4.7 Is bookmarking disabled on secure pages?
1.4.8 Does the key/lock display on status bar for insecure/secure pages?
1.4.9 Is Right Click, View, Source disabled?
1.4.10 Are you prevented from doing direct searches by editing content in the URL?
1.4.11 If using Digital Certificates, test the browser Cache by enrolling for the Certificate and completing all of the required security information. After completing the application and installation of the certificate.
Monday, March 16, 2009
Tuesday, February 24, 2009
Few Recent Bugs
The technology world has seen many interesting bugs in the past few months, so I thought I would list about a few of my favorites and some lessons learned from each.
1.) Zune Leap Year Bug
On December 31, all 30GB Zune music players failed at the same time. The bug turned out to be a flaw in the date calculation code for leap years, but what made this bug interesting was that it happened on a December 31 and not on a February 29. James Whittaker blogged about the Zune date failure, and one of the conclusions he reaches is that testers need a better way to share tester knowledge and information.
See following link for more details:
http://blogs.msdn.com/james_whittaker/archive/2009/01/06/the-zune-issue.aspx
2.) Canon 5D Mark II Black Dot Bug
The Canon 5D Mark II is a high end SLR camera recently released by Canon for a market of professional photographers and very serious amateurs. It’s the second generation of the 5D - the previous model was considered one of the best digital SLRs on the market. Unfortunately, Canon’s newest camera suffered from a very serious bug - tiny black dots would appear next to highlights (very bright parts of the image). While the dots were small (only a few pixels each on a 21 megapixel camera), the problem was apparent for photographers looking to blow up their images or tightly crop a tiny portion.
Like many bugs, this one was discovered by early adopters. However, the bug was in the pre-release cameras as well, indicating that Canon’s engineers simply missed it. “Many eyes” often see small but important details that are missed during regular testing.
Canon recently released a firmware update fixing this problem.
See following link for more details:
http://forums.dpreview.com/forums/read.asp?forum=1032&message=30264865
3.) Security Flaws: Macrumors and Twitter
Two different sites, but two very similar and serious bugs. In the case of Macrumors, somebody hacked the live feed for their coverage of the MacWorld keynote. Right in the middle of the feed, somebody posing as the site authors posted a message saying that Steve Jobs had died. Macrumors is a very well respected Apple news site, so for a few minutes the world was stunned with the news before it became apparent that the news feed had been compromised. The culprit: an admin control panel that had no authentication.
Twitter had a similar issue - hackers were able to gain control of an admin panel and then hijack the accounts of a 33 celebrities including Barack Obama. Twitter has fixed the issue, but many are wondering if there are other serious issues with the site.
Both of these are very serious bugs, and both show that testing needs to be about security as much as anything else.
See following link for more details:
http://www.techcrunch.com/2009/01/06/when-livestreams-go-wrong/
http://www.techcrunch.com/2009/01/05/twitter-gets-hacked-badly/
1.) Zune Leap Year Bug
On December 31, all 30GB Zune music players failed at the same time. The bug turned out to be a flaw in the date calculation code for leap years, but what made this bug interesting was that it happened on a December 31 and not on a February 29. James Whittaker blogged about the Zune date failure, and one of the conclusions he reaches is that testers need a better way to share tester knowledge and information.
See following link for more details:
http://blogs.msdn.com/james_whittaker/archive/2009/01/06/the-zune-issue.aspx
2.) Canon 5D Mark II Black Dot Bug
The Canon 5D Mark II is a high end SLR camera recently released by Canon for a market of professional photographers and very serious amateurs. It’s the second generation of the 5D - the previous model was considered one of the best digital SLRs on the market. Unfortunately, Canon’s newest camera suffered from a very serious bug - tiny black dots would appear next to highlights (very bright parts of the image). While the dots were small (only a few pixels each on a 21 megapixel camera), the problem was apparent for photographers looking to blow up their images or tightly crop a tiny portion.
Like many bugs, this one was discovered by early adopters. However, the bug was in the pre-release cameras as well, indicating that Canon’s engineers simply missed it. “Many eyes” often see small but important details that are missed during regular testing.
Canon recently released a firmware update fixing this problem.
See following link for more details:
http://forums.dpreview.com/forums/read.asp?forum=1032&message=30264865
3.) Security Flaws: Macrumors and Twitter
Two different sites, but two very similar and serious bugs. In the case of Macrumors, somebody hacked the live feed for their coverage of the MacWorld keynote. Right in the middle of the feed, somebody posing as the site authors posted a message saying that Steve Jobs had died. Macrumors is a very well respected Apple news site, so for a few minutes the world was stunned with the news before it became apparent that the news feed had been compromised. The culprit: an admin control panel that had no authentication.
Twitter had a similar issue - hackers were able to gain control of an admin panel and then hijack the accounts of a 33 celebrities including Barack Obama. Twitter has fixed the issue, but many are wondering if there are other serious issues with the site.
Both of these are very serious bugs, and both show that testing needs to be about security as much as anything else.
See following link for more details:
http://www.techcrunch.com/2009/01/06/when-livestreams-go-wrong/
http://www.techcrunch.com/2009/01/05/twitter-gets-hacked-badly/
Wednesday, December 17, 2008
Threat Classification
Classes of Attack
Authentication
The Authentication section covers attacks that target a web site's method of validating the identity of a user, service or application. Authentication is performed using at least one of three mechanisms: "something you have", "something you know" or "something you are". This section will discuss the attacks used to circumvent or exploit the authentication process of a web site.
* Brute Force
A Brute Force attack is an automated process of trial and error used to guess a person's username, password, credit-card number or cryptographic key.
* Insufficient Authentication
Insufficient Authentication occurs when a web site permits an attacker to access sensitive content or functionality without having to properly authenticate.
* Weak Password Recovery Validation
Weak Password Recovery Validation is when a web site permits an attacker to illegally obtain, change or recover another user's password.
Authorization
The Authorization section covers attacks that target a web site's method of determining if a user, service, or application has the necessary permissions to perform a requested action. For example, many web sites should only allow certain users to access specific content or functionality. Other times a user's access to other resources might be restricted. Using various techniques, an attacker can fool a web site into increasing their privileges to protected areas.
* Credential/Session Prediction
Credential/Session Prediction is a method of hijacking or impersonating a web site user.
* Insufficient Authorization
Insufficient Authorization is when a web site permits access to sensitive content or functionality that should require increased access control restrictions.
* Insufficient Session Expiration
Insufficient Session Expiration is when a web site permits an attacker to reuse old session credentials or session IDs for authorization.
* Session Fixation
Session Fixation is an attack technique that forces a user's session ID to an explicit value.
Client-side Attacks
The Client-side Attacks section focuses on the abuse or exploitation of a web site's users. When a user visits a web site, trust is established between the two parties both technologically and psychologically. A user expects web sites they visit to deliver valid content. A user also expects the web site not to attack them during their stay. By leveraging these trust relationship expectations, an attacker may employ several techniques to exploit the user.
* Content Spoofing
Content Spoofing is an attack technique used to trick a user into believing that certain content appearing on a web site is legitimate and not from an external source.
* Cross-site Scripting
Cross-site Scripting (XSS) is an attack technique that forces a web site to echo attacker-supplied executable code, which loads in a user's browser.
Command Execution
The Command Execution section covers attacks designed to execute remote commands on the web site. All web sites utilize user-supplied input to fulfill requests. Often these user-supplied data are used to create construct commands resulting in dynamic web page content. If this process is done insecurely, an attacker could alter command execution.
* Buffer Overflow
Buffer Overflow exploits are attacks that alter the flow of an application by overwriting parts of memory.
* Format String Attack
Format String Attacks alter the flow of an application by using string formatting library features to access other memory space.
* LDAP Injection
LDAP Injection is an attack technique used to exploit web sites that construct LDAP statements from user-supplied input.
* OS Commanding
OS Commanding is an attack technique used to exploit web sites by executing Operating System commands through manipulation of application input.
* SQL Injection
SQL Injection is an attack technique used to exploit web sites that construct SQL statements from user-supplied input.
* SSI Injection
SSI Injection (Server-side Include) is a server-side exploit technique that allows an attacker to send code into a web application, which will later be executed locally by the web server.
* XPath Injection
XPath Injection is an attack technique used to exploit web sites that construct XPath queries from user-supplied input.
Information Disclosure
The Information Disclosure section covers attacks designed to acquire system specific information about a web site. System specific information includes the software distribution, version numbers, and patch levels. Or the information may contain the location of backup files and temporary files. In most cases, divulging this information is not required to fulfill the needs of the user. Most web sites will reveal a certain amount of data, but it's best to limit the amount of data whenever possible. The more information about the web site an attacker learns, the easier the system becomes to compromise.
* Directory Indexing
Automatic directory listing/indexing is a web server function that lists all of the files within a requested directory if the normal base file is not present.
* Information Leakage
Information Leakage is when a web site reveals sensitive data, such as developer comments or error messages, which may aid an attacker in exploiting the system.
* Path Traversal
The Path Traversal attack technique forces access to files, directories, and commands that potentially reside outside the web document root directory.
* Predictable Resource Location
Predictable Resource Location is an attack technique used to uncover hidden web site content and functionality.
Logical Attacks
The Logical Attacks section focuses on the abuse or exploitation of a web application's logic flow. Application logic is the expected procedural flow used in order to perform a certain action. Password recovery, account registration, auction bidding, and eCommerce purchases are all examples of application logic. A web site may require a user to correctly perform a specific multi-step process to complete a particular action. An attacker may be able to circumvent or misuse these features to harm a web site and its users.
* Abuse of Functionality
Abuse of Functionality is an attack technique that uses a web site's own features and functionality to consume, defraud, or circumvents access controls mechanisms.
* Denial of Service
Denial of Service (DoS) is an attack technique with the intent of preventing a web site from serving normal user activity.
* Insufficient Anti-automation
Insufficient Anti-automation is when a web site permits an attacker to automate a process that should only be performed manually.
* Insufficient Process Validation
Insufficient Process Validation is when a web site permits an attacker to bypass or circumvent the intended flow control of an application.
Authentication
The Authentication section covers attacks that target a web site's method of validating the identity of a user, service or application. Authentication is performed using at least one of three mechanisms: "something you have", "something you know" or "something you are". This section will discuss the attacks used to circumvent or exploit the authentication process of a web site.
* Brute Force
A Brute Force attack is an automated process of trial and error used to guess a person's username, password, credit-card number or cryptographic key.
* Insufficient Authentication
Insufficient Authentication occurs when a web site permits an attacker to access sensitive content or functionality without having to properly authenticate.
* Weak Password Recovery Validation
Weak Password Recovery Validation is when a web site permits an attacker to illegally obtain, change or recover another user's password.
Authorization
The Authorization section covers attacks that target a web site's method of determining if a user, service, or application has the necessary permissions to perform a requested action. For example, many web sites should only allow certain users to access specific content or functionality. Other times a user's access to other resources might be restricted. Using various techniques, an attacker can fool a web site into increasing their privileges to protected areas.
* Credential/Session Prediction
Credential/Session Prediction is a method of hijacking or impersonating a web site user.
* Insufficient Authorization
Insufficient Authorization is when a web site permits access to sensitive content or functionality that should require increased access control restrictions.
* Insufficient Session Expiration
Insufficient Session Expiration is when a web site permits an attacker to reuse old session credentials or session IDs for authorization.
* Session Fixation
Session Fixation is an attack technique that forces a user's session ID to an explicit value.
Client-side Attacks
The Client-side Attacks section focuses on the abuse or exploitation of a web site's users. When a user visits a web site, trust is established between the two parties both technologically and psychologically. A user expects web sites they visit to deliver valid content. A user also expects the web site not to attack them during their stay. By leveraging these trust relationship expectations, an attacker may employ several techniques to exploit the user.
* Content Spoofing
Content Spoofing is an attack technique used to trick a user into believing that certain content appearing on a web site is legitimate and not from an external source.
* Cross-site Scripting
Cross-site Scripting (XSS) is an attack technique that forces a web site to echo attacker-supplied executable code, which loads in a user's browser.
Command Execution
The Command Execution section covers attacks designed to execute remote commands on the web site. All web sites utilize user-supplied input to fulfill requests. Often these user-supplied data are used to create construct commands resulting in dynamic web page content. If this process is done insecurely, an attacker could alter command execution.
* Buffer Overflow
Buffer Overflow exploits are attacks that alter the flow of an application by overwriting parts of memory.
* Format String Attack
Format String Attacks alter the flow of an application by using string formatting library features to access other memory space.
* LDAP Injection
LDAP Injection is an attack technique used to exploit web sites that construct LDAP statements from user-supplied input.
* OS Commanding
OS Commanding is an attack technique used to exploit web sites by executing Operating System commands through manipulation of application input.
* SQL Injection
SQL Injection is an attack technique used to exploit web sites that construct SQL statements from user-supplied input.
* SSI Injection
SSI Injection (Server-side Include) is a server-side exploit technique that allows an attacker to send code into a web application, which will later be executed locally by the web server.
* XPath Injection
XPath Injection is an attack technique used to exploit web sites that construct XPath queries from user-supplied input.
Information Disclosure
The Information Disclosure section covers attacks designed to acquire system specific information about a web site. System specific information includes the software distribution, version numbers, and patch levels. Or the information may contain the location of backup files and temporary files. In most cases, divulging this information is not required to fulfill the needs of the user. Most web sites will reveal a certain amount of data, but it's best to limit the amount of data whenever possible. The more information about the web site an attacker learns, the easier the system becomes to compromise.
* Directory Indexing
Automatic directory listing/indexing is a web server function that lists all of the files within a requested directory if the normal base file is not present.
* Information Leakage
Information Leakage is when a web site reveals sensitive data, such as developer comments or error messages, which may aid an attacker in exploiting the system.
* Path Traversal
The Path Traversal attack technique forces access to files, directories, and commands that potentially reside outside the web document root directory.
* Predictable Resource Location
Predictable Resource Location is an attack technique used to uncover hidden web site content and functionality.
Logical Attacks
The Logical Attacks section focuses on the abuse or exploitation of a web application's logic flow. Application logic is the expected procedural flow used in order to perform a certain action. Password recovery, account registration, auction bidding, and eCommerce purchases are all examples of application logic. A web site may require a user to correctly perform a specific multi-step process to complete a particular action. An attacker may be able to circumvent or misuse these features to harm a web site and its users.
* Abuse of Functionality
Abuse of Functionality is an attack technique that uses a web site's own features and functionality to consume, defraud, or circumvents access controls mechanisms.
* Denial of Service
Denial of Service (DoS) is an attack technique with the intent of preventing a web site from serving normal user activity.
* Insufficient Anti-automation
Insufficient Anti-automation is when a web site permits an attacker to automate a process that should only be performed manually.
* Insufficient Process Validation
Insufficient Process Validation is when a web site permits an attacker to bypass or circumvent the intended flow control of an application.
Tuesday, December 9, 2008
SQL Injection
SQL Injection
SQL Injection is an attack technique used to exploit web sites that construct SQL statements from user-supplied input.
Wikipedia Definition:
SQL injection is a technique that exploits a security vulnerability occurring in the database layer of an application. The vulnerability is present when user input is either incorrectly filtered for string literal escape characters embedded in SQL statements or user input is not strongly typed and thereby unexpectedly executed. It is in fact an instance of a more general class of vulnerabilities that can occur whenever one programming or scripting language is embedded inside another.
Structured Query Language (SQL) is a specialized programming language for sending queries to databases. Most small and industrial- strength database applications can be accessed using SQL statements. SQL is both an ANSI and an ISO standard. However, many database products supporting SQL do so with proprietary extensions to the standard language. Web applications may use user-supplied input to create custom SQL statements for dynamic web page requests.
When a web application fails to properly sanitize user-supplied input, it is possible for an attacker to alter the construction of backend SQL statements. When an attacker is able to modify a SQL statement, the process will run with the same permissions as the component that executed the command. (e.g. Database server, Web application server, Web server, etc.). The impact of this attack can allow attackers to gain total control of the database or even execute commands on the system.
The same advanced exploitation techniques available in LDAP Injection can also be similarly applied to SQL Injection.
Example
A web based authentication form might have code that looks like the following: SQLQuery = "SELECT Username FROM Users WHERE Username = '" & strUsername & "' AND Password = '" & strPassword & "'" strAuthCheck = GetQueryResult(SQLQuery)
In this code, the developer is taking the user-input from the form and embedding it directly into an SQL query.
Suppose an attacker submits a login and password that looks like the following: Login: ' OR ''=' Password: ' OR ''=' This will cause the resulting SQL query to become: SELECT Username FROM Users WHERE Username = '' OR ''='' AND Password = '' OR ''=''
Instead of comparing the user-supplied data with entries in the Users table, the query compares '' (empty string) to '' (empty string). This will return a True result and the attacker will then be logged in as the first user in the Users table.
There are two commonly known methods of SQL injection: Normal SQL Injection and Blind SQL Injection. The first is vanilla SQL Injection in which the attacker can format his query to match the developer's by using the information contained in the error messages that are returned in the response.
Normal SQL Injection
By appending a union select statement to the parameter, the attacker can test to see if he can gain access to the database:
http://example/article.asp?ID=2+union+all+select+name+from+sysobjects
The SQL server then might return an error similar to this: Microsoft OLE DB Provider for ODBC Drivers error '80040e14' [Microsoft][ODBC SQL Server Driver][SQL Server]All queries in an SQL statement containing a UNION operator must have an equal number of expressions in their target lists.
This tells the attacker that he must now guess the correct number of columns for his SQL statement to work.
Blind SQL Injection
In Blind SQL Injection, instead of returning a database error, the server returns a customer-friendly error page informing the user that a mistake has been made. In this instance, SQL Injection is still possible, but not as easy to detect. A common way to detect Blind SQL Injection is to put a false and true statement into the parameter value.
Executing the following request to a web site:
http://example/article.asp?ID=2+and+1=1
should return the same web page as:
http://example/article.asp?ID=2
because the SQL statement 'and 1=1' is always true.
Executing the following request to a web site:
http://example/article.asp?ID=2+and+1=0
would then cause the web site to return a friendly error or no page at all. This is because the SQL statement "and 1=0" is always false.
Once the attacker discovers that a site is susceptible to Blind SQL Injection, he can exploit this vulnerability more easily, in some cases, than by using normal SQL Injection.
SQL Injection is an attack technique used to exploit web sites that construct SQL statements from user-supplied input.
Wikipedia Definition:
SQL injection is a technique that exploits a security vulnerability occurring in the database layer of an application. The vulnerability is present when user input is either incorrectly filtered for string literal escape characters embedded in SQL statements or user input is not strongly typed and thereby unexpectedly executed. It is in fact an instance of a more general class of vulnerabilities that can occur whenever one programming or scripting language is embedded inside another.
Structured Query Language (SQL) is a specialized programming language for sending queries to databases. Most small and industrial- strength database applications can be accessed using SQL statements. SQL is both an ANSI and an ISO standard. However, many database products supporting SQL do so with proprietary extensions to the standard language. Web applications may use user-supplied input to create custom SQL statements for dynamic web page requests.
When a web application fails to properly sanitize user-supplied input, it is possible for an attacker to alter the construction of backend SQL statements. When an attacker is able to modify a SQL statement, the process will run with the same permissions as the component that executed the command. (e.g. Database server, Web application server, Web server, etc.). The impact of this attack can allow attackers to gain total control of the database or even execute commands on the system.
The same advanced exploitation techniques available in LDAP Injection can also be similarly applied to SQL Injection.
Example
A web based authentication form might have code that looks like the following: SQLQuery = "SELECT Username FROM Users WHERE Username = '" & strUsername & "' AND Password = '" & strPassword & "'" strAuthCheck = GetQueryResult(SQLQuery)
In this code, the developer is taking the user-input from the form and embedding it directly into an SQL query.
Suppose an attacker submits a login and password that looks like the following: Login: ' OR ''=' Password: ' OR ''=' This will cause the resulting SQL query to become: SELECT Username FROM Users WHERE Username = '' OR ''='' AND Password = '' OR ''=''
Instead of comparing the user-supplied data with entries in the Users table, the query compares '' (empty string) to '' (empty string). This will return a True result and the attacker will then be logged in as the first user in the Users table.
There are two commonly known methods of SQL injection: Normal SQL Injection and Blind SQL Injection. The first is vanilla SQL Injection in which the attacker can format his query to match the developer's by using the information contained in the error messages that are returned in the response.
Normal SQL Injection
By appending a union select statement to the parameter, the attacker can test to see if he can gain access to the database:
http://example/article.asp?ID=2+union+all+select+name+from+sysobjects
The SQL server then might return an error similar to this: Microsoft OLE DB Provider for ODBC Drivers error '80040e14' [Microsoft][ODBC SQL Server Driver][SQL Server]All queries in an SQL statement containing a UNION operator must have an equal number of expressions in their target lists.
This tells the attacker that he must now guess the correct number of columns for his SQL statement to work.
Blind SQL Injection
In Blind SQL Injection, instead of returning a database error, the server returns a customer-friendly error page informing the user that a mistake has been made. In this instance, SQL Injection is still possible, but not as easy to detect. A common way to detect Blind SQL Injection is to put a false and true statement into the parameter value.
Executing the following request to a web site:
http://example/article.asp?ID=2+and+1=1
should return the same web page as:
http://example/article.asp?ID=2
because the SQL statement 'and 1=1' is always true.
Executing the following request to a web site:
http://example/article.asp?ID=2+and+1=0
would then cause the web site to return a friendly error or no page at all. This is because the SQL statement "and 1=0" is always false.
Once the attacker discovers that a site is susceptible to Blind SQL Injection, he can exploit this vulnerability more easily, in some cases, than by using normal SQL Injection.
Monday, December 1, 2008
ISO 9000
ISO 9000 is a family of standards for quality management systems. ISO 9000 is maintained by ISO, the International Organization for Standardization and is administered by accreditation and certification bodies. Some of the requirements in ISO 9001 (which is one of the standards in the ISO 9000 family) include
* a set of procedures that cover all key processes in the business;
* monitoring processes to ensure they are effective;
* keeping adequate records;
* checking output for defects, with appropriate and corrective action where necessary;
* regularly reviewing individual processes and the quality system itself for effectiveness; and
* facilitating continual improvement
A company or organization that has been independently audited and certified to be in conformance with ISO 9001 may publicly state that it is "ISO 9001 certified" or "ISO 9001 registered". Certification to an ISO 9000 standard does not guarantee any quality of end products and services; rather, it certifies that formalized business processes are being applied. Indeed, some companies enter the ISO 9001 certification as a marketing tool.
Although the standards originated in manufacturing, they are now employed across several types of organization. A "product", in ISO vocabulary, can mean a physical object, services, or software. In fact, according to ISO in 2004, "service sectors now account by far for the highest number of ISO 9001:2000 certificates - about 31% of the total."
ISO 9000 family
ISO 9000 includes standards:
* ISO 9000:2000, Quality management systems – Fundamentals and vocabulary. Covers the basics of what quality management systems are and also contains the core language of the ISO 9000 series of standards. A guidance document, not used for certification purposes, but important reference document to understand terms and vocabulary related to quality management systems. In the year 2005, revised ISO 9000:2005 standard has been published, so it is now advised to refer to ISO 9000:2005.
* ISO 9001:2000 Quality management systems – Requirements is intended for use in any organization which designs, develops, manufactures, installs and/or services any product or provides any form of service. It provides a number of requirements which an organization needs to fulfill if it is to achieve customer satisfaction through consistent products and services which meet customer expectations. It includes a requirement for the continual (i.e. planned) improvement of the Quality Management System, for which ISO 9004:2000 provides many hints.
This is the only implementation for which third-party auditors may grant certification. It should be noted that certification is not described as any of the 'needs' of an organization as a driver for using ISO 9001 (see ISO 9001:2000 section 1 'Scope') but does recognize that it may be used for such a purpose (see ISO 9001:2000 section 0.1 'Introduction').
* ISO 9004:2000 Quality management systems - Guidelines for performance improvements. covers continual improvement. This gives you advice on what you could do to enhance a mature system. This standard very specifically states that it is not intended as a guide to implementation.
There are many more standards in the ISO 9001 family (see "List of ISO 9000 standards" from ISO), many of them not even carrying "ISO 900x" numbers. For example, some standards in the 10,000 range are considered part of the 9000 family: ISO 10007:1995 discusses Configuration management, which for most organizations is just one element of a complete management system. ISO notes: "The emphasis on certification tends to overshadow the fact that there is an entire family of ISO 9000 standards ... Organizations stand to obtain the greatest value when the standards in the new core series are used in an integrated manner, both with each other and with the other standards making up the ISO 9000 family as a whole".
Note that the previous members of the ISO 9000 family, 9001, 9002 and 9003, have all been integrated into 9001. In most cases, an organization claiming to be "ISO 9000 registered" is referring to ISO 9001.
Contents of ISO 9001
ISO 9001:2000 Quality management systems — Requirements is a document of approximately 30 pages which is available from the national standards organization in each country. Outline contents are as follows:
* Page iv: Foreword
* Pages v to vii: Section 0 Introduction
* Pages 1 to 14: Requirements
o Section 1: Scope
o Section 2: Normative Reference
o Section 3: Terms and definitions (specific to ISO 9001, not specified in ISO 9000)
* Pages 2 to 14
o Section 4: Quality Management System
o Section 5: Management Responsibility
o Section 6: Resource Management
o Section 7: Product Realization
o Section 8: Measurement, analysis and improvement
In effect, users need to address all sections 1 to 8, but only 4 to 8 need implementing within a QMS.
* Pages 15 to 22: Tables of Correspondence between ISO 9001 and other standards
* Page 23: Bibliography
The standard specifies six compulsory documents:
* Control of Documents (4.2.3)
* Control of Records (4.2.4)
* Internal Audits (8.2.2)
* Control of Nonconforming Product / Service (8.3)
* Corrective Action (8.5.2)
* Preventive Action (8.5.3)
In addition to these, ISO 9001:2000 requires a Quality Policy and Quality Manual (which may or may not include the above documents).
Summary of ISO 9001:2000 in informal language
* The quality policy is a formal statement from management, closely linked to the business and marketing plan and to customer needs. The quality policy is understood and followed at all levels and by all employees. Each employee needs measurable objectives to work towards.
* Decisions about the quality system are made based on recorded data and the system is regularly audited and evaluated for conformance and effectiveness.
* Records should show how and where raw materials and products were processed, to allow products and problems to be traced to the source.
* You need a documented procedure to control quality documents in your company. Everyone must have access to up-to-date documents and be aware of how to use them.
* To maintain the quality system and produce conforming product, you need to provide suitable infrastructure, resources, information, equipment, measuring and monitoring devices, and environmental conditions.
* You need to map out all key processes in your company; control them by monitoring, measurement and analysis; and ensure that product quality objectives are met. If you can’t monitor a process by measurement, then make sure the process is well enough defined that you can make adjustments if the product does not meet user needs.
* For each product your company makes, you need to establish quality objectives; plan processes; and document and measure results to use as a tool for improvement. For each process, determine what kind of procedural documentation is required (note: a “product” is hardware, software, services, processed materials, or a combination of these).
* You need to determine key points where each process requires monitoring and measurement, and ensure that all monitoring and measuring devices are properly maintained and calibrated.
* You need to have clear requirements for purchased product.
* You need to determine customer requirements and create systems for communicating with customers about product information, inquiries, contracts, orders, feedback and complaints.
* When developing new products, you need to plan the stages of development, with appropriate testing at each stage. You need to test and document whether the product meets design requirements, regulatory requirements and user needs.
* You need to regularly review performance through internal audits and meetings. Determine whether the quality system is working and what improvements can be made. Deal with past problems and potential problems. Keep records of these activities and the resulting decisions, and monitor their effectiveness (note: you need a documented procedure for internal audits).
* You need documented procedures for dealing with actual and potential nonconformances (problems involving suppliers or customers, or internal problems). Make sure no one uses bad product, determine what to do with bad product, deal with the root cause of the problem and keep records to use as a tool to improve the system.
1987 version
SO 9000:1987 had the same structure as the UK Standard BS 5750, with three 'models' for quality management systems, the selection of which was based on the scope of activities of the organization:
* ISO 9001:1987 Model for quality assurance in design, development, production, installation, and servicing was for companies and organizations whose activities included the creation of new products.
* ISO 9002:1987 Model for quality assurance in production, installation, and servicing had basically the same material as ISO 9001 but without covering the creation of new products.
* ISO 9003:1987 Model for quality assurance in final inspection and test covered only the final inspection of finished product, with no concern for how the product was produced.
ISO 9000:1987 was also influenced by existing U.S. and other Defense Standards ("MIL SPECS"), and so was well-suited to manufacturing. The emphasis tended to be placed on conformance with procedures rather than the overall process of management—which was likely the actual intent.
1994 version
ISO 9000:1994 emphasized quality assurance via preventive actions, instead of just checking final product, and continued to require evidence of compliance with documented procedures. As with the first edition, the down-side was that companies tended to implement its requirements by creating shelf-loads of procedure manuals, and becoming burdened with an ISO bureaucracy. In some companies, adapting and improving processes could actually be impeded by the quality system.
2000 version
ISO 9001:2000 combines the three standards 9001, 9002, and 9003 into one, called 9001. Design and development procedures are required only if a company does in fact engage in the creation of new products. The 2000 version sought to make a radical change in thinking by actually placing the concept of process management front and center ("Process management" was the monitoring and optimizing of a company's tasks and activities, instead of just inspecting the final product). The 2000 version also demands involvement by upper executives, in order to integrate quality into the business system and avoid delegation of quality functions to junior administrators. Another goal is to improve effectiveness via process performance metrics — numerical measurement of the effectiveness of tasks and activities. Expectations of continual process improvement and tracking customer satisfaction were made explicit.
The ISO 9000 standard is continually being revised by standing technical committees and advisory groups, who receive feedback from those professionals who are implementing the standard.
2008 version
ISO 9001:2008 only introduces clarifications to the existing requirements of ISO 9001:2000 and some changes intended to improve consistency with ISO14001:2004. There are no new requirements. A quality management system being upgraded just needs to be checked to see if it is following the clarifications introduced in the amended version.
Certification
ISO does not itself certify organizations. Many countries have formed accreditation bodies to authorize certification bodies, which audit organizations applying for ISO 9001 compliance certification. Although commonly referred to as ISO 9000:2000 certification, the actual standard to which an organization's quality management can be certified is ISO 9001:2000. Both the accreditation bodies and the certification bodies charge fees for their services. The various accreditation bodies have mutual agreements with each other to ensure that certificates issued by one of the Accredited Certification Bodies (CB) are accepted worldwide.
The applying organization is assessed based on an extensive sample of its sites, functions, products, services and processes; a list of problems ("action requests" or "non-compliances") is made known to the management. If there are no major problems on this list, the certification body will issue an ISO 9001 certificate for each geographical site it has visited, once it receives a satisfactory improvement plan from the management showing how any problems will be resolved.
An ISO certificate is not a once-and-for-all award, but must be renewed at regular intervals recommended by the certification body, usually around three years. In contrast to the Capability Maturity Model there are no grades of competence within ISO 9001.
Auditing
Two types of auditing are required to become registered to the standard: auditing by an external certification body (external audit) and audits by internal staff trained for this process (internal audits). The aim is a continual process of review and assessment, to verify that the system is working as it's supposed to, find out where it can improve and to correct or prevent problems identified. It is considered healthier for internal auditors to audit outside their usual management line, so as to bring a degree of independence to their judgments.
Under the 1994 standard, the auditing process could be adequately addressed by performing "compliance auditing":
* Tell me what you do (describe the business process)
* Show me where it says that (reference the procedure manuals)
* Prove that that is what happened (exhibit evidence in documented records)
How this led to preventive actions was not clear.
The 2000 standard uses the process approach. While auditors perform similar functions, they are expected to go beyond mere auditing for rote "compliance" by focusing on risk, status and importance. This means they are expected to make more judgments on what is effective, rather than merely adhering to what is formally prescribed. The difference from the previous standard can be explained thus:
Under the 1994 version, the question was broadly "Are you doing what the manual says you should be doing?", whereas under the 2000 version, the question is more "Will this process help you achieve your stated objectives? Is it a good process or is there a way to do it better?".
The ISO 19011 standard for auditing applies to ISO 9001 besides other management systems like EMS ( ISO 14001), FSMS (ISO 22000) etc.
Industry-specific interpretations
he ISO 9001 standard is generalized and abstract. Its parts must be carefully interpreted, to make sense within a particular organization. Developing software is not like making cheese or offering counseling services; yet the ISO 9001 guidelines, because they are business management guidelines, can be applied to each of these. Diverse organizations—police departments (US), professional soccer teams (Mexico) and city councils (UK)—have successfully implemented ISO 9001:2000 systems.
Over time, various industry sectors have wanted to standardize their interpretations of the guidelines within their own marketplace. This is partly to ensure that their versions of ISO 9000 have their specific requirements, but also to try and ensure that more appropriately trained and experienced auditors are sent to assess them.
* The TickIT guidelines are an interpretation of ISO 9000 produced by the UK Board of Trade to suit the processes of the information technology industry, especially software development.
* AS 9000 is the Aerospace Basic Quality System Standard, an interpretation developed by major aerospace manufacturers. Those major manufacturers include AlliedSignal, Allison Engine, Boeing, General Electric Aircraft Engines, Lockheed-Martin, McDonnell Douglas, Northrop Grumman, Pratt & Whitney, Rockwell-Collins, Sikorsky Aircraft, and Sundstrand. The current version is AS 9100.
* PS 9000 is an application of the standard for Pharmaceutical Packaging Materials. The Pharmaceutical Quality Group (PQG) of the Institute of Quality Assurance (IQA) has developed PS 9000:2001. It aims to provide a widely accepted baseline GMP framework of best practice within the pharmaceutical packaging supply industry. It applies ISO 9001: 2000 to pharmaceutical printed and contact packaging materials.
* QS 9000 is an interpretation agreed upon by major automotive manufacturers (GM, Ford, Chrysler). It includes techniques such as FMEA and APQP. QS 9000 is now replaced by ISO/TS 16949.
* ISO/TS 16949:2002 is an interpretation agreed upon by major automotive manufacturers (American and European manufacturers); the latest version is based on ISO 9001:2000. The emphasis on a process approach is stronger than in ISO 9001:2000. ISO/TS 16949:2002 contains the full text of ISO 9001:2000 and automotive industry-specific requirements.
* TL 9000 is the Telecom Quality Management and Measurement System Standard, an interpretation developed by the telecom consortium, QuEST Forum. The current version is 4.0 and unlike ISO 9001 or the above sector standards, TL 9000 includes standardized product measurements that can be benchmarked. In 1998 QuEST Forum developed the TL 9000 Quality Management System to meet the supply chain quality requirements of the worldwide telecommunications industry.
* ISO 13485:2003 is the medical industry's equivalent of ISO 9001:2000. Whereas the standards it replaces were interpretations of how to apply ISO 9001 and ISO 9002 to medical devices, ISO 13485:2003 is a stand-alone standard. Compliance with ISO 13485 does not necessarily mean compliance with ISO 9001:2000.
Debate on the effectiveness of ISO 9000
The debate on the effectiveness of ISO 9000 commonly centers on the following questions:
1. Are the quality principles in ISO 9001:2000 of value? (Note that the version date is important: in the 2000 version ISO attempted to address many concerns and criticisms of ISO 9000:1994).
2. Does it help to implement an ISO 9001:2000 compliant quality management system?
3. Does it help to obtain ISO 9001:2000 certification?
Advantages
It is widely acknowledged that proper quality management improves business, often having a positive effect on investment, market share, sales growth, sales margins, competitive advantage, and avoidance of litigation.[3][4] The quality principles in ISO 9000:2000 are also sound, according to Wade,[5] and Barnes, [4] who says "ISO 9000 guidelines provide a comprehensive model for quality management systems that can make any company competitive." Barnes also cites a survey by Lloyd's Register Quality Assurance which indicated that ISO 9000 increased net profit, and another by Deloitte-Touche which reported that the costs of registration were recovered in three years. According to the Providence Business News [6], implementing ISO often gives the following advantages:
1. Create a more efficient, effective operation
2. Increase customer satisfaction and retention
3. Reduce audits
4. Enhance marketing
5. Improve employee motivation, awareness, and morale
6. Promote international trade
7. Increases profit
8. Reduce waste and increases productivity
However, a broad statistical study of 800 Spanish companies [7] found that ISO 9000 registration in itself creates little improvement because companies interested in it have usually already made some type of commitment to quality management and were performing just as well before registration.[3]
In today's service-sector driven economy, more and more companies are using ISO 9000 as a business tool. Through the use of properly stated quality objectives, customer satisfaction surveys and a well-defined continual improvement program companies are using ISO 9000 processes to increase their efficiency and profitability.
Problems
A common criticism of ISO 9001 is the amount of money, time and paperwork required for registration.[8] According to Barnes, "Opponents claim that it is only for documentation. Proponents believe that if a company has documented its quality systems, then most of the paperwork has already been completed."[4]
According to Seddon, ISO 9001 promotes specification, control, and procedures rather than understanding and improvement. [9] [10] Wade argues that ISO 9000 is effective as a guideline, but that promoting it as a standard "helps to mislead companies into thinking that certification means better quality, ... [undermining] the need for an organization to set its own quality standards." [5] Paraphrased, Wade's argument is that total, blind reliance on the specifications of ISO 9001 does not guarantee a successful quality system.
The standard is seen as especially prone to failure when a company is interested in certification before quality.[9] Certifications are in fact often based on customer contractual requirements rather than a desire to actually improve quality.[4][11] "If you just want the certificate on the wall, chances are, you will create a paper system that doesn't have much to do with the way you actually run your business," said ISO's Roger Frost.[11] Certification by an independent auditor is often seen as the problem area, and according to Barnes, "has become a vehicle to increase consulting services." [4] In fact, ISO itself advises that ISO 9001 can be implemented without certification, simply for the quality benefits that can be achieved. [12]
Another problem reported is the competition among the numerous certifying bodies, leading to a softer approach to the defects noticed in the operation of the Quality System of a firm.
Summary
A good overview for effective use of ISO 9000 is provided by Barnes: [4]
"Good business judgment is needed to determine its proper role for a company... Is certification itself important to the marketing plans of the company? If not, do not rush to certification... Even without certification, companies should utilize the ISO 9000 model as a benchmark to assess the adequacy of its quality programs."
* a set of procedures that cover all key processes in the business;
* monitoring processes to ensure they are effective;
* keeping adequate records;
* checking output for defects, with appropriate and corrective action where necessary;
* regularly reviewing individual processes and the quality system itself for effectiveness; and
* facilitating continual improvement
A company or organization that has been independently audited and certified to be in conformance with ISO 9001 may publicly state that it is "ISO 9001 certified" or "ISO 9001 registered". Certification to an ISO 9000 standard does not guarantee any quality of end products and services; rather, it certifies that formalized business processes are being applied. Indeed, some companies enter the ISO 9001 certification as a marketing tool.
Although the standards originated in manufacturing, they are now employed across several types of organization. A "product", in ISO vocabulary, can mean a physical object, services, or software. In fact, according to ISO in 2004, "service sectors now account by far for the highest number of ISO 9001:2000 certificates - about 31% of the total."
ISO 9000 family
ISO 9000 includes standards:
* ISO 9000:2000, Quality management systems – Fundamentals and vocabulary. Covers the basics of what quality management systems are and also contains the core language of the ISO 9000 series of standards. A guidance document, not used for certification purposes, but important reference document to understand terms and vocabulary related to quality management systems. In the year 2005, revised ISO 9000:2005 standard has been published, so it is now advised to refer to ISO 9000:2005.
* ISO 9001:2000 Quality management systems – Requirements is intended for use in any organization which designs, develops, manufactures, installs and/or services any product or provides any form of service. It provides a number of requirements which an organization needs to fulfill if it is to achieve customer satisfaction through consistent products and services which meet customer expectations. It includes a requirement for the continual (i.e. planned) improvement of the Quality Management System, for which ISO 9004:2000 provides many hints.
This is the only implementation for which third-party auditors may grant certification. It should be noted that certification is not described as any of the 'needs' of an organization as a driver for using ISO 9001 (see ISO 9001:2000 section 1 'Scope') but does recognize that it may be used for such a purpose (see ISO 9001:2000 section 0.1 'Introduction').
* ISO 9004:2000 Quality management systems - Guidelines for performance improvements. covers continual improvement. This gives you advice on what you could do to enhance a mature system. This standard very specifically states that it is not intended as a guide to implementation.
There are many more standards in the ISO 9001 family (see "List of ISO 9000 standards" from ISO), many of them not even carrying "ISO 900x" numbers. For example, some standards in the 10,000 range are considered part of the 9000 family: ISO 10007:1995 discusses Configuration management, which for most organizations is just one element of a complete management system. ISO notes: "The emphasis on certification tends to overshadow the fact that there is an entire family of ISO 9000 standards ... Organizations stand to obtain the greatest value when the standards in the new core series are used in an integrated manner, both with each other and with the other standards making up the ISO 9000 family as a whole".
Note that the previous members of the ISO 9000 family, 9001, 9002 and 9003, have all been integrated into 9001. In most cases, an organization claiming to be "ISO 9000 registered" is referring to ISO 9001.
Contents of ISO 9001
ISO 9001:2000 Quality management systems — Requirements is a document of approximately 30 pages which is available from the national standards organization in each country. Outline contents are as follows:
* Page iv: Foreword
* Pages v to vii: Section 0 Introduction
* Pages 1 to 14: Requirements
o Section 1: Scope
o Section 2: Normative Reference
o Section 3: Terms and definitions (specific to ISO 9001, not specified in ISO 9000)
* Pages 2 to 14
o Section 4: Quality Management System
o Section 5: Management Responsibility
o Section 6: Resource Management
o Section 7: Product Realization
o Section 8: Measurement, analysis and improvement
In effect, users need to address all sections 1 to 8, but only 4 to 8 need implementing within a QMS.
* Pages 15 to 22: Tables of Correspondence between ISO 9001 and other standards
* Page 23: Bibliography
The standard specifies six compulsory documents:
* Control of Documents (4.2.3)
* Control of Records (4.2.4)
* Internal Audits (8.2.2)
* Control of Nonconforming Product / Service (8.3)
* Corrective Action (8.5.2)
* Preventive Action (8.5.3)
In addition to these, ISO 9001:2000 requires a Quality Policy and Quality Manual (which may or may not include the above documents).
Summary of ISO 9001:2000 in informal language
* The quality policy is a formal statement from management, closely linked to the business and marketing plan and to customer needs. The quality policy is understood and followed at all levels and by all employees. Each employee needs measurable objectives to work towards.
* Decisions about the quality system are made based on recorded data and the system is regularly audited and evaluated for conformance and effectiveness.
* Records should show how and where raw materials and products were processed, to allow products and problems to be traced to the source.
* You need a documented procedure to control quality documents in your company. Everyone must have access to up-to-date documents and be aware of how to use them.
* To maintain the quality system and produce conforming product, you need to provide suitable infrastructure, resources, information, equipment, measuring and monitoring devices, and environmental conditions.
* You need to map out all key processes in your company; control them by monitoring, measurement and analysis; and ensure that product quality objectives are met. If you can’t monitor a process by measurement, then make sure the process is well enough defined that you can make adjustments if the product does not meet user needs.
* For each product your company makes, you need to establish quality objectives; plan processes; and document and measure results to use as a tool for improvement. For each process, determine what kind of procedural documentation is required (note: a “product” is hardware, software, services, processed materials, or a combination of these).
* You need to determine key points where each process requires monitoring and measurement, and ensure that all monitoring and measuring devices are properly maintained and calibrated.
* You need to have clear requirements for purchased product.
* You need to determine customer requirements and create systems for communicating with customers about product information, inquiries, contracts, orders, feedback and complaints.
* When developing new products, you need to plan the stages of development, with appropriate testing at each stage. You need to test and document whether the product meets design requirements, regulatory requirements and user needs.
* You need to regularly review performance through internal audits and meetings. Determine whether the quality system is working and what improvements can be made. Deal with past problems and potential problems. Keep records of these activities and the resulting decisions, and monitor their effectiveness (note: you need a documented procedure for internal audits).
* You need documented procedures for dealing with actual and potential nonconformances (problems involving suppliers or customers, or internal problems). Make sure no one uses bad product, determine what to do with bad product, deal with the root cause of the problem and keep records to use as a tool to improve the system.
1987 version
SO 9000:1987 had the same structure as the UK Standard BS 5750, with three 'models' for quality management systems, the selection of which was based on the scope of activities of the organization:
* ISO 9001:1987 Model for quality assurance in design, development, production, installation, and servicing was for companies and organizations whose activities included the creation of new products.
* ISO 9002:1987 Model for quality assurance in production, installation, and servicing had basically the same material as ISO 9001 but without covering the creation of new products.
* ISO 9003:1987 Model for quality assurance in final inspection and test covered only the final inspection of finished product, with no concern for how the product was produced.
ISO 9000:1987 was also influenced by existing U.S. and other Defense Standards ("MIL SPECS"), and so was well-suited to manufacturing. The emphasis tended to be placed on conformance with procedures rather than the overall process of management—which was likely the actual intent.
1994 version
ISO 9000:1994 emphasized quality assurance via preventive actions, instead of just checking final product, and continued to require evidence of compliance with documented procedures. As with the first edition, the down-side was that companies tended to implement its requirements by creating shelf-loads of procedure manuals, and becoming burdened with an ISO bureaucracy. In some companies, adapting and improving processes could actually be impeded by the quality system.
2000 version
ISO 9001:2000 combines the three standards 9001, 9002, and 9003 into one, called 9001. Design and development procedures are required only if a company does in fact engage in the creation of new products. The 2000 version sought to make a radical change in thinking by actually placing the concept of process management front and center ("Process management" was the monitoring and optimizing of a company's tasks and activities, instead of just inspecting the final product). The 2000 version also demands involvement by upper executives, in order to integrate quality into the business system and avoid delegation of quality functions to junior administrators. Another goal is to improve effectiveness via process performance metrics — numerical measurement of the effectiveness of tasks and activities. Expectations of continual process improvement and tracking customer satisfaction were made explicit.
The ISO 9000 standard is continually being revised by standing technical committees and advisory groups, who receive feedback from those professionals who are implementing the standard.
2008 version
ISO 9001:2008 only introduces clarifications to the existing requirements of ISO 9001:2000 and some changes intended to improve consistency with ISO14001:2004. There are no new requirements. A quality management system being upgraded just needs to be checked to see if it is following the clarifications introduced in the amended version.
Certification
ISO does not itself certify organizations. Many countries have formed accreditation bodies to authorize certification bodies, which audit organizations applying for ISO 9001 compliance certification. Although commonly referred to as ISO 9000:2000 certification, the actual standard to which an organization's quality management can be certified is ISO 9001:2000. Both the accreditation bodies and the certification bodies charge fees for their services. The various accreditation bodies have mutual agreements with each other to ensure that certificates issued by one of the Accredited Certification Bodies (CB) are accepted worldwide.
The applying organization is assessed based on an extensive sample of its sites, functions, products, services and processes; a list of problems ("action requests" or "non-compliances") is made known to the management. If there are no major problems on this list, the certification body will issue an ISO 9001 certificate for each geographical site it has visited, once it receives a satisfactory improvement plan from the management showing how any problems will be resolved.
An ISO certificate is not a once-and-for-all award, but must be renewed at regular intervals recommended by the certification body, usually around three years. In contrast to the Capability Maturity Model there are no grades of competence within ISO 9001.
Auditing
Two types of auditing are required to become registered to the standard: auditing by an external certification body (external audit) and audits by internal staff trained for this process (internal audits). The aim is a continual process of review and assessment, to verify that the system is working as it's supposed to, find out where it can improve and to correct or prevent problems identified. It is considered healthier for internal auditors to audit outside their usual management line, so as to bring a degree of independence to their judgments.
Under the 1994 standard, the auditing process could be adequately addressed by performing "compliance auditing":
* Tell me what you do (describe the business process)
* Show me where it says that (reference the procedure manuals)
* Prove that that is what happened (exhibit evidence in documented records)
How this led to preventive actions was not clear.
The 2000 standard uses the process approach. While auditors perform similar functions, they are expected to go beyond mere auditing for rote "compliance" by focusing on risk, status and importance. This means they are expected to make more judgments on what is effective, rather than merely adhering to what is formally prescribed. The difference from the previous standard can be explained thus:
Under the 1994 version, the question was broadly "Are you doing what the manual says you should be doing?", whereas under the 2000 version, the question is more "Will this process help you achieve your stated objectives? Is it a good process or is there a way to do it better?".
The ISO 19011 standard for auditing applies to ISO 9001 besides other management systems like EMS ( ISO 14001), FSMS (ISO 22000) etc.
Industry-specific interpretations
he ISO 9001 standard is generalized and abstract. Its parts must be carefully interpreted, to make sense within a particular organization. Developing software is not like making cheese or offering counseling services; yet the ISO 9001 guidelines, because they are business management guidelines, can be applied to each of these. Diverse organizations—police departments (US), professional soccer teams (Mexico) and city councils (UK)—have successfully implemented ISO 9001:2000 systems.
Over time, various industry sectors have wanted to standardize their interpretations of the guidelines within their own marketplace. This is partly to ensure that their versions of ISO 9000 have their specific requirements, but also to try and ensure that more appropriately trained and experienced auditors are sent to assess them.
* The TickIT guidelines are an interpretation of ISO 9000 produced by the UK Board of Trade to suit the processes of the information technology industry, especially software development.
* AS 9000 is the Aerospace Basic Quality System Standard, an interpretation developed by major aerospace manufacturers. Those major manufacturers include AlliedSignal, Allison Engine, Boeing, General Electric Aircraft Engines, Lockheed-Martin, McDonnell Douglas, Northrop Grumman, Pratt & Whitney, Rockwell-Collins, Sikorsky Aircraft, and Sundstrand. The current version is AS 9100.
* PS 9000 is an application of the standard for Pharmaceutical Packaging Materials. The Pharmaceutical Quality Group (PQG) of the Institute of Quality Assurance (IQA) has developed PS 9000:2001. It aims to provide a widely accepted baseline GMP framework of best practice within the pharmaceutical packaging supply industry. It applies ISO 9001: 2000 to pharmaceutical printed and contact packaging materials.
* QS 9000 is an interpretation agreed upon by major automotive manufacturers (GM, Ford, Chrysler). It includes techniques such as FMEA and APQP. QS 9000 is now replaced by ISO/TS 16949.
* ISO/TS 16949:2002 is an interpretation agreed upon by major automotive manufacturers (American and European manufacturers); the latest version is based on ISO 9001:2000. The emphasis on a process approach is stronger than in ISO 9001:2000. ISO/TS 16949:2002 contains the full text of ISO 9001:2000 and automotive industry-specific requirements.
* TL 9000 is the Telecom Quality Management and Measurement System Standard, an interpretation developed by the telecom consortium, QuEST Forum. The current version is 4.0 and unlike ISO 9001 or the above sector standards, TL 9000 includes standardized product measurements that can be benchmarked. In 1998 QuEST Forum developed the TL 9000 Quality Management System to meet the supply chain quality requirements of the worldwide telecommunications industry.
* ISO 13485:2003 is the medical industry's equivalent of ISO 9001:2000. Whereas the standards it replaces were interpretations of how to apply ISO 9001 and ISO 9002 to medical devices, ISO 13485:2003 is a stand-alone standard. Compliance with ISO 13485 does not necessarily mean compliance with ISO 9001:2000.
Debate on the effectiveness of ISO 9000
The debate on the effectiveness of ISO 9000 commonly centers on the following questions:
1. Are the quality principles in ISO 9001:2000 of value? (Note that the version date is important: in the 2000 version ISO attempted to address many concerns and criticisms of ISO 9000:1994).
2. Does it help to implement an ISO 9001:2000 compliant quality management system?
3. Does it help to obtain ISO 9001:2000 certification?
Advantages
It is widely acknowledged that proper quality management improves business, often having a positive effect on investment, market share, sales growth, sales margins, competitive advantage, and avoidance of litigation.[3][4] The quality principles in ISO 9000:2000 are also sound, according to Wade,[5] and Barnes, [4] who says "ISO 9000 guidelines provide a comprehensive model for quality management systems that can make any company competitive." Barnes also cites a survey by Lloyd's Register Quality Assurance which indicated that ISO 9000 increased net profit, and another by Deloitte-Touche which reported that the costs of registration were recovered in three years. According to the Providence Business News [6], implementing ISO often gives the following advantages:
1. Create a more efficient, effective operation
2. Increase customer satisfaction and retention
3. Reduce audits
4. Enhance marketing
5. Improve employee motivation, awareness, and morale
6. Promote international trade
7. Increases profit
8. Reduce waste and increases productivity
However, a broad statistical study of 800 Spanish companies [7] found that ISO 9000 registration in itself creates little improvement because companies interested in it have usually already made some type of commitment to quality management and were performing just as well before registration.[3]
In today's service-sector driven economy, more and more companies are using ISO 9000 as a business tool. Through the use of properly stated quality objectives, customer satisfaction surveys and a well-defined continual improvement program companies are using ISO 9000 processes to increase their efficiency and profitability.
Problems
A common criticism of ISO 9001 is the amount of money, time and paperwork required for registration.[8] According to Barnes, "Opponents claim that it is only for documentation. Proponents believe that if a company has documented its quality systems, then most of the paperwork has already been completed."[4]
According to Seddon, ISO 9001 promotes specification, control, and procedures rather than understanding and improvement. [9] [10] Wade argues that ISO 9000 is effective as a guideline, but that promoting it as a standard "helps to mislead companies into thinking that certification means better quality, ... [undermining] the need for an organization to set its own quality standards." [5] Paraphrased, Wade's argument is that total, blind reliance on the specifications of ISO 9001 does not guarantee a successful quality system.
The standard is seen as especially prone to failure when a company is interested in certification before quality.[9] Certifications are in fact often based on customer contractual requirements rather than a desire to actually improve quality.[4][11] "If you just want the certificate on the wall, chances are, you will create a paper system that doesn't have much to do with the way you actually run your business," said ISO's Roger Frost.[11] Certification by an independent auditor is often seen as the problem area, and according to Barnes, "has become a vehicle to increase consulting services." [4] In fact, ISO itself advises that ISO 9001 can be implemented without certification, simply for the quality benefits that can be achieved. [12]
Another problem reported is the competition among the numerous certifying bodies, leading to a softer approach to the defects noticed in the operation of the Quality System of a firm.
Summary
A good overview for effective use of ISO 9000 is provided by Barnes: [4]
"Good business judgment is needed to determine its proper role for a company... Is certification itself important to the marketing plans of the company? If not, do not rush to certification... Even without certification, companies should utilize the ISO 9000 model as a benchmark to assess the adequacy of its quality programs."
Friday, November 28, 2008
Credit Card Testing
Credit Card Type Credit Card Number
American Express 378282246310005
________________________________________________________________________________
American Express 371449635398431
________________________________________________________________________________
American Express Corporate 378734493671000
________________________________________________________________________________
Australian BankCard 5610591081018250
________________________________________________________________________________
Diners Club 30569309025904
________________________________________________________________________________
Diners Club 38520000023237
________________________________________________________________________________
Discover 6011111111111117
________________________________________________________________________________
Discover 6011000990139424
_________________________________________________________________________________
JCB 3530111333300000
_________________________________________________________________________________
JCB 3566002020360505
_________________________________________________________________________________
MasterCard 5555555555554444
_________________________________________________________________________________
MasterCard 5105105105105100
_________________________________________________________________________________
Visa 4111111111111111
_________________________________________________________________________________
Visa 4012888888881881
_________________________________________________________________________________
Visa 4222222222222
_________________________________________________________________________________
Note : Even though this number has a different character count than the other test numbers, it is the correct and functional number.
Processor-specific Cards
Dankort (PBS) 76009244561
________________________________________________________________________________
Dankort (PBS) 5019717010103742
________________________________________________________________________________
Switch/Solo (Paymentech) 6331101999990016
__________________________________________________________________________________
Here is a list of dummy credit card number that can be used while testing your applications involving credit card transactions:
Visa: 4111-1111-1111-1111
_________________________________________________________________________________
MasterCard: 5431-1111-1111-1111
__________________________________________________________________________________
Amex: 341-1111-1111-1111
__________________________________________________________________________________
Discover: 6011-6011-6011-6611
_________________________________________________________________________________
Credit Card Prefix Numbers:
Visa: 13 or 16 numbers starting with 4
______________________________________________________
MasterCard: 16 numbers starting with 5
______________________________________________________
Discover: 16 numbers starting with 6011
______________________________________________________
AMEX: 15 numbers starting with 34 or 37
American Express 378282246310005
________________________________________________________________________________
American Express 371449635398431
________________________________________________________________________________
American Express Corporate 378734493671000
________________________________________________________________________________
Australian BankCard 5610591081018250
________________________________________________________________________________
Diners Club 30569309025904
________________________________________________________________________________
Diners Club 38520000023237
________________________________________________________________________________
Discover 6011111111111117
________________________________________________________________________________
Discover 6011000990139424
_________________________________________________________________________________
JCB 3530111333300000
_________________________________________________________________________________
JCB 3566002020360505
_________________________________________________________________________________
MasterCard 5555555555554444
_________________________________________________________________________________
MasterCard 5105105105105100
_________________________________________________________________________________
Visa 4111111111111111
_________________________________________________________________________________
Visa 4012888888881881
_________________________________________________________________________________
Visa 4222222222222
_________________________________________________________________________________
Note : Even though this number has a different character count than the other test numbers, it is the correct and functional number.
Processor-specific Cards
Dankort (PBS) 76009244561
________________________________________________________________________________
Dankort (PBS) 5019717010103742
________________________________________________________________________________
Switch/Solo (Paymentech) 6331101999990016
__________________________________________________________________________________
Here is a list of dummy credit card number that can be used while testing your applications involving credit card transactions:
Visa: 4111-1111-1111-1111
_________________________________________________________________________________
MasterCard: 5431-1111-1111-1111
__________________________________________________________________________________
Amex: 341-1111-1111-1111
__________________________________________________________________________________
Discover: 6011-6011-6011-6611
_________________________________________________________________________________
Credit Card Prefix Numbers:
Visa: 13 or 16 numbers starting with 4
______________________________________________________
MasterCard: 16 numbers starting with 5
______________________________________________________
Discover: 16 numbers starting with 6011
______________________________________________________
AMEX: 15 numbers starting with 34 or 37
Saturday, October 18, 2008
ACID Testing
ACID stands for :
Atomicity
Consistency
Isolation
Durability
For OLTP databases and when databases deal with mission-critical business transactions and critical information ACID features are much need for reliability & stability. For customers that demand high quality of databases to maintain confidentiality of their information, ACID features is what you need.
Atomicity:-
Atomicity describes database operations that are defined within a single transactions of nature “all or nothing”. If any single operation fails then the whole transaction fails. This ensures that the database is in a valid state at all times.
For instance, a transaction to transfer money from one bank account to another involves withdrawal & deposit to two different account. If deposit fails, withdrawal should also be reversed, else money will be lost. So these operations should be grouped into single atomic transaction to ensure data integrity. Databases provide atomicity in two ways. 1. Rollback feature, 2. Two phase commit protocol.
Consistency:-
Consistency of data is maintained by allowing only valid transactions that comply to preset database constraints. Some constraints are:
1. Unique constraint on a column - For eg: Only one SSN can be allowed per person record. if a duplicate SSN appears then that transaction should be rejected.
2. Rules: Some rules check if data is in specified format. Eg: Salary should always be numeric and alphabets should be rejected.
Many other features of database help ensure data consistency.
Isolation:-
This concurrent transactions don't overlap each other. Every transaction occurs before or after all other transactions. If huge number of concurrent transactions are expected, then a robust database system is needed which has guaranteed isolation. Databases use Locking mechanism to implement Isolation. The row, table or entire database can be locked when a particular transaction takes place. Although this ensures isolation it poses issues with response times.
Durability:-
A committed or completed transaction should persist regardless of failures of any kind. Eg : Hard disk crash, network unavailability should not affect the saved data in database. Durability is often achieved via
1. Database replication or
2. Load balanced servers where two servers simultaneously run on separate machines. Backups alone cannot provide "durability" of all transactions.
Atomicity
Consistency
Isolation
Durability
For OLTP databases and when databases deal with mission-critical business transactions and critical information ACID features are much need for reliability & stability. For customers that demand high quality of databases to maintain confidentiality of their information, ACID features is what you need.
Atomicity:-
Atomicity describes database operations that are defined within a single transactions of nature “all or nothing”. If any single operation fails then the whole transaction fails. This ensures that the database is in a valid state at all times.
For instance, a transaction to transfer money from one bank account to another involves withdrawal & deposit to two different account. If deposit fails, withdrawal should also be reversed, else money will be lost. So these operations should be grouped into single atomic transaction to ensure data integrity. Databases provide atomicity in two ways. 1. Rollback feature, 2. Two phase commit protocol.
Consistency:-
Consistency of data is maintained by allowing only valid transactions that comply to preset database constraints. Some constraints are:
1. Unique constraint on a column - For eg: Only one SSN can be allowed per person record. if a duplicate SSN appears then that transaction should be rejected.
2. Rules: Some rules check if data is in specified format. Eg: Salary should always be numeric and alphabets should be rejected.
Many other features of database help ensure data consistency.
Isolation:-
This concurrent transactions don't overlap each other. Every transaction occurs before or after all other transactions. If huge number of concurrent transactions are expected, then a robust database system is needed which has guaranteed isolation. Databases use Locking mechanism to implement Isolation. The row, table or entire database can be locked when a particular transaction takes place. Although this ensures isolation it poses issues with response times.
Durability:-
A committed or completed transaction should persist regardless of failures of any kind. Eg : Hard disk crash, network unavailability should not affect the saved data in database. Durability is often achieved via
1. Database replication or
2. Load balanced servers where two servers simultaneously run on separate machines. Backups alone cannot provide "durability" of all transactions.
Monday, September 22, 2008
Linear Code Sequence And Jump (LCSAJ)
LCSAJ means A Linear Code Sequence And Jump, consisting of the following three items (conventionally identified by line numbers in a source code listing): the start of the linear sequence of executable statements, the end of the linear sequence, and the target line to which control flow is transferred at the end of the linear sequence.
LCSAJ testing is a test case design technique for a component in which test cases are designed to execute LCSAJs.
An LCSAJ is defined as unbroken linear sequence of statements:
(a) Which begins at either the start of the program or a point to which the control flow may jump,
(b) Which ends at either the end of the program or a point from which the control flow may jump?
(c) And the point to which a jump is made following the sequence.
LCSAJ coverage = I/L
Where: I = Number of LCSAJs exercised at least once.
L = Total number of LCSAJs.
LCSAJs depends on the topology of a module's design and not just its semantics, they do not map onto code structures such as branches and loops. LCSAJs are not easily identifiable from design documentation. They can only be identified once code has already been written. LCSAJs are consequently not easily comprehensible.
Definitions:
LCSAJ: A Linear Code Sequence And Jump, consisting of the following three items (conventionally identified by line numbers in a source code listing): the start of the linear sequence of executable statements, the end of the linear sequence, and the target line to which control flow is transferred at the end of the linear sequence.
LCSAJ coverage: The percentage of LCSAJs of a component which are exercised by a test case suite.
LCSAJ testing: A test case design technique for a component in which test cases are designed to execute LCSAJs.
LCSAJ testing is a test case design technique for a component in which test cases are designed to execute LCSAJs.
An LCSAJ is defined as unbroken linear sequence of statements:
(a) Which begins at either the start of the program or a point to which the control flow may jump,
(b) Which ends at either the end of the program or a point from which the control flow may jump?
(c) And the point to which a jump is made following the sequence.
LCSAJ coverage = I/L
Where: I = Number of LCSAJs exercised at least once.
L = Total number of LCSAJs.
LCSAJs depends on the topology of a module's design and not just its semantics, they do not map onto code structures such as branches and loops. LCSAJs are not easily identifiable from design documentation. They can only be identified once code has already been written. LCSAJs are consequently not easily comprehensible.
Definitions:
LCSAJ: A Linear Code Sequence And Jump, consisting of the following three items (conventionally identified by line numbers in a source code listing): the start of the linear sequence of executable statements, the end of the linear sequence, and the target line to which control flow is transferred at the end of the linear sequence.
LCSAJ coverage: The percentage of LCSAJs of a component which are exercised by a test case suite.
LCSAJ testing: A test case design technique for a component in which test cases are designed to execute LCSAJs.
Thursday, September 18, 2008
Traceability Matrix
Definition
A traceability matrix is a table that correlates any two base lined documents that require a many to many relationship to determine the completeness of the relationship. It is often used with high-level requirements (sometimes known as marketing requirements) and detailed requirements of the software product to the matching parts of high-level design, detailed design, test plan, and test cases.
Common usage is to take the identifier for each of the items of one document and place them in the left column. The identifiers for the other document are placed across the top row. When an item in the left column is related to an item across the top, a mark is placed in the intersecting cell. The numbers of relationships are added up for each row and each column. This value indicates the mapping of the two items. Zero values indicate that no relationship exists. It must be determined if one must be made. Large values imply that the item is too complex and should be simplified.
To ease the creation of traceability matrices, it is advisable to add the relationships to the source documents for both backward traceability and forward traceability. In other words, when an item is changed in one base lined document, it's easy to see what needs to be changed in the other.
Description
A table that traces the requirements to the system deliverable component for that stage that responds to the requirement.
Size and Format
For each requirement, identify the component in the current stage that responds to the requirement. The requirement may be mapped to such items as a hardware component, an application unit, or a section of a design specification.
Requirements of Traceability Matrix
Traceability matrices can be established using a variety of tools including requirements management software, databases, spreadsheets, or even with tables or hyperlinks in a word processor.
A traceability matrix is created by associating requirements with the work products that satisfy them. Tests are associated with the requirements on which they are based and the product tested to meet the requirement.
Above is a simple traceability matrix structure. There can be more things included in a traceability matrix than shown. In traceability, the relationship of driver to satisfier can be one-to-one, one-to-many, many-to-one, or many-to-many.
Traceability requires unique identifiers for each requirement and product. Numbers for products are established in a configuration management (CM) plan.
Traceability ensures completeness, that all lower level requirements come from higher level requirements, and that all higher level requirements are allocated to lower level requirements. Traceability is also used to manage change and provides the basis for test planning.
Baseline Traceability Matrix
Description
A table that documents the requirements of the system for use in subsequent stages to confirm that all requirements have been met.
Size and Format
Document each requirement to be traced. The requirement may be mapped to such things as a hardware component, an application unit, or a section of a design specification.
Building a Traceability Matrix
Use a Traceability Matrix to:
• verify and validate system specifications
• ensure that all final deliverable documents are included in the system specification, such as process models and data models
• improve the quality of a system by identifying requirements that are not addressed by configuration items during design and code reviews and by identifying extra configuration items that are not required. Examples of configuration items are software modules and hardware devices.
• provide input to change requests and future project plans when missing requirements are identified.
• provide a guide for system and acceptance test plans of what needs to be tested.
Useful Traceability Matrices
Various traceability matrices may be utilized throughout the system life cycle. Useful ones include:
• Functional specification to requirements document: It shows that each requirement (obtained from a preliminary requirements statement provided by the customer or produced in the Concept Definition stage) has been covered in an appropriate section of the functional specification.
• Top level configuration item to functional specification: For example, a top level configuration item, Workstation, may be one of the configuration items that satisfies the function Input Order Information. On the matrix, each configuration item would be written down the left hand column and each function would be written across the top.
• Low level configuration item to top level configuration item: For example, the top level configuration item, Workstation, may contain the low level configuration items Monitor, CPU, keyboard, and network interface card.
• Design specification to functional specification verifies that each function has been covered in the design.
• System test plan to functional specification ensures you have identified a test case or test scenario for each process and each requirement in the functional specification.
Although the construction and maintenance of traceability matrices may be time-consuming, they are a quick reference during verification and validation tasks.
Sample Traceability Matrix
A traceability matrix is a report from the requirements database or repository. What information the report contains depends on your need. Information requirements determine the associated information that you store with the requirements. Requirements management tools capture associated information or provide the capability to add it.
The examples show forward and backward tracing between user and system requirements. User requirement identifiers begin with "U" and system requirements with "S." Tracing S12 to its source makes it clear this requirement is erroneous: it must be eliminated, rewritten, or the traceability corrected.
For requirements tracing and resulting reports to work, the requirements must be of good quality. Requirements of poor quality transfer work to subsequent phases of the SDLC, increasing cost and schedule and creating disputes with the customer.
A variety of reports are necessary to manage requirements. Reporting needs should be determined at the start of the effort and documented in the requirements management plan.
A traceability matrix is a table that correlates any two base lined documents that require a many to many relationship to determine the completeness of the relationship. It is often used with high-level requirements (sometimes known as marketing requirements) and detailed requirements of the software product to the matching parts of high-level design, detailed design, test plan, and test cases.
Common usage is to take the identifier for each of the items of one document and place them in the left column. The identifiers for the other document are placed across the top row. When an item in the left column is related to an item across the top, a mark is placed in the intersecting cell. The numbers of relationships are added up for each row and each column. This value indicates the mapping of the two items. Zero values indicate that no relationship exists. It must be determined if one must be made. Large values imply that the item is too complex and should be simplified.
To ease the creation of traceability matrices, it is advisable to add the relationships to the source documents for both backward traceability and forward traceability. In other words, when an item is changed in one base lined document, it's easy to see what needs to be changed in the other.
Description
A table that traces the requirements to the system deliverable component for that stage that responds to the requirement.
Size and Format
For each requirement, identify the component in the current stage that responds to the requirement. The requirement may be mapped to such items as a hardware component, an application unit, or a section of a design specification.
Requirements of Traceability Matrix
Traceability matrices can be established using a variety of tools including requirements management software, databases, spreadsheets, or even with tables or hyperlinks in a word processor.
A traceability matrix is created by associating requirements with the work products that satisfy them. Tests are associated with the requirements on which they are based and the product tested to meet the requirement.
Above is a simple traceability matrix structure. There can be more things included in a traceability matrix than shown. In traceability, the relationship of driver to satisfier can be one-to-one, one-to-many, many-to-one, or many-to-many.
Traceability requires unique identifiers for each requirement and product. Numbers for products are established in a configuration management (CM) plan.
Traceability ensures completeness, that all lower level requirements come from higher level requirements, and that all higher level requirements are allocated to lower level requirements. Traceability is also used to manage change and provides the basis for test planning.
Baseline Traceability Matrix
Description
A table that documents the requirements of the system for use in subsequent stages to confirm that all requirements have been met.
Size and Format
Document each requirement to be traced. The requirement may be mapped to such things as a hardware component, an application unit, or a section of a design specification.
Building a Traceability Matrix
Use a Traceability Matrix to:
• verify and validate system specifications
• ensure that all final deliverable documents are included in the system specification, such as process models and data models
• improve the quality of a system by identifying requirements that are not addressed by configuration items during design and code reviews and by identifying extra configuration items that are not required. Examples of configuration items are software modules and hardware devices.
• provide input to change requests and future project plans when missing requirements are identified.
• provide a guide for system and acceptance test plans of what needs to be tested.
Useful Traceability Matrices
Various traceability matrices may be utilized throughout the system life cycle. Useful ones include:
• Functional specification to requirements document: It shows that each requirement (obtained from a preliminary requirements statement provided by the customer or produced in the Concept Definition stage) has been covered in an appropriate section of the functional specification.
• Top level configuration item to functional specification: For example, a top level configuration item, Workstation, may be one of the configuration items that satisfies the function Input Order Information. On the matrix, each configuration item would be written down the left hand column and each function would be written across the top.
• Low level configuration item to top level configuration item: For example, the top level configuration item, Workstation, may contain the low level configuration items Monitor, CPU, keyboard, and network interface card.
• Design specification to functional specification verifies that each function has been covered in the design.
• System test plan to functional specification ensures you have identified a test case or test scenario for each process and each requirement in the functional specification.
Although the construction and maintenance of traceability matrices may be time-consuming, they are a quick reference during verification and validation tasks.
Sample Traceability Matrix
A traceability matrix is a report from the requirements database or repository. What information the report contains depends on your need. Information requirements determine the associated information that you store with the requirements. Requirements management tools capture associated information or provide the capability to add it.
The examples show forward and backward tracing between user and system requirements. User requirement identifiers begin with "U" and system requirements with "S." Tracing S12 to its source makes it clear this requirement is erroneous: it must be eliminated, rewritten, or the traceability corrected.
For requirements tracing and resulting reports to work, the requirements must be of good quality. Requirements of poor quality transfer work to subsequent phases of the SDLC, increasing cost and schedule and creating disputes with the customer.
A variety of reports are necessary to manage requirements. Reporting needs should be determined at the start of the effort and documented in the requirements management plan.
Wednesday, September 10, 2008
IEEE 829: Standard For Software Test Documentation
IEEE 829-1998, also known as the 829 Standard for Software Test Documentation, is an IEEE standard that specifies the form of a set of documents for use in eight defined stages of software testing, each stage potentially producing its own separate type of document. The standard specifies the format of these documents but does not stipulate whether they all must be produced, nor does it include any criteria regarding adequate content for these documents. These are a matter of judgments outside the purview of the standard. The documents are:
• Test Plan: a management planning document that shows:
• How the testing will be done (including SUT configurations).
• Who will do it
• What will be tested
• How long it will take (although this may vary, depending upon resource availability).
• What the test coverage will be, i.e. what quality level is required
• Test Design Specification: detailing test conditions and the expected results as well as test pass criteria.
• Test Case Specification: specifying the test data for use in running the test conditions identified in the Test Design Specification
• Test Procedure Specification: detailing how to run each test, including any set-up preconditions and the steps that need to be followed
• Test Item Transmittal Report: reporting on when tested software components have progressed from one stage of testing to the next
• Test Log: recording which tests cases were run, who ran them, in what order, and whether each test passed or failed
• Test Incident Report: detailing, for any test that failed, the actual versus expected result, and other information intended to throw light on why a test has failed. This document is deliberately named as an incident report, and not a fault report. The reason is that a discrepancy between expected and actual results can occur for a number of reasons other than a fault in the system. These include the expected results being wrong, the test being run wrongly, or inconsistency in the requirements meaning that more than one interpretation could be made. The report consists of all details of the incident such as actual and expected results, when it failed, and any supporting evidence that will help in its resolution. The report will also include, if possible, an assessment of the impact upon testing of an incident.
• Test Summary Report: A management report providing any important information uncovered by the tests accomplished, and including assessments of the quality of the testing effort, the quality of the software system under test, and statistics derived from Incident Reports. The report also records what testing was done and how long it took, in order to improve any future test planning. This final document is used to indicate whether the software system under test is fit for purpose according to whether or not it has met acceptance criteria defined by project stakeholders.
Relationship with other standards
Other standards that may be referred to when documenting according to IEEE 829 include:
• IEEE 1008, a standard for unit testing
• IEEE 1012, a standard for Software Verification and Validation
• IEEE 1028, a standard for software inspections
• IEEE 1044, a standard for the classification of software anomalies
• IEEE 1044-1, a guide to the classification of software anomalies
• IEEE 1233, a guide for developing system requirements specifications
• IEEE 730, a standard for software quality assurance plans
• IEEE 1061, a standard for software quality metrics and methodology
• IEEE 12207, a standard for software life cycle processes and life cycle data
• BSS 7925-1, a vocabulary of terms used in software testing
• BSS 7925-2, a standard for software component testing
Use of IEEE 829
The standard forms part of the training syllabus of the ISEB Foundation and Practitioner Certificates in Software Testing promoted by the British Computer Society. ISTQB, following the formation of its own syllabus based on ISEB's and Germany's ASQF syllabi, also adopted IEEE 829 as the reference standard for software testing documentation.
• Test Plan: a management planning document that shows:
• How the testing will be done (including SUT configurations).
• Who will do it
• What will be tested
• How long it will take (although this may vary, depending upon resource availability).
• What the test coverage will be, i.e. what quality level is required
• Test Design Specification: detailing test conditions and the expected results as well as test pass criteria.
• Test Case Specification: specifying the test data for use in running the test conditions identified in the Test Design Specification
• Test Procedure Specification: detailing how to run each test, including any set-up preconditions and the steps that need to be followed
• Test Item Transmittal Report: reporting on when tested software components have progressed from one stage of testing to the next
• Test Log: recording which tests cases were run, who ran them, in what order, and whether each test passed or failed
• Test Incident Report: detailing, for any test that failed, the actual versus expected result, and other information intended to throw light on why a test has failed. This document is deliberately named as an incident report, and not a fault report. The reason is that a discrepancy between expected and actual results can occur for a number of reasons other than a fault in the system. These include the expected results being wrong, the test being run wrongly, or inconsistency in the requirements meaning that more than one interpretation could be made. The report consists of all details of the incident such as actual and expected results, when it failed, and any supporting evidence that will help in its resolution. The report will also include, if possible, an assessment of the impact upon testing of an incident.
• Test Summary Report: A management report providing any important information uncovered by the tests accomplished, and including assessments of the quality of the testing effort, the quality of the software system under test, and statistics derived from Incident Reports. The report also records what testing was done and how long it took, in order to improve any future test planning. This final document is used to indicate whether the software system under test is fit for purpose according to whether or not it has met acceptance criteria defined by project stakeholders.
Relationship with other standards
Other standards that may be referred to when documenting according to IEEE 829 include:
• IEEE 1008, a standard for unit testing
• IEEE 1012, a standard for Software Verification and Validation
• IEEE 1028, a standard for software inspections
• IEEE 1044, a standard for the classification of software anomalies
• IEEE 1044-1, a guide to the classification of software anomalies
• IEEE 1233, a guide for developing system requirements specifications
• IEEE 730, a standard for software quality assurance plans
• IEEE 1061, a standard for software quality metrics and methodology
• IEEE 12207, a standard for software life cycle processes and life cycle data
• BSS 7925-1, a vocabulary of terms used in software testing
• BSS 7925-2, a standard for software component testing
Use of IEEE 829
The standard forms part of the training syllabus of the ISEB Foundation and Practitioner Certificates in Software Testing promoted by the British Computer Society. ISTQB, following the formation of its own syllabus based on ISEB's and Germany's ASQF syllabi, also adopted IEEE 829 as the reference standard for software testing documentation.
Subscribe to:
Posts (Atom)
