CSQA CSQA Certified Software Quality Analyst Questions and Answers

Industry accepted definitions of quality are “conformance to requirements” (from Philip Crosby) and “fit for use” (from Dr. Joseph Juran and Dr. W. Edwards Deming). These two definitions are not inconsistent.

Meeting requirements is a producer’s view of quality. This is the view of the organization responsible for the project and processes, and the products and services acquired, developed, and maintained by those processes. Meeting requirements means that the person building the product does so in accordance with the requirements. Requirements can be very complete or they can be simple, but they must be defined in a measurable format, so it can be determined whether they have been met. The producer’s view of quality has these four characteristics:

Doing the right thing

Doing it the right way

Doing it right the first time

Doing it on time without exceeding cost

Being fit for use is the customer’s definition. The customer is the end user of the products or services. Fit for use means that the product or service meets the customer’s needs regardless ofthe product requirements. Of the two definitions of quality, fit for use, is the more important. The customer’s view of quality has these characteristics:

Receiving the right product for their use

Being satisfied that their needs have been met

Meeting their expectations

Being treated with integrity, courtesy and respect

Reduce the Number of Vendors

Requiring statistical evidence of quality control in the purchase of hardware and software will mean, in most companies, drastic reduction in the number of vendors with whom they deal. Companies will have to consider the cost of having two or more vendors for the same item. A company will be lucky to find one vendor that can supply statistical evidence of quality. A second vendor, if they cannot furnish statistical evidence of their quality, will have higher costs than the one that can furnish the evidence, or they will have to chisel on their quality, or go out of business. A person that does not know his/her costs or whether today's distribution of quality can be repeated tomorrow is not a good business partner.

Use Statistical Methods to Find Sources of Trouble

Which are user faults? Which faults belong to IT? Put responsibility where it belongs. Do not rely on judgment because it always gives the wrong answer on the question of where the fault lies. Statistical methods make use of knowledge of the subject matter where it can be effective, but supplant it where it is a hazard.

Constantly improve the system. This obligation never ceases. Most people in management do not understand that the system (their responsibility) is everything not under the governance of a user.

Institute Modern Aids to Training on the Job

Training must be totally reconstructed. Statistical methods must be used to learn when training is finished, and when further training would be beneficial. A person once hired and trained and in statistical control of his/her own work, whether it be satisfactory or not, can do no better. Further training cannot help. If their work is not satisfactory, move them to another job, and provide better training there.

The Quality Charter should contain the following:

Scope

The scope should involve activities supporting the implementation of the quality policy, and closing the producer and customer gaps.

Objectives

The objectives clarify the role of the quality function, and normally move through three phases as quality processes mature. Product quality is the initial focus, followed by the definition and improvement of work processes. Finally, objectives are oriented towards optimization. The next section covers the maturation of the quality function.

Responsibilities

Responsibilities assigned to the quality function should be important, meaningful, accomplishable and measurable. They should be defined in sufficient detail so that the accomplishment or non-accomplishment of the tasks can be determined. The quality function is not responsible for product quality, but will have responsibilities supporting quality initiatives. These responsibilities should include maintaining current knowledge of quality principles and practices; identifying companies and sources that can provide best processes and practices; training, coaching, and facilitating; and maintaining and analyzing databases of information that will help identify process and product weaknesses as candidates for improvement. Additional responsibilities may include participating on committees; providing process management expertise; serving as a centralized resource for measurement analysis and reporting; acting as custodian for processes by formatting, editing, publishing, distributing, and controlling access to changes, etc. Auditing for process deployment and compliance may also be a responsibility, although this is more desirable separated from QA.

Authority

The quality function needs the authority to perform its responsibilities. This authority should give the quality staff access to all documentation and data that is necessary to analyze and identify problems. In many organizations the quality function receives data that is not available to line management. For example, from product inspections it may know the type and quantity of defects made by individuals in building products. Since inspections are designed to assist the worker in improving quality, the usefulness may be compromised if the worker believes management will use them negatively in a performance appraisal. On the other hand, the quality staff needs that information to identify defect-prone products and processes.

Ongoing Quality Programs

The charter should specify programs that will be performed regularly by the quality function. These might include building and analyzing defect databases, assisting in the development of work processes; summarizing and analyzing quantitative data; training the IT staff in quality principles, practices, and processes; and assisting in the performance of tasks such as software reviews and inspections. These programs may change over time as the quality program matures. For example, initially the quality function may chair reviews and inspections, but when that function matures, the review and inspection responsibilities go to the developers of the products.

Joiner outlined this strategy for reducing common causes of variation:

"Talk to lots of people including local employees, other managers, and staff from various functions.”

“Improve the measurement processes if measuring contributes too much to the observed variation.”

“Identify and rank categories of problems by Pareto analysis.”

If an organization is not ready, and also lacks the sales skills to convince the management group, the probability of success is so low that it is best not to bother with implementation at this time. A ready organization with good salesmanship can take care of itself. The other scenarios present different challenges. Organizations that are not ready, but have the salesmanship, are referred to as

“Challenge 1.” Organizations that are ready, but lack the salesmanship, are referred to as

“Challenge 2.” Each challenge is analyzed separately.

Challenge 1

The solution is to get ready. To get ready, an organization needs the following:

The presence of a competent, trained staff that is capable of performing quality work when building and maintaining systems. Additionally, there must be backup for these individuals. In a moderate-size organization, quality initiatives might be assigned to one person. With the substantial effort to get it going, it can be traumatic if that individual becomes promoted or should leave.

A good IT management team that is capable of setting up the quality function, monitoring it, and making it work. The existence of good management tools, especially project management and systems development methodologies that spell out what to do when,and are specific about where the quality review functions belong in the development cycle and in maintenance.

A limit on the number of internal improvement projects to be done at once.

Good credibility. If there has been a history of not doing things that were promised it may be premature to initiate a quality function until a track record of satisfying promises has been accomplished.

To be mentally prepared for the length of time (measured in months and years, not weeks) to get a new and important function like this into operation and stabilized.

Challenge 2

The process of selling a new quality function is similar to the process of selling anything associated with internally improving an organization. Some items to consider include the following:

To be an issue of real value to the total organization, the quality function must be heartfelt to the salesperson. Without a very strong belief in quality, no one can sell it.

When the proposition is taken to the management group, it is essential for the salesperson to speak the language of the listener.

Management values their time, which places a burden on the presenter to think the proposition through, present the case, ask for a clear decision, and then leave the room when receiving an affirmative answer. Most executive managers will buy into any well thought-out plan that has the necessary controls built in and where they can conceptually see the logic of how the pieces fit together.

It is essential to present a realistic picture. Extreme benefits should not be promised, nor should the salesperson claim that problems could be solved in a very short period of time. Most executive managers are astute enough to see through the unrealistic claims rapidly, weakening the total case.

There is an important principle of getting some things accepted, which is referred to as the Mafia Rule. This is an offer that management can't refuse. Offer that the management group can make all of the decisions on quality issues, while the proponents of the quality function will handle the administrative details.

Ideas are typically accepted only after the person presenting the ideas is accepted. If the salesperson’s personal relationship with management is weak, some "fencemending" should be done before asking management for a major decision.

Using outsiders can sometimes help in getting the right decision. The simple process of paying for advice somehow makes that advice seem more important.

The IT Quality Plan should include the following:

A reference to the organization's quality policy

An assessment of where the organization currently stands in relation to accomplishing the quality policy

Long-term quality goals - these are discussed below

Short-term quality objectives (i.e., programs) - these are discussed below

The means of implementing the quality objectives

Resources required in order to accomplish the short-term objectives and long term-goals

Assure Completeness of Needs Requirements

This step determines whether you have adequately defined your requirements. Your requirements should be defined in terms of the following two categories of outputs:

1. Output Products and Output Reports -Output products and output reports are specific documents that you want produced by the computer system. In many instances, such as the previous payroll check example, the style and format of these output products is important. This does not mean that the specific location of the check has to be defined but, rather, the categories of information to be included on the check. Computer-produced reports may also be important for tax information (e.g., employee withholding forms sent to governmental units), financial statements where specific statements are wanted (e.g., balance sheets or statements of income and expense), or customer invoice and billing forms which you might want preprinted to include your logo and conditions of payment.

2.Management decision information. This category tries to define the information needed for decision-making purposes. In the computer product/report category you were looking for a document; in this case you are looking for information. How that information is provided is unimportant. Thus, the structure of the document, what the documents are, or their size, frequency, or volume are not significant. All you need is information.

Define Critical Success Factor

This step tells whether the COTS software will be successful in meeting your requirements. Critical success factors (CSFs) are those criteria or factors that must be present in the acquired software for it to be successful. You might ask whether the needs are the same as the critical success factors. They are, but they are not defined in a manner that makes them testable, and they may be incomplete. Often the needs do not take into account some of the intangible criteria that make the difference between success and failure. In other words, the needs define what we are looking for, and the critical success factors tell us how we will evaluate that product after we get it. They are closely related and complementary, but different in scope and purpose.

The following list indicates the needs or requirements for the automobile, and is then followed by the CSFs on which the automobile will be evaluated:

Seats six people

Four doors

Five-year guarantee on motor

Gets 20 miles per gallon or greater

Costs under $12,000

Critical success factors:

Operates at 20.5 cents or less per mile

Experiences no more than one failure per year

Maintains its appearance without showing signs of wear for two years

Some of the more common critical success factors for COTS you may want to use are:

Ease of use - The software is understandable and usable by the average person.

Expandability - The contractor plans to add additional features in the future.

Maintainability - The contractor will provide support/assistance to help utilize the package in the event of problems.

Cost-effectiveness - The software package makes money for your business by reducing costs, and so on.

Transferability - If you change your computer equipment the contractor indicates that they will support new models or hardware.

Reliability - In computer language, the system is friendly, meaning that it will help you get your transactions entered into the system so that you can produce your results readily.

Security - The system has adequate safeguards to protect the data against damage (for example, power failures, operator errors, or other goofs that could cause you to lose your data).

Determine Compatibility with Hardware, Operating System, and Other COTS Software

This is not a complex step. It involves a simple matching between your processing capabilities and limitations, and what the contractor of the software says is necessary to run the software package. The most difficult part of this evaluation is ensuring the multiple software packages can properly interface.

This step is best performed by preparing a checklist defining your compatibility needs. Software contractors are generally good about identifying the needed hardware and operating system compatibility. They are generally not good in identifying compatibility with other software packages.

Level 1 – Validation

Validation is testing the software in an operational state. While there are several iterations of testing, they do not occur until the latter part of the system development life cycle. Thus, validation does not take advantage of the control theory stating that the control should be placed as close as possible to where the defect occurs. The most common types of validation are unit testing, integration testing, and system testing. Unit testing tests the smallest operating module of a system; integration testing tests the connecting logic between the modules/units; and the system testing validates that the system executes in the organization's operating environment.

Level 2 – Verification

Verification is the static testing of the system in a nonoperational status. It primarily tests the documentation produced by the project team. The documentation includes requirements, design,code, test, and operating documentation. Verification is performed through a variety of processes, including code analyzers, walkthroughs, reviews, and inspections. These verification processes also interact with the project team, end users, and other interested parties. Level 2 also incorporates acceptance testing, in which the end users define the criteria, which the system must meet in order to, be acceptable, and then test against those criteria.

Level 3 -- Defect Management

This level involves naming defects, counting defects, recording defects in a database, analyzing them, and then managing them throughout the entire life cycle of software. There are two types of defects, just as there are two types of quality. One defect is a variance from requirements/specifications, and the second defect is anything that dissatisfies the end user. For example, if the system was to add A plus B to get C, but in fact added A to X that would be a variance from specification defect. If the output was correct, but hard to use from an end user viewpoint, it would be an ease of use defect. Defects are generally only considered to be defects when they are uncovered in a task beyond the task in which the defect occurred. If the individual who makes the defect finds it during the same task, it is generally not considered to be a defect.

Level 4 -- Statistical Process Control

To be effective, statistical process control needs stabilized processes and defect recording. Some statistical process control techniques will be introduced at Level 3, but their use will be confined more to process improvement than for process management. Good statistical process control techniques require the type of measurement data that is only produced at Level 4. The statistical process control techniques will expand on root cause analysis, use sophisticated statistical tools and analysis, and create dashboards to be used in managing processes.

Level 5 - Preventive Management

This level is focused on preventing defects. It uses the lessons learned from the previous four levels of control/test to address the problems before they can become defects. During the previous four levels, the causes of defects are available and some addressed through process improvement. This level accelerates that trend and in addition provides the professional information staff with methods to modify the way work is performed so that the root causes of defects will not be built into software. This involves building risk models, developing defect expectation charts and/or defect profiles, and processes to customize processes so that the do component of processes can build software that is almost defect free.

Develop a matrix of process results and contributors to show which contributors drive which results. The results should come from the process policy. The contributors can be positive or negative, and involve process, product, or resource attributes. Process attributes includecharacteristics such as time, schedule, and completion. Product attributes include characteristics such as size, correctness, reliability, usability, and maintainability. Resource attributes include characteristics such as amount, skill, and attitude. A cause-and-effect diagram is often used to graphically illustrate the relationship between results and contributors.

Assure process results are aligned to business results. Processes should help people accomplish their organization’s mission. Alignment is subjective in many organizations, but the more objective it is, the greater the chance that processes will drive the mission.

Rank the process results and the contributors from a management perspective. This will help workers make tradeoffs and identify where to focus management attention.

Select metrics for both the process results and contributors, and create two tactical process dashboards: one for process results and one for contributors. These dashboards are used to manage the projects and to control and report project status. Normally results are measured subjectively and contributors are measured objectively. For example, for a result of customer satisfaction, contributors might include competent resources, an available process, and a flexible and correct product. Sometimes, as with customer satisfaction, factors that contribute to achieving the result can actually be used to develop the results metric. In other words, first determine what contributes to customer satisfaction or dissatisfaction and then it can be measured.

While watching the demonstration, let part of your mind be a casual overseer of the entire process. Attempt to get a feel for what is happening and how that might impact your business. You want to end up being able to assess whether you feel good about the software. If you have concerns, attempt to articulate them to the demonstrator as well as possible to determine how the demonstrator responds and addresses those concerns.

The steps below describe a sample defect tracking process. Depending on the size of the project or project team, this process may be substantially more complex.

1. Execute test and log any discrepancies.

The tester executes the test and compares the actual results to the documented expected results. If a discrepancy exists, the discrepancy is logged as a “defect” with a status of “open.” Supplementary documentation, such as screen prints or program traces, is attached if available.

2. Determine if discrepancy is a defect.

The Test Manager or tester reviews the defect log with an appropriate member of the development team to determine if the discrepancy is truly a defect, and is repeatable. If it is not a defect, or repeatable, the log should be closed with an explanatory comment.

3. Assign defect to developer.

If a defect exists it is assigned to a developer for correction. This may be handled automatically by the tool, or may be determined as a result of the discussion in step 2.

4. Defect resolution process.

When the developer has acknowledged the defect is valid, the resolution process begins.

Both the business staff and the quality staff should be involved in IT planning. Involvement is in both strategic and tactical planning.

The objective of this single planning cycle is to ensure that adequate resources and time are available to perform the quality activities. The net result is that the individuals executing the business plan cannot differentiate the quality planning from the business planning. For example, if business planning calls for a quality review prior to the end of each phase of software development, the business staff will assume, that is a logical part of the software development process. If it is not integrated, the review is owned by the quality professional and may not be adequately supported by the IT business staff, such as, system designers and programmers.

Rahail HDD:Users:iMac:Desktop:Screen Shot 2015-01-12 at 6.45.40 PM.png

The figure gives an example of business planning and an example of quality planning. While the blocks show that strategic and tactical business planning and strategic and tactical quality planning occur, the end result is a business plan that incorporates quality activities.

Measure defect rate in defects per thousand function points or defects per thousand lines of code. As processes become better defined, controls become more effective, and as people become more motivated, the defect rate drops significantly.

Rahail HDD:Users:iMac:Desktop:Screen Shot 2015-01-12 at 3.24.25 PM.png

1 – The major advantages are”

Relinquishment of control -The software is developed by individuals who are not employees of the organization,and thus it is difficult to oversee the development process. The contractingorganization cannot direct the employees of the other organization, nor have controlover the many day-to-day decisions that are made in developing software.

Loss of control over reallocation of resources -If work needs to be done to correct problems and/or speed up development, thecontractor cannot take workers off one project and assign them to another project.

2 – The major disadvantages are:

Cultural differences -There may be a significant difference in the culture and values between the contractorand the offshore organization.

Communication barriers -The language of the offshore organization may be different or difficult to comprehendwhich causes difficulty in communicating the needs and desires of the contractor.

A list of five critical success factors required to evaluate COTS software is as follows:

Assure Completeness of Needs Requirements

Define Critical Success Factor

Determine Compatibility with Hardware, Operating System, and other COTS Software

Assure the Software can be Integrated into Your Business System Work Flow

Demonstrate the Software in Operation

When addressing security issues, some general information security principles should be kept in mind, as follows:

Simplicity

Security mechanisms (and information systems in general) should be as simple as possible. Complexity is at the root of many security issues.

Fail Safe

If a failure occurs, the system should fail in a secure manner. That is, if a failure occurs, security should still be enforced. It is better to lose functionality than lose security.

Complete Mediation

Rather than providing direct access to information, mediators that enforce access policy should be employed. Common examples include file system permissions, web proxies and mail gateways.

Open Design

System security should not depend on the secrecy of the implementation or its components. “Security through obscurity” does not work.

Separation of Privilege

Functions, to the degree possible, should be separate and provide as much granularity as possible. The concept can apply to both systems and operators and users. In the case of system operators and users, roles should be as separate as possible. For example, if resources allow, the role of system administrator should be separate from that of the security administrator.