TPI NEXT Checklist

Language: English | Українська

How to use this checklist:

  • You don't have to follow all questions one by one. Just make sure prepare for the audit in advance to cover all topics
  • Be honest
  • Select Yes, No, or N/A for each item
  • Click Notes to add additional comments for any item. Try to make notes as often as possible
  • Your progress is automatically saved in your browser
  • Use the progress bar to track completion
  • Use Reset to clear all selections and start over
  • Check the matrix in the bottom to analyze result
  • Export results into csv or pdf

Credits: This checklist is based on Sogeti’s Test Process Improvement model – TPI NEXT, with AI category added by QAMania.

Privacy Notice: All data is stored locally in your web browser only. No information is sent to or stored on our servers.

Stakeholder commitment

Controlled

Stakeholders commit to and support the test process by granting and delivering negotiated resources

Description Yes No N/A Notes
The principal stakeholder is defined (not necessarily documented) and known to the testers
Budget for test resources is granted by and negotiable with the principal stakeholder
Stakeholders actually deliver the committed resources
The principal stakeholder is responsible for a documented product risk analysis (the input for the test strategy)

Efficient

Stakeholders anticipate the implications of changes, thus enabling the test process to respond adequately

Description Yes No N/A Notes
All relevant stakeholder are defined (not necessarily documented) and known to the testers
Stakeholders actively acquire information on the quality of both the test process and the test object
The stakeholders proactively take action on aspects that affect the test process. This includes changes in the delivery sequence of the test object and changes in the project scope

Optimized

Stakeholders recognize and stimulate process improvement as a shared responsibility

Description Yes No N/A Notes
Line management acknowledges that test process improvement comes with the need for increased learning time for which resources are provided
Stakeholders are willing to adapt their way of working to suit the test process. This includes the software development and requirements management
An adapted way of working by the stakeholder to suit demands of the test process is jointly evaluated by the test organization and the stakeholder

Degree of involvement

Controlled

Test activities are started up early to enable systematic preparations

Description Yes No N/A Notes
The test assignment, scope and approach are negotiated early with the principal stakeholder as one of the first test activities
Test activities are started early, timely before test execution, with the goal of keeping the test activities of the project's critical path
A tester is involved in project planning: dependencies between the test process and other processes are taken into account
A tester is involved in the analysis and mitigation of overall project risks

Efficient

The involvement of testing enables reliable test process output and prevention of defects

Description Yes No N/A Notes
Testers contribute to impact and risk analysis of change requests and changes to the test basis
Testers contribute to the impact analysis of defects
Testers are actively involved in optimizing the test basis (more than a testability review), in which the object under test is described

Optimized

The involvement of testing in the project makes it possible to optimize both the project and the test process

Description Yes No N/A Notes
The test team is involved in the evaluation of the project. The lessons learned from the test process are valued and used for (the set up of) future projects
The test team has an undisputed part in all relevant development activities, being accepted and valued

Test strategy

Controlled

The test strategy enables the distribution of the test efforts and resources among the test levels and test activities

Description Yes No N/A Notes
The principal stakeholder agrees with the documented test strategy
The test strategy is based on a product risk analysis
There is a differentiation in test levels, test types, test coverage and test depth, depending on the analyzed risks
For retests and regression testing a simple strategy determination takes place

Efficient

The test strategy is used to achieve the adequate balance between analyzed product risks, test coverage and available efforts and resources, involving all stakeholders

Description Yes No N/A Notes
All relevant stakeholders agree with the defined (and documented) test strategy
The overlap or gaps in test coverage between test levels or test types are well considered
The test strategy includes adequate test design techniques

Optimized

The method of the test strategy is maintained well to ensure easy and valid application

Description Yes No N/A Notes
The process of creating the test strategy is evaluated regularly and, where necessary, adapted for future use
The test strategy itself is evaluated against metrics on incidents that occur in production

Test organization

Controlled

A test organization enables uniformity in test approach, test products and procedures, agreements and clear test results

Description Yes No N/A Notes
People involved know where to find the persons (or department) responsible for test services
There is a structure of control and accountability within the test organization
Test tasks and responsibilities are defined (and documented) and are assigned to a person or organizational unit
The products and services of the test organization are clear to its clients

Efficient

A test organization enables the right test expertise and experience at the right place

Description Yes No N/A Notes
Different persons or departments that provide test services reconcile the organizing of their test work
The test organization provides the agreed test resources and services to the projects
A well considered choice has been made as where and how to position the test organization
The test policy is followed

Optimized

A test organization leads to continuously improved results by test services

Description Yes No N/A Notes
The products and services of the test organization are regularly evaluated; new services are added when they are cost-effective
The test organization is held accountable for success and failure of test assignments
The test organization performance is regularly compared with external suppliers or similar test organizations

Communication

Controlled

Communication of information to all involved makes it possible to take the appropriate decisions

Description Yes No N/A Notes
Every team member is aware of decisions being made and of internal progress
The test team actively gathers relevant information from stakeholders
It is possible to trace back points of action, agreements and decisions of the test team
The test team deliberates with stakeholders about progress, product quality and risks and signals proactively potential delays as well

Efficient

Concise form and content of information for the target audience contributes to more productive work

Description Yes No N/A Notes
The test team identifies which information needs to be shared with which stakeholder
The test team participates in relevant meetings with other stakeholders
The test team has the different (communication) means necessary to its disposal to communicate with its stakeholders using an appropriate form

Optimized

Communication is a means for team building

Description Yes No N/A Notes
Best practices and lessons learned regarding communication and its efficiency are evaluated during and at the end of a test project for future purposes
The organization investigates the use of new means of communication and defines policies

Reporting

Controlled

The test team and stakeholders agree on what is reported and how, thus preventing a lack of information or an information overload

Description Yes No N/A Notes
The reporting contains aspects of time and/or costs, results and risks
The reporting frequency and content matches the basic demands of stakeholders for the decision making process
Reporting is done in writing

Efficient

Reporting is tuned to specific target groups to support the decision making process

Description Yes No N/A Notes
Fulfilling the reporting demands of stakeholders, needed for an efficient decision making process, is balanced against the effort for providing it
The reporting contains trends and recommendations concerning test process progress and project risks
The reporting contains trends and recommendations concerning test goals and product risks

Optimized

Reporting provides information that can be used to optimize the software development lifecycle

Description Yes No N/A Notes
Reporting provides data and/or measurements that can be used for current and future improvements of the test process and the software development lifecycle
The data and/or measurements for software process improvement are handed over to a line organization at the test project closure

Test process management

Controlled

Proactive management of the test process enables the fulfillment of the test assignment

Description Yes No N/A Notes
At the start of the test project a test plan is created. The test plan includes at least the test assignment, the test scope, the test planning, the roles and responsibilities
The test plan is agreed with the principal stakeholder
Each test activity is monitored and when necessary adjustments are initiated
The test plan is agreed with the relevant stakeholders

Efficient

Managing the test process with clear authorizations makes instant adjustments possible to keep the test project on track

Description Yes No N/A Notes
(Anticipated) Deviations of the test plan are discussed with the principal stakeholder and other relevant stakeholders
Adjustments to the test plan are documented
The test lead has a delegated mandate for the (re-)allocation of resources

Optimized

Lessons learned on test process management advance the effectiveness and efficiency of steering test projects to their required end result

Description Yes No N/A Notes
Test process management is regularly evaluated, internally (by the test organization) and with stakeholders
Lessons learned from earlier test projects are used for improving test process management

Estimating and planning

Controlled

The amount of resources per activity required is predicted

Description Yes No N/A Notes
For test effort estimation, simple techniques are used such as ratios
For each test activity there is an indication of the period in which it runs, the resources required and the products to be delivered. Activities to be identified are: test planning and management, defining test cases and executing test cases
The dependencies between test phases or test activities are plotted in a test planning. A certain overlap of test phases and test activities is allowed
Test estimations and test planning are discussed with the principal stakeholder

Efficient

Formal techniques make estimating and planning reliable

Description Yes No N/A Notes
To be as accurate as possible, at least two estimating techniques are used
The test phases and/or test activities are estimated and planned, using formal techniques
Metrics are used to support the estimating and planning activities
The test planning includes a testability review of the test basis and the evaluation of the test project

Optimized

Estimating is based on experience data from the organization

Description Yes No N/A Notes
The test planning includes the conservation of testware for future reuse
A set of estimation techniques and principles are maintained at an organizational level
Key figures/data for the defined estimation techniques are provided at an organizational level

Metrics

Controlled

With the defined metrics it is possible to estimate and monitor the test process

Description Yes No N/A Notes
In the test process metrics are defined and used to estimate and control the test project
The necessary input for the metrics is recorded uniformly and the defined metrics are systematically stored
The input (data) for metrics is provably accurate

Efficient

The provided objectivity through metrics outweighs the effort of collecting and analyzing the measurement data

Description Yes No N/A Notes
The required effort to collect, analyze and value the necessary data is measured against the benefits
Collecting metrics does not conflict with the progress and quality of the test process
In the test process metrics are defined and used to measure the efficiency of the test process
Conclusions coming from analyzed metrics are discussed with stakeholders and are acted upon

Optimized

Metrics meet the ever changing information need

Description Yes No N/A Notes
The way metrics contribute to the information need is monitored
Changes in information need lead to new or optimization of metrics

Defect management

Controlled

Defects are tracked at an individual level and defect status is monitored

Description Yes No N/A Notes
The defect lifecycle is defined (including a retest) and applied
The following items are recorded for each defect: unique ID, related test case ID (if applicable), person reporting the defect, date, severity category, description (the actions to reproduce the defect, expected and observed result) and defect status
For further handling of defects the responsibilities are defined
All those involved in assessing and solving defects have access to the relevant defect management tool

Efficient

Common aspects of defects are analyzed to find similar defects

Description Yes No N/A Notes
The defect management tool enforces the authorization structure for status transitions of defects
All persons involved in logging and/or tracking defects use the same defect management tool or separate defect management tools with a seamless connection
The defect administration lends itself for extensive reporting possibilities, which means that reports can be selected and sorted in different ways
Trends are identified. For this, more information is recorded about a defect, the subsystem, priority, program and version, test basis and version, root cause, all status transitions and problem solver

Optimized

Defects are analyzed for common properties to avoid future defects

Description Yes No N/A Notes
A set of guidelines for defect management is provided by the line organization or project management, and used for each test project
Defect management is the responsibility of the line or project organization, where the test process provides the necessary data
Defects are analyzed for common properties and recommendations are made to avoid future defects

Testware management

Controlled

All used test and design documents in an approved state, are individually identified and registered

Description Yes No N/A Notes
The test basis, the test object and all testware are identified by name and version
Each test case is related to a test basis document in a transparent way
The test team has access to all items under testware management
The procedure by which testware, the test basis and the test object are managed is explicitly laid down and known to the test team

Efficient

The relations between all (test) artifacts are known and kept up to date

Description Yes No N/A Notes
The test basis, the test object and all testware are referenced by name and version
Traceability is provided between test cases and the requirements
Testware management is supported by a logical storage structure, roles and an authorization structure

Optimized

Testware is made available for reuse in furture test projects and actually reused

Description Yes No N/A Notes
It is agreed at test project start and rethought during the test project which testware will be conserved at the closure of the test project
Guidelines for conserving testware for reuse are available and the reuse of testware is measured
At the end of the project, the testware that will be handed over to maintenance is easily separable from the testware that is not going to be maintained

Methodology practice

Controlled

A described test method enables predictable execution of test activities

Description Yes No N/A Notes
The test process follows a documented test method: the test method describes a set of test activities, the test products delivered by the test project and additional requirements on the way of working
The test method fits the development method applied by the project
The test projects consider the implemented test method to be of practical use

Efficient

The described test method provides practical support for the test projects

Description Yes No N/A Notes
The test method describes for all test activities the goal, the responsible role, any techniques to be used and preconditions
A complete and comprehensive set of templates is provided as part of the test method
Mandatory, conditional and optional elements of the test method are indicated
The mandatory and conditional elements are put into practice

Optimized

Deviations from the test method are evaluated and lead to enhancing the method

Description Yes No N/A Notes
Test teams structurally provide feedback regarding the test method
The implemented test method is continuously enhanced and improved

Tester professionalism

Controlled

With specific test method skills and competences the tester makes the test process more predictable and manageable

Description Yes No N/A Notes
Testers have received specific test training and/or have sufficient experience in the field of structured testing
Testers are familiar with the adopted test method and they apply it
All required expertise - whether industry, business or technical - is available to the test team
Testers are regularly evaluated on specific testing skills as well as general IT abilities in an employee performance appraisal

Efficient

Test personnel have a dedicated role or function for testing and they fulfill their task in line with expectations

Description Yes No N/A Notes
Testers are certified in testing (such as TMap Next or ISTQB)
Testers can explain the rationale behind chosen techniques that have been applied
Test personnel enjoy their job and have a good relationship with other skill groups in the project
Test tasks are defined, allocated and executed in line with expectations

Optimized

Test personnel act from a Quality Assurance point of view thus constantly developing and improving their skills

Description Yes No N/A Notes
Testers actively participate in Special Interest Groups, attend test seminars or read test literature to keep their skills up to date
Test functions are part of the organization’s Human Resource Management and personal career development
Testers strive towards accountability and responsibility for their own work and continuous improvement of their work process

Test case design

Controlled

The test cases make the test execution repeatable and person independent

Description Yes No N/A Notes
The test cases are recorded on a logical level
The test cases consist of a description of: a) initial situation, b) change process = test actions to be performed, c) predicted result
The test cases provide insight into which part of the test basis, describing a specific system behavior, is subject to the test

Efficient

Designing test cases that focus on achieving a specific coverage provides a justified elaboration of the test strategy

Description Yes No N/A Notes
The test cases are understandable to and maintainable by peers within the test organization
The coverage level of the test basis - as reached by the test cases - is known
Formal test design techniques are used to design test cases
Checklists are used for the testing of quality characteristics for which no test cases can be designed

Optimized

Evaluation of test cases, test design techniques and defects provides a way to increase test effectiveness

Description Yes No N/A Notes
Defects that occur in the next phase (the next test level or production) are being analyzed, leading to improvements in the accuracy and effectiveness of test cases
Test cases are checked and evaluated independently on validity and maintainability
The test design techniques are evaluated and adjusted for further re-use

Test tools

Controlled

Test tools needed to execute specific test activities are available and used

Description Yes No N/A Notes
The test tools needed to execute specific test activities and to reach the test goals are available to and used by the test team
Knowledge about the test tools in use is available
Everyone involved, including the purchasing entity, considers the specific test tool used to be beneficial

Efficient

Test tools are used to accelerate specific test activities

Description Yes No N/A Notes
Test tools currently used have been selected for testing faster, cheaper, better or making the test process better manageable
The test tools are at the testers disposal at any required moment
A business case has been created for each introduced test tool within the test process
The use of these test tools is integrated in the test process

Optimized

Test tools, and their use, are continuously evaluated and improved

Description Yes No N/A Notes
The tool policy is created and kept synchronized with the existing test policy
The expertise, best practices and test tools products are collected and published for future projects
Test tools are regularly evaluated on reaching the goals that have been set in the business case for implementing the test tool and the test tool policy

Test environment

Controlled

Changes to the test environment do not happen unexpectedly

Description Yes No N/A Notes
Test environment requirements are documented
Working agreements are made with the parties supplying the test environment. These agreements contain tasks and responsibilities
The test environment is available to the test team during the agreed time
The test manager is timely informed whenever a change to the test environment is planned

Efficient

The test environment is directly related to the demands of the test level or test type at hand

Description Yes No N/A Notes
Acceptance of the test environment takes place using a checklist created in advance
A logical or functional design of the test environment is put together. It addresses applications, systems and their connections, and the use of stubs and drivers (mock-ups)
Supplying parties deliver a technical design of the test environment that is formally accepted by the test manager or the test environment specialist
Agreements with supplying parties have a Service Level Agreement nature

Optimized

Test environments are offered as a service to testers

Description Yes No N/A Notes
The ownership of test environments lies with a distinct department
Usage of test environments is laid down in a standard contract
Services in scope and out of scope with regard to test environments are laid down in a Service Catalogue

AI in testing

Controlled

AI is a game changer in the test process, should be considered and used where applicable

Description Yes No N/A Notes
There is a AI chat / agent available to the test team to support them in their daily work
AI learning and data utilization is handled in accordance with the organization's data privacy and security policies
There are guidelines for the use of AI in the test process, including ethical considerations and compliance with regulations

Efficient

AI tools are used specifically in test routines to improve efficiency and effectiveness

Description Yes No N/A Notes
AI tools are used to analyze test basis to identify potential risks and areas for improvement
AI tools are used to generate test cases based on requirements and design documents
AI tools are used to automate repetitive test tasks, such as test execution and result analysis. It can be AI agent to help AQA
Results from AI tools are validated by human testers to ensure accuracy and relevance
Results from AI tools are preserved as testware after validation for future reference

Optimized

AI tools are continuously evaluated and improved to enhance the test process

Description Yes No N/A Notes
Lessons learned from AI tool usage are documented and shared within the test team and organization
AI tools are regularly updated to incorporate the latest advancements in AI technology and testing practices
The effectiveness of AI tools is measured against predefined metrics, such as test coverage, defect detection rate, and time savings

Test Maturity Matrix

Key Areas Importance Progress by Level
Controlled Efficient Optimized
Stakeholder Relations
Stakeholder commitment
L N H
0%
0%
0%
Degree of involvement
L N H
0%
0%
0%
Test strategy
L N H
0%
0%
0%
Test organization
L N H
0%
0%
0%
Communication
L N H
0%
0%
0%
Reporting
L N H
0%
0%
0%
Test Management
Test process management
L N H
0%
0%
0%
Estimating and planning
L N H
0%
0%
0%
Metrics
L N H
0%
0%
0%
Defect management
L N H
0%
0%
0%
Testware management
L N H
0%
0%
0%
Test Profession
Methodology practice
L N H
0%
0%
0%
Tester professionalism
L N H
0%
0%
0%
Test case design
L N H
0%
0%
0%
Test tools
L N H
0%
0%
0%
Test environment
L N H
0%
0%
0%
AI in testing
L N H
0%
0%
0%

Management view per Key Area priority

Controlled Efficient Optimizing
High
0%
0%
0%
Neutral
0%
0%
0%
Low
0%
0%
0%

Project Information & Summary