Release Notes

10 Cognitive Automation
This planning task is usually undertaken by two to three lead engineers and takes over two weeks to complete.
The goal of this planning task is to determine the fewest number of configurations (contain cost), that provide
the highest test coverage of the newest features and previously delivered functionality, accounts for most-sold
configurations, and factors in the historically challenging configurations. The Test leads perform this analysis
manually and refer to large amounts of proprietary data sets as highlighted in the funnel chart in Figure 4.
Tasks with this sort of profile are now addressable by Analytics and ML-based solutions. By combining Analytics
with ML-based algorithms, the value that can be derived from automating this highly cognitive and analytical
task is twofold:
1) The six-person weeks’ worth of effort is reduced to hours.
2) It enables the team to quickly re-plan a test run when materials or features are delayed at the last
minute.
2.3 Test Suite Planning
Another cognitive testing task that we’ve been working on automating is the Regression Test Suite Planning
model. This analytics-based model uses test case execution history, defect data, and historical SUT
configurations to recommend regression Test Cases & Suites at any given time. It offers a smarter way of
testing that is informed by prior test execution data, and that helps us answer questions such as:
Figure 4 - Test Configuration Planning Model