Release Notes

9 Cognitive Automation
There is a new class of Test Automation tools for GUI applications that have emerged in recent years that
leverage AI techniques such as Computer Vision and Natural Language Processing (NLP). These tools support
the testing of mobile and browser-based applications. They offer the promise of increased coverage across a
broad range of test categories including: Functional, Performance, Usability, and Workflow Testing. All of this
purportedly comes at a dramatic decrease in automation script maintenance costs as compared to conventional
scripted GUI Automation test suites. With these tools, rather than encode both the intent (the ‘what’) and the
precise steps (the ‘how’) of a test scenario using Python, Java, or C#, the Tester first trains the bot on how to
navigate and label the AUT. Next, the Tester authors the intent of each test scenario in a language
understandable by the bot. It leaves the ‘how’ of navigating the AUT to the bot which is abstracted away from
any changes to the DOM of the underlying AUT. Vendors such as Test.AI, Pinklion.AI, Retest, MABL, and
Testim.IO are making strides in reducing the maintenance costs faced by many UI Automation engineers.
While this is encouraging to those testers among us facing the maintenance challenges of GUI Automation, at
present, this appears to be one of the few areas where we see Off-The-Shelf, AI-enabled tools making inroads
in Testing tasks.
2.2 Test Matrix Planning
Organizations that develop Embedded Solutions face complexities with an untenable number of Hardware and
Software configurations for their test efforts.
At Dell EMC Servers, one case that we’ve applied Analytics and ML to is the Test Matrix Configuration Planning
process. This is a deep-think task, relying heavily on the past experiences of our Test Leads and their ability
to analyze large amounts of historical sales and historical test data. After accounting for all of the valid
combinations of new and existing subsystems that go into a new Server, our Test leads face the possibility of
having to test 465 trillion configurations to achieve full test coverage. Of course, due to cost and time, the
realistic number that can be address given time and cost constraints, is closer to ~500 unique configurations
during a new Server release.
Figure 3 - AI-Assisted UI Automation