Artificial Intelligence (AI) in test automation

Sometimes I dream about having more spare time, less time spent on work and more time for hobbies, family and friends. And at this point I start to question where all those robots are that could do our job? Why don’t they exist yet? Why is it taking this long? I mean, we have self-driving cars, so why can’t I have a self-testing system? So, the question is will I get more time for other testing activities soon or will test automation keep me busy in the upcoming decade?

Let’s first note a big difference between two fields here: there is the testing of Artificial Intelligence and testing with the help of Artificial intelligence. The key difference is that when we are testing the AI we have to somehow check of the AI is doing what we expect it to do, which can be difficult since the AI is basically learning on its own. Testing with AI means that we give our work of writing automated tests to a neural network and we will wait on the beach till the AI has written the test code for us. Let’s see it as test automation without any coding, clicking or configuration. In this article we are only focusing on testing with Artificial Intelligence and not testing the Artificial Intelligence itself.

So, what am I looking for? As I said I want to use AI in test automation. I would like to feed an application to an AI tool, go away and when I come back the tool has figured out how to test the application and can run a regression test from that point onwards without braking and that it will update itself. However, hours of searching for a perfect tool has resulted in nothing. There are plenty of tools that claim to use AI for test automation, but none of the tools would actually do the testing for me. So why are they using AI if it’s not taking over my job? Then it’s useless, right?

Let’s start with how AI works in short and why it’s so hard to just build a tool that takes over my job as an test automation engineer. The most used form of AI is machine learning, we feed loads and loads of data to an algorithm and let it analyse it. Based on this, the system starts to recognize patrons and tries to follow these when it has to make decisions on its own. This is nothing magical, it’s an algorithm and maybe some science. So, our first problem of automating test automation with AI is data. I can’t come up with a dataset that contains hundreds of thousands of automated test flows that are readable by an AI platform. And even if I could find it, it will most likely only cover one application or at most one kind. So, in order to use AI to replace us we will need a big data set to train and test AI. And even then, it would be questionable of it would be good enough to test every application. Every company’s business flows are different, they name things differently and they expect different results.

So that means all those tools that claim to be AI test tools are bullshit. I mean, they don’t do my work for me, so why use them? I don’t think that that’s the right thing to say. AI has come a long way in the last couple of years. Mainly because the power of our computers has grown in a way that it gets easier to process huge amounts of data, make predictions and use machine learning. Many test automation tools have started to use AI to make our life easier. They don’t take over our jobs (yet) but they try to make it easier to automate and make the tests more stable.

For example, test automation tools with Artificial Intelligence are now able to make better image comparisons. Where in the past a visual check would fail when a button shifted by just one pixel to the left or the right. Tools that use AI could be able to detect of this is a major change or just a pixel shift that no one cares about. This is will make it a lot easier to do visual comparisons of an UI.

Other tools will try to predict how likely it is this that the new appearing button is replacing the old missing button. In the past a missing element would be failing the test and a new placed button would not be seen by the tool. An AI can create a link between the missing and the new appearing buttons. And then if the AI predicts that it’s likely that the new button is just replacing the old one it won’t fail the test like it used to do in the past, but it will just click the button.

So, as you can see AI is already used in many different test tools, but with small steps, only one at the time. For now, AI isn’t here to replace us, but the AI is here to help us instead. AI has been proven to be useful in test automation and it will be more and more used in different test automation solutions throughout the field. However, we won’t be replaced any time soon. Too bad, because we could spend our time with other testing activities, such as exploratory testing which could feed the AI with useful information. So, we can spend more time with family, friends and hobbies.

Janus van Limpt
Test automation engineer

Would you like to know what possibilities we offer to become a Test automation engineer, please click here.  Are you interested in more blogs on this (and other) subject(s), please click here.