Looking for smarter ways to test software

Software testing is paramount for many applications, but because of its difficulty and invisibility, it is not the most popular part of software engineering. Professor Tanja Vos wants to change that by developing more intelligent testing tools and building better educational modules.

The Software Testing group led by Tanja Vos is unique in spanning two locations in two countries: the Open Universiteit (OU) in Heerlen and the Universidad Politécnica de Valencia (UPV) in Spain. Vos has been an associate professor at UPV since 2003, and in 2016, she also became Professor of Software Engineering at the OU.

‘The Open Universiteit has place- and time-independent teaching education at its core’, says Vos, ‘and then it does not matter whether you collaborate with people working from different locations in the Netherlands or with people working in Valencia. The four PhD students I have at the OU all work from different locations in the Netherlands. In Valencia, there are four people in the group. We communicate with each other in the same way online. We use various chat groups for different purposes and, of course, we meet in person regularly.’

The research group of Vos is specialised in automated testing of desktop, web and mobile applications at the Graphics User Interface (GUI) level. ‘Our flagship is a testing tool called TESTAR,’ says Vos, ‘which we have continuously developed since 2010. TESTAR is a form of scriptless testing in which you do not have to write a test script in advance, but in which TESTAR decides which actions to select to create a test. We construct a very large collection of all possible actions in a given software state, and then we choose one of them to go the next state.’

In its default configuration, TESTAR only does random testing. This may sound strange but in practice, it turns out to be complementary to scripted testing. Vos: ‘We have tested software with TESTAR at several companies, including the mobile banking app of ING, and we found that TESTAR covers different parts of the software than human written tests do.’

Still, the question remains whether there are better ways to test than defining a script in advance or choosing random test actions. ‘Indeed’, says Vos, ‘that is why we are investigating if and how we can let TESTAR itself learn the best way to select actions. This involves using techniques from artificial intelligence. Another future research direction is to see whether we can use a ChatGPT type of tool trained on a large number of existing test scripts to predict the best test action to select next. We are constantly looking for smarter ways of testing software.’

Agents and models

Beatriz Marín is one of the senior researchers in the Valencia part of the Software Testing group. She was a professor in Santiago (Chile) for many years, and in 2021, Vos persuaded her to come and join the research group in Valencia as an expert in software quality. ‘During the past few years, I have worked on a project that combines the use of intelligent agents and models in testing’, says Marín. ‘An additional reason Tanja asked me to join the group is that I had experience using serious games to motivate students for software testing. Whether you work in a company or in academia, software testing is often considered difficult, and too few people want to do it. But of course, good testing is extremely important, so our group sees testing and quality as inseparable parts of the education in software engineering.’

Marín has just started working on a new three-year EU-funded project called ENACTEST, to improve the education of testing. The goal of this project is to identify and design seamless teaching materials for testing that are aligned with industry and learning needs. Marín: ‘We have four people from our research group working in ENACTEST. Nine different European partners are involved, including universities, vocational centres and small and medium enterprises. In the long run, this project will improve the software quality on which our digitalised society relies.’

Better education

As a PhD student of the Open Universiteit, Niels Doorn is researching how to improve testing education. Doorn works on his doctoral project three days a week; the other two days, he works as a lecturer and researcher in IT and computer science at the NHL Stenden University of Applied Sciences in Emmen. ‘The goal of my PhD project is to create effective methods for teaching software testing that are supported by scientific evidence’, says Doorn, ‘because at present, we know very little about what works well and what not.’

Doorn uses both a theoretical and a practical approach. For example, he compares how test experts test software to how students do that. Doorn: ‘What conceptual knowledge and experiences do they use? How can we best integrate testing into programming assignments without compromising the programming concepts we want to teach students? Among other things, we have already created a website with exercises ready for use by teachers.’

Doorn does the day-to-day supervision and most meetings online, but the group members also meet up several times a year. ‘In May, we had a computer science day where the group members got together. We conduct workshops for OU students. And we take occasions like the opening of the academic year, an inauguration or a conference to get together for one or two days. Although the group is spread across two countries, the collaborations are going really well.’

Group passport – Software Testing at OU and UPV

Research fields: Software testing, automated testing, software quality, education in software testing
Institution: The Software Testing group is spread over two locations: Open Universiteit (OU) in Heerlen, and Universidad Politécnica de Valencia (UPV).
Websites:
Prof. Tanja Vos
Open Universiteit (OU)
Universidad Politécnica de Valencia (UPV)
TESTAR (Automated system testing of desktop, web and mobile applications at the Graphical User Interface Level)
ENACTEST

By Bennie Mols
Images Ivar Pel