Do you have a fascination for software testing? A fascination with the underlying simplicity which is juxtaposed against the real world complexity? We do too!
Underlying everything is that simple concept of a set of actions, some expected output and a pass/fail result. Then Scale that up in the real world and everything suddenly becomes a lot more complicated. Bolt automated testing on top and that seems to take things in to a new dimension of complexity.
The Simplicity
Let’s face it. There’s something quite fascinating about the simplicity of software testing. At it’s core is the simplest of concepts. The concept of one action, one expected result and a pass/fail result. Such simplicity. And a simplicity that on the surface should scale with ease. More product to cover, add more actions, more expected results and more passes, more fails. When everything else in software engineering is beset with complexity this would seem like a refreshing aberration.
Indeed for most in software engineering, at an organisational level, we’ve been able to run testing in a parallel universe separate to the development. Anyone outside of the QA team has been able to ignore most of the complexity. In that waterfall world where development and test seem to live separate lives everyone else (except the test team) could hide from the complexity of testing. They could keep living with that simple belief that testing is done separately, in parallel. Keep believing that there’s nothing complicated in there to worry about. So long as those test reports, charting progress, keep popping up then everything is okay.
The Complexity
For decades though the testers at the coal face have recognised, and worked diligently, to manage that real heavy weight complexity that makes up this discipline. And it is complex. Really damn complex. Cans of worms spring to mind. And most really don’t want to take the lid off that can. They don’t want to because deep down they know that once those worms start to escape it’s going to be damn hard to get those worms back in. The reality of what you face, if you want to do the job properly, needs to be addressed.
What is Jenkins and How to Install Jenkins
Checking Test System Environment Before Starting Your Tests
Test Automation Execution based on App Version Updates
Re-running A Failed Automated Test Automatically With Jenkins
Running Data Driven Tests with the Matrix Project Plugin
Trouble is we’re in an agile world now. An agile world where everyone has been thrown together. Those worms are starting to escape. Those parallel worlds of dev and QA have converged. There’s no silos in which to hide complexity in now. There’s a realisation that this simple ‘testing’ concept really isn’t as simple as you want to believe.
Those outside of the test discipline (yes I’m talking about those like developers, business analysts and project managers) are been forced to face up to the amount of testing that’s required and the complexity that challenge presents. Even the simplest needs, like linking test cases to requirements, start to become difficult as your projects scale. There’s a long list of complexities that come with managing test cases in Agile projects (we won’t bore you with them here – give us a call and we’ll happily talk about them all day with you).
Complexity Can’t Be Hidden in Agile
Just understand that there’s a complexity that’s been hidden for a long time now. That complexity visible to all now. And that presents a new challenge. How to simplify that complexity for those in the team that really don’t care (I use that phrase lightly because I know they do care really) and don’t see it as a priority with all the other bits of complexity presenting themselves in your agile project.
You can try and simplify if you want. You can remove traceability, cross project reporting, managing regression sets and things like integrated reporting. Removing this just removes large amounts of data and tracking you need to manage projects effectively though. Like sweeping the dust under the carpet. It’s still there. It’s not going away. It’s a delusion that we really need to wake up to.
Are we delusional?
We came to the conclusion a while back that the world of Test Automation is generally doing a pretty damn good job of deluding it’s self. Deluding it’s self that it’s a worthwhile investment when it’s NOT. An uncomfortable truth for many out there? Probably!
Don’t get us wrong. There are teams out there that are seeing a decent return on their investment in the pursuit of effective Automated testing. Teams that are doing a damn good job of increasing the quality of the products they release in the process.
In general though, let’s face it, most teams struggle and waste a huge amount of time and money. It’s a world of headless chickens, running through a lot of smoke and bumping into quite a few mirrors. If you stopped and took stock of the amount of effort going in for the return delivered we’d be shocked.
The waste comes from the massive overhead in implementing and maintaining the frameworks/infrastructure you need in place. The complexity and time it takes to write each individual test. It’s an aspect of the process that everyone seems to just accept when, if you look closely, these aspects are killing any chance you have of seeing an RoI.
A Solution?
There’s a balance to be had between having the system/process that tracks everything you need to track. Balanced against keeping everything simple enough so that everyone in the team knows what’s going on and how to follow that process.
The merging of dev and test in Agile teams means there’s an opportunity to confront these challenges together and come up with some innovative solutions that deal with this simplicity. And it’s just that challenge we found ourselves dealing with.
The challenge that software Test Automation really should be simpler!
Agile teams don’t want the complexity of running another development project in parallel to their main development project. As a team all you want is the ability to write effective, high quality, automated tests that you can run reliably from one sprint to the next. And you want to do all of this fast…. and you don’t want to be dependent on one single person, a dedicated Test Automation engineer. You just need your stuff tested, in the simplest, fastest most effective way possible?
So why are you wrestling with a significant sideline automation project that is just another complex development project in disguise!?
This is the problem we like to think we’ve solved with LeapWork. With LeapWork we set out to remove a whole area of complexity so that everyone in your agile team can write and run automated tests.
Forget building frameworks. Forget leaving automation to one or two specialists. Forget the struggle to integrate the Test Automation process into your agile process. Start thinking smarter.
Wasting hours, days, months developing the automation process and frameworks is NOT a good use of time and resources. Automation is expensive. It’s expensive because there’s a massive overhead in building and maintaining everything that supports the execution of (lets face it) a handful of automated tests. Itís just not worth it when you take everything into consideration.
Is It Worth It?
IT ONLY BECOMES WORTH IT when you’re adding and running larger numbers of automated tests. It’s all arse about face on most automation projects. 80% of the time spent maintaining frameworks and infrastructure, 20% of the time spent writing and running tests.
IF YOU WANT TO SEE A RETURN ON YOUR INVESTMENT IN AUTOMATION THIS NEEDS TO BE TURNED ON IT’S HEAD. You need to be spending 80% of your time writing new tests and 20% of your time maintaining what’s there to support the development and execution.
And don’t forget how much time it usually takes just to write one automated test. I mean the typical figure is that it might take 10 times as long to write an automated test as it will to run it manually. So you have to run the test at least 10 times before you see a return on that investment…. but that’s if the calculations don’t include all the time you waste managing the infrastructure. It’s a con!
The reality is that you need to be down at 2 or 3 multiples and spending only 20% of your time managing and maintaining the infrastructure. If you’re not in this ball park we’ve come to the conclusion that you’re never going to see a return on your investment. May as well go out and hire a bigger team of manual testers. Stop deluding yourselves. Take a reality check.
Yes there are many benefits of working closer together in Agile teams. Just be aware that working closer together means that real complexity hidden in the traditional test team is now going to be exposed to all of those within your agile team. It means working together to resolve those complexities. Lot’s of complexity that your agile team tn
One of those complexities being automated testing. Traditionally what should be thought of as a development project in it’s own right. Now it’s no longer the domain of a dedicated test team. Now it’s the domain of everyone in your sprint team. But wait! If it’s a development project and your BA’s can’t code, those in the manual testing domain struggle to code and your developers have absolutely no time to work on coding automation… where do you go?
Time to simplify
The software engineering project you’re working on is complex enough in it’s own right. You need the benefits of automated testing, but you need it without the complexity. You need it without the complexity because testing, automated testing, is everyone’s jobs now. And you’re only going to succeed making it everyone’s job if you simplify it. If you make it easy enough for the BA to write automated tests (sorry … no disrespect to BA’s). If you make it simple enough for a manual tester to write an automated test in the same time it took them to write a manual test (the holly!! Grail of automation).