Busting the 10 Biggest Myths in Test Automation

“To test or not to test?”

Well, pretty much everyone who uses enterprise software knows that you have to test. In 2023, the question has evolved into “to automate or not to automate?” Most people would agree that test automation saves time and money on testing, but only 5% of organizations carry out fully automated testing. As for future plans, up to 73% of companies would like to reach a ‘manual to automation’ proportion of 50:50 or 25:75. Only 14% reported that they wish to eliminate manual testing in full. An overall takeaway? The majority of enterprises are not satisfied with their testing programs. 

Why is the adoption of test automation slower than the adoption of automation in other areas like manufacturing, copywriting, and retail? 60% of all corporate data is now stored in the Cloud. The average enterprise uses 1,295 cloud services. That’s a lot of migrations, a lot of ERPs, a lot of updates–basically a lot of testing that could be automated, but is remaining manual. The reason for this?

Misconceptions.

These misunderstandings about the protocols, cost, and requirements of automated testing often get in the way of clients making the changes they need to make to stay competitive and up to speed with their packaged app. Let's check out the top ten:

Myth #1: Test automation is expensive and requires a lot of resources.

Reality: While it’s  true that there’s an upfront investment required for test automation tools and resources, the benefits of automation far outweigh the costs. Automated testing saves major time, which means it reduces labor costs. It also ensures you don’t have an expensive disruption in service by improving the overall quality of software. Gartner reports that system downtime costs an average of $300,000 per hour. That means just one instance of downtime due to poor testing could put you in a financial hole.

Myth #2: Test automation totally eliminates the need for manual testing and replaces jobs.

Reality: Test automation is not a replacement for manual testing. Automated testing is best used to complement manual testing, not replace it entirely. While automated testing can help speed up the testing process and catch certain types of defects, it cannot replace the human judgment that is needed for certain types of testing (such as exploratory testing). 

‍

See this high-level overview of automation testing: A Comprehensive Guide on Automation Testing

Myth #3: Test automation is only suitable for large projects with ample resources.

Reality: Test automation works for any software or digital ecosystem. Automation can be used on projects of any size, and it can be especially useful for smaller projects that have limited resources and tight deadlines. Opkey provides testing for all sorts of custom applications, mobile interfaces, and websites. (In addition to our expertise 15+ packaged apps, including Oracle Cloud, Oracle EBS, Salesforce, SAP, and Workday.)

Myth #4: It’s better to use Selenium or another open-source tool to test. 

Reality: Selenium is an open-source test automation tool that allows APIs to automate tests carried out in web browsers. Any action can be performed by writing a piece of custom code. Organizations are often attracted to it at first because it’s free. 

However, we found that the same test that takes 4 hours to code with Selenium takes three minutes when powered by Opkey automation. Multiply that time proportion across every single test in your app. 

Yeah. Pretty big difference. 

Any money it seems you would save at first is quickly eaten up in labor costs as manual testers spend weeks and months testing simple changes. See more about the comparison points in our blog “Opkey vs. Selenium” here.

Myth #5: You need to be a technical expert to utilize automation.

Reality: Nope! That’s the beauty. Automated testing tools are not limited to IT teams, or even to technical users. With no-code tool like Opkey, no coding knowledge is required. This allows the actual end users, who know their business processes best, to contribute to testing. 

Myth #6: Test automation can be done by absolutely anyone.

Reality: Let’s not swing too far to the other end of the pendulum. Test automation does require an understanding of the layout, metrics, and functions of your ERP ecosystem. If a user doesn’t have this basic knowledge, they may not be able to correctly interpret the results. They don’t need to know how to code, but they do need to be familiar with your packaged app. 

Myth #7: Test automation is just a fad. 

Reality: Not so. The industry has permanently changed. The increasing pace of software development and the increasing use of methodologies like agile and DevOps means that manual testing is simply unfeasible. 

Human hands can’t keep up, and the errors that human minds produce can derail an ERP and cost millions of dollars. The automated testing market (global) was estimated at $22.4 billion in 2022 and is expected to reach around $57 Billion by 2030. That’s some serious growth, driven by some serious need. 

We surveyed over 300 IT leaders about their ERP testing needs. See the results here in our State of ERP Report. 

Myth #8: Test automation is a one-time activity.

Reality: Test automation is a routine need for your system, like upkeeping a car. Changes, updates, and the natural growth of a business require systems to expand and be tested in new ways. Automated tests must be reviewed regularly to ensure they are still relevant and effective.

Myth #9: Test automation is only for regression testing.

Reality: Test automation is not only for regression testing. It can (and should!) be used for a variety of testing activities, including:

Myth #10: Test automation is inflexible and cannot adapt to changes in software requirements. It is hard to maintain. 

Reality: Test automation can change with your system, faster than manual methods can. Automated tests are designed to be flexible and adaptable to changes in software and business requirements. With the right tools and processes in place, automated tests can be updated quickly and easily, often at the single click of a button.

(Bonus) Myth #11: Companies are always happy with their SI partners and packaged apps. 

Reality: Not so! Unfortunately, partnering with SI partners to implement, migrate, or improve a software can sometimes come with drawbacks. Here are some of those drawbacks:

Additionally, ERP ecosystems can become a drag on resources without proper testing. Inflexible or outdated testing methods can delay progress and create conflicts in authority. According to our data from the State of ERP report, only 7% of respondents are extremely satisfied with their current test coverage. That leaves a lot of room for improvement. We have seen that automated testing, implemented strategically, improves company-partner relations as well as packaged app satisfaction.  

There you have the major myths that are floating around the test automation space. But what’s the overarching truth that busts them all? Test automation is a move that creates ease, not complication. 

It’s an achievement and advancement of modern technology that saves you time, allows you to accomplish higher value activities, and secures your ERP in an uncertain market. For example, we see Oracle Cloud patch update cycles being cut to 3 days from 4 weeks. That’s a measurable instance of a major win with automation, and it’s happening across the 14+ packaged apps that we service. 

We recommend making the change as soon as possible, before the software lifecycle speeds up even more and manual testing becomes increasingly unable to keep up with the pace. Bust the myths, make the change–and your team, budget, and leadership will thank you.