SAN MATEO, Calif., April 7, 2020 /PRNewswire/ -- Applitools,
the leading provider of Visual AI powered test automation, today
announced an industry report that reveals the impact of Visual AI
on the performance of the most popular open source test frameworks
- Cypress, Selenium, and WebdriverIO. Data was sourced from 288
quality engineers across 101 countries, spending an average of 11
hours during November 2019. This
amounts to a combined 80 work weeks, or well over a year and a half
of quality engineering hours, making it the largest industry study
of its kind to date.
To download the "Impact of Visual AI on Test Automation
Report" visit:
(https://www.applitools.com/impact-of-visual-ai/)
"The Impact of Visual AI on Test Automation Report" provides
important learning including:
- 5.8x Faster Test Creation: When using existing open
source code-based frameworks in isolation, it took testers 7 hours
to create test code in contrast to just 1.2 hours using Visual AI.
As a result, testers are able to greatly expand test coverage while
simultaneously releasing faster than ever before.
- 3.8x Increase in Test Code Stability: Participants used
32 locators and labels on average to provide test coverage using
code based frameworks alone. This number dropped to just 7 with
Visual AI. This increased stability greatly reduces test flakiness,
broken CI/CD builds, and time spent on maintenance. Better test
stability leads to efficiency gains along whatever dimension the
quality engineering team values most - either more coverage, more
time to manage quality, or even faster test cycles in support of
CI/CD.
- 45% More Efficient at Catching Bugs: Participants caught
19 out of 20 bugs (95%) using Visual AI compared to just 13 (65%)
when using code based frameworks in isolation. When test builds are
faster, provide more coverage, and are more stable and easily
maintained, testers catch more bugs pre-production while still
delivering more features, faster than previously thought
possible.
- Visual AI Helped Testers Succeed in Their Work When Used in
Conjunction with Open Source Frameworks: All 288 testers that
participated in the study achieved success rates of 79% when using
their preferred code-based framework vs. 88% with Visual AI. This
is a nine point jump in their performance after just a single hour
of Visual AI training. Eric
Davidson, Software Engineer from Chick-Fil-A comments,
"This was an incredible opportunity to experience the power of
Visual AI. Applitools has created a tool that easily integrates
with existing frameworks and quickly augments functional test
suites..."
To dive deeper into this study, register for the upcoming
webinar on April 23, 2020, "The
Impact of Visual AI on Selenium, Cypress, and WebdriverIO:
Side-by-Side Test Code Comparison From 288 Quality Engineers"
hosted by Raja Rao and James Lamberti of Applitools
(https://go.applitools.com/200423-Hackathon-insights-webinar.html).
"Visual AI is a tremendous efficiency multiplier for
automated testing," said Gil Sever,
CEO of Applitools. "It improves the productivity of
developers and test automation engineers more than 5X, delivers
huge time and money savings in the app delivery cycle, removes the
testing bottleneck in the CI/CD process, and helps deliver software
applications with unprecedented visual and functional quality on
any computer and mobile device."
"Testers need to look at Visual AI as a superpower. A virtual
assistant that can access every screen and page in minutes, inspect
it for changes and interpret every one of them the same way the
human eye and brain would, but do so without ever getting tired or
making any mistakes. We believe this capability is a major part of
the future of test automation."
According to Gartner's Critical Capabilities for Software
Test Automation (December 18,
2019), "61% of [its 2019 Software Quality Tools and
Practices Survey] respondents said that AI/ML features would be
very valuable in software testing tools. Improved defect detection
(48%), reduction in test maintenance cost (42%) and improved test
coverage (41%) were seen as the top benefits expected (multiple
answers were allowed)." "The Impact of Visual AI on Test Automation
Report" provides empirical evidence that Applitools Visual AI
delivers exactly what the market expects from the addition of AI/ML
into automated testing.
Data was sourced via the "Global Visual AI Rockstar Hackathon"
which challenged software testers to create test suites for five
real-world functional testing use-cases using their preferred
code-based approach, including Selenium, Cypress, or WebdriverIO,
and then repeat the process after adding Applitools Visual
AI. This approach provided an unbiased, highly controlled
environment from 288 quality engineers from 101 different countries
during the 30 days ending November 30,
2019.
"Completing the hackathon was a keystone achievement in my
career!" said Tracy Mazelin, QA
Engineer at Paylocity. "I learned more by participating in this
hackathon than any other automation instruction I've taken in the
past number of years. I'm now 100 percent convinced that Visual AI
testing is an essential tool for efficiently validating web and
mobile software applications."
"This was the most interesting and useful event of the year
in the field of test automation," said Viktar Silakou, Lead QA
Automation Engineer at Globant S.A. "This allows you to take a
look at test automation from a different point of view and gives an
opportunity to radically improve your existing approaches."
About Applitools
Applitools enables Visual AI powered
test automation to help teams release high-quality web and mobile
apps faster and more efficiently.
Applitools Visual AI modernizes important test automation use
cases -- Functional Testing, Visual Testing, Web and Mobile UI/UX
Testing, Cross Browser Testing, Responsive Web Design Testing,
Cross Device Testing, PDF Testing, Accessibility Testing and
Compliance Testing -- to transform the way organizations deliver
innovation at the speed of CI/CD at a significantly lower Total
Cost of Ownership (TCO).
Hundreds of companies from verticals such as Software, Banking,
Insurance, Retail, Pharmaceuticals, and Publishing -- including 50
of the Fortune 100 -- use Applitools to deliver the best possible
digital experiences to millions of customers on any device and
browser, and across every screen size and operating system.
Applitools is headquartered in San
Mateo, California, with an R&D center in Tel Aviv, Israel. For more information, please
visit applitools.com.
Media Contact:
Jeremy
Douglas
Catapult PR-IR
+1 303-581-7760, ext. 16
jdouglas@catapultpr-ir.com
View original content to download
multimedia:http://www.prnewswire.com/news-releases/impact-of-visual-ai-on-test-automation-shows-5-8x-faster-authoring-5-9x-better-test-code-efficiency-and-a-4-6x-increase-in-test-stability-301036643.html
SOURCE Applitools