Best Automation Testing Tools for 2018 (Top 10 reviews)

best testing tools

Here is a nice review of the top 10 best automation testing tools circa 2018.  It covers the following:

  1. Selenium
  2. Katalon Studio
  3. Unified Functional Testing (UFT)
  4. Watir
  5. IBM Rational Functional Tester (RFT)
  6. TestComplete
  7. TestPlant eggPlant
  8. Tricentis Tosca
  9. Ranorex
  10. Robot framework

If you are just setting up the QA team or department and want to know what’s new and hot, or old and tested in the world of automated testing, have a look at these tools.

On Empathy & Pull Requests

I’ve trained more people on the subject of pull requests than I care to remember.  But I’ve never came close to explaining the best practices as well as this Slack Engineering blog post does:

Basically, your reviewer is totally missing context, and it is your pull request’s job to give them that context. You have a few options:

  • Give it a good title, so people know what they’re getting it into before they start.
  • Use the description to tell your reviewer how you ended up with this solution. What did you try that didn’t work? Why is this the right solution?
  • Be sure to link to any secondary material that can add more context — a link to the bug tracker or a Slack archive link can really help when describing the issue.
  • Ask for specific feedback — if you are worried that the call to the `fooBarFrobber` could be avoided, let them know that so they can focus their effort.
  • Finally, you should explain what’s going on for your reviewer. What did you fix? Did you have any trouble fixing the bug? What are some other ways you could’ve fixed this, and why did you decide to fix it this way?

Not every pull request needs every single one of those things, but the more information you give your reviewer, the better they will be able to review your code for you. Plus, if someone ever finds a problem in the future and tracks it down to this pull request, they might understand what you were trying to do when they make a follow-up fix.

Give your reviewer all the context they need to get up to speed with your bug so they can be an informed, useful code reviewer. It’s all about getting your reviewer onto the same page as yourself.

CakePHP with NightwatchJS on Travis CI

My colleague Andrey Vystavkin has been setting up a testing environment for our CakePHP projects recently.  We had one before, of course, using PHPUnit.  But this time we wanted to add Google Chrome headless browser with some form of JavaScript test suite, so that we could cover functional tests and a bit of front-end.  Andrey described the configuration of NightwatchJS on TravisCI in this blog post.  If you are more of a “show me the code on GitHub” person, have a look at this Pull Request (still work in progress) on our project-template-cakephp project.

Once we are happy with the TravisCI configuration, we’ll be bringing this setup to our BitBucket Pipelines environment as well.

The setup is also based around CakePHP framework, but it’s easy enough to adopt it to any other framework, PHP or not.

Integrated Package for better testing in CakePHP

Viraj Khatavkar wrote this blog post showing how to use Integrated Package for better testing in CakePHP.  Testing in general is not a simple subject, so anything to assist with it is very very welcome.

I’m sure we’ll be trying it at work in the next week or two.

Scheduled pipelines now available in Bitbucket Pipelines

BitBucket blog announces the support for scheduled Bitbucket Pipelines.  This is super cool and has been on the wishlist for a while now.  Here are a few examples of how this feature is useful:

  • Nightly builds that take longer to run
  • Daily or weekly deployments to a test environment
  • Data validation and backups
  • Load tests and tracking performance over time
  • Jobs and tasks that aren’t coupled to code changes

Making “Push on Green” a Reality

Making “Push on Green” a Reality is an insider look at how Google handles continuous deployment.  Very few teams and companies need to deal with such level of complexity, but the overall principals still probably apply.

Updating production software is a process that may require dozens, if not hundreds, of steps. These include creating and testing new code, building new binaries and packages, associating the packages with a versioned release, updating the jobs in production datacenters, possibly modifying database schemata, and testing and verifying the results. There are boxes to check and approvals to seek, and the more automated the process, the easier it becomes. When releases can be made faster, it is possible to release more often, and, organizationally, one becomes less afraid to “release early, release often”. And that’s what we describe in this article—making rollouts as easy and as automated as possible. When a “green” condition is detected, we can more quickly perform a new rollout. Humans are still needed somewhere in the loop, but we strive to reduce the purely mechanical toil they need to perform.

PHP Smart Analyzer

PHP Smart Analyzer (or PHPSA for short) is yet another item in a growing list of tools for PHP code static analysis.  It’s in an early alpha state, but looking at the list of goals, it’s quite promising.

If that’s up your valley, have a look also at PHPQA and PHPStan, which I wrote about earlier.

BitBucket Pipelines improved support for Docker

Here are some exciting news from the BitBucket Pipelines blog: Bitbucket Pipelines now supports building Docker images, and service containers for database testing.

We developed Pipelines to enable teams to test and deploy software faster, using Docker containers to manage their build environment. Now we’re adding advanced Docker support – building Docker images, and Service containers for database testing.

PHPUnit Snapshot Assertions – a way to test without writing actual test cases

phpunit-snapshot-assertions – is an interesting addition to the PHPUnit assertions which allows testing against previously created snapshots.  This is particularly useful for testing the outputs of API end-points, format conversion functions, and the like.  Instead of testing the actual functionality, these assertions allow to compare the output of the current test run with the known good output of a previously created snapshot.

This works well for generic text, but even better for widely used formats like JSON and XML, where, in case of a failed assertion, a meaningful difference can be provided.

Here is a blog post providing some more details on philosophy and methodology.

PHPQA all-in-one Analyzer CLI tool

PHPQA all-in-one Analyzer CLI tool.  This project bundles together all the usual PHP quality control tools, and then some.  It simplifies the installation and configuration of the tools and helps developers to push up the quality control bar on their projects.

The tools currently included are: