Best Automation Testing Tools for 2018 (Top 10 reviews)

best testing tools

Here is a nice review of the top 10 best automation testing tools circa 2018.  It covers the following:

  1. Selenium
  2. Katalon Studio
  3. Unified Functional Testing (UFT)
  4. Watir
  5. IBM Rational Functional Tester (RFT)
  6. TestComplete
  7. TestPlant eggPlant
  8. Tricentis Tosca
  9. Ranorex
  10. Robot framework

If you are just setting up the QA team or department and want to know what’s new and hot, or old and tested in the world of automated testing, have a look at these tools.

CakePHP with NightwatchJS on Travis CI

My colleague Andrey Vystavkin has been setting up a testing environment for our CakePHP projects recently.  We had one before, of course, using PHPUnit.  But this time we wanted to add Google Chrome headless browser with some form of JavaScript test suite, so that we could cover functional tests and a bit of front-end.  Andrey described the configuration of NightwatchJS on TravisCI in this blog post.  If you are more of a “show me the code on GitHub” person, have a look at this Pull Request (still work in progress) on our project-template-cakephp project.

Once we are happy with the TravisCI configuration, we’ll be bringing this setup to our BitBucket Pipelines environment as well.

The setup is also based around CakePHP framework, but it’s easy enough to adopt it to any other framework, PHP or not.

Integrated Package for better testing in CakePHP

Viraj Khatavkar wrote this blog post showing how to use Integrated Package for better testing in CakePHP.  Testing in general is not a simple subject, so anything to assist with it is very very welcome.

I’m sure we’ll be trying it at work in the next week or two.

Improving the output of PHPUnit

Via “Improving the output of PHPUnit” I’ve learned about PHPUnit Emoji Result Printer.  Obviously, this sounds like fun, but I’m also sure there is some really smart way of using this.

Too bad I couldn’t try it on one of our work projects just yet, as this requires PHPUnit 6+, while we are still on PHPUnit 5.7.  But we’ll get there.

Using non-breakable spaces in test method names

Using non-breakable spaces in test method names is a great example of how something can start as a joke and quickly turn into something very practical and useful.

if we decide to not follow PSR-2 naming for test methods because of readability, we might as well use non-breakable spaces since it’s even more readable…

Regex101 – online regex editor and debugger

Regex101 is an online regular expression editor and debugger.  You can test your regular expressions against sample data, see if the expression worked, watch it matched, and so on.  Having an explanation for each part of the regular expression dynamically generated, and a quick reference nearby is super handy too.

PHP Smart Analyzer

PHP Smart Analyzer (or PHPSA for short) is yet another item in a growing list of tools for PHP code static analysis.  It’s in an early alpha state, but looking at the list of goals, it’s quite promising.

If that’s up your valley, have a look also at PHPQA and PHPStan, which I wrote about earlier.

BitBucket Pipelines improved support for Docker

Here are some exciting news from the BitBucket Pipelines blog: Bitbucket Pipelines now supports building Docker images, and service containers for database testing.

We developed Pipelines to enable teams to test and deploy software faster, using Docker containers to manage their build environment. Now we’re adding advanced Docker support – building Docker images, and Service containers for database testing.

Wireshark Layer 2-3 pcap Analysis w/ Challenges (CCNP SWITCH)

Johannes Weber, a networking and security professional, has done something really cool while preparing for his CCNP SWITCH exam.  He has built a lab with some networking equipment, configured it all, and captured network traffic, featuring a variety of level 2 and 3 protocols.  He has published his setup, the captured traffic, and a variety of challenges, that helped him to prepare, and which can help others.

While preparing for my CCNP SWITCH exam I built a laboratory with 4 switches, 3 routers and 2 workstations in order to test almost all layer 2/3 protocols that are related to network management traffic. And because “PCAP or it didn’t happen” I captured 22 of these protocols to further investigate them with Wireshark. Oh oh, I remember the good old times where I merely used unmanaged layer 2 switches. 😉

In this blogpost I am publishing the captured pcap file with all of these 22 protocols. I am further listing 45 CHALLENGES as an exercise for the reader. Feel free to download the pcap and to test your protocol skills with Wireshark! Use the comment section below for posting your answers.

Of course I am running my lab fully dual-stacked, i.e., with IPv6 and legacy IP.

I think these are great for several reasons:

  • A feature-rich and complete networking setup, which is not easily available to everyone.
  • A fixed set of data (captured network traffic).
  • Plenty of very specific, testable, and verifiable questions.
  • Overall, very helpful resource from an experience professional, for anybody who wants to know about networks.
  • Overall, a great set of questions and challenges for those interviewing networking candidates.

The lab setup includes the following:

  • 1x Cisco Catalyst 2960, (C2960-LANBASEK9-M), Version 15.0(2)SE9
  • 2x Cisco Catalyst 2950, (C2950-I6K2L2Q4-M), Version 12.1(22)EA14
  • 1x Cisco Catalast 3560, (C3560-IPSERVICESK9-M), Version 12.2(55)SE10
  • 3x Cisco Router 2811, (C2800NM-ADVENTERPRISEK9-M), Version 15.1(4)M9
  • 2x old Notebooks, Dell or somewhat, running either Ubuntu or Knoppix Linux

Personally, I am not very involved with networks these days.  But even for more me the above setup serves as a reminder of how complex underlying technology infrastructure has got in recent years – hardware, software, protocols, and all.

PHPUnit Snapshot Assertions – a way to test without writing actual test cases

phpunit-snapshot-assertions – is an interesting addition to the PHPUnit assertions which allows testing against previously created snapshots.  This is particularly useful for testing the outputs of API end-points, format conversion functions, and the like.  Instead of testing the actual functionality, these assertions allow to compare the output of the current test run with the known good output of a previously created snapshot.

This works well for generic text, but even better for widely used formats like JSON and XML, where, in case of a failed assertion, a meaningful difference can be provided.

Here is a blog post providing some more details on philosophy and methodology.