Test automation with Gauge
Gauge is an open source test automation tool created by ThoughtWorks (the company behind Selenium).
It works by creating a readable Specification "spec" file and a test script. Spec files use Markdown.
Test scripts can be written in JavaScript, C#, Java, Ruby or Python.
Gauge is similar to Cucumber, but not quite. Cucumber is a project description tool intended for collaboration with the project stakeholders. Gauge's purpose is automated testing.
In this article I will try to explain why I fell in love with this tool.
It all started with an eCommerce website. Every time we deployed to QA and production, we manually tested every "flow".
This approach is tiresome and unsustainable, so automated testing was the natural response. At some point, there was the initiative of using Cucumber alongside Nightwatch API. The goal was to automate the web testing of each flow.
Cucumber works by describing scenarios of an expected behavior of a system, using Gherkin. Scenarios are saved in "feature" files. Cucumber's structure uses three statements: Given-When-Then. These statements go inside "scenarios", which are your test cases.
Feature: Main Page Validations
Scenario: Main page
Given I open eShop "Home" page
Then the title is "My page title"
Scenario: Lower banner
Given I open eShop "Home" page
When I see the main banner
Then the width is 700 and height is 300
Once the feature is written you connect it with a test script. Test scripts perform the actual job.
import { client } from 'nightwatch-api';
import { Given, When, Then } from 'cucumber';
Given(/^I open eShop "([^"]*)" page$/, async text => {
await client.url("http://www.mysiste.com"]);
});
Then(/^the title is "(.*?)"$/, async text => {
await client.expect.title().to.equal(text);
});
When(/^I see the main banner$/, async () => {
await client.expect.element("#selector").to.be.visible;
});
// Each test case has a Cucumber statement "attached" to it
You can reuse feature statements and code statements as you see fit.
While this approach seemed to work, personally, I encountered many problems.
Cucumber is an excellent tool, but its purpose is to describe features collaboratively. What happens behind the scenes is another story.
You can use Cucumber with many programming languages, such as Java or C#, alongside Selenium WebDriver. We used Nightwatch API, an interface that controls NightwatchJS that uses the W3C WebDriver.
When using Nightwatch API I realized that its documentation is scarce. Moreover, writing "simple" test scripts ended being very tricky and troublesome.
When doing assertions on the UI, first you need to locate its elements. You can use CSS selectors or XPath. It is recommended to use a Page Object Model design pattern. This means, using a repository pattern to locate Web UI elements.
// Example Page Object Model in JavaScript
module.exports = {
elements : {
// My page selectors
aFormElement: {
selector: '.form__radio_label',
index: 0
},
someButton: {
selector: '.btn',
index: 1
},
exampleXPathSelector: {
selector: '//input[@type='text']',
index: 0
},
}
};
You can locate some elements easily, but finding others is a nightmare. If a given element has an id, then there is no problem. If not, using CSS selectors or XPath is the way to go.
Once you locate the elements, write the feature file then you can actually start writing the test code snippets.
For example, in Selenium, a simple Google Search requires the following.
// Selenium example
var browser = new webdriver.
Builder().
usingServer().
withCapabilities({'browserName': 'chrome' }).
build();
browser.get('http://www.google.com');
// Uses HTML locators, elements, form submissions etc.
browser.findElement(webdriver.By.name('q')).sendKeys('My Google Search!');
browser.findElement(webdriver.By.name('btnG')).click();
And that's when we lost it. We realized that writing both feature files and code was unsustainable, at least for us.
Simple actions like finding elements, clicking or asserting certain UI elements, like a banner, required tremendous work. Finishing a "flow" took almost 3 days, and we had at least 30 flows or more.
And one day I found Gauge. So, why I fell in love with Gauge? <3
It's very easy to get started
I started with the Getting Started tutorial and to see what all the fuzz was about. I started with the equivalent "Hello World" tutorial. Our previous Google Search now looks like this:
openBrowser()
goto("https://www.google.com")
// API for testing like a user
write('My Google Search!')
press('Enter')
I was amazed. In just two hours or less I could finish a full "flow" for our eCommerce website.
Many of our developers are Jr. Developers, so coaching them with Gauge meant less training time.
Specs are readable
Spec files use natural language. Cucumber feature files are tied to a Given-When-Then structure. Gauge is all about writing reusable and readable specs.
Consider the following Cucumber example:
Given I open Google
When I type 'My Google Search!'
And I hit enter
Then I should see the results
It is natural up to a certain point. In Gauge, you do not need the Given-When-Then structure. It works by simply writing "steps".
* I open Google
* I search 'My Google Search'
And that's it. You can write anything you want in any language. You can describe as much or as little as you want.
* Abro Google
* Busco 'Mi búsqueda en Google'
Out-of-the-box multilingual functionality. Spanish example. With Cucumber you need to download language packages.
With Cucumber, the problem we found is that you need to train the stakeholders that write and read the feature files. It often meant arguments about who should write them.
Business analysts? Programmers? Product owners? Testers? UX Designers? Cucumber uses a strict syntax, that may be easy for a programmer to understand, but maybe not so much for a Product Owner.
Cucumber recommends a three amigos approach, meaning meetings with Product Owners, Programmers and Testers only for feature definition.
With Gauge, everyone can read and write spec files. And I think it's more flexible. What if you have no dedicated test team? Or what if the test task is solely entitled to the test team?
Regardless of the tool you use, I think it's better to actually describe what you want to test and what you need as a result.
Gauge is so flexible, that even if you wanted you could still use Cucumber's syntax!
Meet Taiko, Gauge's secret weapon
Taiko is an open source browser automation tool built by ThoughtWorks. It is a NodeJS library that automates Chrome.
It addresses two problems that I personally saw with other test tools: finding UI elements and wait calls.
Consider the following example
<button class="btn" type="button" href="https://www.mylink.com"> Continue</button>
That button class is shared with 10 other buttons in our HTML page. It is nested inside a form, a grid and five other divs. It has no id.
The way we could find that button via CSS or Xpath is the following:
CSS: .abc-grid:nth-child(4) .btn
XPath: //button[@type='button'])[8]
XPath: //button[contains(text(),'Continue')]
The XPath and CSS index is different, so we either use one or another, or write a helper in our Page Object Model.
Using the text match via XPath could be another solution. And, since this button was inside a form, the actual click() and submitForm() behaviour is different.
// Nightwatch example
When(/^Click on some element$/, async () => {
await client.click(".abc-grid:nth-child(4) .btn");
});
// Taiko Example
step("Click on some element", async () => {
await click("Continue");
});
So, notice anything? We could not locate the button using the .btn class, or .btn.button or nothing similar, but rather, using the .abc-grid:nth-child(4) .btn selector (this is a real example, btw). We obtained our button class with the aid of some tools, using Chrome's Dev Tools was not enough.
With Nightwatch, we can only locate elements with CSS selectors or XPath.
With Taiko, on the other hand, it lets you find elements with CSS selectors, XPath or with Smart Selectors.
With the above example, the Nigthwatch example clicks the element associated to the CSS or XPath selector I needed to perform the action.
In the Taiko example, it simply clicked where it found the text "Continue". It made a smart search and found the button by it's actual text. No need of selectors. That is a smart selector.
Taiko knows how to perform the action based on the current focus or with a given element.
Smart Selectors in Taiko can find elements by proximity, by its type (button, checkbox, text box, etc.), content or its selector.
In some cases it's useful to find an element by its type. Imagine you have a button and a link with the text "Continue", so the simple click("Continue") is not enough.
await click('Continue')
// These two look for different elements
await click(button('Continue'))
await click(link('Continue'))
// In any case, you can still use CSS selectors, which are still useful, specially if you have an id
await click($('#id-continue'))
await click($('#id-continue',near('username'),below('login')))
// Or XPath
await click($('//*[text()='Continue']').exists())
// And so on. It's very, VERY powerful
We literally spent hours to perform a literal click with Nightwatch/Selenium, and it didn't work.
The second aspect I love about Taiko is its implicit wait calls. Consider the following example.
- Click a button, "Hello"
- Button performs an Ajax call
- When Ajax completes, it shows another button
- Click second button, "Continue"
In Taiko, it would read like this:
await click(button('Hello'))
await click(button('Continue'))
In Nightwatch, it could look like this:
// Assuming given buttons have an id
await client.click("#button-hello");
await client.pause(1000);
await client.expect.element("#button-continue").to.be.visible;
await client.click("#button-continue");
Taiko implicitly knows when it should wait in order to perform the next action (implicit wait calls). In other frameworks, you need to manually input a wait time in these scenarios.
Manually specifying a wait time is not sustainable. What if an action varies from 500 to 1500 milliseconds? Taiko simply knows how and when to wait, and you can set a timeout if it takes too long for some reason.
Taiko's API is more powerful. There are many more features that I haven't tried myself. It also has out-of-the-box parallel execution, sharing state between steps, HTML reporter, and much, much more.
Compare Taiko's API vs Nightwatch JS yourself.
Dreaded autocomplete text boxes
Autocomplete text boxes for some reason don't work sometimes with other frameworks/tools. Even with visual tools such as UIPath, Selenium IDE and Kantu they just don't work (even when using XType).
For us, autocomplete text boxes actually became some sort of "moment of truth" when evaluating any testing tool.
In our eCommerce site, the autocomplete text box functionality was done using jQuery's UI Autocomplete.
jQuery's UI autocomplete works internally with jQuery's own keyup() and keydown() events and by attaching a "datasource" to it. So, setting a value to a text box is not enough, you have to actually "press" a key.
In our site, when selecting an option from the autocomplete text box, you then select an option from another list. It is a crucial step in our "flow" and you cannot skip it.
In Nightwatch API you have the setValue() and sendKeys() methods. Supposedly, the difference is that one simply sets a value and the other one mimics a user typing.
For some reason, we could not get it to work. After our third try I actually tried to manually trigger the behavior by injecting a script, but the widget and JavaScript in general does not work that way.
Taiko also has two similar methods, write(), which simply sets a value and press(), which mimics a user typing. But this time it worked, like magic!
Summarizing
For me, the charm of Gauge is that it just works. It works brilliantly and with little effort.
Even small things that shouldn't be a problem meant no trouble with Gauge/Taiko.
Writing test cases should not only be easy, but also productive and useful. We definitively achieved that with Gauge and with so much ease.
You should definitively give it a try!