Small Talks/2013/July/ZK Testing with Selenium IDE

Author
Robert Wenzel, Engineer, Potix Corporation

Date
July 2013

Version
ZK 6.5 (or later)

This page is under construction, so we cannot guarantee the accuracy of the content!

Introduction

About this article

nice application, how to test ... refer to zats... some things need to be tested in a real browser, so selenium comes into play... how to create tests

programmatically... link to older Small Talk?? or use Selenium-IDE to record/tweak/maintain!! and replay/debug -- this article :D

Environment

• Firefox (used with 21.0)
• Selenium IDE 2.0.0
• JDK 6/7
• ZK 6.5.3 6/7
• Maven 3.0

Steps

Initialize your testing environment

• download Selenium IDE 2.0.0 link
  • docu http://wiki.openqa.org/display/SIDE/Home
• Maven 3.0
  • download link
  • setup maven

• download link (test project)
• maven: mvn jetty:run
• localhost:8080

(- war file ) ???

Initial Situation

1. Launch Selenium IDE CTRL+ALT+S
2. open example app localhost:8080
3. Record login-test and replay it

New Test
open	/selenium_testing/login.zul
type	id=jXDW5	test
type	id=jXDW8	test
clickAndWait	id=jXDWb

Looks hard to read / maintain and doesn't even work :'(

Looking at the recorded commands and comparing with page source we notice the IDs are generated and changing with every request. So no chance to record / replay this way.

A way to make the IDs predictable is shown in the next Section.

Custom Id Generator

As mentioned in other Small Talks about testing one strategy is to use a custom IdGenerator implementation to create predictable, readable (with a business meaning) and easily selectable (by selenium) component ids.

public class TestingIdGenerator implements IdGenerator {

	public String nextComponentUuid(Desktop desktop, Component comp,
			ComponentInfo compInfo) {
        int i = Integer.parseInt(desktop.getAttribute("Id_Num").toString());
        i++;// Start from 1
        
        StringBuilder uuid = new StringBuilder("");
        
        desktop.setAttribute("Id_Num", String.valueOf(i));
        if(compInfo != null) {
        	String id = getId(compInfo);
        	if(id != null) {
        		uuid.append(id).append("_");
        	}
        	String tag = compInfo.getTag();
        	if(tag != null) {
        		uuid.append(tag).append("_");
        	}
        }
        return uuid.length() == 0 ? "zkcomp_" + i : uuid.append(i).toString();
    }

The IDs will look like this:

• {id}_{tag}_{##} (for components with a given id)
• {tag}_{##} (for components without a given id - e.g. listitems in a listbox)
• {zkcomp}_{##} (for other cases, just to make them unique)

e.g. a button in zul-file

<button id="login" label="login" />

will become something like this in HTML in browser (number can vary):

<button id="login_button_12" class="z-button-os" type="button">login</button>

In order to separate production from test configuration we can include an additional config file at startup to enable the TestingIdGenerator only for testing. In ZK this is possible by setting the library property org.zkoss.zk.config.path (also refer to our Testing Tips)

e.g. via VM argument -Dorg.zkoss.zk.config.path=/WEB-INF/zk-test.xml

Windows:

set MAVEN_OPTS=-Dorg.zkoss.zk.config.path=/WEB-INF/zk-test.xml
mvn jetty:run

Linux:

export MAVEN_OPTS=-Dorg.zkoss.zk.config.path=/WEB-INF/zk-test.xml
mvn jetty:run

TODO: improve using maven profiles...

Now we get nicer and deterministic IDs when recording a test case.

After recording login-test again and we get this:

login test
open	/selenium_testing/login.zul
type	id=username_textbox_6	test
type	id=password_textbox_9	test
clickAndWait	id=login_button_12

This already looks nice, and self explaining - Unfortunately still fails to replay :(

Why this happens and how to fix it? See next section ...

ZK specific details

ZK heavily uses JavaScript and Selenium IDE does not record all events by default.

in our case here the "blur" events of the input fields have not been recorded, but ZK relies on them to determine updated fields. So we need to manually add selenium commands "fireEvent target blur" unfortunately Selenium does not offer a nice equivalent like "focus" action.

login test
open	/selenium_testing/login.zul
type	id=username_textbox_6	test
fireEvent	id=username_textbox_6	blur
type	id=password_textbox_9	test
fireEvent	id=password_textbox_9	blur
clickAndWait	id=login_button_12
assertLocation	glob:*/selenium_testing/index.zul
verifyTextPresent	Welcome test
verifyTextPresent	Feedback Overview

Replaying the script finally works :D, and we see the overview page (also added a few verifications for that).

Selenium Extensions

If you find the syntax of "fireEvent" hard to remember or think it is too inconvenient to add an additional line just to update an input field there's help using a Selenium Core extension. The extension - file usually called user-extensions.js and can be used in both Selenium-IDE and Selenium RC when running the tests outside of Selenium IDE (please refer to [selenium documentation]).

This snippet shows 2 simple custom actions:

blur

convenience action to avoid "fireEvent locator blur"

typeAndBlur

combines the type with a blur event, to make ZK aware of the input change automatically

Selenium.prototype.doBlur = function(locator) {
    // All locator-strategies are automatically handled by "findElement"
    var element = this.page().findElement(locator);
    // Fire the "blur" event
    triggerEvent(element, "blur", false);
};

Selenium.prototype.doTypeAndBlur = function(locator, text) {
    // All locator-strategies are automatically handled by "findElement"
    var element = this.page().findElement(locator);

    // Replace the element text with the new text
    this.page().replaceText(element, text);
    // Fire the "blur" event
    triggerEvent(element, "blur", false);
};

To enable it just download the example file, and add it to the Selenium-IDE configuration. (the file contains another extension ... more about that later in "making tests more robust and maintainable to UI updates")

(Selenium-IDE Options > Options... > [General -Tab] > Selenium Core extensions)

Then restart Selenium-IDE - close the window, and reopen it - e.g. by [CTRL+ALT+S]

Adapted test using the blur action:

login test
open	/selenium_testing/login.zul
type	id=username_textbox_6	test
blur	id=username_textbox_6
type	id=password_textbox_9	test
blur	id=password_textbox_9
clickAndWait	id=login_button_12

Same test using the typeAndBlur:

login test
open	/selenium_testing/login.zul
typeAndBlur	id=username_textbox_6	test
typeAndBlur	id=password_textbox_9	test
clickAndWait	id=login_button_12

It is not required to use either of these extensions, but saving 1 line for each input is my personal preference.

Another thing to keep in mind is robustness and maintenance effort of test-cases when changes in the UI happen ... the generated numbers in the end of each ID are still an obstacle on this way as they are prone to change everytime a component is added or removed above that component.

Wouldn't it be nice to avoid having to adapt test cases that often? Read on...

Improve robustness and maintainability of Test Scripts

Locators in Selenium

To remove the hard coded running numbers of the component IDs from our test cases Selenium offers e.g. XPath or CSS locators.

Using XPath is it possible to select a node by its ID-prefix:

//input[starts-with(@id, 'username_')]

This will work as long as the prefix "username_" is unique on the page, otherwise will perform the action on the first element found. So it is a good idea for this scenario to give widgets suitable IDs (plus: it will also improve the readability of source code).

In more complex cases one can select nested components to distinguish components with the same ID (e.g. in different ID spaces). e.g. if the "street" component appears several times on the page use this:

//div[starts-with(@id, 'deliveryAddress_')]//input[starts-with(@id, 'street_')]
//div[starts-with(@id, 'billingAddress_')]//input[starts-with(@id, 'street_')]

Another example using locating the delete button in the currently selected row of a listbox using ID prefixes, CSS-class and text comparison.

//div[starts-with(@id, 'overviewList_')]//tr[contains(@class, 'z-listitem-seld')]//button[text() = 'delete']

NOTE: Make sure not too use the "//" operator too extensively in XPath, as it might perform badly. (for me this was never a matter sofar, as there are much worse performance bottle necks, affecting your test execution speed - improve test execution speed).

This sheet provided by Michael Sorens is an excellent reference with many helpful examples how to select page elements in various situations.

So now we can change the test case to use this kind of XPath locator, searching only by ID prefix:

login test
open	/selenium_testing/login.zul
typeAndBlur	//input[starts-with(@id, 'username_')]	test
typeAndBlur	//input[starts-with(@id, 'password_')]	test
clickAndWait	//button[starts-with(@id, 'login_')]

Done that, the order of the components may change without breaking the test, as long as the actual IDs are not changed. Of course this requires some manual work, but the effort invested once will pay off quickly as your project evolves.

Custom Locator and LocatorBuilder

If you don't want to change the locator to XPath manually everytime after you recorded a test case Selenium IDE has more to offer. Even improving the readability.

Custom Locator

In the user-extensions.js from above you'll find a custom locator that will hide the XPath complexity for simple locate scenarios. It is based on the format of IDs generated by TestingIdGenerator (from above).

// The "inDocument" is a the document you are searching.
PageBot.prototype.locateElementByZkTest = function(text, inDocument) {
    // Create the text to search for
	
    var text2 = text.trim();

	var separatorIndex = text2.indexOf(" ");
	
	var elementNameAndIdPrefix = (separatorIndex != -1 ? text2.substring(0, separatorIndex) : text2).split("#");
	var xpathSuffix = separatorIndex != -1 ? text2.substring(separatorIndex + 1) : "";
	
	var elementName = elementNameAndIdPrefix[0] || "*";
	var idPrefix = elementNameAndIdPrefix[1];

	var xpath = "//" + elementName + "[starts-with(@id, '" + idPrefix + "')]" + xpathSuffix;

    return this.xpathEvaluator.selectSingleNode(inDocument, xpath, null, this._namespaceResolver);
};

this locator will work in 2 forms

1. zktest=#login_
2. zktest=input#username_

Internally generated/queried XPaths will be

1. //*[starts-with(@id, 'login_')]
2. //input[starts-with(@id, 'username_')]

1. will look for any element with an ID starting with "login_"
2. will also test the elements html tag, and find the <input> element with the ID-prefix "username_..."

Custom LocatorBuilder

Now to make this really handy Selenium IDE also offers extensions (don't mix this up with Selenium Core Extension)

Add file TODO:(download file) zktest-Selenium-IDE-extension.js to selenium config

(Selenium IDE Options > Options... > [General -Tab] > Selenium IDE extensions)

and move it up in the Locator Builder priority list.

(Selenium IDE Options > Options... > [Locator Builders - Tab])

It contains the following LocatorBuilder which will generate the locator in form "zktest=elementTag#IdPrefix" mentioned above using only the first part of the ID (the ID prefix - its business name). If an ID does not contain 3 tokens separated by "_" it will use the full ID instead including the sequential number, this will indicate to you that the element has no ID specified and another location approach is maybe better, or just give it an ID in the zul file.

 
LocatorBuilders.add('zktest', function(e) {
	var idTokens = e.id.split("_")
	if (idTokens.length > 2) {
		idTokens.pop();
		idTokens.pop();
		return "zktest=" + e.nodeName.toLowerCase() + "#" + idTokens.join("_") + "_";
	} else {
		return "zktest=" + "#" + e.id;
	}
	return null;
});

Restart Selenium-IDE and record the test case again... and you'll get this (after changing the "type" to "typeAndBlur" events).

New Test
open	/selenium_testing/login.zul
typeAndBlur	zktest=input#username_	test
typeAndBlur	zktest=input#password_	test
clickAndWait	zktest=button#login_

More Tests

Let's record a test case for edit and save an existing "feedback item" in the list, and verify the results, and a logout test. These tests introduce new challenges testing an ajax applications with Selenium - and their solution or workarounds.

Now that we are actually changing data I adapted the login test, to user with a random user each time. So that the Mock implementation will serve fresh data for each test run, without having to restart the server, or having to cleanup (there are other strategies possible to do the same - that's just what I use here).

login test
store	javascript{'testUser'+Math.floor(Math.random()*10000000)}	username
open	/selenium_testing/login.zul
typeAndBlur	zktest=input#username_	${username}
typeAndBlur	zktest=input#password_	${username}
clickAndWait	zktest=button#login_
assertLocation	glob:*/selenium_testing/index.zul
verifyTextPresent	logout ${username}

Edit and Save Test

After recording and applying the typeAndBlur actions we get the following.

edit save test
open	/selenium_testing/index.zul
click	zktest=#button_27
typeAndBlur	zktest=input#subject_	Subject 1 (updated)
typeAndBlur	zktest=input#topic_	consulting
select	zktest=select#priority_	label=low
click	zktest=#listitem_62
typeAndBlur	zktest=textarea#comment_	test comment (content also updated)
click	zktest=button#submit_
assertTextPresent	Feedback successfully updated
verifyTextPresent	subject=Subject 1 (updated)
verifyTextPresent	[text=test comment (content also updated)]
verifyTextPresent	priority=low
click	zktest=button#backToOverview_
assertTextPresent	Feedback Overview

We still see some hard-coded numbers. I'll just delete the "click zktest=#listitem_62" as the "select" will command will just do fine (Selenium-IDE will sometimes record more than you need, and sometimes just not enough - by default).

Interesting for now is the locator "zktest=#button_27" - one of the the "edit" button. If we try to give it an ID in the zul code it will fail to render (duplicate ID), because the button is rendered multiple times - once for each <listitem>. (one workaround could be to surround it by an IDspace component or include the buttons from a different file). But it is not necessary - using the right locator XPath.

Locating an Element inside a Listbox

If you just want to locate the first edit button in the list you can use: it will find the button by its text and not by its ID

pure XPath:

//*[starts-with(@id, 'overviewList_')]//button[text() = 'edit']

ok combine with the "zktest" selector from the selenium extension above

zktest=#overviewList_ //button[text() = 'edit']

more interesting is it to locate by index, e.g. exactly the second edit button

xpath=(//*[starts-with(@id, 'overviewList_')]//button[text() = 'edit'])[2]

or even more fun to select the "edit" button inside a <listitem> containing a specific text in a cell

xpath=//*[starts-with(@id, 'overviewList_')]//*[starts-with(@id, 'listitem_')][.//*[starts-with(@id, 'listcell_')][text() = 'Subject 2']]//button[text() = 'edit']

This can get infinitely complex, and reduce the readability of your testcase. That's why it is also a good idea to write comments in your test case (yes test cases can have comments too).

Even though quite complex this last locator is still quite stable to changes in the UI. Especially when the <listbox> content changes, you can still use it to find the same <listitem> (and its button) several times in a test case.

Using Variables

Repeating them in the code will give you a headache maintaining them, when e.g. the label of the button changes from "edit" to "update". It is better to "store" locator strings or parts of them in variables, which can be reused throughout the test.

store	xpath=//your/complex/locator[text() = 'edit']	editButton
...	...	...
verifyVisible	${editButton}
click	${editButton}

TODO screenshot with comments and variables

These are just a few general ideas, about what can be done to reduce the work maintaining test cases increase stability or make them more readable - at reasonable investment when setting them up (recording and adjusting) initially.

Waiting for AJAX

However rerunning the test suite at full speed (which is what we should always aim for) will - or worse, might sometimes - fail, because we don't wait long enough for ajax responses to render... there is no full page reload so Selenium-IDE does not wait automatically.

An idea might be to reduce the test speed... which would affect all test cases, so we try to avoid that. Also there is a longer operation when clicking the "submit" button (2 seconds). Delaying every Step by 2 seconds is not acceptable.

Another possibility is using the pause command and specify the number of milliseconds to wait. Also here the execution speed might vary so we are either waiting too short, so that errors still occur, or we wait too long and waste time - both not desirable.

Luckily Selenium comes with a variety of waitFor... actions, which stop the execution just until a condition is met. A useful subsection is:

• waitForTextPresent - wait until a text is present anywhere on the page
• waitForText - wait until an element matching a locator has the text
• waitForElementPresent - wait until an element matched by a locator becomes rendered
• waitForVisible - wait until an element matched by a locator becomes visible

The previously recorded "edit save test" would likely fail in line 3 just after the click on the edit button. Between Step 2 and 3 ZK is updating the content of the page, without reloading the Page causing a minor delay.

edit save test
open	/selenium_testing/index.zul
click	zktest=#overviewList_ //button[text() = 'edit']
typeAndBlur	zktest=input#subject_	Subject 1 (updated)
typeAndBlur	zktest=input#topic_	consulting
select	zktest=select#priority_	label=low
typeAndBlur	zktest=textarea#comment_	test comment (content also updated)
click	zktest=button#submit_
assertTextPresent	Feedback successfully updated
...	...	...

So we have to wait e.g. until the new Headline "New/Edit Feedback" is present. And wait after submitting the feedback article accordingly - replace the "assertTextPresent" with "waitForTextPresent". Also the last assert... can be replaced.

edit save test
open	/selenium_testing/index.zul
click	zktest=#overviewList_ //button[text() = 'edit']
waitForTextPresent	New/Edit Feedback
typeAndBlur	zktest=input#subject_	Subject 1 (updated)
typeAndBlur	zktest=input#topic_	consulting
select	zktest=select#priority_	label=low
typeAndBlur	zktest=textarea#comment_	test comment (content also updated)
click	zktest=button#submit_
waitForTextPresent	Feedback successfully updated
verifyTextPresent	subject=Subject 1 (updated)
verifyTextPresent	[text=test comment (content also updated)]
verifyTextPresent	priority=low
click	zktest=button#backToOverview_
waitForTextPresent	Feedback Overview

Now this test can run as fast as possible adapting automatically, to any speedup, or slowdown of the test server.

Logout Test

This test should be very simple, but still there is a catch. When trying to record this test, we notice that Seleniume-IDE won't record clicking on the logout button. Selenium-IDE is not very predictable about which events are recorded and which are ignored. In this case the it is a <toolbarbutton> which is rendered as a <div> so maybe that's why.

(There is a [stackoverflow question about this] and also an [IDE extension] available to enable recording of ALL clicks in a test. I tried it this one - and didn't like the big number of clicks recorded. In the end I leave it up to you to uncomment this in the IDE extension for Locator Builder provided above and judge yourself - maybe you want to test exactly this.)

As we know its name (or we can easily find out with tools like firebug) we just add another click event manually and change the locator to "zktest=div#logout_"

logout test
click	zktest=div#logout_
waitForTextPresent	Login (with same username as password)

Recording Other Components

Other components require some extra attention too when recording events on them. (sometimes you might wonder why the results are varying). So here just a few examples.

Menu

Recording events on a <menubar> (even with submenus) is generally very simple. Unless you click the "wrong" areas while recording highlighted in the image below. If you just click the menu(item)-text Selenium-IDE will create nice locators, otherwise it will fallback to some other strategy, using CSS or generated XPath which might surprise you (this is no bug in Selenium-IDE, it is just another html element receiving the click event).

menu test (wrong)
click	css=td.z-menu-inner-m > div
click	css=span.z-menuitem-img

What you get avoiding the areas.

menu test (right)
click	zktest=button#feedbackMenu_
click	zktest=a#newFeedbackMenuItem_