Monthly Archives: April 2014

Getting Better

Exploratory-testing evangelist Jon Bach once told me this:

  • Only one tester in ten that he’s interviewed claims to do anything at all toward professional improvement.

One way to become a better tester is to read.

From time to time I like to review one of the more important books about a tool or technique I’m working with. Among such books are the Cookbooks from O’Reilly.

Right now I’m working with C#, so before long I’ll spend an hour or two just leafing through the C# Cookbook, reviewing its recipes. When I do this, I almost always find something useful that I didn’t need before, but now do.

Here’s a link to some of these cookbooks on Amazon.

Enjoy!

Improve!

Something for a Friday Afternoon

From satirist Jonathan Swift:

So nat’ralists observe, a flea
Hath smaller fleas that on him prey,
And these have smaller fleas that bite ’em,
And so proceed ad infinitum.

Logician Augustus De Morgan goes him one better:

Great fleas have little fleas
Upon their backs to bite ’em,
And little fleas have lesser fleas,
And so ad infinitum.

And the great fleas themselves, in turn,
Have greater fleas to go on,
While these again have greater still,
And greater still, and so on.

Clean and DRY Verifiers

In a Coded UI Test (CUIT), a test method is a method that has attribute TestMethod. A test method is what many might call a test script. It’s the outermost method in the test, and directs the test steps.

Some say that only the test method itself should perform verifications, that a method in a page object (or other supporting object) should not perform verifications automatically:

The usual reason given is that such automatically performed verifications definitely will affect performance, and may not even be wanted or needed in a particular test context.

I agree, but with one addition: a method can appropriately perform verification at the request (direct or indirect) of the test method. So the request for verification should originate in the test method.

No matter where the actual verification is performed, the verifier method must log the expected value, the actual value, the verdict (pass or fail), and a descriptive message.

Question: Where is the best place to perform the actual verification?

Answer: Wherever it will be clean and DRY (Don’t Repeat Yourself).

A that will be where the verification method has the fewest and simplest parameters passed to it: in a page object!

A page object encapsulates its entire page, so it already has access to the HTML control that has the actual value for the verification. That means that a verification method in the page object need not pass the actual value. And that means, in turn, that a call to the verification method has fewer parameters: at most, just the expected value and a message string. That’s pretty DRY.

But wait, there’s more!

When the expected value is a constant (a table column header, for example), that value can also be stored in the page object. So in that case, the verification method would have no parameters at all. That’s really DRY.

Examples:

  • Home page verifies logged-in user’s name:
    public Boolean VerifyUserName(String expectedValue, String message = "User name")
  • User page verifies user data:
    public Boolean VerifyUser(User expectedValue, String message = "User")
  • Users page verifies that user does not exist:
    public Boolean VerifyUserNotExist(User expectedValue, String message = "User does not exist")
  • Page object knows its own column headers: public VerifyColumnHeaders()
  • Page object knows its own URL: public VerifyUrl()

Finally, I have a special-purpose verifier:

  • Verify that the locators in a page object correspond to actual controls in the UI: public Boolean VerifyLocators()

So performing verification in a page object, under the supervision of the test method, is easy. And doing so improves both cleanliness and DRYness.

Object not Found? Log the Context!

“Object not found.”

That’s what a GUI test tool is likely to log when an object is, well, not found. And many times no useful additional information — context — is available.

But there are situations when context is available, but usually not reported: and that situation is when some sort of selection fails.

Examples:

  • Menu item not found.
  • Tree view or cascaded menu item not found.
  • Radio button not found.
  • Select list option not found.

In these situations, it’s very useful for the test log to report what was found:

  • Menu: items found in the menu and the item not found.
  • Tree view or cascaded menu: nesting-level of the failure, items found at that level, the item not found, and the items successfully found farther up the tree.
  • Radio button: buttons found in the set and the item not found.
  • Select list option: options found in the list and the option not found.

This can really matter.

Suppose, for example, that the spelling (or even the casing) of an item is changed. You might have to breakpoint the test and run it for minutes, just to see what’s going on. But if the context of the failure — the items that were found — are logged, you’d immediately see what’s wrong.

So how to do this? In the GUI encapsulator, discussed in post
“Encapsulate the GUI Tool?”

For example, suppose in the GUI encapsulator you have a method whose job it is to select a given option from a given select list:

  • Create a new method that logs all the options in a given select list.
  • There will already be code to search for the relevant control. Around that code, place a try block.
  • In the catch block, call the new logger method, then re-raise the exception.

Now when a desired select option is not found, the log will contain all the items that were in the list, which you can now examine without re-running the test. Time-saver!

Try it! You’ll like it!

Location, Location, Location

In my Coded UI Test (CUIT) page objects, I encapsulate locator data into a Locator object. A locator defines and specifies the search for a specific HTML control in the target web application.

The Locator object has:

  • A locator name.
  • One or more name/value pairs, each of which indicates an attribute name and value to search for.
  • A search criterion: Contains or EqualTo.
  • A status:
    • Required: The control is expected to be on the page now; it is an error if it is not present.
    • Forbidden: The control is expected to not be on the page now; it is an error if it is present.
    • Allowed: The control is allowed (but not required) to be on the page now; it is not an error whether or not it is present.

Locator Creation

A page object creates locators when it is instantiated. Many of the locators will be Required. Some may be Forbidden or Allowed.

Locator Maintenance

The page object is responsible for maintaining the status of its locators. For example, if a locator is initially Forbidden, but some Javascript creates the corresponding control, the page object must change the status to Required.

More specifically, I have a user page with a delete button. If the test presses that button, the application will put up a display with buttons for confirming or cancelling the deletion. The locators for these two buttons, which have been Forbidden must now be changed to Required.

Conversely, when one of those two buttons is clicked, their containing display is removed, and the two locators must be returned to Forbidden.

The reason for this strict locator maintenance is that the page object may be called upon at any time to verify its locators’ controls: to confirm that each Required locator’s is present, and that each Forbidden locator’s control is absent.

In the Page Object: Locators, not Controls

A page object operates on an HTML element by passing a locator to a lower-level library method. That method finds the control and performs the operation. For example, a page object can call a button-click method, passing a locator; the button-click method uses the locator to find the button, then clicks it.

The page object does not retrieve a control, nor, I think, should it. It’s not clear to me when or why the object might to go “stale” (no longer reflect the state of the UI), so I prefer to get a fresh object for each operation. And, I think, doing so is just good data-hiding.

A Thorn in My Side

A very few of the HTML elements I’m interested in do not have sufficiently unique attributes to support unambiguous location. For example, there may be several h2 elements on the page, each with no attributes at all.

Perhaps the most fragile way to get at one of these is to search for all h2 elements, then take the one with the appropriate index. A change in the number or positions of the elements can break the test.

A way that’s only a little better, and one that I’ve felt obliged to use occasionally, is to find a nearby element (one that can easily be located), then “walk” the DOM (accessing parent and child elements) to get to the desired element.

I hate having this DOM-walking code in my page object, even though it’s factored into a method. Doing things this way means that there are two completely different ways the page can operate on a control:

  • The right way: Call for the operation to be done, passing a locator object.
  • The wrong way: User a locator to retrieve a control, walk the DOM to get to the desired control, then call for the operation to be done, passing a control.

Note that this means that for the same operation on different HTML elements, I need two method overloads: one that accepts a locator object, and another that accepts a control object.

A Partial Solution

A partial solution I’m considering is enhancing class Locator so that it stores one of the following:

  • Name/value pairs, representing the usual attributes in the element to be searched for.
  • The method that’s called finds the control and performs the operation.

  • A reference to a method that returns a control. The method that’s called calls the function, gets the control, and performs the operation.

Doing this would at least mean that a page object always works with a locator, and never with method that returns a control object. And that in turn would mean that if (when?) the HTML is improved, the locator could be adjusted without requiring other changes.

Thoughts, anyone?

Something for a Friday Afternoon

[On Fridays, I’ll try to post something, usually off-topic, that I think is interesting.]

As part of our preparation for travelling to Northern India in ’11, The Dean (that’s my wife) and I viewed the film Jodhaa Akbar, wherein a Hindu princess is betrothed to a Muslim emperor (which all ends well).

The entire film has a wonderful soundtrack. This stunning scene has some of the best music from the film.

Enjoy!

Encapsulate the GUI Tool?

It’s pretty well established that a GUI test library should encapsulate the web pages. See Keep Your Page Objects DRY.

But encapsulate the GUI tool itself?

Well, yeah.

First off, what does it mean to encapsulate the GUI tool? It means that only one class makes direct calls to methods in the GUI tool. That class “wraps” methods in the GUI tool, performing any desired actions before and after calling into the tool. All other classes, including the page-object classes, call into the wrapper class, and not into the GUI tool’s classes.

How does this matter? Let me count the ways.

Diagnostics

I’ve recently worked with a GUI tool that, when it could not locate a control, simply raised an exception saying “Object not found,” together with the search criteria it used.

On any such failure, a method should return something useful. Often this can be what it did find:

  • Select list option not found: return the options found in the list.
  • Treeview or cascading menu item not found: return the items found in the tree or menu.
  • Radio button not found: return the labels of the buttons found.

This information can save lots of time — time not spent running the test up to the point of failure, just to see what’s there.

Btw, in my case I’ve named the class WebGui, because my GUI tool also has an interface to Windows applications.

So in my class WebGui, there are methods that log diagnostic messages. Method WebGui.SelectOption, for example, first calls the GUI tool’s method to select the option. If the selection is not found, WebGui.SelectOption logs an error together with all the options that were found.

This can mean a big time saving when, for example, the difference is just a spelling change.

Return Values

Consider whether a wrapper method can return something useful. For a setter method, it’s always useful to return the previous value, to allow the value to be restored. See Getters and Setters.

In my method OpenHyperlink, I’m returning the URL of the opened link.

Instrumentation

You can add instrumentation to the wrappers, so that useful information gets captured in the test log. One implementation is to use an environment variable to control the verbosity of the logging done in the wrappers. By setting the environment variable, you can “turn on the rally lights” in the test, and get much more detail than usual.

I also like to add timestamps and durations to the logged calls, which can help identify trouble spots in the tests.

Options

You can put options onto the wrapper methods that are lacking in the GUI tool:

  • Retry an action some number of times, or for some time period.
  • Confirm that a setter method actually worked (raise exception if not).
  • Optionally use a strategy different from the default strategy; for example, send keystrokes instead of clicking.

In Conclusion

The point here is not that you’ll want to use all of these ideas. The point is that if you encapsulate the GUI tool, you’ve left open the door to doing so.