arnley

Update: SDN (the official SAP Developer Network) published my article about using Totori as an Automated Testing Tool for SAP Enterprise Portal!

Abstract

Totori is an automated User Acceptance testing workbench that can be used for Portal behavior testing and automated non-regression testing for building, extending and maintaining long-term SAP Portal applications.

Context

You are starting a mission for a reputable company customer, implementing a new Enterprise Portal with a high business impact.

You know for a fact that there will be quantity and complexity: lots of user interfaces, lots of processes, lots of customizing, from-scratch developments in SAP technologies (Web DynPro/Java, BSP, and Enterprise Portal Applications).

Problem

In such a context, where technical complexity meets quantity meets fast responsiveness, the quality of the delivery can suffer, whereas it should never be the case.

If you identify yourself (or your current project) to one of the following situations:

• While developing a new Portal Web Application, you always wonder if you “got it right” and never see the end of the plethora of “business rules” and “use cases”.

• In a SOA architecture, lots of rework on the lower layers create instability in the upper layers (front-end, portal), forcing you to repeat yet-again boring, repetitive and time-consuming non-regression testing.

• You have difficulties delivering evolutions and corrections on time, in a maintenance project for example, whereas you and your colleagues are the A-team and no-one can surpass you at what you do. All because of non-documented developments.

It’s time to think of a new approach which is quality-, quantity-, and complexity-friendly.

But before that, let’s take a look at what we’re used to.

How these problems were addressed until now

Unit testing

Unit testing brings quality and removes uncertainty when they fully cover the code. Once done manually, it soon become evident that its automation would require less time and provide more information, since it could be run at will.

Examples of Unit testing frameworks: JUnit, JTiger

Functional testing

Functional testing brings quality and removes uncertainty over business use cases. It addresses integration issues. Its automation is rare and even recent approaches are quite overwhelming: Rspec, for example, helps documenting tests with plain English, but this documentation is mixed with the code, forcing the developer to maintain the code and the documentation of its features.

Usability testing

Usability testing is the Rolls Royce of tests since it requires end users trying to frenetically break the applications. These tests tend to be documentable, since they aim to meet initial requirements.

I never saw a project automating these tests, nor saw tests being conducted without human interventions.

What changed?

The apparition of multiple advanced programming languages / technologies / frameworks:

• Ruby, a simple yet productive scripting language
• Watir, a library for automated browser testing
• Cucumber, the most advanced Behavior Driven Development (BDD) implementation

Mixing these three interesting assets could well let us achieve something bigger…

A new approach

I wanted to approach the subject with quite an innovative alternative, with the potential of getting rid of uncertainty, adding high value to developments and increasing the project ROI:

Automating behavior tests with features documented in a plain-English manner

Using all the three assets together (Ruby, Watir and Cucumber), this goal can be attained for sure.

Totori is an open source implementation of this goal.

It acts in two steps:

1. First, define a feature that the web application should meet
2. Then, implement corresponding steps in ruby code

The following example, packaged with Totori as a plugin, highlights these two steps with a simple scenario: logging in and off an Enterprise Portal.

Feature definition

Feature: Portal logging in and logging off
  In order to check access to the portal
  As a user
  I want to log into the portal and log off

  Scenario Outline: Unauthorized credentials
    Given I am on the main page
    And I am not logged in
    When I log in with credentials  and

    Then I should see the error message "User authentication error"

    Examples:
      | user           | password |
      | unknown_user_1 | toto     |
      | unknown_user_2 | toto     |

  Scenario Outline: Authorized credentials
    Given I am on the main page
    And I am not logged in
    When I log in with credentials  and

    Then I should be logged in

    Examples:
      | user           | password |
      | user_01        | abcd1234 |

  Scenario: Logging off
    Given I am logged in
    When I log off
    Then I should be logged off

Steps implementation

Given /I am on the main page/ do
  @portal.goto(@config.root_url)
end

Given /I am not logged in/ do
  if @portal.logged_in?
    @portal.logoff
  end
end

Given /I am logged in/ do
  if !@portal.logged_in?
    Given("I am on the main page")
    user = @config.valid_credential
    Given("I log in with credentials #{user[:name]} and #{user[:password]}")
  end
  if !@portal.logged_in?
    raise "Authentication error"
  end
end

When /I log in with credentials (.*) and (.*)/ do |user, password|
  @portal.connect(user, password)
end

When /I log off/ do
  @portal.logoff
end

Then /I should see the (.+) message "(.+)"/ do |severity, text|
  case severity
    when "error" then @portal.reports_error?(text)
    when "warning" then @portal.reports_warning?(text)
    when "success" then @portal.reports_success?(text)
  end
end

Then /I should be logged in/ do
  @portal.logged_in?
end

Then /I should be logged off/ do
  !@portal.logged_in?
end

Generated report

The following report is the result of running an other scenario:

In this example, 4 scenarios passed. Passing and failing scenarios make use of the green and red colors in order to quickly identify if things went well or not.

More about Totori

You can find more information about Totori on http://totori.org. It still is very young (pre-alpha) but very promising.

Going further

• Behavior Driven Development (BDD): http://behaviour-driven.org
• Try out the Totori Workbench: http://totori.org