Automatic acceptance testing with Selenium Webdriver, JBehave, Spring, Jenkins and Maven

The story of getting an internationalized, automated acceptance test suite up and running.

I had been assigned a task of automating regression tests of the graphical interface of an administration application. The project already had a fair coverage of unit and integration tests on WS level. I needed to simulate the administrator’s tasks to ensure stability.

This post will highlight some of the issues I dealt with.

I started off with a skeleton Maven project set up by a colleague. He had used the PageObject design pattern and a template setup found online. I took my JBehave experience and leveraged the project into a running scenario mapped to a JUnit unit test. So far so good.

Behavioral Driven Development, BDD, was a key input into the assignment and as such I wanted to translate the scenarios, i.e. the keywords (the rest I just plain regex matching), into our native language. Native language usually means UTF-8 and you all know where this is heading. Where did I need to put all my encoding instructions?

Anyway, I started out with the following test class.

package se.cag.cd.test;
<imports>

@RunWith(SpringAnnotatedEmbedderRunner.class)
@Configure(using = SeleniumConfiguration.class, pendingStepStrategy = FailingUponPendingStep.class)
@UsingEmbedder(embedder = Embedder.class, generateViewAfterStories = true, ignoreFailureInStories = true, ignoreFailureInView = false, storyTimeoutInSecs = 6000, threads = 1, metaFilters = "-skip")
@UsingSpring(resources = { "cd-steps.xml" })
public class CDStories extends InjectableEmbedder {
@Test
public void run() throws Throwable {
  Keywords keywords = new LocalizedKeywords(locale());
  CrossReference crossReference = new CrossReference().withJsonOnly().withOutputAfterEachStory(true).excludingStoriesWithNoExecutedScenarios(true);
  ContextView contextView = new LocalFrameContextView().sized(800, 80);
  SeleniumContext seleniumContext = new SeleniumContext();
  SeleniumStepMonitor stepMonitor = new SeleniumStepMonitor(contextView, seleniumContext, crossReference.getStepMonitor());
  Format[] formats = new Format[] { new SeleniumContextOutput(seleniumContext), CONSOLE, TXT, WEB_DRIVER_HTML, XML };
  StoryReporterBuilder reporterBuilder = new StoryReporterBuilder().withCodeLocation(codeLocationFromClass(this.getClass()))
.withFailureTraceCompression(true).withRelativeDirectory("jbehave").withDefaultFormats().withFormats(formats).withKeywords(keywords).withCrossReference(crossReference).withFailureTrace(true);
  Configuration configuration = injectedEmbedder().configuration();
  configuration
  .useKeywords(keywords)
  .useStoryParser(new RegexStoryParser(keywords))
  .useStepCollector(new MarkUnmatchedStepsAsPending(keywords))
  .useStepMonitor(stepMonitor)
  .useStepFinder(new StepFinder(new StepFinder.ByLevenshteinDistance()))
  .useDefaultStoryReporter(new ConsoleOutput(keywords))
  .useStoryReporterBuilder(reporterBuilder)
  .useParameterConverters(
    new ParameterConverters().addConverters(new ParameterConverters.ExamplesTableConverter(new ExamplesTableFactory(keywords))));
  if (configuration instanceof SeleniumConfiguration) {
    SeleniumConfiguration seleniumConfiguration = (SeleniumConfiguration) configuration;
    seleniumConfiguration.useSeleniumContext(seleniumContext);
  }
injectedEmbedder().runStoriesAsPaths(storyPaths());
}

private Locale locale() {
  return new Locale("sv", "SE");
}

  protected List<String> storyPaths() {
    String searchIn = codeLocationFromClass(this.getClass()).getFile();
    List<String> foundPaths = new StoryFinder().findPaths(searchIn, asList("**/" + System.getProperty("storyFilter", "*") + ".story"), asList("**/" + System.getProperty("storyFilter", "*test*") + ".story"));
    return foundPaths;
  }
}

All of a sudden, all my steps were Pending. What? Here comes UTF-8 encoding. Dove into the maven pom and add

<properties>
  <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
</properties>

And under the maven-failsafe-plugin add

<configuration>
  <encoding>UTF-8</encoding>
  <inputEncoding>UTF-8</inputEncoding>
  <outputEncoding>UTF-8</outputEncoding>
  <argLine>-Dfile.encoding=UTF-8</argLine>
</configuration>

Which got it working again.

Having come this far, the tests are running on my local laptop. Not much automation going on there. I need to put it on our Jenkins server.

Obstacles:

  1. Server is a RHEL5 which doesn’t support very modern Firefox  versions.
  2. No screen to view the interface on.

Actions:

  1. Asked IT to set up a RHEL6 for me. Now I have a Firefox 17 at least.
  2. Set up the job to initiate headless screen support. I used the Jenkins Xvfb plugin and configured it to start before job and end after job. Furthermore I set up the selenium-maven-plugin in a Jenkins specific maven profile. It starts the selenium server before integration test phase. See below.
    1. Needing a step prior to tests meant I had to configure my tests to be run in the integration test phase and NOT in the test phase as defaulted by JUnit.
<profile>
  <id>jenkins</id>
  <build>
    <plugins>
      <plugin>
        <groupId>org.codehaus.mojo</groupId>
        <artifactId>selenium-maven-plugin</artifactId>
        <version>2.0</version>
        <executions>
          <execution>
            <id>selenium</id>
            <phase>pre-integration-test</phase>
            <goals>
              <goal>start-server</goal>
            </goals>
            <configuration>
              <background>true</background>
            </configuration>
          </execution>
        </executions>
      </plugin>
    </plugins>
  </build>
  <activation>
    <activeByDefault>false</activeByDefault>
  </activation>
</profile>

<plugin>
  <groupId>org.apache.maven.plugins</groupId>
  <artifactId>maven-surefire-plugin</artifactId>
  <dependencies>
    <dependency>
      <groupId>org.apache.maven.surefire</groupId>
      <artifactId>surefire-junit47</artifactId>
      <version>2.13</version>
    </dependency>
  </dependencies>
  <configuration>
    <excludes>
      <exclude>**/*.class</exclude>
    </excludes>
  </configuration>
</plugin>
<plugin>
  <artifactId>maven-failsafe-plugin</artifactId>
  <version>2.12</version>
  <dependencies>
    <dependency>
      <groupId>org.apache.maven.surefire</groupId>
      <artifactId>surefire-junit47</artifactId>
      <version>2.13</version>
    </dependency>
  </dependencies>
  <configuration>
    <encoding>UTF-8</encoding>
    <inputEncoding>UTF-8</inputEncoding>
    <outputEncoding>UTF-8</outputEncoding>
    <argLine>-Dfile.encoding=UTF-8</argLine>
  </configuration>
  <executions>
    <execution>
      <goals>
        <goal>integration-test</goal>
      </goals>
      <configuration>
        <includes>
          <include>**/*.class</include>
        </includes>
      </configuration>
    </execution>
  </executions>
</plugin>

Having the tests run in automated fashion on a Jenkins server means no one is there to watch errors occur. How do we report them? Luckily JBehave web offers functionality to capture screenshots on failure. Just insert the following in your Spring beans file:

<bean id="screenshotOnFailureDriver" class="org.jbehave.web.selenium.WebDriverScreenshotOnFailure">
  <constructor-arg ref="driverProvider" />
</bean>

As you can see it takes a WebDriverProvider as constructor arg. I started out using a WebDriver to control my scenarios and couldn’t understand why the screenshooting functionality didn’t kick in. Eventually I found that it needed WebDriverProvider to be initiated. Now it works J

So, finally, I have my tests working. Still waiting for a few things to be worked out before actual automation can start though.

  1. Due to our application I need a localized version of Firefox installed on the Jenkins server.
  2. I run my tests on an existing test environment. I do not setup and deploy to a temporary controlled environment. License issues etc. are preventing that.

Other quirks

I tried to localize the step parameters in my Java files but it seems that the StepMatcher from JBehave doesn’t support that yet. Or it’s an encoding issue I won’t bother dig deeper into.

We have AJAX updates on the site and sometimes I noticed that the tests ran to fast for the updates so that I got stale elements on the page. So even when using WebDriverWaits I needed to use some Thread.sleep in order to not get a handle to an element which later got updated.

Publicerad i Continuous Delivery, Java, Test Taggar: , , , , ,

Kategorier

WP to LinkedIn Auto Publish Powered By : XYZScripts.com