Automated tests for SOAP API in Java

Don’t think the story will be short, but it’s definitely interesting. So, I got a new project where I need to cover a SOAP API with automated tests written on Java. As a starting point I have a URL to a WSDL file to work with. The “tricky” part is in the environment configuration. Our test environment is hidden behind a SOCKS proxy server. So, if you want to work with a test environment, you need to make your requests through that proxy, otherwise you’ll “talk” to another environment that we don’t need right now. Not a usual configuration, but it had it’s own reasons for doing so. My first problem was to decide how to work with WSDL and in what direction my future framework would go. After some Googling and trials, I stopped on the JAX-WS library. This is handy because with the “maven wsimport” task, you can easily generate all needed classes and objects where your stubs will be held. Here’s how it looks like in pom.xml:

                <plugin>
                    <groupid>org.jvnet.jax-ws-commons</groupid>
                    <artifactid>jaxws-maven-plugin</artifactid>
                    <version>${jaxws.plugin.version}</version>
                    <executions>
                        <execution>
                            <id>{put_some_id}</id>
                            <phase>validate</phase>
                            <goals>
                                <goal>wsimport</goal>
                            </goals>
                            <configuration>
                                <!--<wsdldirectory>${project.basedir}\src\wsdl</wsdlDirectory>-->
                                <wsdllocation>{your_wsdl_location}</wsdllocation>
                                <destdir>${project.basedir}\src\main\java\Stubs\oop</destdir>
                                <wsdlfiles>
                                    <wsdlfile>${project.basedir}\src\main\resources\wsdl\myWSDL.wsdl</wsdlfile>
                                </wsdlfiles>
                                <packagename>Stubs.oop</packagename>
                                <!-- Without this, multiple WSDLs won't be processed -->
                                <stalefile>${project.build.directory}/jaxws/stale/wsdl.done</stalefile>
                            </configuration>
                        </execution>
                        <execution>
                            <id>{put_some_id}</id>
                            <phase>validate</phase>
                            <goals>
                                <goal>wsimport</goal>
                            </goals>
                            <configuration>
                                <!--<wsdldirectory>${project.basedir}\src\wsdl</wsdlDirectory>-->
                                <wsdllocation>{your_wsdl_location}</wsdllocation>
                                <destdir>${project.basedir}\src\main\java\Stubs\wsdl2</destdir>
                                <wsdlfiles>
                                    <wsdlfile>${project.basedir}\src\main\resources\wsdl\your_wsld2.wsdl</wsdlfile>
                                </wsdlfiles>
                                <packagename>your_package_name</packagename>
                                <!-- Without this, multiple WSDLs won't be processed -->
                                <stalefile>${project.build.directory}/jaxws/stale/wsdl.done</stalefile>
                            </configuration>
                        </execution>
                    </executions>
                </plugin>

It was a bit hard for me to get the WSDL file from the URL, so I decided to save it locally and read it from there then. Since changes on the API are not frequent, I can easily accept this. This configuration allows to add as many WSDL sources as I want and to store stubs for each of them in project. First problem was solved. Now, I had to figure out how to make my framework to use proxy for it’s calls. My first trial was basic and most common in the Java world:

System.getProperties().put("proxySet", "true");
System.getProperties().put("proxyHost", getProxyHost());
System.getProperties().put("proxyPort", getProxyPort());

But I kept getting errors and no successful connection. My next guess was that problem could be in the SSL, since we are using HTTPS. So, I started the browser, imported the security certificates (they were self-signed since it’s a test environment), added them to java “cacerts” keystore, and added next to my code:

System.setProperty("javax.net.ssl.keyStore", "keystore.jks");
System.setProperty("javax.net.ssl.trustStrore", "cacerts.jks");
System.setProperty("javax.net.ssl.keyStorePassword", "changeit");

I tried to debug and to make sure that the JVM is using my proxy settings and it was true – everything was there. What would be the next step in debugging for me? I decided to install a local proxy and try to put my requests through it, so I could get more details. I installed Charles – a fantastic proxy server that helped me a lot in the past with REST API automation. It has a life changing ( for me in this case) “External Proxy” option . I set it to proxy all requests to our SOCKS proxy and I put my Java API calls through Charles as well. And it worked! You can’t even believe how happy I was because of it. But now I got another problem – Charles cost $50 for one license. So, there were two questions that I needed to solve:

  • Why my Java calls works fine with Charles and without it they fail?
  • Where to find free and easy to setup HTTP proxy?

With help from one of our developers we could find out why Java kept getting authentication errors and didn’t want to connect to test environment properly. The problem was that Java was connecting to the SOCKS proxy properly, but it was resolving remote DNS to another environment to which I didn’t have proper certificates and credentials for authorization! I found only one article on StackOverflow regarding this issue and from that point I new that I need to have local HTTP proxy that is:

  • free
  • can forward my requests to SOCKS proxy

I’ve spent whole day trying to find an easy-to-use and adequate proxy server for Windows ( it’s my Workstation that I need to use), I looked over 15 different apps and failed. In the end, I installed VirtualBox with Ubuntu and install there Squid3 proxy server. It’s the most popular and common proxy server for Unix, from what I saw on Google. But this crazy application has a configuration file with more then 3000 lines! It’s not that easy to make it run as I want, it’s not even that easy to restart it. So, after couple hours I gave up on it and started looking for more solutions. Luckily, I found Polipo – tiny, easy to install and setup proxy server. It only has a few main options that I had to setup and make everything work as a charm:

  • proxyAddress – set it to IP address of your local virtual box, or your local machine, if you use Linux or Mac
  • allowedClients – list of IP addresses from which Polipo will allow to access it and forward requests towards another proxy or web directly
  • socksParentProxy = “host name and port of your proxy to which I needed to forward my requests”
  • socksProxyType = socks5

Save changes and restart – that’s it! After that I pointed my Java framework to the local proxy and got green tests! To set the proxy for my tests I used custom MyProxySelector class:

                      package Base;

                      import java.net.*;
                      import java.util.ArrayList;
                      import java.util.HashMap;
                      import java.io.IOException;

                      public  class MyProxySelector extends ProxySelector {
                          // Keep a reference on the previous default
                          public ProxySelector defsel = null;

                          /**
                           * Inner class representing a Proxy and a few extra data
                           */
                          class InnerProxy {
                              Proxy proxy;
                              SocketAddress addr;
                              // How many times did we fail to reach this proxy?
                              int failedCount = 0;

                              InnerProxy(InetSocketAddress a) {
                                  addr = a;
                                  proxy = new Proxy(Proxy.Type.HTTP, a);
                              }

                              SocketAddress address() {
                                  return addr;
                              }

                              Proxy toProxy() {
                                  return proxy;
                              }

                              int failed() {
                                  return ++failedCount;
                              }
                          }

                          /**
                           * A list of proxies, indexed by their address.
                           */
                          HashMap<SocketAddress, InnerProxy> proxies = new HashMap<SocketAddress, InnerProxy>();

                          public MyProxySelector(ProxySelector def, String host, int port) {
                              // Save the previous default
                              defsel = def;

                              // Populate the HashMap (List of proxies)
                              InnerProxy I = new InnerProxy(new InetSocketAddress(host, port));
                              proxies.put(i.address(), i);
                          }

                          /**
                           * This is the method that the handlers will call.
                           * Returns a List of proxy.
                           */
                          public java.util.List select(URI uri) {
                              // Let's stick to the specs.
                              if (uri == null) {
                                  throw new IllegalArgumentException("URI can't be null.");
                              }
                              /**
                               * If it's a http (or https) URL, then we use our own
                               * list.
                               */
                              String protocol = uri.getScheme();
                              if ("http".equalsIgnoreCase(protocol) ||
                                      "https".equalsIgnoreCase(protocol)) {
                                  ArrayList l = new ArrayList();
                                  for (InnerProxy p: proxies.values()) {
                                      l.add(p.toProxy());
                                  }
                                  return l;
                              }

                              /**
                               * Not HTTP or HTTPS (could be SOCKS or FTP)
                               * defer to the default selector.
                               */
                              if (defsel != null) {
                                  return defsel.select(uri);
                              } else {
                                  ArrayList l = new ArrayList();
                                  l.add(Proxy.NO_PROXY);
                                  return l;
                              }
                          }

                          /**
                           * Method called by the handlers when it failed to connect
                           * to one of the proxies returned by select().
                           */
                          public void connectFailed(URI uri, SocketAddress sa, IOException ioe) {
                              // Let's stick to the specs again.
                              if (uri == null || sa == null || ioe == null) {
                                  throw new IllegalArgumentException("Arguments can't be null.");
                              }

                              /**
                               * Let's lookup for the proxy
                               */
                              InnerProxy p = proxies.get(sa);
                              if (p != null) {
                                      /**
                                       * It's one of ours, if it failed more than 3 times
                                       * let's remove it from the list.
                                       */
                                  if (p.failed() >= 3)
                                      proxies.remove(sa);
                              } else {
                                      /**
                                       * Not one of ours, let's delegate to the default.
                                       */
                                  if (defsel != null)
                                      defsel.connectFailed(uri, sa, ioe);
                              }
                          }
                      }

And to turn the proxy on and off I wrote next switch methods:

            private ProxySelector defaultProxy = ProxySelector.getDefault();

            public void setLocalProxy(){
                MyProxySelector ps = new MyProxySelector(ProxySelector.getDefault(),
                                                         localProxyName,localProxyPort);
                ProxySelector.setDefault(ps);
            }

            public void disableProxy(){
                ProxySelector.setDefault(defaultProxy);
            }

That’s it. Now I can run my tests easily with control over when to the use the proxy and when not to. In the future I’ll move my tests to the CI server (Jenkins most probably) and will setup Polipo HTTP proxy on that environment in two minutes. It’s always nice to solve such non-ordinary problems. Most probably my solution is not very elegant and “right”, but it works right now and for this moment this is all that matters to me, since I can start writing automated tests rather then fighting with configuration issues.

Share

Testing your project with PHPUnit and Selenium

PHPUnit is a very handy tool you can use to test and detect possible mistakes in your PHP project.

Functional testing is also possible with PHPUnit using default headless browser (crawler). It works perfect as long as you don’t need to test functionality which uses JavaScript. For this purpose, you can use PHPUnit and Selenium.

Selenium is a powerful test tool that allows you to perform tests for web applications in any programming language using any mainstream browser.

Installation requirements

Before you start using Selenium to test your PHP project, you need to install the following:

  1. Selenium Server
  2. PHPUnit_Selenium package

1. Installing Selenium Server

Perform the following steps to install the Selenium Server:

    1. Download a distribution archive of Selenium Server.
    2. Unzip the distribution archive and copy selenium-server-standalone-2.35.0.jar (check the version suffix) to a folder from where you will run the server (such as /usr/local/bin).
    3. Start the Selenium Server by running the following command:
java -jar /usr/local/bin/selenium-server-standalone-2.35.0.jar

2. Installing PHPUnit_Selenium package

      The PHPUnit_Selenium package is necessary for natively accessing the Selenium Server from PHPUnit.
      To install it, run the following command:
pear install phpunit/PHPUnit_Selenium

Using Selenium in PHPUnit tests

There are two Selenium test cases:

    • PHPUnit_Extensions_Selenium2TestCase
    • PHPUnit_Extensions_SeleniumTestCase

PHPUnit_Extensions_Selenium2TestCase test case allows you to use the WebDriver API (partially implemented).

<?php
class WebTest extends PHPUnit_Extensions_Selenium2TestCase
{
    protected function setUp()
    {
        $this->setBrowser('firefox');
        $this->setBrowserUrl('http://www.example.com/');
    }

    public function testTitle()
    {
        $this->url('http://www.example.com/');
        $this->assertEquals('Example WWW Page', $this->title());
    }

}
?>

PHPUnit_Extensions_SeleniumTestCase test case implements the client/server protocol to talk to Selenium Server as well as specialized assertion methods for web testing.

<?php
require_once 'PHPUnit/Extensions/SeleniumTestCase.php';

class WebTest extends PHPUnit_Extensions_SeleniumTestCase
{
    protected function setUp()
    {
        $this->setBrowser('*firefox');
        $this->setBrowserUrl('http://www.example.com/');
    }

    public function testTitle()
    {
        $this->open('http://www.example.com/');
        $this->assertTitle('Example WWW Page');
    }
}
?>

Using different browsers with WebDrivers

For running tests with different browsers you should have WebDriver of that browser. As an example, we’ll try to use the Google Chrome browser and its WebDriver.

  1. Download WebDriver.
  2. To make Selenium Server aware of this WebDriver, perform one of the following tasks:
    • Store the Chrome WebDriver binary in the system path
    • Start the Selenium Server with -Dwebdriver.chrome.driver=path/to/your/chromedriver

You can now set Chrome as a browser for your functional test:

...
$this->setBrowser('chrome');
...

Running headless Selenium Server with Xvfb

Sometimes you don’t want the browser to be launched at your desktop during testing, or you may not be using Xserver. To make it work, one of the easiest solutions is to use Xvfb.

Xvfb is an X11 server that performs various graphical operations in memory, without displaying any screen output.

So, let’s try it. At first ensure that you have Xvfb installed on your server. If not, you can install it from your OS repository. For example, to install it on Ubuntu, run the following command:

sudo apt-get install xvfb

Once installed, to run your Selenium Server in Xvfb, run the following command:

DISPLAY=:1 xvfb-run java -jar selenium-server-standalone-2.35.0.jar

Once the server starts, run any of the test case examples. The output should display as follows:

PHPUnit 3.8.0 by Sebastian Bergmann.

F

Time: 4 seconds, Memory: 6.25Mb

There was 1 failure:

1) WebTest::testTitle
Failed asserting that two strings are equal.
--- Expected
+++ Actual
@@ @@
-'Example WWW Page'
+'Example Domain'

/your/project/Tests/Functional/WebTest.php:14

FAILURES!
Tests: 1, Assertions: 1, Failures: 1.

That’s all for now. Enjoy testing!

Source references:

  1. PHPUnit and Selenium (official documentation)
  2. Selenium Server downloads list
Share

BDD with Behat for Symfony2

Behavior Driven Development is an exciting new method of creating software. At LeaseWeb we work with roles like Product Owners, Functional Designers and Testers. These people have no access to the code repository. With BDD (Behavior Driven Development) this might change. BDD allows the Product owner to write the user stories in files with a .feature extension. An example of a feature definition is written below:

Feature: customer_list
  In order to reproduce a customers problem
  As a LeaseWeb employee working in dashboard
  I need to see a list of customers

Scenario: List customers in the dashboard
  Given I am logged in as a employee with username "bob" and password "i<3alice"
  And there is a customer with number "1234567890"
  And there is a customer with number "1234567892"
  When I get redirected after login
  Then I should get a list showing:
    """
    1234567890
    1234567892
    """

As you can see the product owner clarified the user story by describing a scenario. He might add more scenario’s and even backgrounds. Wouldn’t it be really cool if the software developers could implement these scenario’s and the product owner could see the progress by running this file? This is exactly what Behat offers. Software developers implement the methods that Behat generates. By default all generated functions throw a PendingException:

 /**
  * @Given /^I am logged in as a employee with username "([^"]*)" and password "([^"]*)"$/
  */
public function iAmLoggedInAsAEmployeeWithUsernameAndPassword($username, $password)
{
  throw new PendingException();
}

Software developers will replace those throw statements with real implementation like this (this example is using Mink):

 /**
  * @Given /^I am logged in as a employee with username "([^"]*)" and password "([^"]*)"$/
  */
public function iAmLoggedInAsAEmployeeWithUsernameAndPassword($username, $password)
{
  $this->visit('/login');
  $this->fillField('_username', $username);
  $this->fillField('_password', $password);
  $this->pressButton('login');
}

After implementing this function the product owner can run the feature with the “behat” command of Symfony2. The product owner sees the feature where every step of the scenario is colored. Green steps are implemented, brown steps are not yet implemented (Pending) and blue ones are skipped because they depend on pending steps. At the end of the run Behat shows a summary:

1 scenario (1 pending)
5 steps (1 passed, 3 skipped, 1 pending)
0m0.855s

Why BDD might make a huge difference:

  • Stories and scenario’s are added to the repository and can: easily be found, verified and related to a version or release.
  • It enables the product owner to get a visual and understandable view of the progress of the team.
  • The team will write good user stories and scenario’s since they are an important part of the development process.

And in a way it makes user stories and scenario’s “fun” for developers and product owners.

Read more about this topic:

Installation instructions:

Add git repositories to the “deps” file and update vendors afterwards with “bin/vendors install”. If your project is based on a Standard Edition that includes vendors you cloud consider cloning the repositories manually into the target directories.

In the “deps” file add:

...

[BehatBundle]
  git=https://github.com/Behat/BehatBundle.git
  target=/bundles/Behat/BehatBundle

[MinkBundle]
  git=https://github.com/Behat/MinkBundle.git
  target=/bundles/Behat/MinkBundle

You also need to install pear and the required packages.
IMPORTANT: sometimes this process hangs for several minutes, be prepared to have a lot of patience!

$ sudo apt-get install php-pear php5-curl php5-dev
$ sudo pear upgrade pear
$ sudo pear channel-discover components.ez.no
$ sudo pear channel-discover pear.symfony-project.com
$ sudo pear channel-discover pear.phpunit.de
$ sudo pear install --alldeps phpunit/PHPUnit
$ sudo pear channel-discover pear.symfony.com
$ sudo pear channel-discover pear.behat.org
$ sudo pear install --alldeps behat/behat
$ sudo pear install --alldeps behat/mink

In app/AppKernel.php:

public function registerBundles()
{
  ...
  if ($this->getEnvironment() === 'test') {
    $bundles[] = new Behat\BehatBundle\BehatBundle();
    $bundles[] = new Behat\MinkBundle\MinkBundle();
    if (!defined('BEHAT_AUTOLOAD_SF2')) define('BEHAT_AUTOLOAD_SF2', false);
    require_once 'behat/autoload.php';
    require_once 'gherkin/autoload.php';
    require_once 'mink/autoload.php';
  }
  return $bundles;
}

In app/autoload.php:

$loader->registerNamespaces(array(
  ...
  'Behat\\BehatBundle'=>__DIR__.'/../vendor/bundles',
  'Behat\\MinkBundle'=> __DIR__.'/../vendor/bundles',
));
Share