A PHP SOAP client for the command line

Here at LeaseWeb, we work a lot with SOAP web-services to integrate our internal applications with each other. Especially during development and testing of our applications as we need the ability to practice with SOAP API’s.

If you are into fancy GUI applications then maybe SoapUI is something you could use to familiarise yourself with the API you are integrating with.

If you like simplicity and the command line, or dislike heavy GUI applications you can give php soap client a try.

php soap client is a command line application written in PHP and packed using the popular .phar format. Once installed you can start exploring soap web-services.

There are two ways of installing the client. To get the latest stable version download soap_client.phar from here or do:

$ curl -sS http://leaseweb.github.io/php-soap-client/installer | php

This will download the phar file to the current working directory and make it executable so you can use start using it right away by invoking:

$ ./soap_client

To install the latest master version you can get the source code directly from GitHub, package your own .phar file and install it — using GNU Make.
In order to be able to create the .phar file you need to have composer installed. To read more about composer refer to their excellent documentation here.

# Install php soap client
$ git clone https://github.com/LeaseWeb/php-soap-client.git
$ cd php-soap-client
$ composer.phar install
$ make
$ sudo make install

If you are getting a Failed to compile phar exception while running make you need to set phar.readonly = Off in your php.ini. On a development machine this is fine to do but please be ware of the security risks when setting phar.readonly to Off.

The above make install command will install the soap_client application to /usr/local/bin and make it executable so you can easily call it like this:

$ soap_client
php-soap-client version 2.1.3

Usage:
  [options] command [arguments]

Options:
  ...

Available commands:
  call           Call the remote service with the `method` specified and output the reponse to stdout.
  help           Displays help for a command
  list           Lists commands
  list-methods   Get a list of available methods to call on the remote.
  request        Generate an xml formatted SOAP request for the given method and output to stdout.
  wsdl           Get the WSDL of a soap service.

From this point onwards we assume you have installed the soap_client.phar on your system in /usr/local/bin/soap_client and that the directory /urs/local/bin is in your $PATH.

Lets say we would like to see what methods are available on the remote service http://www.webservicex.net/ConvertTemperature.asmx. We could issue the following command:

$ soap_client --endpoint='http://www.webservicex.net/ConvertTemperature.asmx?WSDL' list-methods

Which will output the following:

ConvertTemp

If you run the above command with the -vvv option you will get more verbose output.
In this case the only available method is ConvertTemp. Let’s see how a SOAP XML request looks like for this method:

$ soap_client --endpoint='http://www.webservicex.net/ConvertTemperature.asmx?WSDL' request ConvertTemp

<SOAP-ENV:Envelope xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/" xmlns:ns1="http://www.webserviceX.NET/">
  <SOAP-ENV:Body>
    <ns1:ConvertTemp>
      <ns1:Temperature>0</ns1:Temperature>
      <ns1:FromUnit></ns1:FromUnit>
      <ns1:ToUnit></ns1:ToUnit>
    </ns1:ConvertTemp>
  </SOAP-ENV:Body>
</SOAP-ENV:Envelope>

If you want to make a SOAP request to the ConvertTemp method on the remote service use the call sub command:

$ soap_client --endpoint='http://www.webservicex.net/ConvertTemperature.asmx?WSDL' call --editor ConvertTemp

Notice the --editor option after the call sub command. If you use the --editor flag soap_client will open up the editor specified in your environment variable $EDITOR so you are able to modify the request XML before sending it.

If you issue the same request multiple times, you could save a soap request as a local XML file and pass it to /dev/stdin of the soap_client call command:

# Get the request xml and store it locally
$ soap_client --endpoint='http://www.webservicex.net/ConvertTemperature.asmx?WSDL' request ConvertTemp &gt; my_sample_request.xml

# Now edit my_sample_request.xml

# Now you can call the ConvertTemp method with this pre-prepared request
$ soap_client --endpoint='http://www.webservicex.net/ConvertTemperature.asmx?WSDL' call ConvertTemp < my_sample_request.xml

Since you will be repeating soap_client commands frequently in a short time while exploring a remote web service you can save yourself some time by setting an environment variable SOAPCLIENT_ENDPOINT that contains the URL to the WSDL. When this environment variable is set you can omit the --endpoint command line option. Let’s do this now and call the ConvertTemp method:

$ export SOAPCLIENT_ENDPOINT='http://www.webservicex.net/ConvertTemperature.asmx?WSDL'
$ soap_client call ConvertTemp &lt; my_sample_request.xml

I wanted to know how much 107.6 degrees Fahrenheit is in Celsius, so my my_sample_request.xml contains:

$ cat my_sample_request.xml

<SOAP-ENV:Envelope xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/" xmlns:ns1="http://www.webserviceX.NET/">
  <SOAP-ENV:Body>
    <ns1:ConvertTemp>
      <ns1:Temperature>107.6</ns1:Temperature>
      <ns1:FromUnit>degreeFahrenheit</ns1:FromUnit>
      <ns1:ToUnit>degreeCelsius</ns1:ToUnit>
    </ns1:ConvertTemp>
  </SOAP-ENV:Body>
</SOAP-ENV:Envelope>

And the result:

$ soap_client call ConvertTemp < my_sample_request.xml

stdClass Object
(
    [ConvertTempResult] => 42
)

The answer is 42.

If you rather see the responses in XML format you can use the --xml command line option:

$ soap_client call --xml ConvertTemp < my_sample_request.xml

<?xml version="1.0" encoding="utf-8"?>
<soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
  <soap:Body>
    <ConvertTempResponse xmlns="http://www.webserviceX.NET/">
      <ConvertTempResult>42</ConvertTempResult>
    </ConvertTempResponse>
  </soap:Body>
</soap:Envelope>

This tutorial should give you enough information to get started with exploring, testing and/or developing SOAP API’s.
In a future blog post, I will continue the topic of the php soap client. We are currently working on packing the .phar archive for the web.

php-soap-client for the web
php soap client for the web

So you would be able to drop the soap_client.phar somewhere in your apache DocumentRoot and explore SOAP services from your browser.

Share

Automated tests for SOAP API in Java

Don’t think the story will be short, but it’s definitely interesting. So, I got a new project where I need to cover a SOAP API with automated tests written on Java. As a starting point I have a URL to a WSDL file to work with. The “tricky” part is in the environment configuration. Our test environment is hidden behind a SOCKS proxy server. So, if you want to work with a test environment, you need to make your requests through that proxy, otherwise you’ll “talk” to another environment that we don’t need right now. Not a usual configuration, but it had it’s own reasons for doing so. My first problem was to decide how to work with WSDL and in what direction my future framework would go. After some Googling and trials, I stopped on the JAX-WS library. This is handy because with the “maven wsimport” task, you can easily generate all needed classes and objects where your stubs will be held. Here’s how it looks like in pom.xml:

                <plugin>
                    <groupid>org.jvnet.jax-ws-commons</groupid>
                    <artifactid>jaxws-maven-plugin</artifactid>
                    <version>${jaxws.plugin.version}</version>
                    <executions>
                        <execution>
                            <id>{put_some_id}</id>
                            <phase>validate</phase>
                            <goals>
                                <goal>wsimport</goal>
                            </goals>
                            <configuration>
                                <!--<wsdldirectory>${project.basedir}\src\wsdl</wsdlDirectory>-->
                                <wsdllocation>{your_wsdl_location}</wsdllocation>
                                <destdir>${project.basedir}\src\main\java\Stubs\oop</destdir>
                                <wsdlfiles>
                                    <wsdlfile>${project.basedir}\src\main\resources\wsdl\myWSDL.wsdl</wsdlfile>
                                </wsdlfiles>
                                <packagename>Stubs.oop</packagename>
                                <!-- Without this, multiple WSDLs won't be processed -->
                                <stalefile>${project.build.directory}/jaxws/stale/wsdl.done</stalefile>
                            </configuration>
                        </execution>
                        <execution>
                            <id>{put_some_id}</id>
                            <phase>validate</phase>
                            <goals>
                                <goal>wsimport</goal>
                            </goals>
                            <configuration>
                                <!--<wsdldirectory>${project.basedir}\src\wsdl</wsdlDirectory>-->
                                <wsdllocation>{your_wsdl_location}</wsdllocation>
                                <destdir>${project.basedir}\src\main\java\Stubs\wsdl2</destdir>
                                <wsdlfiles>
                                    <wsdlfile>${project.basedir}\src\main\resources\wsdl\your_wsld2.wsdl</wsdlfile>
                                </wsdlfiles>
                                <packagename>your_package_name</packagename>
                                <!-- Without this, multiple WSDLs won't be processed -->
                                <stalefile>${project.build.directory}/jaxws/stale/wsdl.done</stalefile>
                            </configuration>
                        </execution>
                    </executions>
                </plugin>

It was a bit hard for me to get the WSDL file from the URL, so I decided to save it locally and read it from there then. Since changes on the API are not frequent, I can easily accept this. This configuration allows to add as many WSDL sources as I want and to store stubs for each of them in project. First problem was solved. Now, I had to figure out how to make my framework to use proxy for it’s calls. My first trial was basic and most common in the Java world:

System.getProperties().put("proxySet", "true");
System.getProperties().put("proxyHost", getProxyHost());
System.getProperties().put("proxyPort", getProxyPort());

But I kept getting errors and no successful connection. My next guess was that problem could be in the SSL, since we are using HTTPS. So, I started the browser, imported the security certificates (they were self-signed since it’s a test environment), added them to java “cacerts” keystore, and added next to my code:

System.setProperty("javax.net.ssl.keyStore", "keystore.jks");
System.setProperty("javax.net.ssl.trustStrore", "cacerts.jks");
System.setProperty("javax.net.ssl.keyStorePassword", "changeit");

I tried to debug and to make sure that the JVM is using my proxy settings and it was true – everything was there. What would be the next step in debugging for me? I decided to install a local proxy and try to put my requests through it, so I could get more details. I installed Charles – a fantastic proxy server that helped me a lot in the past with REST API automation. It has a life changing ( for me in this case) “External Proxy” option . I set it to proxy all requests to our SOCKS proxy and I put my Java API calls through Charles as well. And it worked! You can’t even believe how happy I was because of it. But now I got another problem – Charles cost $50 for one license. So, there were two questions that I needed to solve:

  • Why my Java calls works fine with Charles and without it they fail?
  • Where to find free and easy to setup HTTP proxy?

With help from one of our developers we could find out why Java kept getting authentication errors and didn’t want to connect to test environment properly. The problem was that Java was connecting to the SOCKS proxy properly, but it was resolving remote DNS to another environment to which I didn’t have proper certificates and credentials for authorization! I found only one article on StackOverflow regarding this issue and from that point I new that I need to have local HTTP proxy that is:

  • free
  • can forward my requests to SOCKS proxy

I’ve spent whole day trying to find an easy-to-use and adequate proxy server for Windows ( it’s my Workstation that I need to use), I looked over 15 different apps and failed. In the end, I installed VirtualBox with Ubuntu and install there Squid3 proxy server. It’s the most popular and common proxy server for Unix, from what I saw on Google. But this crazy application has a configuration file with more then 3000 lines! It’s not that easy to make it run as I want, it’s not even that easy to restart it. So, after couple hours I gave up on it and started looking for more solutions. Luckily, I found Polipo – tiny, easy to install and setup proxy server. It only has a few main options that I had to setup and make everything work as a charm:

  • proxyAddress – set it to IP address of your local virtual box, or your local machine, if you use Linux or Mac
  • allowedClients – list of IP addresses from which Polipo will allow to access it and forward requests towards another proxy or web directly
  • socksParentProxy = “host name and port of your proxy to which I needed to forward my requests”
  • socksProxyType = socks5

Save changes and restart – that’s it! After that I pointed my Java framework to the local proxy and got green tests! To set the proxy for my tests I used custom MyProxySelector class:

                      package Base;

                      import java.net.*;
                      import java.util.ArrayList;
                      import java.util.HashMap;
                      import java.io.IOException;

                      public  class MyProxySelector extends ProxySelector {
                          // Keep a reference on the previous default
                          public ProxySelector defsel = null;

                          /**
                           * Inner class representing a Proxy and a few extra data
                           */
                          class InnerProxy {
                              Proxy proxy;
                              SocketAddress addr;
                              // How many times did we fail to reach this proxy?
                              int failedCount = 0;

                              InnerProxy(InetSocketAddress a) {
                                  addr = a;
                                  proxy = new Proxy(Proxy.Type.HTTP, a);
                              }

                              SocketAddress address() {
                                  return addr;
                              }

                              Proxy toProxy() {
                                  return proxy;
                              }

                              int failed() {
                                  return ++failedCount;
                              }
                          }

                          /**
                           * A list of proxies, indexed by their address.
                           */
                          HashMap<SocketAddress, InnerProxy> proxies = new HashMap<SocketAddress, InnerProxy>();

                          public MyProxySelector(ProxySelector def, String host, int port) {
                              // Save the previous default
                              defsel = def;

                              // Populate the HashMap (List of proxies)
                              InnerProxy I = new InnerProxy(new InetSocketAddress(host, port));
                              proxies.put(i.address(), i);
                          }

                          /**
                           * This is the method that the handlers will call.
                           * Returns a List of proxy.
                           */
                          public java.util.List select(URI uri) {
                              // Let's stick to the specs.
                              if (uri == null) {
                                  throw new IllegalArgumentException("URI can't be null.");
                              }
                              /**
                               * If it's a http (or https) URL, then we use our own
                               * list.
                               */
                              String protocol = uri.getScheme();
                              if ("http".equalsIgnoreCase(protocol) ||
                                      "https".equalsIgnoreCase(protocol)) {
                                  ArrayList l = new ArrayList();
                                  for (InnerProxy p: proxies.values()) {
                                      l.add(p.toProxy());
                                  }
                                  return l;
                              }

                              /**
                               * Not HTTP or HTTPS (could be SOCKS or FTP)
                               * defer to the default selector.
                               */
                              if (defsel != null) {
                                  return defsel.select(uri);
                              } else {
                                  ArrayList l = new ArrayList();
                                  l.add(Proxy.NO_PROXY);
                                  return l;
                              }
                          }

                          /**
                           * Method called by the handlers when it failed to connect
                           * to one of the proxies returned by select().
                           */
                          public void connectFailed(URI uri, SocketAddress sa, IOException ioe) {
                              // Let's stick to the specs again.
                              if (uri == null || sa == null || ioe == null) {
                                  throw new IllegalArgumentException("Arguments can't be null.");
                              }

                              /**
                               * Let's lookup for the proxy
                               */
                              InnerProxy p = proxies.get(sa);
                              if (p != null) {
                                      /**
                                       * It's one of ours, if it failed more than 3 times
                                       * let's remove it from the list.
                                       */
                                  if (p.failed() >= 3)
                                      proxies.remove(sa);
                              } else {
                                      /**
                                       * Not one of ours, let's delegate to the default.
                                       */
                                  if (defsel != null)
                                      defsel.connectFailed(uri, sa, ioe);
                              }
                          }
                      }

And to turn the proxy on and off I wrote next switch methods:

            private ProxySelector defaultProxy = ProxySelector.getDefault();

            public void setLocalProxy(){
                MyProxySelector ps = new MyProxySelector(ProxySelector.getDefault(),
                                                         localProxyName,localProxyPort);
                ProxySelector.setDefault(ps);
            }

            public void disableProxy(){
                ProxySelector.setDefault(defaultProxy);
            }

That’s it. Now I can run my tests easily with control over when to the use the proxy and when not to. In the future I’ll move my tests to the CI server (Jenkins most probably) and will setup Polipo HTTP proxy on that environment in two minutes. It’s always nice to solve such non-ordinary problems. Most probably my solution is not very elegant and “right”, but it works right now and for this moment this is all that matters to me, since I can start writing automated tests rather then fighting with configuration issues.

Share

Symfony2 SOAP support

Where RESTful API’s are the standard in modern day web applications, SOAP (Simple Object Access Protocol) used to be a very popular protocol for RPC calls, and in some circles it might still be popular: SOAP is the form of RPC that Microsoft supports (best) in their .net framework. It consists of a WSDL file and a HTTP POST request with a “SOAPAction” header set, when invoking a remote procedure. It can be found in “$_SERVER[‘HTTP_SOAPACTION’]” in vanilla PHP and in “$this->getRequest()->server->get(‘HTTP_SOAPACTION’)” in a Symfony2 controller.

PHP had no support for SOAP for a long time and since this was not an easy protocol to implement, this effectively meant you couldn’t use SOAP in PHP based applications. Symfony1 used to have SOAP support and in Symfony2 SOAP support can be added by installing a bundle. This bundle is called “BeSimpleSoapBundle” and is based on some BeSimple and the Zend Framework SOAP classes. It has a really nice WSDL generator and it works very good (we tried it). Installing the BeSimpleSoapBundle is not hard (and is well documented), but the project hasn’t been updated in the last two years. So if you want to play it safe you might want to refrain from using it.

You may want to choose the new (PHP >5.2) SOAP Server implementation as suggested by the official Symfony2 documentation. It consists (like BeSimpleSoapBundle) of two parts: a client and a server. The server takes care of decoding the WSDL file, parsing this incoming XML and calling a method in a said object. The client takes care of parsing the WSDL file and being able to call the methods with the right parameters. The only thing you are missing when not using BeSimpleSoapBundle is a WSDL generator. Fortunately there are many WSDL generators online.

But the problem persists that every time you run your application on a different machine you have to regenerate the WSDL file. Changing this hostname when moving from one development VM to another was to cumbersome for us so we came up with another solution: let PHP (the Symfony2 project) generate the WSDL file and fill in the “base” URL wherever nessecary. We altered the book example by adding a conditonal statement, so that if the SOAPAction header is not present it just serves the WSDL file. In the controller we also added a client implementation for debugging purposes.

We used the WSDL generator by nbsoftware to generate the WSDL file and used “webservice service(“http://localhost/soap/server”)” as the webservice definition. The resulting WDSL file was put into the “Resources/views” directory of the Bundle as a twig template called “wsdl.xml.twig” and we replaced “###SERVER_ADDRESS###service” with “http://localhost/soap/server” and all the occurences of with  “http://localhost” with “{{ base_uri }}”. We serve the file conditionally by using the “$this->render()” call with the template identifier and the “base_uri” as arguments.

When you want to call another controller from that controller you can instantiate the controller by using the “new” keyword and after that call the “setContainer” method with “$this->container” as argument. After that you can set the newly created controller as the object that handles the SOAP calls. If you create a public method with a name that is also present in the WSDL file you can call that method using the SOAP client.

This method makes offering SOAP functionality really easy and with a little effort you can offer your RESTful calls via SOAP to .net applications as well. Below you find the full controller code that we use:


    class SoapController extends Controller
    {

        /**
         * Function to retrieve current applications base URI for WSDL
         */
        private function getBaseUri()
        {    // get the router context to retrieve URI information
            $context = $this->get('router')->getContext();
            // return scheme, host and base URL
            return $context->getScheme().'://'.$context->getHost().$context->getBaseUrl();
        }

        /**
         * Serves WSDL on GET and handles SOAP calls when given
         */
        public function serverAction()
        {
        // init data format
        $this->getRequest()->attributes->set('_format', 'xml');
            // retrieve base URI for WSDL file location
        $base_uri = $this->getBaseUri();
        $request = $this->getRequest();
        // if this is not a SOAP request, serve the WSDL file
        if (!$request->server->has('HTTP_SOAPACTION'))
        {
            return $this->render('LswSoapBundle:Soap:wsdl.xml.twig', compact('base_uri'));
        }
        // load the mapping from the config file
        $mapping = $this->container->getParameter('soap.mapping');
        // get rid of the extra quotes around the soap action name
        $soapAction = array_pop(explode('/',trim($request->server->get('HTTP_SOAPACTION'),'"')));
        // intiantiate the (controller) class that is mapped
        $object = new $mapping[$soapAction]();
        // make sure the controller has access to the container
        $object->setContainer($this->container);

        // initialize the SOAP server
        $server = new \SoapServer($base_uri.'/soap/server.xml');
        // point to the controller that has a method with the same name as the soap action
        $server->setObject($object);

        try
        {   // try to execute the method
            $data = $server->handle();
        }
        catch (\Exception $e)
        {   // catch any exception and make a proper SOAP error response
            $code = $e->getCode();
            $message = $e->getMessage();
            return $this->render('LswSoapBundle:Soap:error.xml.twig', compact('code', 'message'));
        }
        // on success render the response
        return compact('data');
        }

        /**
         * Client for WSDL calls, for debugging only
         */
        public function clientAction()
        {
            $base_uri = $this->getBaseUri();
            try {
                    $client = new \SoapClient($base_uri.'/soap/server.xml', array('trace' => 1));
                    $a = array('message'=>'hello world!');
                    $response = $client->__soapCall('helloWorld', array($a));
                    var_dump($response);
                    die();

                } catch (\SoapFault $e) {
                    var_dump($e->getMessage(), $client->__getLastResponse()); die();
            }
        }

    }

NB: One of the ceveats is that the wsdl file will be cached (by PHP) and that you may want to run “rm /tmp/wsdl-*” before debugging a nasty problem. This could save you a lot of time.

Share