SOAP Builders Round III – Web Services Interoperability Testing

Overview

The goal of these tests is to demonstrate WSDL interoperability between SOAP-based web service toolkits. WSDL is often described as equivalent to the Interface Definition Languages (IDLs) used by RPC, CORBA and COM. To define what WSDL interoperability means consider the following scenario:

  1. Company X creates a service using tool A and exposes wsdl A.wsdl generated with the tool A
  2. Customers that use tools B and C create client code using tools B and C
  3. Companies Y and Z that use tools B and C respectively want to create equivalent services that would be used by customers with the same client code as in 2. So they have to create services with equivalent wsdls.
    Definition: Two wsdls are equivalent if their corresponding SOAP messages on the wire look the same.

To insure that this scenario would work, we need to make sure that tools B and C:

Since company X could take any of the tools A, B and C to create original wsdl, we have N^3 test matrix to ensure full interoperability.

Earlier rounds of interop testing on the SOAPBuilders list were targeting steps 1 and 2 of the scenario above. They have shown that clients can consume WSDL from many different servers, but no tests have been done to demonstrate that a wide range of web service client and server tools can process the equivalent WSDL definitions (step 3 in scenario above).

Looking from another perspective, the classic RPC development model is IDL-centric; the expectation is that developers will start with interface definitions and use them to build both clients and servers. This model allows industry groups to define standard problem-domain specific interface suites without having to implement either clients or servers and it needs to be supported in the web service world.

The primary purpose of these tests is to demonstrate support for all steps 1,2,3 and therefore support for classic style of development. This testing is necessary if WSDL is to prove successful as a standard for describing web service interfaces. Of course we are not able to cover all N^3 tests, so we decided to split the tests into 2 Groups: Group D and Group E.

Group D targets the classic classic RPC development model, that is, tests are defined by a given wsdl definition that is used to generate both servers and clients. This way we have partial coverage of steps 2-3, taking predefined wsdl instead of that produced by the tool A.

Group E targets steps 1 and 2 to ensure coverage of mainstream WSDL features. These tests rely on a server-centric development model, i.e., ensuring that clients can consume WSDL generated by servers and matching certain criteria (for instance, using a document/literal binding).

The difference in the development models for the two Groups is shown in the figure below.

We also introduce Group F for WSDL extensibility features and WSDL support for soap headers. In this Group we will verify that clients can consume predefined wsdls with extensibility elements and SOAP header declarations without failure. No functional support for headers and extensibility declarations will be checked.

WSDL Interop face-to-face Agenda

The Groups D, E and F are grouped in priority order. Our goal is to complete as much of the Groups as possible in the time allotted with Group D being the core testing that we would like to accomplish at a minimum.

Group D

The Group D tests are defined in terms of a set of WSDL documents. In order to pass these tests toolkit should be able to not only generate clients for predefined WSDL service descriptions, but also servers that implement them. This is a marked difference from the original SOAPBuilders tests in which many endpoints redefined the binding of the test interfaces (e.g., different SOAPAction headers). The toolkit should be able to generate clients and servers using the normal techniques developers are expected to use.

The procedure

Here are the steps for each test and a corresponding diagram that illustrates the code being generated and the calls being made.

  1. Start with predefined WSDL
  2. Generate client from predefined WSDL
  3. Test generated client against pre-built server
  4. Generate server from predefined WSDL
  5. Test generated client against generated server
  6. Generate second client from generated server's WSDL
  7. (some clients can do this dynamically)
  8. Test second generated client against generated server
  9. Test second generated client against pre-built server

    Cross-check with other vendors' generated clients and servers

    The tests

    Test

    Description

    Binding Style/Use

    EmptySA

    echoString operation with empty ("") soapAction

    Rpc/encoded

    Import1

    echoString operation, WSDL imports portType definition from imported\import1B.wsdl (different targetNamespace)

    Rpc/encoded

    Import2

    echoStruct operation, WSDL imports portType definition from imported\import2B.wsdl that in its turn imports import2B.xsd with schema definitions

    Rpc/encoded

    AbsoluteImport2

    Version of Import2 test that imports files based on absolute URLs (Cannot be generated until tests are deployed)

    Rpc/encoded

    Import3

    Uses messages from import2.wsdl, reimplements method

    echoStruct and adds method echoStructArray

    Rpc/encoded

    Compound1

    Use of attributes in SOAP payload, including

    attribute on element of simpleType

    Document/literal

    Compound2

    Two schema sections, types in 1st schema references types in the 2nd schema

    Document/literal

    InteropTestDocLitParameters

    Reduced version of SOAPBuilders Interop test wsdl with “parameters” way of describing rpc requests in Document/Literal (Document/Literal – Wrapped). Version has operations echoString, echoArrayOfString and echoStruct

    Document/literal

    InteropTestDocLit

    Reduced version of SOAPBuilders InteropTest test, document/literal mode. Version has operations echoString, echoArrayOfString and echoStruct

    Document/literal

    InteropTestRpcEncoded

    Reduced version of SOAPBuilders InteropTest test, rpc/encoded mode. Version has operations echoString, echoArrayOfString and echoStruct

    Rpc/encoded

     

    Group E

    The Group E tests are defined in terms of a set of WSDL documents. In order to pass these tests toolkit should be able to expose services with the same operation names, parameters (same type and order) and binding style/use as the predefined WSDL service descriptions. All other details of the WSDL definitions may vary.

    The procedure

    Here are the steps for each test and a corresponding diagram that illustrates the code being generated and the calls being made.

     

    1. Start with predefined pseudo-code service description

    2. Implement a server by hand with correct operations, parameters and specified binding style/use

    3. Generate WSDL describing server

    4. Generate client from implemented server’s generated WSDL

    5. Test generated client against implemented server

    6. Cross-check by generating clients for other vendors' implemented servers

    The tests

    Test

    Description

    Binding Style/Use

    InteropTestDocLit

    Reduced version of SOAPBuilders InteropTest test, document/literal mode. Version has operations echoString, echoArrayOfString and echoStruct

    Document/literal

    InteropTestRpcEncoded

    Reduced version of SOAPBuilders InteropTest test, document/literal mode. Version has operations echoString, echoArrayOfString and echoStruct

    Rpc/Encoded

    InteropTestList

    list echoLinkedList (list a);

    struct list {

    int varInt;

    string varString;

    list child; //nillable

    }

     

    Rpc/Encoded

    Group F

    The Group F tests are defined in terms of WSDL documents. In order to pass these tests toolkit should be able to generate and execute client code based on given wsdl without error. Resulted message should be consumable by rpc/encoded echoString service.

    The procedure

     

    Here are the steps for each test and a corresponding diagram that illustrates the code being generated and the calls being made.

    1. Generate and execute client code based on given wsdl.
    2. Execute client against every service for Group D InteropTestDocLit test.

    The tests

    Test

    Description

    Binding Style/Use

    InteropTestHeaders

    Modified version of SOAPBuilders InteropTest test, document/literal mode Version has one operation echoString with 2 headers defined.

    Document/literal

    InteropTestExtensibility

    Modified version of SOAPBuilders InteropTest test, document/literal mode Version has one operation echoString with extensibility elements everywhere they are allowed.

    Document/Literal

    InteropTestExtensibilityRequired

    Same as InteropTestExtensibility, but with wsdl:required attribute on the ext:binding element. According to the spec, Toolkit should reject such wsdl.

    Document/Literal

     

     

    Test Results

     

    The test results should be reported using the standard format shown below (example for Group D).

     

    Toolkit A

    1st generated client, pre-built server (Steps 1, 2, 3)

    1st generated client, generated server (Steps 4, 5)

    2nd generated client, generated server (Steps 6, 7)

    2nd generated client, pre-built server (Step 8)

    EmptySA.wsdl

    Pass

    Pass

    Pass

    Pass

    Import1.wsdl

    Pass

    Pass

    Pass

    Pass

    Import2.wsdl

    Pass

    Fail

    Fail

    Fail

    Import3.wsdl

    Pass

    Not Supported

    Not Supported

    Not Supported

    :

    :

    :

    :

    :

    There are four results for each WSDL file, each corresponding to a set of steps in the testing procedure. Each toolkit can pass, fail or not support each set of steps for a given WSDL file. A toolkit passes the test if everything works correctly. A toolkit fails a test if it is designed to support the test scenario, e.g., it supports document/literal bindings, but it does not work correctly. A toolkit doesn’t support a test if it is not designed to support the test scenario, e.g., it does not support document/literal bindings.