Getting Testy with SOA 11g – Part 2: The Mock Web Service Pattern

Despite what the title implies, I’m not really cranky with SOA 11g. I promise. I certainly wouldn’t have spent the last few years working with it if I wasn’t constantly impressed by what this vast framework can do. The best thing about having had a chance now to go through a number of significant SOA 11g projects has been discovering new and even better patterns to make development easier and increase the overall quality of our finished product.

This series is all about discussing some of those patterns – specifically around the best way I’ve found so far to set up an automated and highly repeatable testing framework for SOA 11g service projects.

Why would you worry about this, you might ask? Well, unless you want to unleash mostly untested code on your hapless production users only to suffer thru a sea of frantic help tickets, it might be worth investing up front in automated testing. This is the only way to be sure your code is as bullet-proof as possible before you release the hounds and deploy your code to your production environment.

Part 1 of this series covered the Integration Service Pattern, which is a best practice for setting up your SOA projects in a highly modular, highly reusable way.

Benefits of the Mock Web Service Pattern

Here in Part 2 of this series, we’re going to take advantage of those structures we put in place in Part 1 to create mock web services and then to set up SOAP UI test suites to consume and exercise them as rigorously as possible. I’ve tried a number of different approaches when trying to set up automated tests for SOA 11g and this pattern has easily been the most successful. You’ll discover a number of advantages:

Easy to set up
Easy to set up test requests and mocked responses
Easy to create exception handling scenarios
Easy to test for schema compliance
Easy to exclude variable portions of your response xml in your test (like the current timestamp)

At this point you’re probably sensing a theme beginning to emerge. This pattern is easy. Once you understand how it works, it’s quick to set up and simple to maintain.

So, if you’re brave enough, let’s take the big plunge and get started.

Setting up a Sample Project

If you’d like to see a simple project that demonstrates specifically how this works in SOA 11g version 11.1.1.6, download GettingTestyPart2.zip. For this demo, I’m running Oracle’s Pre-built Virtual Machine for SOA Suite and BPM Suite 11g.

Within the GettingTestyPart2.zip file, you should see 3 main directories:

AvioOrderEntry – Contains the SOA demo application we’ll be using
DataModel – Contains sql scripts to set up your database for the demo
SOAPUIProject – Contains the SOAP UI project used in this demo

If you already went thru Part 1 of this series, you can skip the database setup. However, for those of you who skipped Part 1 here are the instructions to set up the database. Inside the DataModel directory, you’ll find the following 2 files:

1-MasterDataLoader.sql
2-CreateUsersAndTables.sql

Log into your local database as the SYSTEM user and run 1-MasterDataLoader.sql. This will set up a new user: ORDER_TRACKING_DB as well as a table we’ll be using for this demo.

Next, you’ll need to set up a WebLogic datasource and data connection pool for both users. If you’re a little fuzzy on how this works, check out a blog article I wrote on this topic: Oracle SOA Database Adapter 101 – WebLogic Configuration Steps.

Here is the setup information for users:

Username: ORDER_TRACKING_DB
Password: ORDER_TRACKING_DB
Hostname: soabpm-vm
Port: 1521
SID: XE
Data Source JNDI: jdbc/ORDER_TRACKING_DB
Connection Pool JNDI: eis/DB/OrderTrackingDB

Beginning the Mock Web Service Project

Create a new SOA project called AOEIntegrationMockWS
Copy the AOEIntegrationwsdlconsumedCustomerService directory to AOEIntegrationMockWSwsdlCustomerService. The idea here is that we’re going to create a mocked version of the Integration project’s consumed service. Because we did such a good job structuring our wsdl and xsd data in the Integration project, we can simply copy that into our new MockWS project without making any further changes

In the AOEIntegrationMockWS composite, let’s create a BPEL process that implements this WSDL. In order to minimize rework, be sure to pay specific attention to how this is constructed:
1. BPEL 2.0
2. Namespace: Name it after the operation you’re targeting: “UpdateCustomerRecord”. You’ll have one BPEL process per operation.
3. Template: Base on a WSDL
4. Service Name: Name your service just like the Consumed Service for clarity. In this case, name it “CustomerService”
5. WSDL URL: This is the wsdl representing the mocked version of your service: wsdl/CustomerService/CustomerService.wsdl

Setting up File Adapters for Response XMLs

Create a new File Adapter External Reference called: “UpdateCustomerRecordRequestFiles”
Click Next. Select the radio option for “Define from operation and schema (specified later)”

Click Next. Select the radio option for “Synchronous Read File”

Click Next. Select the radio option for “Logical Name” to specify your directory name. Set the logical name in a standardized way as “UPDATE_CUSTOMER_RECORD_MOCK_WS_RESPONSE”

Click Next. Set the file name to “test.xml.” This serves as a default file name for the adapter. Since we’re going to always set the file name for our xml file within the UpdateCustomerRecord BPEL process, this setting essentially is irrelevant and only serves to make the adapter happy.

Click Next. When choosing the URL for your schema element, you want to set this to your service’s response schema. The idea here is that the UpdateCustomerRecord BPEL process will determine which xml response to find based upon a unique value within the request that’s passed into the process. It’s tempting to just look up the wsdl and then grab the response schema from that, however, I’ve found this to be problematic. The better way that always works is to extract the response schema from the xsd itself as shown below.

Click OK, Next and Finish. Then be sure to “Save All”
Be sure to go into the jca file to switch the DeleteFile property to false. This is critical because otherwise your test xmls will get deleted every time you use them which will quickly become a problem when you’re trying to re-run SOAP UI test suites. In our example, this file can be found at: AOEIntegrationMockWSUpdateCustomerRecordRequestFiles_file.jca.jca

Before doing the next step be sure that your BPEL process is closed or else you will encounter problems.
Drag a new reference from the UpdateCustomerRecord BPEL to the UpdateCustomerRecordRequestFiles File Adapter and “Save All”

Setting up File Adapters for Fault XMLs

At this point we’ve got a basic structure in place that will eventually be able to retrieve normal response xml files. The only problem with what we have here is … in a word … problems. So far, we have no way to deal with faults. Since these utilize a different schema, we will need to implement a separate file adapter to retrieve fault xml files. The steps are similar to what was shown above. Here is a quick synopsis:
1. Create a new File Adapter External Reference called: “UpdateCustomerRecordFaultFiles”
2. Click Next. Select the radio option for “Define from operation and schema (specified later)”
3. Click Next. Select the radio option for “Synchronous Read File”
4. Click Next. Select the radio option for “Logical Name” to specify your directory name. Set the logical name in a standardized way as “UPDATE_CUSTOMER_RECORD_MOCK_WS_FAULT”
5. Click Next. Set the file name to “test.xml”
6. Click Next. Set the URL to “wsdl/CustomerService/xsd/UpdateCustomerRecord.xsd.” As a reminder, search for this based upon the xsd – not the wsdl!!
7. Click OK, Next and Finish. Then be sure to “Save All”
8. Be sure to go into the jca file to switch the DeleteFile property to false. This is critical because otherwise your test xmls will get deleted every time you use them which will quickly become a problem when you’re trying to re-run SOAP UI test suites. In our example, this file can be found at: AOEIntegrationMockWSUpdateCustomerRecordFaultFiles_file.jca
9. Be sure that your BPEL process is closed
10. Drag a new reference from the UpdateCustomerRecord BPEL to the UpdateCustomerRecordFaultFiles File Adapter
11. Be sure to Save All

Creating the DVM for fault XML files

We already know that our BPEL process will utilize a unique value within the request xml to determine which response xml to retrieve. However, how will the process know whether it needs a response xml or a fault xml? The easiest way to do this is by using a Domain Value Map (DVM). For simplicity, create one DVM per BPEL process in your MockWS project. Create a AOEIntegrationMockWSdvm directory
When you create your DVM, name it UpdateCustomerRecord.dvm and be sure to browse for the AOEIntegrationMockWSdvm directory you created in the previous step

Open up the source view of your shiny new DVM so we can make some changes. As the name implies, this is simply a map. The key needs to be a unique value within your request xml. This can be something like an account number, social security number or credit card number. In or case, the closest thing we have is the email address. Simply replace the xml in your DVM source with the following. This will represent each of the emails in your requests that will return fault responses. Be sure to save this when you’re done.

Completing the BPEL process

So far, so easy. Right? Now let’s take a look at our UpdateCustomerRecord BPEL process. The first thing we’ll need is a variable to hold our xml response (or fault) filename as well as what type it is (response or fault). We’ll need to create a small xsd for this that will conceivably be used for every future mocked web service so we should put it in a common xsd folder: AOEIntegrationMockWSxsdcommonDynamicFileName.xsd

Once you create a BPEL variable based on the DynamicFileName element, you will need to go into the CustomerService.wsdl and update the DynamicFileName.xsd schema location. This can easily be fixed by clicking on the following tag in the wsdl and updating the schema location in the Property Inspector window:

Create an Assign Activity called “SetFileNameAndResponseType” where we’ll utilize our DVM to determine the response type based upon the email address and then concatenate the file name from “updateCustomerRecord-” + the first portion of the email address + “.xml”

After the Assign Activity, create an “If” Structured Activity. This will branch depending upon the value in the responseType that you just retrieved from the DVM – either “response” or “fault”
In the “response” branch, create an Invoke activity called “GetMockResponse” to call the UpdateCustomerRecordRequestFiles File Adapter. After designating your input and output variables, be sure to click on the Properties tab to set the jca.file.FileName to the value of the fileName field in your dynamicFileName variable. This, here is the point of the entire exercise because this is where we retrieve the response file specifically matched to the corresponding request file.

Unfortunately, there is one gotcha here. If you eventually deploy the project with the fileName references as a variable, an artificial space is introduced in front of that file name which will corrupt your path, meaning that the File Adapter will not be able to find your xml file. I know. It’s annoying. The easy way around this is to do the following
1. Copy the variable string: $dynamicFileName/ns4:fileName
2. Select the “expression” radio button
3. Paste the string into the “expression” field: $dynamicFileName/ns4:fileName

After the invoke, create an Assign Activity called “SetOutputResponse” to map the output xml from the file adapter to the process’s outputVariable. There is no need for a Transform Activity because this is a very simple mapping where we know that the schemas are identical.

Move the replyOutput right after the “SetOutputResponse” Assign Activity
Now, onto the fault branch. Create an Invoke Activity called “GetFaultResponse.” This will be exactly like Step 22, except we will obviously input and output variables corresponding to the “UpdateCustomerRecordFaultFiles” File Adapter request/response elements.
1. Don’t forget to set the jca.file.FileName to the value of the fileName field in your dynamicFileName variable.
2. Don’t forget to copy the “variable” string to the “expression” field like we did in Step 23.
After the “GetFaultResponse” Invoke Activity, create an Assign Activity called “SetOutputFault.” In this case, map the output of the “UpdateCustomerRecordFaultFiles” File Adapter to the faultResponse variable representing the UpdateCustomerRecordFault message type from CustomerService.wsdl

Use a Web Service Reply immediately after the “SetOutputFault” Assign Activity. This will return the “faultResponse” variable you set in the last step. Be sure to designate the Namespace URI and Local Part as shown below:

As BPEL processes go, this one is pretty painless. Be sure that everything is saved and that your project compiles successfully before moving on

Create XML Fault and Response Files

Create the following new “xml” directory structure within the AOEIntegrationMockWS project. This will be used to hold the fault and response XML files for your tests. An example of this structure can be found below:

Now we need xml response and fault files to fill our new directory structure. Although there are many ways to get this, the best way is to call the actual service if possible. This will ensure that your test data is as close to actual data returned from the service as possible. In our case, be sure to deploy the CustomerService project. Then use SOAP UI to call http://soabpm-vm:7001/soa-infra/services/default/CustomerService/CustomerSvc?WSDL. Don’t be alarmed on the off chance you get a fault from this service. It’s programmed to return a normal response for 30 seconds and then various faults for the next 30 seconds. It was coded purposely like this for Part 3 of this series. The following is a sample of a good response from this simple service:

From this response, copy all of the xml inside of the body tags into a new file called: AOEIntegrationMockWSxmlupdateCustomerRecordresponseupdateCustomerRecord-johnsmith.xml. It’s always good form to include the xml tag at the top of the file as well:

In order to ensure that the mocked responses are coming thru correctly, I added a second file called: updateCustomerRecord-markbrody.xml
In order to generate a fault file, simply create an email address in excess of 30 characters. As an example, I created a record for Herman Munster with the following email address: hermanmunsterisareallyscaryguy@halloween.com. This is clearly going to exceed the database’s 30 character limit for this field, thus generating a fault. For the fault xml file, take all of the XML inside of the detail tag and copy this to a new file called: AOEIntegrationMockWSxmlupdateCustomerRecordfaultupdateCustomerRecord-hermanmunsterisareallyscaryguy.xml. I also created a second fault file called updateCustomerRecord-gomezaddams.xml. Remember, that the unique identifier (in our case email addresses) from the fault xmls need to be entered into our DVM:
1. Raw fault:

updateCustomerRecord-gomezaddams.xml:

Generate MockWS Config Plan

Now that we have our MockWS project ready as well as our XML files, it’s tempting to deploy the project … and find out that it doesn’t work yet. There are a couple of simple reasons for this. First, we need to find a place on the VM to put our XML files. Even though we’re storing them in our project directory, we will need to have them in a specific spot in the Linux file system. If you’ve never set up a file share on the Oracle VM, not to worry. It’s very easy and well documented. Just go to Page 30 of the Oracle SOA Suite/BPM Suite VirtualBox Appliance: Introduction and Readme.
Once you’ve set up your shared folder, copy the AOEIntegrationMockWSxml there. From the VirtualBox GUI, you should be able to see the files in /media/sf_share/xml:

This is the last step before deploying the MockWS project. It’s always a best practice to generate a Config Plan, but this is especially true when using File Adapters. This is the easiest way to designate the logical directory names. Right click the composite.xml and choose “Generate Config Plan”
Within the config plan, you’ll see a section where we can designate the actual directory paths for UPDATE_CUSTOMER_RECORD_MOCK_WS_RESPONSE and UPDATE_CUSTOMER_RECORD_MOCK_WS_FAULT

Deploy and Test the MockWS Project

Deploy the AOEIntegrationMockWS project utilizing the config file we just created
Test out your MockWS project in SOAP UI: http://soabpm-vm:7001/soa-infra/services/default/AOEIntegrationMockWS/CustomerService_ep?WSDL
At this point, you should successfully get the correct response xmls from John Smith and Mark Brody and the correct fault xmls from Herman Munster and Gomez Addams. If you don’t, go back thru the instructions to be sure you didn’t miss something. Even though these steps are easy, there are unfortunately a few gotchas that could cause problems if they are overlooked.

Configuring the Integration Project to Call the Mock Web Service

Now that we have a fully functional mock web service, we need to configure our Integration project to call this in place of the actual consumed web service. Go to your AOEIntegration project and generate a config plan
Open up the config plan and insert a search/replace in the wsdlAndSchema block to swap out the actual CustomerService endpoint for the mocked Customer Service endpoint. Be sure to deploy the AOEIntegration project with the new config plan

At this point, when testing the AOEIntegration service, you should successfully get the correct response xmls from John Smith and Mark Brody and the correct fault xmls from Herman Munster and Gomez Addams. Its well worth noting that no matter how many times you invoke these services, you’ll always get the same response. This would not be the case if you ran the same tests against the actual service because (just like in real life) you would potentially get different responses with each call. This is the point of creating mock services – to enable highly repeatable tests.

Setting up the SOAP UI Test Suite

We’re finally ready to set up our test suite. Before diving in, we need to consider all of the positive and negative scenarios that our service should respond to. In our case, there are 4 possibilities we need to test for:
1. SUCCESS
2. RETRY fault
3. NO RETRY fault
4. Service is not reachable – which is another RETRY fault handled in the CatchAll
In your GettingTestyPart2.zip, open up the SOAP UI project and navigate to the one called “GettingTesty-AOEIntegration.” In there, you’ll find a MockService request titled “JohnSmith” which is one of the Happy Path tests described above in Steps 31 and 32. Right click on this and select “Add to TestCase”

Name your Test Suite. In this case, I named it “TestSuite – AOEIntegration”
Name your Test Case. Always be sure that these names are highly descriptive. In this case, I named it “UpdateCustomerRecord-TC1-HappyPath”
Name your request something meaningful. In this case, I called it “UpdateCustomerRecord-TC1-HappyPath Request.” Also, save some work for yourself by automatically adding some assertions:
1. SOAP Response
2. Schema Compliance
3. Not SOAP Fault

If you open up your test and run it, you should see a successful response. Even more importantly, if you click on the “Assertions” tab on the bottom of the screen, you should see 3 valid assertions as shown in the screenshot below:

So far, we’ve ensure that this is a valid SOAP response that is schema complaint and not a SOAP Fault. However, to ensure that future code changes don’t impact the response, it’s always best to actually assert the actual text of the response. Click on the “Add Assertion” icon shown below:

From the “Property Content” tab on the left, select the option for “XPath Match”

Once you select the XPath Match assertion, you must put in all of the declarations from your service response.
1. The basis for these is highlighted in yellow below:

In the “Declare” text box, enter each of the namespace declarations as shown below
After the declarations, enter the path describing where the content is being extracted from (based on the text highlighted in green above)
Assuming you have a successful response already retrieved, you can click on the “Select from current” button to automatically grab the text of the result. This is much easier than typing or even copying it in yourself since SOAP UI will make some slight changes that can sometimes be a little tricky to guess:

Now let’s work on the negative test scenarios. Go back to the GettingTesty-AOEIntegration project and click on the “HermanMunster” MockService request. Then select the option to add this request to your test suite. Name the new test “UpdateCustomerRecord-TC2-RetryFault”

For this test set up the following assertions:
1. SOAP Response
2. Schema Compliance
3. SOAP Fault
4. XPath Match. Set up your “Declare” text box like this:

Create a 3^rd test case from the GomezAddams request called “UpdateCustomerRecord-TC3-NoRetryFault” and set up the assertions exactly as you did in the last step
For the final test case, you may be wondering how we could easily simulate a service outage. In this case, if we simply send in a request that doesn’t have a corresponding test xml file, the resulting fault will be caught in the AOEIntegration catchAll block. The assertions are set up exactly like Steps 55 and 56

You are now the proud and happy owner of your very own integration test suite.

Conclusion

The best part about this pattern is that it is now extremely easy to add new mock web services that can then be used for additional tests within your SOAP UI Test Suite. This is incredibly useful in large projects to ensure that changes cutting across many composites can quickly be regression tested to ensure there are no adverse side effects. &nb

+1.972.608.4777