+1.972.608.4777

AVIO Consulting

Getting Testy with SOA 11g – Part 1: The Integration Service Pattern

Sep 10, 2013 | BPM

Most developers that I’ve met really dislike automated testing – hence the title.  In the course of the SOA 11g projects I’ve worked on, I’ve found a number of patterns that have really made setting up automated tests for services easy.  Honest.  While this, of course, requires a little bit of extra work, it’s not as much as you’d think and the benefits far outweigh the additional coding.  For starters, have you ever made a small change to a really complex service on a tight timeline hoping and praying you didn’t inadvertently hatch any new bugs while fixing your current one?  If so, you’ll find this pattern is a much better way to calm your nerves rather than a prescription from your doctor because a framework that supports automated testing can quickly exercise every possible scenario in your service without being shackled to external dependencies. 

The best way to easily set up a good testing framework is to ensure that your project is set up in the most efficient way possible – which is the focus of Part 1 of this series.  Whenever your project requires calls to an external dependency such as a database, web service or messaging queue the best practice is to use a product like Oracle Service Bus to manage these interactions.  However, if you don’t have a license to OSB like many shops I’ve encountered, you should utilize the Integration Service Pattern instead.  This involves creating a separate SOA project within your application that will utilize a BPEL process to invoke your external dependency.  This BPEL process should then be exposed as a web service so it can be accessed from other projects.   

Benefits of using the Integration Service Pattern

This pattern promotes modularity, which is one of the core tenets of Service Oriented Architecture.  A well-coded, simple service can be written once and then used repeatedly by other projects.  Even though they say “practice makes perfect,” this is a much more manageable solution then coding and testing the same interface repeatedly in numerous processes and projects.

We’ll also discover later in this series that automated tests for this Integration service will help to speed assembly testing for any parent services that invoke this integration service.  This means that even though you’re using this same service in 10 places, you only have to have one test suite for the interface.

Finally, there are a number of exception handling benefits to using the Integration Service pattern that we’ll cover later in this series that allow greater flexibility in dealing with exceptions.

Setting up a Sample Project

If you’d like to see a simple project that demonstrates specifically how this works, download GettingTestyPart1.zip.  For this demo, I’m running Oracle’s Pre-built Virtual Machine for SOA Suite and BPM Suite 11g.

Within the GettingTestyPart1.zip file, you should see 3 main directories:

  • AvioOrderEntry – Contains the SOA demo application we’ll be using
  • DataModel – Contains sql scripts to set up your database for the demo
  • SOAPUIProject – Contains the SOAP UI project used in this demo

First, let’s set up your database.  Inside the DataModel directory, you’ll find the following 2 files:

  • 1-MasterDataLoader.sql
  • 2-CreateUsersAndTables.sql

Log into your local database as the SYSTEM user and run 1-MasterDataLoader.sql.  This will set up a new user: ORDER_TRACKING_DB as well as a table we’ll be using for this demo. 

Next, you’ll need to set up a WebLogic datasource and data connection pool for both users.  If you’re a little fuzzy on how this works, check out a blog article I wrote on this topic: Oracle SOA Database Adapter 101 – WebLogic Configuration Steps.

Here is the setup information for users:

  • Username: ORDER_TRACKING_DB
  • Password: ORDER_TRACKING_DB
  • Hostname: soabpm-vm
  • Port: 1521
  • SID: XE
  • Data Source JNDI: jdbc/ORDER_TRACKING_DB
  • Connection Pool JNDI: eis/DB/OrderTrackingDB

Within the GettingTestyPart1.zip, you should be able to find the SOA demo application in the “AvioOrderEntry” directory.  Deploy the project named “CustomerService”.  For now, we’re not going to explore this project any further except to say that it will play the role of an “external” web service that our application will be utilizing. 

In this demo, we’re going to build a very small order entry application.  In Part 1 of this series, the application will be very simple but will evolve to become more complex with each new entry in the series.  To place an order in our new system, we need to have 3 components:

  1. Customer information – For our purposes this will also serve as the shipping address.
  2. Item to be ordered
  3. Order quantity

OK.  Enough of the boring setup stuff.  Let’s start banging out the Integration layer. 

Structuring the Integration Layer

  1. Create a new SOA project called “AOEIntegration” within your application.
  2. Create a “wsdl” directory within AOEIntegration.  If you’re asking why we would bother to do this, it’s because JDeveloper by default will put all of its wsdls into the project’s root directory.  This is fine if your project is small, but if it ever becomes large, this quickly leads to a huge, disorganized directory – like London’s “fatberg” that was recently in the news.
  3. Within your “wsdl” directory, create a “consumed” directory and an “exposed” directory.  This is important because it’s helpful if you name your integration service (the exposed service) like the consumed service to reduce confusion.  The only issue with that is it requires a bit more organization to keep everything straight since your “exposed” wsdl will be structured a little differently than your “consumed” wsdl.  Stay tuned for more on that later.
    1. Consumed  – refers to the actual wsdls (and xsds) of the services you’re invoking
    2. Exposed– refers to the wsdls (and xsds) that the Integration Service is exposing.  These are the wsdls we’ll be using later on in our ProcessOrder project.
  4. Within your “consumed” directory, create a “CustomerService” directory.  The idea here is that each consumed service in your Integration project will have its own directory.
  5. In the “CustomerService” directory create an “xsd” directory.  This will be used to hold any schemas that the consumed wsdl utilizes. 

Adding Consumed and Exposed WSDLs to your Integration Project

  1. Now that you have your directory structure in place, it’s tempting to create your web service external reference and allow it to automatically pull in the wsdl along with all of its dependencies.  This is fine if there is no other way to get this information, but be aware that it tends to bloat up your project with extra files.  If possible a better option is to get the wsdl and supporting xsds straight from the owners of that web service.  In our case, we’re in luck because we know who built the CustomerService wsdl – we did!  You can copy the files from AvioOrderEntryCustomerServicewsdlCustomerService.  Be sure that all of the references within your wsdl and xsds are linked properly. 
  2. Create the same directory structure in the “exposed” directory and copy the wsdl and xsd files from the corresponding “consumed” directory: AOEIntegrationwsdlexposedUpdateCustomerRecord
  3. Your finished directory should look like this:

  1. It’s critical that we change the namespace for the exposed wsdl and schema because they cannot be the same as what is used for the consumed wsdl.  Within AOEIntegrationwsdlexposedCustomerServicexsdUpdateCustomerRecord.xsd, update the namespace to refer to “AOEIntegration” versus “AvioOrderEntry”:

  1. In addition, let’s also delete the updateCustomerRecordFault element and the UpdateCustomerRecordFaultType complexType.  These will not be required in our exposed schema and will be replaced by other faults.  This is part of the reason for having an Integration service in the first place:

  1. Before we update the exposed CustomerService.wsdl, we need to first add a new xsd to the project which will contain our “Retry” and “NoRetry” faults which will be used in the exposed web service.  These will become important when setting up our fault policies in Part 3 of this series.  Since these faults should be used for any of the exposed services within the Integration project, they should be in a “Common” folder within the project’s xsd directory.  Therefore, create the following file: AOEIntegrationxsdCommonFaults.xsd:

  1. Within AOEIntegrationwsdlexposedCustomerServiceCustomerService.wsdl we need to update the namespace to refer to “AOEIntegration” versus “AvioOrderEntry” just like we did with the corresponding xsd:

  1. Now, let’s set up our shiny new faults within the AOEIntegrationwsdlexposedCustomerServiceCustomerService.wsdl.  First we will need to delete the fault type that was copied over from the consumed service:

  1. Now we can add in the references to our “Retry” and “NoRetry” faults:

  1. If your consumed wsdl is missing the binding and service elements you should set these up before inserting the external reference to your web service in the composite.  You should be able to pull up the running wsdl in a browser to find these sections.  In our example, this wsdl would be found at: http://soabpm-vm:7001/soa-infra/services/default/CustomerService/CustomerSvc?WSDL.  Be sure to copy the binding and service blocks into bottom of AOEIntegrationwsdlconsumedCustomerServiceCustomerService.wsdl:

  1. Don’t forget that it may be necessary to add a namespace reference for the WSDL SOAP binding:

  1. With that minor housekeeping done, we can finally insert an external reference to the CustomerService web service within your Integration project.  This is sort of helpful if we actually would like to make a call to this service.
    1. Be sure to reference the consumed service: AOEIntegrationwsdlconsumedCustomerServiceCustomerService.wsdl
    2. Since we just went to the trouble of setting up our consumed wsdl, be sure to uncheck the “copy wsdl” option.  

 

Creating the Integration BPEL process

  1. Now it’s time to create your BPEL process.  In order to minimize rework (because that’s personally my favorite thing to do), we need to be careful how we set up our BPEL process:
    1. BPEL 2.0
    2. Namespace: Name it after the operation you’re targeting: “UpdateCustomerRecord”.  You’ll have one BPEL process per operation.  If a service supports multiple operations then the best practice is to put a mediator in front of the multiple BPEL processes representing each operation so they can be represented as one “external” service in the Integration project.  For now, we’re going to stick to dealing with one operation for simplicity.  In our case, name your BPEL process: “UpdateCustomerRecord”
    3. Template: Base on a WSDL
    4. Service Name: Name your exposed service slightly differently than the consumed service.  We need to do this to eliminate confusion within the composite.  In this case, let’s call it “Customer”
    5. WSDL URL: This is the exposed wsdl – the one that will be used from this point forward to invoke this service.  In our case, use AOEIntegrationwsdlexposedCustomerServiceCustomerService.wsdl

  1. You’ll notice once you click OK that JDev will slightly alter the Exposed Service name to “Customer_ep.”  If you’re like me and this bugs you, change it back to “Customer” within the Exposed Service dialog:

  1. Now you can drag a reference from your BPEL process to the External Reference.  At this point your project should compile successfully and your composite should look like this:

  1. Before delving into the UpdateCustomerRecord.bpel, we should set up our DVM file.  This is a quick and easy way to group fault codes into categories (in our case, retryable and non-retryable).  To quickly explain this file:
    1. ServiceName – Enables this file to be used to hold data for multiple services
    2. ServiceResponseCode – In our case, we have a service that will throw a response code (‘00008’ for system exceptions and ‘00010’ for business exceptions)
    3. ExplanationCode –  Defines precisely what the problem is. 
    4. TargetResponseCode  –  If the incoming fault matches the aforementioned fields, then the resulting code will be determined here.  In our case we are only listing Retry Faults (“00001”).  When we actually call the DVMLookup in our xslt, we will put in a default value of “00002” to return a NoRetry Fault if no correct combinations were found in this file
    5. Create the following file: AOEIntegrationdvmResponseCodes.dvm:

  1. At long last we’re finally ready to take a look at our UpdateCustomerRecord.bpel:

  1. You’ll notice in this process that if no fault is thrown there are very simple transforms to map input and output data from the service call.  Even though it’s taken a little bit of time to explain the pattern, ultimately this should be a very thin layer within your project.
  2. If the consumed service throws a defined fault this is where the bulk of our Integration coding will go:

  1. In the AnalyzeFault transform, utilize a DVM Lookup Value function to determine if this is a retryable fault or not

  1. For each fault type (in our case Retry and NoRetry), insert a Reply activity into the end of each “if” branch.  Also insert a Retry activity into your CatchAll.  Most of the time, this will probably intercept binding faults that occur when the invoked service is unavailable

  1. When we deploy and test this service we’ll find that the output looks like this in several different scenarios. 
    1. Success 

​

  1. Retry fault

  1. NoRetry fault

Utilizing the Integration Service

  1. Now that we have our Integration Service all set up, we can finally start using it within our Order Entry web service.  Create a SOA project named AOEOrderEntryService.
  2. Since we did such a good job structuring our wsdl and xsd directories, it’s now very easy to copy them from project to project.  In this case, simply copy everything from AOEIntegrationwsdlexposed to AOEOrderEntryServicewsdlconsumed.  This might be a little confusing at first, but if you think about it, from the perspective of the AOEOrderEntryService, it is consuming the wsdl from our Integration service. 
  3. In addition, copy AOEIntegrationxsdCommon to  AOEOrderEntryServicexsdCommon:

  1. Be sure to add in the binding and service blocks from the deployed AOEIntegration Customer wsdl as well as the reference for the WSDL SOAP binding like we did in Steps 15 and 16:  http://soabpm-vm:7001/soa-infra/services/default/AOEIntegration/Customer?WSDL
  2. Here is what this piece of the ProcessOrder BPEL process looks like for now.  This is our Order Entry BPEL process that will first call the Customer service to update a customer record:

  1. If you run this service (which sometimes will ‘accidentally’ throw faults), we will see the following from the Order Entry Service
    1. Success

  1. Retryable fault

  1. Non-Retryable fault

I Love It When A Plan Comes Together

The great thing about his pattern is that once you have the Customer integration service completed, it can be used in other BPEL or BPMN processes long after your initial project has been completed.  This means that even though the Customer integration service was originally written for the Order Entry process, it now can be used for a completely different process without any changes at all. 

Be sure to stay tuned for the rest of this series:

  • Part 2: Shows how a Mock Web Service can be constructed which will allow us to thoroughly test this Integration Service in a SOAP UI test suite
  • Part 3: Discusses building SOAP UI tests for asynchronous service calls.  Also touches on using the Integration Service to filter faults into Retry and NoRetry faults for easier asynchronous error handling
  • Part 4: Details how to build mock  services and tests for a process utilizing a messaging queue.

Be sure to check out my blog posts for other articles around SOA process patterns, software development best practices and development methodologies.