Imagine your SOA/BPM application is consuming a service (internal or third party) that takes more than 5 minutes to reply, which is more than the default JTA timeout. What would happen in this situation? The result is a faulted instance.
Last month at a Oracle BPM project, we faced a similar situation. After doing some research, we realized that not only do you have to update the EJBs or JTA timeouts, but there are also several other timeouts to tweak. In fact, six different timeouts need to be updated. Five of them related to the Oracle SOA Suite, plus the DB timeout. Here’s an overview of the 6 timeouts that we had to update and how to do so!
Web Service Binding Timeout
First, you need to identify all composites involved with the long-running service. This will include the composites where the service is called, and the composites calling these composites. Then, in each composite go to the source view and add the following properties to the web service binding (tag <binding.ws/>):
- oracle.webservices.httpConnTimeout (milliseconds)
- oracle.webservices.httpReadTimeout (milliseconds)
- oracle.webservices.local.optimization (false)
Local optimization is enabled by default in SOA Suite to optimize synchronous calls between BPEL processes deployed on the same container. We need to disable the local optimization, otherwise, the HTTP timeouts won’t be considered.
BPEL Timeout
Set the property “SyncMaxWaitTime” (value is in seconds). Go to the Enterprise Manager, then in the navigation menu on the left, expand the “SOA” folder and click on the “soa-infra (<SOA server name>)” item.
In the details page, expand the “SOA Infrastructure” dropdown. Then go to “SOA Administration” and finally, click on “BPEL Properties” option.
The “BPEL Service Engine Properties” page is shown. From there, click on the link “More BPEL Configuration Properties…”
In the “Application Defined MBeans” page, you can update the value of “SyncMaxWaitTime” property– keep in mind this value is in seconds.
EJB’s Timeouts
Open the WebLogic console, then go to Deployments, find “soa-infra” and expand it by clicking on the plus sign.
You have to update the timeout of the following EJB’s:
- BPELEngineBean
- BPELDeliveryBean
- BPELActivityManagerBean
- BPELServerManagerBean
- BPELProcessManagerBean
- BPELInstanceManagerBean
- BPELFinderBean
Typically, the way to update the timeout of these EJB’s would be clicking on the name, then in the Configuration tab, increasing the value of “Transaction Timeout” and finally, activating the changes. But first, make sure to stop all SOA Servers before doing that. According to Oracle Support Doc ID 1271369.1, if you update these BPEL EJB’s while the SOA servers are running, your SOA Infra won’t start up anymore and you’ll experience the pain.
If you did update the transaction timeouts with the SOA Servers running and now your SOA environment is broken, don’t fret. Try the following steps to resolve the issue:
- Shutdown the entire domain.
- In a console, go to the <your domain folder>/servers/AminServer and rename the “tmp” folder to “tmp_OLD”.
- Start the Admin Server and then the SOA Servers.
This should bring your SOA environment back to life.
JTA Timeout
In the WebLogic console, go to the Domain Structure panel and click on the domain name. Then in the domain settings page, go to “Configuration” and then “JTA” tab. From there, you can update the value of “Timeout Seconds” property.
SOADataSource XA Transaction Timeout
In the WebLogic console, go to the Domain Structure panel, then “Services” and then “Data Sources”. Find “SOADataSource” on the list and click it. Select the “Configuration” tab and then “Transaction” tab. From there, make sure that “Use XA Data Source Interface” is checked and “XA Transaction Timeout” is set to zero (which means the WebLogic Server Transaction Manager passes the global WebLogic Server transaction timeout in seconds).
Database Distributed Transaction Timeout
Last but not least, update the DB transaction timeout. For that, ask the DBA to help you to update the DISTRIBUTED_LOCK_TIMEOUT value of your Oracle Database.
Having faulted instances can be extremely frustrating and time-consuming if you don’t know how to update them efficiently. So, next time when your Oracle BPM project faces the same situation, I hope that this how-to guide will help you identify what is causing the issue and you’ll be able to quickly update the timeouts and resolve the issue!