AVIO Consulting

AVIO ICS Maven Plugin

Mar 16, 2018 | Connectors, Oracle

AVIO ICS Maven Plugin. I began getting my feet wet (shameless self-advertising) and took in the whole ICS experience.

It has been a fun adventure working in ICS, mastering the art of deactivating, editing, fixing, saving, activating, and finally testing… only to curse at a simple forgotten step and repeat the process again. Some tasks that used to be simple became harder, some tasks that were complicated and confusing became simpler.

What do you mean my changes are lost? (Heavy sigh)

AVIO Lock Is Lost

Sometimes when making changes to an integration or rearranging a layout, something becomes corrupt and ICS does not activate, or fails to edit properly, or loses my lock! I haven’t dug into the root cause, and it easily could be my fault, but my first instinct is to make sure I protect myself in case it happens again.

So I began manually exporting the integration after each development accomplishment. I would date these exports and keep them locally on my computer to re-import later if anything went wrong. Eventually, I started expanding them and putting them into source control, which allowed me to monitor changes to files. The process wasn’t complicated, but it required too much time and way too many mouse clicks.

Why can’t this be done programmatically?

Of course, it can! Oracle has exposed REST APIs to integrate with ICS. There are 2 versions available, v1 and v2 (click for Oracle Documentation). As a general rule, use v2; however, some features aren’t available in v2, and therefore v1 has to be used for some features (for example, activate and deactivate).

REST APIs Version 2

Soon, it became tedious for me to even use the REST APIs, and so I created a maven plugin to do the work for me.

 

Enter ics-maven-plugin.

AVIO Consulting has created a maven plugin, aptly named ics-maven-plugin, which is currently available through GitHub at https://github.com/avioconsulting/ics-maven-plugin. The plugin will soon be available through Maven Central as well (I’ll update when that happens).

*Update (03-27-2018): This plugin is now available on Maven Central – https://repo.maven.apache.org/maven2/com/avioconsulting/maven/ics-maven-plugin/1.0/

This new plugin has several features that make the development lifecycle much easier to bear!

Keep in mind, none of these goals/phases makes development any simpler, you still have to develop the integrations and connections from the portal, but it does help significantly with the development lifecycle.

Also, as a side note, sending messages to my colleagues saying that I had to write a Mojo was mildly entertaining.

Phases

This plugin defines a new packaging type (iar) and hooks into the Maven lifecycle at the following phases:

  • generate-resources
    • By default, this does nothing.
    • If the ‘export’ property is set to true (-Dexport=true), it will run the ‘export’ goal and will export the integration defined in the POM file
    • If the ‘connection’ property is set to the name of a connection (-Dconnection=EBS), a template JSON config file will be created
  • package
    • Runs the ‘package’ goal, which generates an iar file

Additional individual goals

Additional goals are provided for supporting tasks but are not part of the standard maven lifecycle.

  • activate – Activates an integration
  • deactivate – Deactivates an integration
  • import
    • Deactivates the integration if it is present and active
    • Imports (or updates) the integration (iar file)
    • Generates the connections / lookups / schedule components as necessary
    • Updates the connection properties
    • Activates the integration
  • updateConnection – Updates the connection properties for a specific connection

See the ics-maven-plugin documentation for the latest functionality and detailed usage instructions.

Exported Project Structure

The exported IAR has a non-standard folder structure. This plugin moves some folders around and puts them into a more common structure. Additionally, the folder that contained the version number has been removed. When the package phase is executed this structure is converted back to the ICS standard structure within the IAR file.

Here are the basic folder mappings that occur:

  • icspackage/project/AVIO_FTP_INTEGRATIO_01.00.0000 → src/main/iar
  • icspackage/appinstances → src/main/resources/connections
  • icspackage/dvm → src/main/resources/lookups
  • icspackage/schedule → src/main/resources/schedule

Below is an example of an exported integration that would get checked into source control.

Export Folder Structure

ICS Development Lifecycle

First design and implement your integration within ICS, adding the invokes and actions as well as configuring your connections. Once you have something working, its time to back it up and save it in source control.

Export

Let’s start with the basic function so that a copy of the integration can be stored in source control.

The export goal will:

  • Use the API to download an export IAR file
  • Expand the archive file locally
  • Restructure the folders into the standard maven project structure (src/main directories)

The exported integration also includes details of dependent artifacts. So now an Integration project will be self-contained, with references to Connections, Lookups, and Schedules.

kevin@kking-lt3:~/code/avio/demos/packages/avio-integration/AVIO_FTP_INTEGRATIO$ mvn generate-resources -Dexport=true -Denv=DEV
[INFO] Scanning for projects...
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building AVIO_FTP_INTEGRATIO 01.00.0000
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] --- maven-enforcer-plugin:1.4:enforce (enforce-property) @ AVIO_FTP_INTEGRATIO ---
[INFO]
[INFO] --- maven-resources-plugin:2.7:copy-resources (copy-project-properties) @ AVIO_FTP_INTEGRATIO ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 2 resources
[INFO]
[INFO] --- properties-maven-plugin:1.0-alpha-2:read-project-properties (default) @ AVIO_FTP_INTEGRATIO ---
[INFO]
[INFO] --- ics-maven-plugin:1.0-SNAPSHOT:export (default-export) @ AVIO_FTP_INTEGRATIO ---
[INFO] Exporting integration AVIO_FTP_INTEGRATIO_01.00.0000 from https://avioconsultingcloud.integration.us2.oraclecloud.com
[INFO] Expanding ICS export, use -Dexpand=false to disable it. Optionally add -Dclean=true to remove existing files before expanding.
[INFO] [Integration.exportIntegration] Starting
[INFO] [RestUtilities.invokeService] Invoking REST Service
[INFO] [RestUtilities.invokeService] Calling GET on https://avioconsultingcloud.integration.us2.oraclecloud.com/icsapis/v2/integrations/{id}/archive
[INFO] [RestUtilities.invokeService] Using params: {id=AVIO_FTP_INTEGRATIO|01.00.0000}
[INFO] [RestUtilities.invokeService] Service returned 200 in 1533 ms.
[INFO] [Integration.expandIar] Decompression for target/AVIO_FTP_INTEGRATIO_01.00.0000.iar has commenced.
[INFO] [Integration.expandIar] Decompression for target/AVIO_FTP_INTEGRATIO_01.00.0000.iar has completed.
[INFO] [Integration.iarToProjectStructure] Starting, copying from target/iar
[INFO] [Integration.iarToProjectStructure] Finished
[INFO] [Integration.exportIntegration] Complete
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 3.042 s
[INFO] Finished at: 2018-02-26T14:56:06-06:00
[INFO] Final Memory: 18M/253M
[INFO] ------------------------------------------------------------------------

At this point, make sure you add the exported files to source control! Keep in mind the plugin also does not identify the differences from a previous version in source control, specifically any removed files.

Once exported, this allows for advanced editing of XSLTs within JDeveloper as well.

Import into ICS

Now that the integration code is in source control, let’s import it back into our environment or promote it to a new environment.

The import goal will now:

  • Deactivate existing integrations
  • Import (new) or update (existing) the integration
  • Create connections and lookups (with configuration updates)
  • Activate the integration

Now that saves time and a whole bunch of clicks! Yay!

kevin@kking-lt3:~/code/avio/demos/packages/avio-integration/AVIO_FTP_INTEGRATIO$ mvn initialize ics:import -Denv=DEV
[INFO] Scanning for projects...
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building AVIO_FTP_INTEGRATIO 01.00.0000
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] --- maven-enforcer-plugin:1.4:enforce (enforce-property) @ AVIO_FTP_INTEGRATIO ---
[INFO]
[INFO] --- maven-resources-plugin:2.7:copy-resources (copy-project-properties) @ AVIO_FTP_INTEGRATIO ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 2 resources
[INFO]
[INFO] --- properties-maven-plugin:1.0-alpha-2:read-project-properties (default) @ AVIO_FTP_INTEGRATIO ---
[INFO]
[INFO] --- ics-maven-plugin:1.0-SNAPSHOT:import (default-cli) @ AVIO_FTP_INTEGRATIO ---
[INFO] Importing integration AVIO_FTP_INTEGRATIO_01.00.0000 to https://avioconsultingcloud.integration.us2.oraclecloud.com
[INFO] [Integration.importIntegration] Starting
[INFO] [Integration.retrieveIntegrationDetails] Starting
[INFO] [RestUtilities.invokeService] Invoking REST Service
[INFO] [RestUtilities.invokeService] Calling GET on https://avioconsultingcloud.integration.us2.oraclecloud.com/icsapis/v1/integrations/{integration}/{version}
[INFO] [RestUtilities.invokeService] Using params: {integration=AVIO_FTP_INTEGRATIO, version=01.00.0000}
[INFO] [RestUtilities.invokeService] Service returned 200 in 1212 ms.
[INFO] [Integration.retrieveIntegrationDetails] Complete
[INFO] [Integration.importIntegration] Current status: ACTIVATED
[INFO] [Integration.deactivate] Starting
[INFO] [RestUtilities.invokeService] Invoking REST Service
[INFO] [RestUtilities.invokeService] Calling POST on https://avioconsultingcloud.integration.us2.oraclecloud.com/icsapis/v1/integrations/{integration}/{version}/deactivate
[INFO] [RestUtilities.invokeService] Using params: {integration=AVIO_FTP_INTEGRATIO, version=01.00.0000}
[INFO] [RestUtilities.invokeService] Service returned 200 in 3151 ms.
[INFO] [Integration.deactivate] Complete
[INFO] [RestUtilities.invokeService] Invoking REST Service
[INFO] [RestUtilities.invokeService] Calling PUT on https://avioconsultingcloud.integration.us2.oraclecloud.com/icsapis/v2/integrations/archive
[INFO] [RestUtilities.invokeService] Using params: {}
[INFO] [RestUtilities.invokeService] Service returned 200 in 624 ms.
[INFO] [Integration.importIntegration] Complete
[INFO] [Integration.getConnections] Finding connections.
[INFO] [Integration.retrieveIntegrationDetails] Starting
[INFO] [RestUtilities.invokeService] Invoking REST Service
[INFO] [RestUtilities.invokeService] Calling GET on https://avioconsultingcloud.integration.us2.oraclecloud.com/icsapis/v1/integrations/{integration}/{version}
[INFO] [RestUtilities.invokeService] Using params: {integration=AVIO_FTP_INTEGRATIO, version=01.00.0000}
[INFO] [RestUtilities.invokeService] Service returned 200 in 466 ms.
[INFO] [Integration.retrieveIntegrationDetails] Complete
[INFO] [Integration.getConnections] Integration AVIO_FTP_INTEGRATIO has 1 unique connections.
[INFO] [RestUtilities.invokeService] Invoking REST Service
[INFO] [RestUtilities.invokeService] Calling GET on https://avioconsultingcloud.integration.us2.oraclecloud.com/icsapis/v2/connections/{id}
[INFO] [RestUtilities.invokeService] Using params: {id=AVIO_FTP}
[INFO] [RestUtilities.invokeService] Service returned 200 in 238 ms.
[INFO] [Connection.retrieveConnectionDetails] Complete
[INFO] Connection AVIO_FTP has status CONFIGURED
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 1 resource
[INFO] [RestUtilities.invokeService] Invoking REST Service
[INFO] [RestUtilities.invokeService] Calling POST on https://avioconsultingcloud.integration.us2.oraclecloud.com/icsapis/v2/connections/{id}
[INFO] [RestUtilities.invokeService] Using params: {id=AVIO_FTP}
[INFO] [RestUtilities.invokeService] Service returned 200 in 1166 ms.
[INFO] [Connection.updateConnection] Complete
[INFO] [Integration.activate] Starting
[INFO] [RestUtilities.invokeService] Invoking REST Service
[INFO] [RestUtilities.invokeService] Calling POST on https://avioconsultingcloud.integration.us2.oraclecloud.com/icsapis/v1/integrations/{integration}/{version}/activate?enablePayloadTracing={payloadTrace}&enableTracing={enableTrace}
[INFO] [RestUtilities.invokeService] Using params: {integration=AVIO_FTP_INTEGRATIO, payloadTrace=true, enableTrace=true, version=01.00.0000}
[INFO] [RestUtilities.invokeService] Service returned 200 in 9943 ms.
[INFO] [Integration.activate] Complete
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 18.261 s
[INFO] Finished at: 2018-02-26T15:01:44-06:00
[INFO] Final Memory: 22M/255M
[INFO] ------------------------------------------------------------------------

 

Conclusion

This plugin now automates some tedious tasks and implements the release process into a CI/CD pipeline.

I hope this helps others in their ICS projects!