Starting Up
The arrival of the Jenkins Pipeline feature coincided with the the addition of Maven in the Oracle Fusion Middleware 12c release. Jenkins pipelines now easily leverage Maven phases and goals for Fusion Middleware application deployments, including SOA/BPM composite applications and Service Bus projects. This is the first in a series of blogs that will present three deployment workflows (pipelines) implemented in Jenkins for Oracle SOA projects. The workflows will utilize Git source control, Git Flow branching methodology and scripts. It also assumes the use of a managed Maven repository where application and FMW Maven artifacts are centralized. Although the blogs primarily address SOA, there will be discussions about Metadata Services (MDS) and Service Bus (OSB) deployments.
Although the approach defined here in Jenkins is leverages open source tools organizations should weigh the approach outlined here to the benefits (e.g. support and upgrades) that a commercial product brings to the table, like MyST from Rubicon Red. MyST provides both platform provisioning and application release management tools that enable you to move the associated WebLogic artifacts (like data sources, queues, topics) when the application is promoted to higher environments. Both are versioned nicely in MyST and easily migrated in the MyST release management dashboard.
Jenkins however is the most widely adopted open source build server available, highly scalable, with a very large library of plugins to accomplish virtually any build task. Jenkins provides a way to automate the tasks associated with building and deploying a application component using the Groovy language that leverages an extensive plugin library exposed through Java, referred to as a “pipeline”. The pipeline provides a graphical view of the deployment, as a series of stages, e.g. “Checkout”, “Package”, “Deploy”, “Test”. Additionally there are manual gateways that can be implemented in a pipeline that only certain groups or users can performed, in the form of role-based assignments.
Why Git and Git Flow?
Although there are many successful FMW implementations using SVN and the Maven Release plugins there seems to be a shift in their use in favor of the Git source code management system and the Git Flow scripts. The Maven Release plugin is seen by many as needlessly complex and redundant in its builds and source code updates. (This blog questions why people are still using the Maven Release plugin.) Git’s more flexible merge capabilities, coupled with a robust branching methodology in the form of Git Flow are being utilized increasingly adopted today. SVN proponents would argue that similar workflows can be implemented in SVN but not many (OK, none) are implemented in SVN client tools, as is the case for Git Flow in tools like GitKraken (Axosoft) and SourceTree (Atlassian).
In the interest of fairness to all I’ll provide an unfavorable link to the use of Git Flow, so you may judge for yourself. Be sure to read the comments people posted in blog as the responses turn the arguments around against the use of Git Flow.
The Pipelines
The three pipelines to be covered in this blog series are referred to as “CI” (Continuous Integration), “CR” (Create Release) and “DR” (Deployment Release) and the details for each follow in the sections below. They provide a full life cycle for the release of software from development, through test and production environments. Although, the pipelines focus on SOA composites and MDS projects, the pipelines can be extrapolated to include OSB, JAX-WS, and ADF deployments.
Custom builds for JMS, data sources (using WebLogic WLST plugin for example) are not within the scope of this topic but pipelines should be created to automate these tasks as well.
Continuous Integration (CI) Pipeline
Much is already written about Continuous Integration (see Martin Fowler topics for good overview). When the Git project repository “develop” branch is updated on the remote Git repository it triggers a build of the CI pipeline for the project. The build server CI job is polling source control at a specified interval. The Developer role on the build server may manually submit or cancel a build. The goal of the build is to automatically deploy the SOA project to the SOA-DEV environment where automated unit or integration tests can be run. More times than not SOA tests are integrated, that is, they rely on the SOA deployment and are not run in isolation as a unit test. Regardless, tests should be written and run in every CI build. In the event of testing failures the developer (team) should be notified so the errors can review the errors in Jenkins.
Create Release (CR) Pipeline
The CR pipeline is principally a Git Flow nuance, to run the “git flow release start” script that captures the current “develop” branch to create a “release” branch. However, after the release is created in the Git repository, there are additional steps that follow to update the Maven artifacts in the newly created release, to remove the “-SNAPSHOT” verbiage. Once this is done the branch is checked back into the Git project repository. This facilitates the DR pipleline creation of a common SOA archive (JAR) that is used for both TEST and PROD deployment.
Deployment Release (DR) Pipeline
Although the pipeline looks quite busy it’s chief purpose is to create a deployable SOA archive that will be built once, stored in the Maven artifact repository (Artifactory) and then used for the TEST and PROD deployments when they are approved.
The DR pipeline will use the current “release” branch in the Git repository and will prompt the Test Admin when it is ready for deployment to TEST. There is polling performed on the “release” branch as developers will be committing fixes to the “release” branch so any number of builds may be done before it is deployed to TEST. Only the Test Admin can approve the deployment to TEST. The Test Admin may wait unit end-of-day before approving the deployment to TEST.
After deployment to TEST then testing proceeds and developers fix defects in the release branch. When a change is committed by a developer to the “release” branch it will trigger another build. Once again the Test Admin will be prompted whether the build should be deployed to TEST. Perhaps this is only done at end of day and there may be any number of release build initiated (but not deployed) until the Test Admin OK’s it. It also provides a Prod Admin with the ability to OK a deployment to production. The latter is always optional and as a release must be first be “accepted” before the Prod Admin will be notified of the deployment opportunity.
In case you were wondering what the “Accept Release” step is, read on in the blog series!
Finishing Up
While Continuous Integration implementation are relatively easy to implement in Jenkins, it is important to ensure automated tests are also incorporated into the feedback look to ensure what is considered “done” from a development perspective will pass muster. In addition to the Opitz Framework for SOA testing, SoapUI (see my other blog topic on this subject) can be leverage easily as well.