Restoring SOA 12.2.1.1 Maven Functionality

With the release of Oracle SOA Suite 12.2.1.1, the ability to create and build SOA projects with Maven has been broken.  Customers who want an official fix should contact Oracle Support and reference Bug 24710353.  For those that choose this official path, you should stop reading now.  For anyone else interested in a temporary workaround, keep reading.

As with any of my posts, this post should not be construed as an official, Oracle-supported way of fixing the product.  This post simply represents what I did to restore the functionality for my Oracle SOA Suite 12.2.1.1.0 Quick Start installation.  The high-level steps are:

  • Fix the SOA 12.2.1.1 Maven metadata
  • Sync the Maven repository with the updated Oracle Home
  • Recreate the SOA applications and projects

Fixing the SOA 12.2.1.1 Maven Metadata

The first thing we need to do to fix the Maven metadata in the SOA 12.2.1.1 Oracle Home.  I am using an Oracle Home created by the SOA 12.2.1.1 Quick Start installer, though the steps should be similar for any Oracle Home containing SOA Suite 12.2.1.1.

The high-level steps are:

  • Fix the parent POM
  • Fix the archetypes
  • Remove the extra, unnecessary Maven metadata files (optional)

Fixing the sar-common Parent POM

The SOA archetypes create Maven projects with their parent set to the com.oracle.soa:sar-common parent POM.  Unfortunately, the SOA 12.2.1.1 version of this POM has a version of 12.2.1-0-0 so the first fix is to set its version to 12.2.1-1-0.

To correct the metadata, do the following steps:

  1. In the $ORACLE_HOME/soa/plugins/maven/com/oracle/soa/sar-common/ directory, make a copy of the 12.2.1-0-0 directory and name it 12.2.1.
  2. In the $ORACLE_HOME/soa/plugins/maven/com/oracle/soa/sar-common/12.2.1/ directory, rename the sar-common-12.2.1-0-0.pom to sar-common-12.2.1.pom.
  3. Open the sar-common-12.2.1.pom in a text editor and change the <version> element value from 12.2.1-0-0 to 12.2.1-1-0.

When finished, the top portion of the sar-common-12.2.1.pom file should look what is shown below.  The change made to the file is highlighted in red.

 <modelVersion>4.0.0</modelVersion>
 <groupId>com.oracle.soa</groupId>
 <artifactId>sar-common</artifactId>
 <version>12.2.1-1-0</version>
 <packaging>pom</packaging>

 <parent>
   <groupId>com.oracle.maven</groupId>
   <artifactId>oracle-common</artifactId>
   <version>12.2.1-1-0</version>
   <relativePath></relativePath>
 </parent>

At this point, you have made all the changes needed for this artifact.  Optionally, you can delete the following directories as they are not needed:

  • $ORACLE_HOME/soa/plugins/maven/com/oracle/soa/sar-common/12.2.1-0-0
  • $ORACLE_HOME/soa/plugins/maven/com/oracle/soa/sar-common/12.2.2-0-0

Next, you need to update the two SOA archetypes.

Fixing the SOA Archetypes

SOA has two main archetypes:

  • oracle-soa-project
  • oracle-soa-application

Fixing the archetypes is a bit more involved since we need to edit files inside of the archetype JAR in addition to its POM.

Fixing the oracle-soa-project Archtetype

Again, we need to change the version numbers from 12.2.1-0-0 to 12.2.1-1-0 so that the archetype references the proper artifacts that exist in the Oracle Home.  To correct the archetype’s metadata, do the following steps:

  1. In the $ORACLE_HOME/soa/plugins/maven/com/oracle/soa/archetype/oracle-soa-project directory, make a copy of the 12.2.1-0-0 directory and name it 12.2.1.
  2. In the $ORACLE_HOME/soa/plugins/maven/com/oracle/soa/archetype/oracle-soa-project/12.2.1/ directory, rename the JAR and POM files to:
    • oracle-soa-project-12.2.1.jar
    • oracle-soa-project-12.2.1.pom
  3. Open oracle-soa-project-12.2.1.pom in a text editor and change the <version> element value from 12.2.1-0-0 to 12.2.1-1-0.

When finished, the top portion of the oracle-soa-project-12.2.1.pom file should look what is shown below.  The change made to the file is highlighted in red.

 <modelVersion>4.0.0</modelVersion>

 <groupId>com.oracle.soa.archetype</groupId>
 <artifactId>oracle-soa-project</artifactId>
 <version>12.2.1-1-0</version>
 <packaging>jar</packaging>

Now, we are ready to edit the files in the oracle-soa-project-12.2.1.jar by doing the following steps:

  1. Open a shell in the $ORACLE_HOME/soa/plugins/maven/com/oracle/soa/archetype/oracle-soa-project/12.2.1/ directory.
  2. Make a directory called temp and change into the newly created directory.
  3. Extract the JAR contents by running the following command:jar xf ../oracle-soa-project-12.2.1.jar
  4. In the temp/archetype-resources/ directory, open pom.xml in a text editor and change:
    1. The <parent> stanza’s <version> from 12.2.1-0-0 to 12.2.1-1-0.
    2. In the <Build> section, change the oracle-soa-plugin <version> from 12.2.1-0-0 to 12.2.1-1-0.
  5. In the temp/META-INF/maven/com.oracle.soa.archetype/oracle-soa-project/ directory, open pom.properties in a text editor and change the version property value from 12.2.1-0-0 to 12.2.1-1-0.
  6. In the temp/META-INF/maven/com.oracle.soa.archetype/oracle-soa-project/ directory, open pom.xml in a text editor and change the <version> element’s value from 12.2.1-0-0 to 12.2.1-1-0.
  7. From the temp directory, run the following commands to replace the oracle-soa-project-12.2.1.jar with its updated contents:
    1. rm ../oracle-soa-project-12.2.1.jar
    2. jar cf ../oracle-soa-project-12.2.1.jar *
    3. cd ..
    4. rm -rf temp

At this point, you have made all the changes needed for this archetype.  Optionally, you can delete the following directories as they are not needed:

  • $ORACLE_HOME/soa/plugins/maven/com/oracle/soa/archetype/oracle-soa-project/12.2.1-0-0
  • $ORACLE_HOME/soa/plugins/maven/com/oracle/soa/archetype/oracle-soa-project/12.2.2-0-0

Fixing the oracle-soa-application Archetype

Again, we need to change the version numbers from 12.2.1-0-0 to 12.2.1-1-0 so that the archetype references the proper artifacts that exist in the Oracle Home.  The steps required are very similar to the oracle-soa-project archetype.  To correct the archetype’s metadata, do the following steps:

  1. In the $ORACLE_HOME/soa/plugins/maven/com/oracle/soa/archetype/oracle-soa-application directory, make a copy of the 12.2.1-0-0 directory and name it 12.2.1.
  2. In the $ORACLE_HOME/soa/plugins/maven/com/oracle/soa/archetype/oracle-soa-application/12.2.1/ directory, rename the JAR and POM files to:
    • oracle-soa-application-12.2.1.jar
    • oracle-soa-application-12.2.1.pom
  3. Open oracle-soa-application-12.2.1.pom in a text editor and change the <version> element value from 12.2.1-0-0 to 12.2.1-1-0.

When finished, the top portion of the oracle-soa-project-12.2.1.pom file should look what is shown below.  The change made to the file is highlighted in red.

 <modelVersion>4.0.0</modelVersion>

 <groupId>com.oracle.soa.archetype</groupId>
 <artifactId>oracle-soa-application</artifactId>
 <version>12.2.1-1-0</version>
 <packaging>jar</packaging>

Now, we are ready to edit the files in the oracle-soa-application-12.2.1.jar by doing the following steps:

  1. Open a shell in the $ORACLE_HOME/soa/plugins/maven/com/oracle/soa/archetype/oracle-soa-application/12.2.1/ directory.
  2. Make a directory called temp and change into the newly created directory.
  3. Extract the JAR contents by running the following command:jar xf ../oracle-soa-application-12.2.1.jar
  4. In the temp/archetype-resources/__projectName__/ directory, open pom.xml in a text editor and change:
    1. The <parent> stanza’s <version> from 12.2.1-0-0 to 12.2.1-1-0.
    2. In the <Build> section, change the oracle-soa-plugin <version> from 12.2.1-0-0 to 12.2.1-1-0.
  5. In the temp/META-INF/maven/com.oracle.soa.archetype/oracle-soa-application/ directory, open pom.properties in a text editor and change the version property value from 12.2.1-0-0 to 12.2.1-1-0.
  6. In the temp/META-INF/maven/com.oracle.soa.archetype/oracle-soa-application/ directory, open pom.xml in a text editor and change the <version> element’s value from 12.2.1-0-0 to 12.2.1-1-0.
  7. From the temp directory, run the following commands to replace the oracle-soa-project-12.2.1.jar with its updated contents:
    1. rm ../oracle-soa-application-12.2.1.jar
    2. jar cf ../oracle-soa-application-12.2.1.jar *
    3. cd ..
    4. rm -rf temp

At this point, you have made all the changes needed for this archetype.  Optionally, you can delete the following directories as they are not needed:

  • $ORACLE_HOME/soa/plugins/maven/com/oracle/soa/archetype/oracle-soa-application/12.2.1-0-0
  • $ORACLE_HOME/soa/plugins/maven/com/oracle/soa/archetype/oracle-soa-application/12.2.2-0-0

At this point, all required editing of the SOA 12.2.1.1 Maven metadata is complete.  Before moving on, let’s look at one last optional step for removing soem extra baggage that is not needed.

Removing Unnecessary Maven Metadata

The last step is to remove extra versions of the oracle-soa-plugin that are not needed.  Simple do the following steps:

  • Delete the $ORACLE_HOME/soa/plugins/maven/com/oracle/soa/plugin/oracle-soa-plugin/12.2.1-0-0/ directory
  • Delete the $ORACLE_HOME/soa/plugins/maven/com/oracle/soa/plugin/oracle-soa-plugin/12.2.2-0-0/ directory
  • Rename the $ORACLE_HOME/soa/plugins/maven/com/oracle/soa/plugin/oracle-soa-plugin/12.2.1-1-0/ directory to $ORACLE_HOME/soa/plugins/maven/com/oracle/soa/plugin/oracle-soa-plugin/12.2.1/
  • Rename the files within the $ORACLE_HOME/soa/plugins/maven/com/oracle/soa/plugin/oracle-soa-plugin/12.2.1/ directory as follows:
    • oracle-soa-plugin-12.2.1.jar
    • oracle-soa-plugin-12.2.1.pom

At this point, we are ready to repopulate the Maven repository with the corrected artifacts.

Syncing the Maven Repository

Now that the SOA 12.2.1.1 Oracle Home’s Maven metadata is correct, we simply need to push it into the Maven repository using the oracle-maven-sync plugin.

Before doing this, make sure that the oracle-maven-sync plugin is installted in the Maven repository by doing the following steps:

  1. Open a shell in the $ORACLE_HOME/oracle_common/plugins/maven/com/oracle/maven/oracle-maven-sync/12.2.1/ directory.
  2. Run the command
    mvn install:install-file -Dfile=oracle-maven-sync-12.2.1.jar -DpomFile=oracle-maven-sync-12.2.1.pom

Now, run the oracle-maven-sync plugin to push the corrected Maven artifacts into the Maven repository like this:

mvn com.oracle.maven:oracle-maven-sync:12.2.1-1-0:push -DoracleHome=/path/to/soa/oracle/home

If you happen to get a failure from the oracle-maven-sync plugin because of duplicate POMs, simply re-run the command above after adding the -DpushDuplicates=true argument.

At this point, we are ready to test the work we have done.

Create a New SOA Application

While creating or fixing broken SOA applications and projects is beyond the scope of this article, we do need a simple way to test that our fix was successful.  As such, we will use the oracle-soa-application archetype to create a new SOA application and then package the resulting SAR to make sure that the oracle-soa-plugin is able to find all of its dependencies.

To test our work, open a shell somewhere (e.g., /tmp) and run the following commands

  1. mvn archetype:crawl -Dcatalog=~/.m2/archetype-catalog.xml
  2. mvn archetype:generate -DinteractiveMode=false
    -DarchetypeGroupId=com.oracle.soa.archetype
    -DarchetypeArtifactId=oracle-soa-application
    -DarchetypeVersion=12.2.1-1-0
    -DgroupId=test
    -DartifactId=soa-test
    -Dversion=1.0
    -DprojectName=soa-project
  3. cd soa-test
  4. mvn  package

In this article, we learned about a problem in the Oracle SOA Suite 12.2.1.1 Maven functionality and how to restore functionality ourselves.  Remember, this is not an official Oracle-supported solution and anyone looking for such needs to contact Oracle Support to obtain an official patch.

Restoring OSB 12.2.1 Maven Functionality

It seems that testing of the Oracle Service Bus (OSB) Maven functionality in the new 12.2.1 release failed to catch a few issues that make the OSB Maven plugin unusable out of the box.  Oracle is aware of the issue and working to create a patch for this.  In the meantime, users can work around the problem with a few simple changes.  This blog documents those changes.

Fixing the com.oracle.servicebus:client POM

The first change we need to make is to edit the com.oracle.servicebus:client POM.  This POM can be found at ${ORACLE_HOME}/osb/plugins/maven/com/oracle/servicebus/client/12.2.1/client-12.2.1.pom.  Open this file in a text editor and make the following changes.

  1. Delete the <dependency> stanza for the com.oracle.weblogic:wlthint3client.  This artifact does not exist.  Fortunately, it is is not needed since the weblogic-server-pom dependency provides all of the connectivity to WebLogic Server that is required.
  2. In the weblogic-server-pom dependency, change the <version> element’s value from “LATEST” to “[12.2.1,12.2.2)” (without the double quotes).
  3. In the com-bea-core-xml-xmlbeans dependency, change the <artifactId> element’s value from “com-bea-core-xml-xmlbeans” to “com.bea.core.xml.xmlbeans” (without the double quotes).
  4. In the com-bea-core-xml-xmlbeans dependency, change the <version> element’s value from “LATEST” to “[12.2.1,12.2.2)” (without the double quotes).

After making these changes, the relevant section of the file should look like the one shown here.

...
<dependencies>
    <dependency>
        <groupId>com.oracle.weblogic</groupId>
        <artifactId>weblogic-server-pom</artifactId>
        <version>[12.2.1,12.2.2)</version>
        <type>pom</type>
    </dependency>
    <dependency>
        <groupId>com.oracle.weblogic</groupId>
        <artifactId>com.bea.core.xml.xmlbeans</artifactId>
        <version>[12.2.1,12.2.2)</version>
    </dependency>
    <dependency>
        <groupId>com.oracle.osb.common</groupId>
        <artifactId>oracle.servicebus.common</artifactId>
        <version>[12.2.1,12.2.2)</version>
    </dependency>
    ...

Fixing the com.oracle.servicebus:sbar-project-common Parent POM

The next change we need to make is to edit the com.oracle.servicebus:sbar-project-common POM.  This POM can be found at ${ORACLE_HOME}/osb/plugins/maven/com/oracle/servicebus/sbar-project-common/12.2.1/sbar-project-common-12.2.1.pom.  Open this file in a text editor and make the following change.

  1. Change the <parent> section’s <version> element value from “12.1.4-0-0” to “12.2.1-0-0” (without the double quotes).

After making these changes, the relevant section of the file should look like the one shown here.

... 
<parent>
    <groupId>com.oracle.maven</groupId>
    <artifactId>oracle-common</artifactId>
    <version>12.2.1-0-0</version>
</parent>
...

Fixing the com.oracle.servicebus:sbar-system-common Parent POM

The last change we need to make is to edit the com.oracle.servicebus:sbar-system-common POM.  This POM can be found at ${ORACLE_HOME}/osb/plugins/maven/com/oracle/servicebus/sbar-system-common/12.2.1/sbar-system-common-12.2.1.pom.  Open this file in a text editor and make the following change.

  1. Change the <parent> section’s <version> element value from “12.1.4-0-0” to “12.2.1-0-0” (without the double quotes).

After making these changes, the relevant section of the file should look like the one shown here.

... 
<parent>
    <groupId>com.oracle.maven</groupId>
    <artifactId>oracle-common</artifactId>
    <version>12.2.1-0-0</version>
</parent>
...

Populating the Maven Repository with the Changes

Now that you have made all of the required changes, simply run the com.oracle.maven:oracle-maven-sync plug-in to synchronize your Maven repository with the changed artifacts.  If you are unfamiliar with the Oracle Maven Sync plug-in, please refer to the documentation here.  If you already have the plug-in installed, simply run the command shown below (replacing the “${ORACLE_HOME}” with the actual path to your OSB 12.2.1 installation directory).

mvn com.oracle.maven:oracle-maven-sync:12.2.1-0-0:push -DoracleHome=${ORACLE_HOME}

At this point, the OSB Maven functionality should be restored for the local machine. If you have a Maven Repository Manager, you can use the Oracle Maven Sync plug-in to populate it with the corrected artifacts.

WLST in Fusion Middleware 12.2.1

Starting in Fusion Middleware (FMW) 12.2.1, Oracle has made changes to the WebLogic Scripting Tool (WLST) to simplify its usage.  With these changes, it becomes easier to write provisioning scripts in environments where multiple FMW products are in use.  This article tries to capture some of the more important changes for WLST users to understand.

Which WLST to Use?

In previous versions of FMW, each product suite had its own customized version of WLST.  As discussed in the How to use WLST as a Jython 2.7 module blog,1 figuring out which product’s version of WLST to use and what the specific environment it creates can be challenging.  For example, if you want to write scripts that interact with OSB, you need to use  $ORACLE_HOME/osb/common/bin/wlst.sh to execute them but if your scripts interact with SOA Suite (e.g., BPEL), you must use $ORACLE_HOME/soa/common/bin/wlst.sh.  These product-specific shell scripts customize the CLASSPATH and loads the product’s WLST scripts, if any, from the $ORACLE_HOME/<product-home>/common/wlst/ directory before delegating to  $ORACLE_HOME/oracle_common/common/bin/wlst.sh.  In turn, the $ORACLE_HOME/oracle_common/common/bin/wlst.sh script also customizes the CLASSPATH and loads its WLST scripts (except in the WLS only installation case) before running WLST.  This complexity makes it very difficult to write WLST scripts that manipulate multiple products since doing so would require creating a custom WLST environment that combines the environments of all of the relevant products.

Starting in FMW 12.2.1, Oracle has unified WLST so that there is a single version of WLST that automatically loads the necessary environment for all products installed in the Oracle Home from which WLST was invoked.  The new unified version of WLST is accessed by running $ORACLE_HOME/oracle_common/common/bin/wlst.sh.  There are a number of changes that were made to support this, each of which I will discuss in the subsequent sections.

The Java CLASSPATH

The first thing users who inspect the unified WLST script will notice is that the CLASSPATH is greatly simplified.  In almost all cases, the CLASSPATH contains only two JAR files:

  1. $JAVA_HOME/lib/tools.jar
  2. $ORACLE_HOME/wlserver/modules/features/wlst.wls.classpath.jar

The wlst.wls.classpath.jar contains the base WLST classpath required for WLS support and does not change regardless of what other products might be installed in the ORACLE_HOME.  How is WLST able to support other products with only the base WLS classes in the Java CLASSPATH?  The answer is that it does this by moving WLST customization from a twisty maze of shell scripts for each product to a plugin model for WLST extensions.  Before we talk about the new WLST plugin model, let’s look at another user visible change.

The WLST PY Scripts

As mentioned previously, previous versions of WLST loaded a product’s custom PY scripts out of their $ORACLE_HOME/<product-home>/common/wlst/ directory at WLST startup.  Which products’ common/wlst directories would be loaded at startup was controlled by the path specified using the weblogic.wlstHome Java System property.  Each product’s customized WLST scripts modified the value of this property to add their own common/wlst directory.

Starting in 12.2.1, you will notice that the common/wlst directories no longer contain the PY scripts that were there in previous releases.  Instead, these PY files are now packaged inside of JAR files.  For example, the wlstModule.py and wlstModule_core.py files that previously existed in the $ORACLE_HOME/wlserver/common/wlst/modules/ directory are no longer in the file system in WLS 12.2.1 (they are packaged and loaded from inside one of the base WLST JAR files).  By packaging them inside the core WLST JAR, WLST ensures that these PY files are available prior to loading any WLST extension containing PY files that might depend on wlstModule.py or wlstModule_core.py.

At startup, WLST scans its CLASSPATH looking for “/wlstScriptDir” resources (e.g., any JAR that contains a top-level directory named “wlstScriptDir”).  It treats each wlstScriptDir that it finds the same as it previously would have treated a common/wlst directory on the search path specified by the weblogic.wlstHome property.  That is:

  • Any wlstScriptDir/modules/ directory is added to the Jython path so that other scripts that import modules can find the PY scripts in these directories
  • Any PY files in the wlstScriptDir/lib/ directory are imported by WLST at startup
  • Any PY files in the wlstScriptDir/ directory itself are executed by WLST at startup

This not only simplifies the WLST extension model (e.g., no need for the weblogic.wlstHome property), it also allows WLST to work better with build tools like Maven (more about that later).  Now, let’s look at the WLST plugin model.

The WLST Plugin Model

Starting in FMW 12.2.1, WLST now uses a plugin model to load a single, Oracle Home-wide WLST environment.  It does this through a plugin model and scanning the Oracle Home for all WLST plugins.  What exactly is a WLST plugin?

A WLST plugin is a JAR file in a specific location that includes the classes and PY files that WLST loads at startup.  The JAR may contain the classes and/or PY files directory, or it may reference some or all of them via a Manifest Class-Path entry in the JAR.  The PY files should be appropriately located in the JARs’ top-level wlstScriptDir directories, as described previously.

A Note about Ordering

In Java, the order of entries on the CLASSPATH typically doesn’t matter (unless there are duplicate artifacts across the different JARs in the CLASSPATH).  The reason for this is because Java loads each class on demand by searching the CLASSPATH in the order in which it was defined.  WLST PY file loading works differently.

At startup, WLST searches the CLASSPATH for entries that contain “/wlstScriptDir” by calling ClassLoader.getResources(), which returns the entries in a CLASSPATH-determined order.  It then proceeds to process the wlstScriptDir entries, as previously described, in order.  What this means is that if a PY file in one JAR file depends on modules or functions defined by a PY file in another JAR, the PY file that depends on externally-defined modules or functions must be loaded after the modules on which it depends.  The primary way to accomplish this is by CLASSPATH ordering to make sure the externally-defined modules appear earlier in the CLASSPATH than those depending on them.  This is important to remember as we discuss what happens during WLST startup.

The WLST Startup Process

When starting a WLST interpreter in FMW 12.2.1 (or newer), WLST will search the Oracle Home to locate all of the WLST plugins installed in the Oracle Home and put them on the “WLST CLASSPATH” (more about that in a minute).  In general, the WLST plugin model doesn’t provide ordering guarantees between JARs in the same location or across locations.  However, there are two important exceptions to this rule:

  1. The $ORACLE_HOME/oracle_common/plugins/wlst/ directory will always be processed first, assuming it exists.
  2. The $ORACLE_HOME/oracle_common/plugins/wlst/fmwshare-wlst-dependencies.jar, if it exists, will always be the first JAR on the “WLST CLASSPATH.”

These exceptions allow FMW products to rely on shared code in a WLST environment.2

Once the “WLST CLASSPATH” is determined, WLST continues its startup process including searching the “WLST CLASSPATH” for PY files to process.  Let’s take a closer look at the “WLST CLASSPATH” and how it works

The WLST CLASSPATH

Starting with WLST in FMW 12.2.1, WLST uses a custom ClassLoader to load the WLST plugin JARs.  It is important to understand how this works to prevent typical classloading errors that can occur.  During the WLST startup process, WLST creates the following ClassLoader hierarchy:3

Java System ClassLoader
	- WLS "Descriptor" ClassLoader
	        - WLST Plugin ClassLoader

Now, let’s look at what is loaded by each ClassLoader.

Java System ClassLoader

When running the provided shell script $ORACLE_HOME/oracle_common/common/bin/wlst.sh (or wlst.cmd), the shell script typically sets the Java CLASSPATH to include the following JARs:

  1. $JAVA_HOME/lib/tools.jar
  2. $ORACLE_HOME/wlserver/modules/features/wlst.wls.classpath.jar

These two JARs comprise the base classes needed to start a bare-bones WLST interpreter.  In previous releases, the wlst.wls.classpath.jar did not exist and instead, weblogic.jar and a few other JARs were used to set the CLASSPATH.  This simplification was introduced to make it easier for customers using “java weblogic.WLST” or embedded WLST to ensure that they get the same CLASSPATH as the WLST shell scripts.2

WLS Descriptor ClassLoader

The WLS Descriptor ClassLoader is not new.  For those of you who have written custom WLS Security providers, this classloader is the secret sauce behind the semantics of the $ORACLE_HOME/wlserver/server/lib/mbeantypes directory.  It loads all of the security provider JARs in this location.  I mention it here only because it now sits between the Java System ClassLoader and the WLST Plugin ClassLoader in the classloading hierarchy.

WLST Plugin ClassLoader

The WLST Plugin model works because of the WLST Plugin ClassLoader.  The plugin JARs found during WLST startup are used to create the “WLST CLASSPATH” this classloader uses.  In general, the classloading hierarchy will be completely invisible to most WLST users.  However, it is critical to understand in at least two situations:

  1. Any modification of the Java CLASSPATH (used by the Java System ClassLoader) should not include classes used in a WLST plugin or that uses classes in a WLST plugin.
  2. Any users creating WLST plugins must understand that they cannot assume their classes are loaded by the Java System ClassLoader.  This impacts certain use cases for built-in Java classes where the developers need to use the methods in those packages designed to specify the classloader in use (e.g., java.util.ResourceBundle).

Should you run into problems with this new plugin model, or just happen to be curious, you can turn on WLST initialization debugging.  Setting the wlst.debug.init Java System property to true will cause WLST to spit out a lot of debug-level information during startup.  Inside this information, you can see the JARs it finds with wlstScriptDir entries and the order in which they are being loaded.  To enable this output with the WLST shell script, simply do the following:

WLST_PROPERTIES=-Dwlst.debug.init=true; export WLST_PROPERTIES
$ORACLE_HOME/oracle_common/common/bin/wlst.sh

This concludes our whirlwind tour of the new WLST Plugin mechanism. Now, let’s look at the various ways to use WLST.

WLST Usage

The four different ways to invoke WLST have not fundamentally changed.  They include:

  • Run the provided WLST shell script
  • Run the weblogic.WLST java class
  • Create an embedded WLST interpreter in your Java program
  • Run one of the weblogic-maven-plugin’s WLST-related goals

WLST shell scripts

The wlst.sh (and wlst.cmd) shell scripts are the easiest way to use WLST.  As discussed earlier, previous releases of FMW products had multiple WLST shell scripts, each of which created a customized environment for that product.  Those scripts lived inside the various product directories at common/bin/wlst.sh (and wlst.cmd); for example, $ORACLE_HOME/soa/common/bin/wlst.sh.  In 12.2.1, those scripts have been deprecated.  The scripts that exist in those locations now print out a deprecation message and call the WLST script in $ORACLE_HOME/oracle_common/common/bin.

Another visible change to the WLST shell scripts is that they no longer use the value of the CLASSPATH environment variable coming from the environment used to execute them.  The reason for this is simply to try to prevent accidentally adding unintended classes to the Java CLASSPATH, which has the potential to introduce insidious classloading issues due to the new WLST Plugin model and its classloader hierarchy.  If you really want to modify the Java CLASSPATH being used to run WLST, simply set the WLST_EXT_CLASSPATH environment variable prior to invoking the shell script and its value will be prepended to the Java CLASSPATH used to start WLST.

As in previous releases, the WLST shell scripts honor the WLST_PROPERTIES environment variable as a mechanism to add other arguments (e.g., Java System Property definitions) to the Java command-line used to start WLST.

java weblogic.WLST

Some long-time WLS users are used to running setWLSEnv.sh and then running “java weblogic.WLST” to start WLST.  While still fully supported, there is really not a lot of reason to do this these days.  Starting WLST in this way is not guaranteed to provide a proper WLST environment for using other FMW products, though the changes in 12.2.1 make tremendous improvements in this area. In all cases, it is the user’s responsibility to ensure the proper environment exists when starting WLST this way.2

Embedded WLST

Embedded WLST is an API that allows users to create a WLST interpreter inside their own Java program and send commands to it to execute Jython code.  Changes in 12.2.1 make this task much easier than in previous releases by:

  • Simplifying the Java CLASSPATH required to run WLST
  • Eliminating most of the Java System properties required to run WLST

To run your Java program using embedded WLST, you only need to emulate the environment created by the WLST shell script.  For example, the following is an excerpt from a script that should provide a fully functional WLST environment for your custom Java application.

APP_CLASSPATH=/path/to/myapp.jar
JAVA_HOME=/path/to/my/jdk/installation
ORACLE_HOME=/path/to/my/oracle/installation
CLASSPATH=${JAVA_HOME}/lib/tools.jar:${ORACLE_HOME}/wlserver/modules/features/wlst.wls.classpath.jar:${APP_CLASSPATH}
PATH=${JAVA_HOME}/bin:${PATH}
export JAVA_HOME ORACLE_HOME CLASSPATH PATH

java -DORACLE_HOME=${ORACLE_HOME}/oracle_common com.example.MyApp $@

Now, in your Java application, you can simply do something like the following.

import java.util.Hashtable;
import weblogic.management.scripting.util.WLSTInterpreter;

...

Hashtable wlstProperties = new Hashtable();
wlstProperties.put(WLSTInterpreter.ENABLE_SCRIPT_MODE, Boolean.TRUE);
WLSTInterpreter wlst = new WLSTInterpreter(wlstProperties);
wlst.exec("connect(wlsUser, wlsUserPassword, wlsUrl)");

...

wlst.exec("disconnect()");

Just like running “java weblogic.WLST” directly, any user of embedded WLST is responsible for providing the required WLST environment prior to creating a WLSTInterpreter, as shown above.2

The weblogic-maven-plugin’s WLST-Related Goals

Starting in WebLogic Server 12.1.2, Oracle has provided a WebLogic Server Maven plugin that works the way Maven users would expect (e.g., uses Maven dependency management, provides POMs for WebLogic Server, etc.).4  This plugin has support for running WLST scripts from a Maven build.  Current versions of the plugin provide two WLST-related goals:

  • wlst – This goal allows the user to run the WLST shell script from their locally-installed Oracle Home.
  • wlst-client – This goal allows the user to run WLST scripts directly from Maven where WLST is executing from JARs in the user’s local Maven repository rather than a locally-installed Oracle Home.

The wlst Goal

In 12.2.1, the weblogic-maven-plugin’s wlst goal is mostly unchanged and supports all WLST use cases.  The one minor change is that the goal no longer honors the wlstVersion configuration parameter, which provided the user control over which WLST shell script the goal would run.  Given the WLST Unification changes we have been discussing, that is no longer useful and the goal will always run the WLST shell script in $ORACLE_HOME/oracle_common/common/bin.  The rest of this section is devoted to discussing the wlst-client goal and how it works with the new 12.2.1 WLST changes.

The wlst-client Goal

The wlst-client goal was introduced in WLS 12.1.3 as an alternative way to invoke WLST scripts from Maven.  The primary advantage of this goal was that it did not require a locally-installed Oracle Home to execute WLST online scripts.  That means that Maven builds could run WLST scripts against a remote server installation without having WebLogic Server installed on the local machine.

WLST offline scripts still required a local Oracle Home.  However, when you think about it, that is not really a limitation since WLST offline scripts, by definition, execute on a locally-available domain directory.  Since every domain directory depends on a locally-available Oracle Home, there is little reason to try to support WLST offline operations without a locally-available Oracle Home.

Starting in 12.2.1, the wlst-client goal fully supports both online and offline scripts for all products.  The execution of offline commands requires setting the middlewareHome configuration parameter to point to the locally-available Oracle Home.  If it is not set and an offline command is encountered, the goal will fail with an error stating that offline commands are not supported (due to the lack of a known location for a locally-available Oracle Home).

It is important to note that the wlst-client goal does NOT leverage the WLST Plugin classloading mechanism on WLST startup.  Instead, it relies on the normal Maven dependency mechanisms.  To run a WLST script that, for example, manipulates an Oracle SOA Suite domain, the Maven POM’s plugin declaration must add dependencies on the required WLST Plugin artifacts.  Since SOA Suite’s WLST environment also depends on the FMW infrastructure WLST environment, the Maven POM’s plugin’s declaration has to include both the fmwshare-wlst-dependencies and soa-wlst-dependencies artifacts as dependencies.  Due to the PY file ordering problem we discussed earlier, the fmwshare-wlst-dependencies dependency MUST come before the soa-wlst-dependencies declaration, as shown here.

<build>  
  <plugins>
    <plugin>  
      <groupId>com.oracle.weblogic</groupId>
      <artifactId>weblogic-maven-plugin</artifactId>
      <version>12.2.1-0-0</version>
      <configuration>
        <middlewareHome>/path/to/my/oracle/installation</middlewareHome>
      </configuration>
      <executions>
        <execution>
          <id>create-my-soa-domain</id>
          <phase>pre-integration-test</phase>
          <goals>
            <goal>wlst-client</goal>
          </goals>
          <configuration>
            <fileName>${project.basedir}/src/test/scripts/createSoaDomain.py</fileName>
            <scriptArgs>
              <arg>/path/to/soa/domain</arg>
              <arg>${oracleAdminUser}</arg>
              <arg>${oracleAdminPassword}</arg>
            </scriptArgs>
          </configuration>
        <execution>
        ...
      </executions>
      <dependencies>
        <dependency>
          <groupId>com.oracle.fmwshare</groupId>
          <artifactId>fmwshare-wlst-dependencies</artifactId>
          <version>12.2.1-0-0</version>
        </dependency>
        <dependency>
          <groupId>com.oracle.soa<groupId>
          <artifactId>soa-wlst-dependencies</artifactId>
          <version>12.2.1-0-0</version>
        </dependency>
      </dependencies>
    </plugin>
    ...
  </plugins>
</build>

WLST Best Practices

In working with WLST, there are a few best practices that should be kept in mind to make your WLST scripts as portable and modular as possible.

Always Use Explicit Imports for Java Classes

Historically, WLST has gone out of its way to make it easy to use WebLogic Server Java classes by scanning a set of WebLogic Server JARs at WLST startup.  Doing this allows WLST users to write scripts that reference the scanned classes without importing them.  For example, one could write the following script to get information about the WLS installation without explicitly importing any Java classes.

...
wlHomePath = weblogic.Home.getPath()
wlVersion = weblogic.version.getReleaseBuildVersion()
...

However, doing this reduces the portability of the WLST script due to its reliance on WLST package scanning.  For example, WLST package scanning does not work properly when running the wlst-client goal from Maven, due to the WebLogic Server JARs not being in the expected directory structure when executing from the local Maven repository.  A simple change to the script, shown below, will ensure that the WLST script works anywhere.

import weblogic.Home as wlsHome
from weblogic import version as wlsVersion
...
wlHomePath = wlsHome.getPath()
wlVersion = wlsVersion.getReleaseBuildVersion()
...

Note that when running the above script with wlst-client goal, the wlHomePath variable will be set to None if the middlewareHome configuration parameter has not been supplied.

WLST 12.2.1 deprecates this WebLogic Server package scanning capability so it is a good idea to always use explicit imports even if you have no intention of using WLST from Maven.

Import wlstModule for Modularity

WLST provides a set of functions and global variables for WLST scripts to use for working with the WebLogic Server MBean servers.  While basic scripts and command-line use allow the use of these functions directly, doing so in your WLST scripts will reduce their modularity.  The easiest way to explain this is to look at an example.

Imagine that your company does not allow data center operations staff to access WebLogic Server administrative functions requiring the administrator credentials directly but instead requires the use of provided WLST scripts to do so.  Imagine that you, as the WLST script developer, have devised a secure way to retrieve the administrator credentials and connect to the production Admin Server.  This code might look something like the following.

adminUrl = sys.argv[1]
wlUser = ... # secure retrieval of admin user
wlPassword = ... # secure retrieval of admin password

connect(wlUser, wlPassword, adminUrl)

This code works great when you embed it directly in your test script.  Now, you want to factor this code out into its own WLST PY function that other scripts use rather than copy.  The natural way in Jython to do this is to create a function in a PY file that your other scripts import. Your first step is to move this into a function inside your test script, as shown here.

def wlsConnect(adminUrl):
    wlUser = ... # secure retrieval of admin user
    wlPassword = ... # secure retrieval of admin password
    connect(wlUser, wlPassword, adminUrl)

This all seems to work fine so now the next step is to move the function (and others) into their own PY file that can be imported into other scripts. Imagine that you created a wlsAdminUtils.py file and added your function to it. Now, you can simply do this in all of your scripts needing an administrative connection to the server.

import wlsAdminUtils
...
wlAdminUrl = sys.argv[1]
wlsConnect(wlAdminUrl)
...

That all looks good but to your surprise, when you try to run your new script, you get a “Jython NameError: connect” pointing to the call to the WLST connect() function in your wlsAdminUtils.py-defined wlsConnect function. This is due to the fact that the code is now running in a different context, due to the import, than it was previously. Fortunately, the fix is simple and will allow the function to run in any context: You need to import the wlstModule and use it to qualify the name of the WLST functions, as shown here.

import wlstModule as wlst

def wlsConnect(adminUrl):
    wlUser = ... # secure retrieval of admin user
    wlPassword = ... # secure retrieval of admin password
    wlst.connect(wlUser, wlPassword, adminUrl)

Importing wlstModule in your WLST scripts will allow the script to be executed directly or imported without breaking.

Avoid Use of WLST Global Variables for Modularity

WLST global variables have a similar modularity problem to the one described in the previous section.  However, there is no easy fix for the WLST global variables as there was for WLST functions.  If you try to import a script that uses WLST global variables into your script, the same Jython NameError will result for the variable name.  Importing wlstModule and qualifying the variable with it will eliminate the NameError but it creates another problem.

The problem is that this “global” variable is recreated in the new imported script’s context.  That is, the global variable is not truly global and now you have the same variable defined in two different contexts.  This can cause the value of the global variable to get out of sync so that its value can no longer be trusted.  For example, the WLST variable mbs is supported to contain a reference to the MBeanServerConnection corresponding to the current location in the MBean hierarchy.  Imagine a scenario where your outer.py script calls the WLST connect() function and then calls a function in your imported inner.py module that uses the mbs global variable.  If inner.py imports wlstModule and uses it to reference the mbs variable, the mbs variable value in outer.py will be the MBeanServerConnection object you expect but inside inner.py, the mbs variable value will be None.

Fortunately, most scripts can be written without explicitly using the WLST global variables.  For example, it is common to see WLST scripts that do something like this following.

cd(someLocation)
cmo.setListenAddress('7001')

The previous code using the WLST cmo global variable can easily be replaced as follows.

cd(someLocation)
set('ListenAddress', '7001')

That way, you can write your script using wlstModule as follows so that the script could be imported by some other script at a later point in time without the script breaking, as shown here.

import wlstModule as wlst
...
wlst.cd(someLocation)
wlst.set('ListenAddress', '7001')

This concludes our discussion of using WLST in 12.2.1.  The next section talks about some thoughts I have around possible enhancements to make WLST even easier to use.

Possible Enhancements

Before I discuss my ideas for making WLST even easier to use, I am required to remind you of the following facts:

1.) All views expressed on this site are completely my own and in no way represent the views of my employer.

2.) Any ideas expressed in this section are completely my own and in no way represent any form of promise or commitment by my employer to make any of these changes to their products at any point in the future.

With those caveats, here are a few ideas for how the changes in WLST 12.2.1 could be further enhanced to make things easier for its users.

Simplifying the WLST environment

WLST 12.2.1 has gone a long way toward unifying and simplifying the WLST environment for users by doing the following.

  • Eliminating the need for users to determine the proper WLST shell script to use to start the WLST environment needed to execute their WLST scripts by unifying the WLST environment under a single script in the $ORACLE_HOME/oracle_common/common/bin directory.
  • Reducing the need for users of “java weblogic.WLST” and embedded WLST to reverse engineer the WLST shell scripts to determine the necessary WLST environment required to successfully run their WLST scripts.  The WLST plugin model, the codification of the WLST classpath in the wlst.wls.classpath.jar, and the elimination of most of the required Java System properties made a huge impact in this area.
  • Allowing Maven users to executing any WLST script using the wlst-client goal.

One thing that would simplify the use of “java weblogic.WLST” or embedded WLST even further would be to create an explicit setWlstEnv script that would work in the same way and set up the exact same environment as the WLST shell scripts do.  This would allow those users to simply call that script to prepare their environment before running the relevant Java program.

Another thing that would help Maven users would be to create a single Maven artifact per product that a user could add to their POM’s weblogic-maven-plugin dependency declarations to eliminate the error-prone approach where the user needs to understand each product’s WLST dependencies and the order in which they must appear to successfully execute WLST scripts from Maven.

Simplifying User Extensions to WLST

The new WLST Plugin model simplifies the use of WLST by unifying the entire Oracle Home’s WLST environment.  It allows Oracle Products to easily extend WLST with their modifications as they are installed into an existing Oracle Home.  Users can even create their own WLST Plugin JARs and add them to the Oracle Home to load their own customizations into WLST.  However, there are a couple of limitations with the current implementation:

  1. WLST Plugins must be located in the Oracle Home to be loaded.
  2. WLST doesn’t provide the user any load ordering guarantees for their plugins beyond the two discussed earlier.

Putting user-written plugins into the Oracle Home could be viewed as a feature or as a significant limitation.  On the positive side, it prevents users without write access to the Oracle Home from adding plugins to the WLST environment.  However, it also means that the Oracle Home now includes user customizations so if an administrator removes the Oracle Home and reinstalls it, the user customizations could be lost.

One idea is to add one or more other locations outside the Oracle Home where WLST should look for WLST plugins.  These locations could be specified in an environment variable and/or Java system property prior to starting WLST.

The load ordering is not a huge limitation for user-provided scripts provided that they do not rely on other WLST plugins’ modules or functions.  User-provided plugins can safely use any modules or functions defined by WLST plugins in the $ORACLE_HOME/oracle_common/plugins/wlst directory but not any other directories.  That means, for example, a user cannot safely create a WLST plugin with WLST scripts that depend on WLST modules or functions defined by the SOA WLST plugin (due to the lack of load ordering guarantees).

One idea to address this limitation would be to ensure that user-provided WLST plugins are always placed after the Oracle-provided WLST plugins on the WLST CLASSPATH.  Using the previous idea to specify alternate locations for user-provided WLST plugins, WLST could ensure that the user-provided WLST plugins provided with this mechanism are always placed at the end of the WLST CLASSPATH–after all of the Oracle-provided WLST plugins.  By doing this, it would ensure that user-provided WLST plugins could safely rely on any Oracle-defined WLST modules and functions.

Improving the Modularity of WLST Global Variables

As discussed in the WLST Best Practices section, the current WLST global variables have some limitations when it comes to modularity. In Jython, the recommended way of creating truly global variables that work in any context is to put them inside a module and have everyone that uses them import that module. For example, WLST global variables could be moved into the wlstGlobals module so that the variables could be referenced as shown here.

import wlstModule as wlst
import wlstGlobals
...
wlst.cd(someLocation)
wlstGlobals.cmo.setListenAddress('7001')

Since the WLST cd function changes the value of cmo, the example above assumes that the cd function has been changed to import wlstGlobals and sets the value of wlstGlobals.cmo appropriately. This problem with this is that it will break all existing scripts that use the WLST global variables in their unqualified state.

My idea to introduce these new WLST global variables without breaking all existing scripts is simply to make the internal WLST functions that modify the global variable values change both the existing and new copies of the variables. If this could be accomplished, it has the potential to allow existing scripts to continue to work while giving people time to move to the new global variable definitions.


1 I do not recommend following Maarten’s approach in this blog to upgrade WLST to use a newer version of Jython. While you may get it to work for simple things, his step-by-step guide leaves out some important details of what is required to make his approach more robust.
2 See the Possible Enhancements section for more discussion on this topic.
3 There are a few exceptions where this hierarchy is different but other than the one we discuss in the WLST and Maven section, they should be transparent to the user.
4 See the WebLogic Server 12.2.1 documentation for required setup and more detailed information at  https://docs.oracle.com/middleware/1221/wls/WLPRG/maven.htm.