Migration from WebSphere Message Broker / IBM Integration Bus to MuleSoft.com‎ Mule Open Source ESB

Migration from IIB to another ESB vendor

Part of the reason for industry standards to exist is to avoid vendor lock-in.
Such was the premise of JEE, that is mostly fulfilled at present. It is not that hard to take well
written JEE application and migrate it from one vendor JEE run-time to another.
Unlike JEE or BPMN and BPEL in the business process management space, the ESB does not have
any industry standard for the flow markup language, hence all ESB products use proprietary
languages to implement the integration logic.
However migrating from one vendor ESB to another is not a complete rewrite.
Such migration would still be faster than if you had to build that application from scratch. Here is
why:

  • WSDL, XSD, XPath and XSL transformations can be reused with relatively little rework.
  • Overall flow logic already built can be re-implemented.
  • Custom built Java code can potentially be reused.
  • Databases, JMS and MOM configuration and queues can also be easily reused.

Being the convinced proponent of practical approach and persuaded that all white papers and slideware
is nothing but vaporware and hot air talks worth precisely zero if not implemented writing real
executable code, we will apply this to practice.
Let migrate the message flow for the alternative scenario of MySQL to MC and MS Dynamics
CRM integration discussed earlier.

Introduction

The success of ESB has resulted in a flood of ESB products, both open source and proprietary.
Open source projects like Mule ESB leverage the power of open standards and open source
communities.
ESB’s open source community like Mule ESB is made up of hundreds of users and developers with
real-world integration problems. If a developer needs to support an obscure format, and writes a
new connector to do so, they can then contribute the connector back to the codebase.
Likewise, if a problem is found, while MuleSoft offers support for enterprise clients, the bug will
be quickly identified and is supposed to be fixed by the community.

So if You feel that You are ready for going postal open source, then read on !

Scenarii

Natural language description:

We take the Subscriber GBO object from input queue, then do a look up in the MS Dynamics CRM
and if we do not find any Account with the same primary email we create a new one, if we find
exactly one Account we update it in the CRM and if there are more then one accounts with the
same primary email, we send the input message to the error queue for further manual processing.

Diagram :

Implementation considerations

In order to use Mule ESB first of all we have to download and install AnyPoint Studio development
and test execution environment.
Next step is to create the empty integration project
File->New->Mule Project
Project name : GBO2MSFCRU_APP
Then we have to configure connectors to the WebSphere MQ and to MS Dynamics CRM.
Go to the “Global Elements” tab → Press create -> Connector endpoints → WMQ
and capture the necessary parameters for the QMGR connection

In order to test WebSphere MQ connection we have to create four necessary queues.
In our case we are going to reuse queues created for IIB implementation of the flow:
MSMC.GBO2MCFCRU.IN
MSMC.MSCRE.REQ
MSMC.MSUPD.REQ
MSMCERROR
Then we must allow remote connection to the QMGR.

runmqsc QLWSRV2008R2X64VL2
DEFINE CHANNEL (CHANNEL1) CHLTYPE (SVRCONN) TRPTYPE (TCP)
SET CHLAUTH(CHANNEL1) TYPE(ADDRESSMAP) ADDRESS('<qmgr host name>') MCAUSER('<user name>');

or for development purpose only, just disable channel authentication completely:

SET qmgr chlauth(disabled)

Next step is to copy all the jar files from the MQ Install path/java/lib on our classpath and
reference them on Java Build Path of the project


While the connector for MQ is shipped with default installation of AnyPoint Studio, we have to
download and install the MS CRM Connector from AnyPoint connectors update site.

During the next step we must configure the Kerberos connection to the MS Dynamics CRM:

In some cases default connection configuration can not be retrieved and in this case the creation of
krb5.config and jaas config files in the src/main/resources directory might be necessary:
krb5.config

[libdefaults]
default_realm = CORP.ABTIMO.COM
default_tkt_enctypes = rc4-hmac aes128-cts des3-cbc-sha1 des-cbc-md5 des-cbc-crc
default_tgs_enctypes = rc4-hmac aes128-cts des3-cbc-sha1 des-cbc-md5 des-cbc-crc
permitted_enctypes = rc4-hmac aes128-cts des3-cbc-sha1 des-cbc-md5 des-cbc-crc
[realms]
CORP.ABTIMO.COM = {
kdc = 192.168.0.26
default_domain = corp.abrimo.com
}
[domain_realm]
corp.abrimo.com = CORP.ABTIMO.COM
.corp.abrimo.com = CORP.ABTIMO.COM

jaas.config

Kerberos {
com.sun.security.auth.module.Krb5LoginModule required debug=true
}

After successful connector configuration let’s implement the magic logic:
Retrieval of the GBO Subscriber

<?xml version="1.0" encoding="UTF-8"?>
<Subscriber>
<email>titi@toto.com</email>
<id>12345</id>
<title>Me.</title>
<first_name>Alyona</first_name>
<last_name>Subbotina</last_name>
<status>new</status>
<created_ad>13 FEB 1990</created_ad>
<address1>115 East 57th Street</address1>
<address2>11 floor</address2>
<region>NY</region>
<postcode>10022</postcode>
<city>New York</city>
<lang>en</lang>
<os>iOS</os>
<updated_at>3 MAR 2014</updated_at>
<UDID>12345678910111213141516</UDID>
<device>iPhone</device>
<appname>iHit</appname>
<appversion>3.0</appversion>
<country>USA</country>
</Subscriber>

from the input queue:

Store the retrieved object in the flow variable


Transform retrieved XML to JSON then to Java object , with return class java.lang.Object:

Store subscriber’s email to the flow variable :

<set-variable variableName="theEmail" value="#[message.payload.Subscriber.email]" doc:name="Variable"/>

Then retrieve the list of the accounts with the same email from MS Dynamics CRM

Once more store retrieved list and the number of records in it in the flow variables:

<set-variable variableName="accqty" value="#[payload.size()]" doc:name="Variable"/>
<set-variable variableName="qcrmres" value="#[payload]" doc:name="Variable"/>

Next make a choice based on number of accounts retrieved :

Main scenario: Brand new Account.

Retrieve the content of the variable from the flow to the payload

<expression-transformer expression="#[message.payload = flowVars.theGBO; message.payload]" doc:name="Expression"/>

Put the object in the queue for creation.

<wmq:outbound-endpoint queue="MSMC.MSCRE.REQ" connector-ref="wmqConnector" doc:name="MSMC.MSCRE.REQ">
<wmq:transaction action="JOIN_IF_POSSIBLE"/>
</wmq:outbound-endpoint>

Alternative 1: Account already exists.

Put the account number of the existing Account to update in the flow variable.
Retrieve the payload from flow variable in the same way as for the main scenario
Transform the GBO to the ASBO CRMUpdate, put the AccountNumber in it.
This one was not very easy .
In fact the easiest way as it supposed to be is the usage of the Data Mapper. This is the MULE
transformer that is supposed to act pretty much as business map in the IIB. It can create the visual
mapping between two objects. The problem is that in MULE ESB this component is poorly
implemented and has many drawbacks. It works particularly bad if you have co convert two xml
objects described by .xsd schemas. In addition it is buggy and produce annoying warnings even if
everything is supposed to be correct
See for example the warning message below:

WARN 2015-04-13 14:55:04,012 [DispatchThread:
[com.ibm.mq.jmqi.remote.impl.RemoteSession[connectionId=414D5143514C575352563230303852327D092855200D990
1]]] org.jetel.exception.ConfigurationStatus: Issue in component [XML WRITER:EXT_XML_WRITER0]
Invalid mapping (Port binding to a root element may produce invalid XML file. Set 'Records per file' or 'Max
number of records' component attributes to '1'.)

There is a lot of comments about it in different MULE user groups as well as on StackOverflow. After applying some shamanism you can almost get rid of the warning but we can not rely on this because we are professional developers an have to produce reliable results without silly dancing with tambourine around wigwam. So the natural decision is to reuse the xslt transformation. Unfortunately again AnyPoint Studio does not provide any tool for xslt transformation assistance. Hopefully doing the migration from IIB we can use the XSLT tooling from IIB Toolkit.
Open perspective → XML
Create XSL template
Run debugger:

After some testing we produce the template GBO2CRMU.xsl that does the required transformation.
We could use it in IIB version of the flow as well.
We need to modify the template in order to be able to retrieve the flow parameter duriung the transformation as we need to enrich the output with Account number value retrieved from MS Dynamics CRM:

<mulexml:xslt-transformer name="xslt1" doc:name="xslt1">
<mulexml:xslt-text>
<xsl:stylesheet version="2.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
<xsl:output method="xml"/>
<xsl:param name="accid"/>
<xsl:template match="Subscriber">
<xsl:element name="CRMUpdate">
<xsl:element name="AccountNumber">
<xsl:value-of select = "$accid" />
</xsl:element>
<xsl:if test="city">
<xsl:element name="Address1_City">
<xsl:value-of select="city"/>
</xsl:element>
</xsl:if>
<xsl:if test="country">
<xsl:element name="Address1_Country">
<xsl:value-of select="country"/>
</xsl:element>
</xsl:if>
<xsl:if test="region">
<xsl:element name="Address1_County">
<xsl:value-of select="region"/>
</xsl:element>
</xsl:if>
<xsl:if test="address1">
<xsl:element name="Address1_Line1">
<xsl:value-of select="address1"/>
</xsl:element>
</xsl:if>
<xsl:if test="address2">
<xsl:element name="Address1_Line2">
<xsl:value-of select="address2"/>
</xsl:element>
</xsl:if>
<xsl:element name="Address1_Name">
<xsl:value-of select="title"/>
<xsl:value-of select="first_name"/>
<xsl:value-of select="last_name"/>
</xsl:element>
<xsl:if test="postcode">
<xsl:element name="Address1_PostalCode">
<xsl:value-of select="postcode"/>
</xsl:element>
</xsl:if>
<xsl:if test="email">
<xsl:element name="EMailAddress1">
<xsl:value-of select="email"/>
</xsl:element>
</xsl:if>
<xsl:element name="Name">
<xsl:value-of select="title"/>
<xsl:value-of select="first_name"/>
<xsl:value-of select="last_name"/>
</xsl:element>
</xsl:element>
</xsl:template>
</xsl:stylesheet>
</mulexml:xslt-text>
<mulexml:context-property key="accid" value="#[flowVars.accountNumber]"/>
</mulexml:xslt-transformer>

Alternative 2 (Default): More than one existing account with the same email.

As main scenario, but use the error message queue.

Conclusion

As you can see migrating ESB implementations is not as hard as it may seem, despite the lack of the
standard “ESB flow” language.

Technical migration is only part of the issue and often not the hardest one to overcome. Skills,
culture, experience, risk, even politics play significant role. On the financial side there is a lot of
elements to consider, like payload format, and volume of messages to process. It’s to say that the
paid solution from first class established vendor can be cheaper than Open Source and less potent
and vice versa, Open Source may be more expensive and easier to use.

If you are an Open Source user and like to get your software free of charge with optional paid
support, you still may consider IBM software.

You can download and use IBM Integration Bus for Developers V9.0 for free

In the context of a bigger picture, the cost of license
and support over the life time of the project is not as significant as one might think. I would say that
open source software in the enterprise domain has been for years very similar to the so called free to
play approach in the smart phone gaming industry. It is free to start and do some basic things but
achieving more challenging goals becomes costly and drowning yourself in the famous Marc
Fleury’s open source pool can make it very difficult to reemerge.

Using licensed paid solution on the other hand is like using professional tools versus low end tools
for office network and phone cable deployment or for home renovation project. The ROI is easily
noticeable, especially when the low quality tool breaks in the midst of the project.

Leave a Reply

You must be logged in to post a comment.



 
       
iBudget    |   No More Money Falling Through Your Fingers        Home page
 
App store
Available on the iPhone
 
    Why Using Feedburner in Blogs Is So Important?