ABSI - far beyond system integration

On 31 January 2017     By Sven Delporte      Dell , Salesforce

In almost all Salesforce projects we do, customers ask us: How can I get my Salesforce data into my ERP system, and vice versa, in an easy and controllable way?

There is quite a lot of answers to this question as there is a variety of tools and methods both free and paying that can do this job.

We use a range of tools depending on the customer needs and budget, but most of the time it comes to 4 possible flows.

1. Batch files processing using the free and easy to use Apex DataLoader. 
With Apex DataLoader you can insert/update/"upsert" records into Salesforce from a csv file. The Dataloader tool is capable of spitting out Success and Error files so you can track what went wrong and what was successful. Now this works perfectly if you don’t have a lot of flows and if your volumes of data are manageable. Monitoring your integration flows can be a challenge as you need to dig into the error files generated by Dataloader.

2. Real-time web services between Salesforce and your ERP System. 
This scenario is mostly done if you have a couple of flows that need real-time integration. For example if you create an account in Salesforce and you transfer the account from prospect to customer, then you could create this account also in your ERP system in a real-time way. As a response of the web service call you could update the CRM Account with an ERP customer ID. Which is then useful in the later flows. The downside of this approach is that you need developers at both ends of the systems.

Dell Boomi which can handle real-time flows as well as batch flows. The decision to go for an Integration Platform is best made based on the number and complexity of your integration flows. We generally recommend our customers who want to connect 2 systems together - where the flows are pretty straight forward - not to use an Integration Platform as this comes with a license cost, but rather go for option 1 or 2 above. As soon as you have multiple enterprise applications a platform like Dell Boomi becomes a very interesting option. To check if Dell boomi's multi-purpose PaaS is right for your company, you can go here

4. Using Salesforce Lightning Connect to plug into your ERP system using an OData connector from Salesforce.

At ABSI we have implemented all 4 of these possible approaches and are aware that they all have pros and cons. We evaluate each case individually and propose the best approach. This may start with a short term Apex DataLoader integration to get the data into the system but evolves into a more robust middleware solution in the long run.

As customers and projects mature the need for integration flows typically rises rapidly. The benefit of multiple integrations becomes clearly visible and valued when business users become more demanding to move fast and flexible between the systems they use. Data and information needs to be available and up to date, this is critical for more and more data every day. This is when we recommend a strategic choice for the adoption and implementation of a reliable integration platform like Dell Boomi. My colleague Doubi wrote more than 2 years ago that he considers that Boomi will do to integration what Salesforce did to CRM years ago.

Integration use-case

In the following example we will walk through a common integration use-case where we will update a Salesforce Account and mark this account as customer, by changing the Account Type to ‘Customer’. This change will fire a workflow rule that will send out an Outbound Message. The Outbound Message will be captured by Dell Boomi and the account data can be processed into the ERP system. In our example we don’t actually have an ERP system so we will write the account data to a CSV file and send the file to an FTP site.

Step 1. Setting up Salesforce Outbound Message

In Salesforce you can set up a workflow rule that sends out an outbound message based on a particular action (change a field for example). In this example we will trigger the workflow if we change the picklist field Type on account to “Customer”.

Notice on the Outbound Message screenshot that we are using a Putsreq.com endpoint, this is solely used for testing purposes. PutsReq is a great website for testing out HTTP requests. It captures the header and displays the raw body of our message. In our example we use it to capture the xml output which will later be used in our Dell Boomi xml profile.

Workflow rule screenshot

OutboundMessage screenshot

Putsreq.com screenshot

Capture the xml output and paste it in a text file so we can use it later in our Dell Boomi xml profile setup.

When the outbound message is set up we can go to the Boomi side to set up our integration flow there.

Step 2. Dell Boomi - Build integration process

In Boomi we will set up a web server listener which you receive in the Professional plus license or with the Professional license + the Services Enablement add-on which is a key element if you want to set-up a listener process. It allows you to create a public endpoint that you can use in your integration flows.

When we create a process in Boomi it will ask us what we want to use as a Start Shape. In our example we will choose the Web Services Server. We can then create an operation for it where we define what needs to happen. This is where our previously saved xml file from putsreq.com will come in handy.

Click here on the ‘+’ sign on the Request Profile to create a new xml profile from our previously saved xml file.

By clicking the import button on the top right corner we can upload an xml file or even a complete xml schema. The xml profile will be auto-generated for us and we will receive an output profile like this:

After saving it we will need to create a response profile, as the Outbound Message also expects an acknowledgement. Otherwise it will remain in the Salesforce Queue for 24 hours. This is a very handy feature in case the Dell Boomi or the ERP system is not responding for any reason. The queued message will be re-sent on regular intervals until the server responds, or it will be dropped if the 24 hours are passed.

The response xml should look like this:
After adding this into a message shape we can return the documents to the Salesforce platform.

Our process now looks like this:

We can start by adding a target connector where we can again import our existing target structure so that we don’t need to create this manually.
In our example we are using an easy to use FTP connector where we will output a csv file with our Account data in it. In a real life example this would be our ERP system using a SOAP or REST interface or maybe one of the 50+ standard application connectors available on the Boomi platform.

Now that our target connector FTP is configured, we need a mapping between the Salesforce message and the csv output. This can be achieved by dragging the map shape onto the canvas. When we do this we will have an empty map shape where we can select our source profile and our target profile. As a source profile we can click on Choose and then navigate to our OutboundMsg XML profile we created earlier. As a target profile we need to create a new flat file profile as we haven’t done this just yet.

On the CSV profile we can also import an example CSV to get the profile we want (see result below).

From now on it is just a matter of dragging a line from a left side field to a right side field to create the mapping between 2 systems. We can also use <b>Boomi Suggest</b> which will do the mapping automatically for us.

Now we are almost there as we just need to connect the shapes and add an extra process property to give our CSV file a name. The process property step allows you to give your file a proper name and extension. You can add variables like date and time but you can also use static text.

Our complete process looks now like this:

Now that our process is built, it is ready to deploy to an Atom. An Atom in Boomi is a single tenant, single node AtomSphere engine that can either run in the cloud or on premise. We can use Windows/Linux and - now also since the newest release - Docker which is a great addition and very easy to use.
Once we have deployed our process, there is still one very important thing to configure: the Endpoint url for our web server. This can be set in the shared web server settings of the atom which is also the place where we configure the security options.

In our example we will use basic authentication with a username and a token. Which we need to convert in a base64 string to use in the url. The username and password need to be set as username:token and be converted to base64 string using an online encoder like base64encode.org.

The Boomi endpoint in Salesforce looks like this:

This is of course when we use the Cloud Atom from Boomi. If we use a local atom we can define the url ourselves.

When everything is setup you can test the process by changing the Account Type to customer in Salesforce and in Dell Boomi monitoring we will see that the message arrives on the web server.

If our process is configured correctly we should have a file on our FTP server as well looking like this:

From the FTP server the file could be processed by our ERP system, or just as easily we could have connected directly to our ERPs API with one of the many application connectors provided in Dell Boomi.

Using an integration platform we can really speed up the build of our integration processes. Everything is visible in one place and we don’t need to be a developer to monitor, debug, and/or adapt the integration flows. If you don’t have developers in your company that can maintain the integration flows, a middleware tool like Dell Boomi can really help you in achieving your integration goals.

If you want to know more how Dell Boomi can help your company in integrating your application landscape don’t hesitate to ask us more information, we can help you in setting up a proof of concept.

Also you can go here and read my blog from more then 3 years ago.

Sven Delporte
@Sven Delporte