Replacing SOA API calls by EDA request/replies using Redis

This post aims at simplifying and explaining Enterprise Application Integration (EAI) concepts using more pictures and examples than words.


The main difference between Event Driven Architecture (EDA) and Service Oriented Architecture (SOA) is that EDA is asynchronous and SOA synchronous.


In the picture above you see how SOA typically connects anything to anything directly.


The EDA pattern, on the other hand, uses a single communication channel (message bus or queueing server).

Ideally we stop thinking about API calls (request/response) and we move to “events”. “Events” are things that happen in your application that might be of interest to other applications. When a customer needs to be created by your website you can make an API call and wait for an “OK”, or you could rewrite your application to “fire and forget” and handle errors separately. But what if an application has a question that needs to be answered? For example: What if you have a “customer reference” and want to find the corresponding “email address”?

The solution is to use the Request/Reply Enterprise Integration Pattern (EIP). Most Enterprise Messaging Systems (EMS) offer an Enterprise Service Bus (ESB) on which the message format (set of optional and required headers) is specified, but the data format is free. It is up to the applications to run their own integration engine, which has adapters for different data types for the various applications it wants to talk with.

A SugarCRM example

The following message could be in the “SugarCRM/Requests” queue:

Timestamp:  1382055093876 ms
MessageId:  345781632423
ReplyQueue: SugarCRM/Replies
Data:       {"method":"GET","url":"/Accounts/f1eeca5f-c0eb-891e-db92-52323b958a87"}

The SugarCRM application has an integration engine that monitors the “SugarCRM/Requests” queue. It will receive the above message, transforms it into a API request and executes it. The result that it receives will be transformed into the following message and put in the “SugarCRM/Replies” queue:

Timestamp:  1382055096943 ms
MessageId:  345781632567
ReplyTo:    345781632423
Data:       {"id":"f1eeca5f-c0eb-891e-db92-52323b958a87","name":"RR. Talker Co","date_entered":"2013-09-12T18:09:00-04:00","date_modified":"2013-09-12T18:09:00-04:00","modified_user_id":"1","modified_by_name":"Administrator","created_by":"1","created_by_name":"Administrator","description":"","deleted":false,"assigned_user_id":"seed_jim_id","assigned_user_name":"Jim Brennan","team_count":"","team_name":[{"id":"East","name":"East","name_2":"","primary":true}],"linkedin":"","facebook":"","twitter":"","googleplus":"","account_type":"Customer","industry":"Electronics","annual_revenue":"","phone_fax":"","billing_address_street":"67321 West Siam St.","billing_address_street_2":"","billing_address_street_3":"","billing_address_street_4":"","billing_address_city":"Santa Fe","billing_address_state":"NY","billing_address_postalcode":"44150","billing_address_country":"USA","rating":"","phone_office":"(949) 400-8060","phone_alternate":"","website":"","ownership":"","employees":"","ticker_symbol":"","shipping_address_street":"67321 West Siam St.","shipping_address_street_2":"","shipping_address_street_3":"","shipping_address_street_4":"","shipping_address_city":"Santa Fe","shipping_address_state":"NY","shipping_address_postalcode":"44150","shipping_address_country":"USA","email":[{"email_address":"","invalid_email":false,"opt_out":false,"primary_address":true,"reply_to_address":false},{"email_address":"","invalid_email":false,"opt_out":false,"primary_address":false,"reply_to_address":false}],"email1":"","parent_id":"","sic_code":"","parent_name":"","email_opt_out":false,"invalid_email":false,"campaign_id":"","campaign_name":"","my_favorite":false,"_acl":{"fields":{}},"following":true,"_module":"Accounts"}

Tutorial: EDA with Redis

One of the more KISS – but less conventional – ways of implementing the above scheme would be to use Redis to store queues. The “SugarCRM/Requests” queue can be implemented as a “List” of “MessageId” values. The messages can be stored with “Hash” data types in the global namespace using the Redis “HMSET” command, where the headers can be individual keys. The unique “MessageId” values can be generated using the Redis “INCR” command. The “SugarCRM/Replies” queue can be implemented as a “Hash” data type as well, where  the “ReplyTo” can be used as the key and the “MessageId” as the value.


2 thoughts on “Replacing SOA API calls by EDA request/replies using Redis”

  1. Could EDA be a workaround for using a fat server as API (the problem you described in a previous post)?

  2. @Mark: Good comment, let me try to give my take on it. I think that even with EDA you may need synchronized requests and fat servers will still be a problem in that case. Does that make sense?

Leave a Reply

Your email address will not be published. Required fields are marked *