Thursday, 19 January 2012

Enterprise Application Integration: Part 3

Click here to download the code from Codeplex. Please make sure you click on 'Code for Part 3'. This example cannot be run using SQL Server Express, it must be run on SQL Server. Please run the script in the SQL Scripts solution folder to create the required databases (even if you have done this for the last part).

General Refactoring

Database Scripts

The databases scripts were creating the database files to a specified directory. This has been rectified.

Response Objects

I now no longer favour the generic Response objects returned by the Application Layer. The need to cast this to the actual type adds complexity for the consumer. The point was to return other types in various circumstances (ie. validation failures or if an entity is not found). Instead, Fault Exceptions will handle these circumstances (although I do wonder whether a failed validation is really exceptional?). I still favour Data Transfer Objects.

On some projects, I always have a request object, even if it only has one property. The advantage of this is that if a new version of the service is released, it will be backwardly compatible with existing consumers of the service.

WCF Layer

The Remote Façade Layer has been renamed the WCF layer, because this is more meaningful, and it is not truly an implementation of the Remote Façade pattern. I am wondering if this layer is truly needed - the Application layer is already polluted with WCF related stuff.

Logging and Infrastructure Layer

I have added in some logging. This is in a new layer - the Infrastructure layer. This is not yet perfected - if the Domain needed to log events, it would need a dependency on the Infrastructure layer, violating a principle of the Domain Model Pattern. Each service operation logs the content of the request, and errors are also logged. The logging still relies on SQL - I may change this in future to harness NHibernate, to keep it consistent with the rest of the project.

Note that depending on which binding you are using, you can't always use Application_Error in Global.asax, so instead a Behaviour Attribute is added to each service to log errors. Hopefully, I will figure out a way to log service calls with attributes, removing the need to add _log.Add() at the start of each service, forming a type of AOP. Also note that setting up behaviour extensions in web.config is affected by a bug that has existed since the dawn of WCF and has not yet been fixed.

Also, Application_EndRequest does not work for some bindings, so I have moved this to the end of each service call. Hopefully I will be able to fix this with a Behaviour Attribute.

Integration Refactoring

One thing I haven't been happy with when using XML or WCF Web Services in the past is the point-to-point connections between applications, which makes the applications tightly coupled. This leads to many design problems, including entities being duplicated between systems - like the Site entity in the account system.

I have been reading up on SOA, and while I don't claim to be building an SOA here, I am borrowing some concepts. One borrowed concept is the composite service. In this system, some composite services consume the services from the Client Data System and the Accounts System, and any integration is performed in the composite services. This removes the reference from the Accounts System to the Client Data System, removes the need for Service Gateways, removes the need for any Client Data System entities or properties from the Accounts System, removes the need for the consequent mapping files, and removes the need for the views in the database. The two systems have become completely decoupled.

The only connection that remains are foreign keys in the Accounts System to the Client data System (for example - Site ID in the Account entity). I read somewhere even this is frowned upon, but I'm not at all sure how this would work? I'll leave this for now.

There is some business logic in the composite services> I'm not sure this is the best place for it, and may review it later. Some validation is occurring in the composite services too, and again I will review this. For example, the Account composite service checks if the client is established before calling the Account Service to create it - an alternative would be to pass the Client entity regardless to the Account Service and let it decide whether to create the account?

The validation is all a bit spread out - maybe this could go in the UI.

Another bonus is that I have avoided replicating the FullName business logic.

The Contact entity may appear to be replicated in the Accounts System, but in fact this only holds information about a contact that is specific to the Accounts System, and does not replicate any properties from the Client data System (except the ID to tie them together).

Wednesday, 4 January 2012

Enterprise Application Integration: Part 2

Warning! This post and related code is something I began a while ago but did not get round to completing. Since starting this I have read up on EAI techniques and also on SOA. I now understand that while the examples here may be suitable for integrating two or three small applications, they would not scale for anything other than a very small business. However, as one of the primary functions of this blog is to record my learning process, I have decided to include it.

Click here to download the code from Codeplex. Please make sure you click on 'Code for Part 2'. This example cannot be run using SQL Server Express, it must be run on SQL Server. Please run the script in the SQL Scripts solution folder to create the required databases (even if you have done this for the last part).

Firstly, I have removed the integration layer and incorporated it into the Application layer to reduce complexity.

Extending Properties of an Entity

A new business requirement has been specified: contacts must have a payment authorisation level (low, medium and high) and this must be displayed in the drop down list in the create payment page.

Clearly, the Contact entity needs a new field: PaymentAuthorizationLevel, but should this go in the Client Data System? This information is specific to the Accounts System, and if every system stored its own information relating to a contact in the Client Data System, the Client Data System would become soon bloated, and the constant change would be hard to manage. This solution is unscalable.

So the solution to this I have implemented in this version is have an entity in the Accounts System also called 'Contact' which holds all the fields from the Client Data System and also the new PaymentAuthorizationLevel field. There is a table in the database for the Accounts System called 'Contact', which contains only an Id and the PaymentAuthorizationLevel field. The Id is not auto-incrementing: it is the primary key but it is also a foreign key to the Id in the Client Data System's Contact table.

There are now two mappings for Contact in the Data Layer: one with an entity name of 'ReadContact': This maps to a view which is a join of both Contact tables, and therefore populates all fields in the Accounts System. This mapping is read-only:

The other mapping is 'WriteContact', and is for writing. This saves the Contact object to the Accounts System database, but only the Id and PaymentAuthorizationLevel fields. In order to save a new Contact, the ContactService of the Account System must call the CreateContact method of the ContactService in the ClientDataSystem, return the Id, build a Contact entity in the Accounts System with the returned ID and PaymentAuthorizationLevel and then save that:

Edit: I've later noticed that the Contact Mapping (Read) in the above diagram should say Contact Mapping (Write)!

As mentioned in my disclaimer, this is not a perfect solution (The CreateContact method of the AccountService is very bloated), and I have already learnt new methods, but I thought I would document this as part of my learning process, and it may be useful for some smaller integration problems.