Monday, 19 December 2011

Enterprise Application Integration: Part 1

Warning! This post and and related code is something I began a while ago but did not get round to completing. Since starting this I have read up on EAI techniques and also on SOA. I now understand that while the examples here may be suitable for integrating two or three small applications, they would not scale for anything other than a very small business. However, as one of the primary functions of this blog is to record my learning process, I have decided to include it.

Click here to download the code from Codeplex. Please make sure you click on 'Code for Part 1. This example cannot be run using SQL Server Express, it must be run on SQL Server. Please run the script in the SQL Scripts solution folder to create the required databases.

The solution contains the Client Data System from previous posts, and also the new Accounts System, which integrates with it. Both systems have an added layer: the Remote Facade layer, which provides a WCF wrapper around the Application Services. The UI then calls these WCF services.

This example demonstrates integration at three levels: at the UI level, the Domain level, and the Data level. Each of these forms of integration has its own motivations and features.

Integration at the UI Level

The Accounts System UI layer has references to both the Accounts WCF services Client Data System WCF services. The Client/Index page displays a list of clients, and to populate this, the Client controller has uses the Client Data System's WCF services to return the list of clients. There is no contact with the Account System's WCF services or domain at all.

Integration at the Data Level

From the client index screen, the user can click the 'Accounts' link for a particular client and this will display a list of accounts associated with that client. There is a business requirement that each account must be associated with a particular site, and the list of accounts lists the relevant site along side each account. The account record only has the ID of the site, but the list has to display the first line of the address. One way to do this would be to get the list of accounts from the Accounts System, and then for each record, call the Client Data System to get the site. However, this would result in n+1 database calls, and would therefore perform very badly. Another option would be for the UI to get the list of accounts, and then when it has all the site ID's, it could get all the sites in one go, and then join them in the UI controller. However, this would push too much code into the controller. (This could however be done by a service composition or possibly a task service. I will cover these in later posts.)

The method that has been adopted in this code is at the data level. A copy of the site entity exists in the domain (only with get/set properties - no logic), and also a mapping file that points to the Client Data System's database. The site entity in the Accounts System is read only - any edits to sites should be done though the Client Data System's services. This way, a query to return a collection of accounts can join to the sites.

Integration at the Domain Level

There is a business requirement that only established client can have accounts. Currently, an established client is on that has been a client for at least three months, but this is a possible area of change, so the logic must remain in the client entity in the Client Data System. When the validation logic in Accounts System needs to check if the client is established, it needs to use the services of the Client Data System so it can access the business logic in the domain to verify that the client is established. However, the domain cannot directly consume the Client Data System's as this would break the rule that the domain cannot have any dependencies. Instead, it consumes it though an interface. Another layer, the 'Integration Layer' contains the implementation of this gateway to make the service call, and this is injected in by the Application Layer. This is an example of the Gateway Pattern.

Another possibility would be for a service composition to use the Client Data Service to check the client's established status, and if it is established, it will then call the Accounts Service. This will be covered in a later post.

Friday, 7 October 2011

Enterprise Software Architecture: How To Do It Part 6

This is part 6 in a series. For earlier and later posts in this series, please see here: Enterprise Software Architecture: How To Do It. For the accompanying code, click here.

UI Layer

Click here to download the code from Codeplex, and be sure to download 'Code For Part 6'. You will see a new assembly added called Lucid.ESA.ClientDataSystem.UI.

It's just a simlpe MVC UI on top of the Application Layer. Not much to say here, and that's as it should be. The UI simply delegates to the Application Layer's services. Sometimes there may be more complex presentation logic, which would necessitate ViewModel classes, but I'll leave that for another time.

Thursday, 29 September 2011

Enterprise Software Architecture: How To Do It Part 5

This is part 5 in a series. For earlier and later posts in this series, please see here: Enterprise Software Architecture: How To Do It. For the accompanying code, click here.

The Application Layer

Click here to download the code from Codeplex, and be sure to download 'Code For Part 5'. You will see a new assembly added called Lucid.ESA.ClientDataSystem.Application.

The job of the application layer is to provide an entry point to the functionality of the system. It consists of completely stateless, self contained services, with each call fully encapsulating the functionality of a use case. Ideally, they should accept and return dumb Data Transfer Objects with no behaviour (no methods that can be called), rather than entities.

The Application Layer does not contain business logic, and is not concerned with the implementation of persistance. All it does is orchestrate the Domain and Data Layer. A typical application service will follow the pattern of 'get entities, call business logic on entities, save entities'. This is explained in the comments of the code. I would probably not normally include these comments: I am moving towards the school of thought that limits comments. If a class follows the Single Responsibility Principle, it should be clear what it does. A Repository class should not need a definition.

You may also notice a new assembly containing Integration Tests has been added.

Inversion Of Control

Rather than creating an instance of the repositories using the new keyword, the services use the ObjectFactory (part of StructureMap) to get an instance of the interface. This way, the executing assembly can decide what concrete class to return. If you look at the AssemblyInitializer class of the integration tests, you will see that it instructs StructureMap to return the concrete classes, where as in  the UnitTests/Application/ClientServiceTests, A RhinoMocks mock is injected instead. For more information on StructureMap and RhinoMocks, please follow the links.

Data Transfer Objects

You may notice that I map all my entities to Data Transfer Objects (DTOs). Many people think this is an unnecessary overhead, but I favour it for two reasons. Firstly, it removes all problems with serialising entities, cause by uninitialised NHibernate proxies. This becomes a problem when serialising for WCF, and can also be a problem data binding or serialising to ViewState in ASP.NET. These issues outweigh the overhead of mapping to DTOs. Secondly, it removes any posibility of calling any of the methods containing business logic from outside the application layer, ensuring that all business logic is encapsulated within service calls. Mapping to DTOs is simplified with AutoMapper.


The services communicate using a combination of the Request/Response pattern and the Special Case Pattern. The services contracts are designed to return a base response, which contains only a Status field. If it is a Create service, and the creation was successful, it returns an inherited CreateResponse object, which contains the new Id of the created entity. However, if the created entity failed validation, the inhertied Response is a ValidationErrorResponse, containing the validation messages. A Get service could return a NotFoundResponse if there was no entity matching the supplied Id, but that has not been implemented here.

The Request objects supplied for services that require many parameters (such as Create and Edit) are converted to the Parameter objects required for the factory methods of the entities. All primitive types are simply mapped using AutoMapper, and required entities are loaded into the Parameters based on the corresponding Ids in the Request. This follows the usual pattern of 'load what you need, perform business logic, and save'.

What Not To Do

public Response UpdateSite(SiteDto siteDto)
  ISiteRepository siteRepository =
  Mapper.CreateMap<SiteDto, Site>();
  Site site = Mapper.Map<SiteDto, Site>(site);

  using (ITransactionProvider transactionProvider =

  return new CreateResponse(site.Id.Value);


The problem with this is that the SiteDto has been passed directly into the service. This means that the properties for the Site have been set externally to the service, and consequently any business decisions will have been made externally. This sort of service helps to ensure that business logic is pushed out of the domain, and possibly up into the UI.

Sunday, 25 September 2011

Enterprise Software Architecture: How To Do It Part 4

This is part 4 in a series. For earlier and later posts in this series, please see here: Enterprise Software Architecture: How To Do It. For the accompanying code, click here.

Persisting the Entities

Any meaningful Enterprise system will need some way of persisting valid entities. A new assembly has been added to the code - the data layer. The data layer has a reference to the domain so the domain cannot reference the data layer.

Click here to download the latest code from Codeplex. Please download the code marked 'Code For Part 4'. I supplied a zip file last time but this time you will need to download straight from the downloads section - I'm still learning to use Codeplex properly!

The data layer follows the Repository Pattern, also defined here. The repository behaves like a collection, from which you can retrieve entities, add entities and query for entities.

You may remember the domain contained RepositoryContracts. This is because the domain defines what the data layer needs to do, but is not concerned with the actual persistence itself. It also means the domain can access the repository, but opinion is divided on whether you should do this. For now, the rule is you should never access the data layer from the domain.

It is the responsibility of the data layer to implement these repositories and provide the functionality that will save and retrieve entities. The persistence medium can take a number of forms, such as binary files, XML documents, an object database, but most commonly it will take the form of a relational database, such as SQL Server. Because the relational model of the database is different to the object model of the domain, we will need an Object Relational Mapper (ORM) to map between the two. In this case I have used NHibernate. I won't provide you with a tutorial on NHibernate, but instead point you to NHForge, where you can download the assemblies and then follow the Getting Started Guide.

Again, following the tests, this time in Lucid.ESA.ClientDataSystem.UnitTests/Data should demonstrate how to use the repository classes. Note that the test copies a fresh instance of the database each time it runs a test, so that the database is in a known state. This can be found in the MyTestInitialize method of each test.

Another thing to note is the SessionProvider class in Lucid.ESA.ClientDataSystem.Data/Common. This ensures that one NHibernate session is created per Http Request in the case of ASP.NET, one session per call is created in the case of WCF, and in this case, one session per test for unit testing, although to ensure this it is required to set the session in MyClassInitialize, and close the session in MyTestCleanup

Wednesday, 21 September 2011

Enterprise Software Architecture: How To Do It Part 3

This is part 3 in a series. For earlier and later posts in this series, please see here: Enterprise Software Architecture: How To Do It. For the accompanying code, click here.

Coding the Domain

Rather than endlessly explaining how and why things should be done with the odd code snippet, I'll jump straight in and supply some code and the explain what I did and why I did it:

Click here to download Client Data System at CodePlex. It's the first change set: 3875.

If you open the solution and expand the Lucid.ESA.ClientDataSystem.Domain project, and then expand the entities folder, you should be able to recognise the entity classes, all the properties they hold, and all their behaviours.

You may have some questions...

What are all these things I don't recognise?

Firstly, some things that you should not worry about for now: Ignore Common/IRepository.cs and everything in the RepositoryContact folder. Ignore the fact that all the entities inherit from a base Entity class, except for the fact that the Id property (which you may have noticed was missing from the entities) is in here. We'll come to all these later.

Why do the entities all have a static Create method? Why can't the creation of the object be done in the constructor?

Eric Evans and the DDD community refer to this as a Factory, which is based on the Gang of Four's Factory Method pattern. There can be many factory methods for an entity. This way, there can be different ways of creating an entity depending on which one is called. If this was done with constructors, each constructor would need a different signature. Also, a factory method does not have to return the base type of the entity. It could return a sub type, and which sub type is returned could be decided by business logic in the factory making decisions based on the information passed through in the parameters. An alternative to the Factory Method is an a dedicated Factory Class or an Abstract Factory.

Why bother having these complex parameter classes that are passed in to the Create and Edit methods? Why not just set the properties of the entity?

Remember this is a simple example. In this particular case, the majority of the functionality of the Create and Edit methods is to set the member attributes based on the properties of the parameter objects (one exception is that _deleted is defaulted to false). In most cases, more complex business decisions would be made in here based on what was passed in through the parameters. For example, the Site entity may have an _overseas attribute, which would have to be calculated based on the address passed in in the parameters, and as mentioned before, it may want to return a sub type rather than the base type, depending on what parameters are passed in. All this business logic has to be done somewhere, and that somewhere should be in the domain. If the properties of the entity were set from the outside, it is possible (and usually likely) that the business logic would get pushed out of the domain and into the service layer, or worse, the UI.

Running the Tests

All the logic in the domain should be tested in the form of unit tests. In this case, these are in Lucid.ESA.ClientDataSystem.UnitTests. I have to say, I have not followed any strict testing methodologies here. I have yet to read Roy Osherove's The Art of Unit Testing but it is on my to read list, and anyone who wants to know how to unit test properly should read this. However, these tests demonstrate how to use the domain: build a 'create' parameter object, call the Create method to return an entity, and call Validate on the entity to ensure it is valid (if it is, no records in the returned collection will be present). Then you can build an 'edit' parameter object, pass that into the Edit method of the entity and Validate it again.

Next post: Persisting the Entities.

Monday, 19 September 2011

Enterprise Software Architecture: How To Do It Part 2

This is part 2 in a series. For earlier and later posts in this series, please see here: Enterprise Software Architecture: How To Do It


I'm going to start with a simple data entry system to record details about a company's clients. The requirements specify the following use cases:
  • View details for a client.
  • Add a new client.
  • Edit an existing client.
  • Mark a client as deleted.
  • Add a site to the client. One client can have many sites.
  • Edit a site.
  • Delete a site.
  • Add a contact for a client. One client can have many contacts.
  • Edit a contact.
  • Delete a contact.
Many people will be concerned that 'add', 'view', 'edit' and 'delete' are synonymous for 'create', 'read', 'update' and 'delete', and that this will simply be a CRUD system. Firstly, this is a simple example application and the more complex use cases will come later, but mainly this is very often (at least in my experience) what specified use cases look like. It's no good saying developers should not create CRUD systems when that is what they are being asked to do.

Choosing the Architectural Pattern

The architectural pattern I will be using will be the Domain Model pattern. This is sometimes described as something new and ground-breaking, when in fact all it is is good old fashioned Object Oriented Programming. The Domain Model pattern is not the same thing as Domain Driven Design, which is an approach to application design, rather than a specific pattern. Other patterns are available: Active Record, Transaction Script and Table Module are patterns generally recommended for simpler applications. However, I don't believe that Domain Module adds significant overhead, and often small utility applications can slowly creep into huge applications that run the enterprise, so I tend to use Domain Model as my default.

Much is often said of 'n-tier' architecture, placing UI at the top, followed by the Application Layer, The Domain/Business Logic Layer and the Data Layer. I don't find this description helpful or accurate. Instead I prefer to think in terms of The Onion Architecture.

Defining the Domain Model

First the entities must be identified. The entities are classes representing an element of the business. In this case, the entities are Client, Site and Contact. The realations and multiplicity between the entities also needs to be defined.

The properties of each entity should be defined, as should the behaviours. In this simple example, the behaiours will generally reflect the use cases, along with a Validate method on each class which will be used to return a list of validation errors.

This is the model we end up with:

In reality, there would probably be a more complex relationship between Site and Contact, which I have decided to omit for this simple example.

Enterprise Software Architecture: How To Do It

Articles in this series:
Enterprise Software Architecture: How To Do It
Enterprise Software Architecture: How To Do It Part 2
Enterprise Software Architecture: How To Do It Part 3
Enterprise Software Architecture: How To Do It Part 4

Most of my technical reading for the past year or so has focused on enterprise software architecture. Much of this material has been theoretical and vague. It has often included philosophical explanations about how to design a system, along with some detailed diagrams, but there is usually one glaring omission: code samples. This information has generally answered questions such as 'what you should consider' and 'how you should approach a problem', which are important, but it often misses a more fundamental question: 'how to do it'.

Maybe many people think it should just go without saying, that it should just be built in knowledge that every developer comes with, but there seems to be very few resources that describe the basic architectural framework for enterprise systems. And it doesn't seem to be taught in colleges or universities. This is what has led to an abundance of badly conceived 'Webforms over Stored Procedures' type systems. These are commonly unmaintainable, unscalable and often do not function according to specification.

Of course, there is no universal agreement on enterprise software architecture, and each case is different (try asking a software architect a question that is not answered with 'it depends'), but there are a few fundamental rules and patterns that can be followed. I intend to illustrate these on this blog.

This is not so much 'how it's done', more 'how I do it'. People will disagree, and I welcome any constructive criticism. I am after all still learning, and probably will be until the end of my career. So if you think I'm doing something wrong, please tell me and the world how to do it right.

What I hope to provide here is a basic template for enterprise software architecture that can tweaked according to the specific problem in hand. This is not an end to end journey through requirements gathering, analysis and modelling of a complex system. For that, I recommend you take a look at Ayende's excellent Macto posts.