Enterprise Example Part 5

The code for this is available on Github, commit 9012233152.

This commit adds the 'ItOps' service. Following Udi's philosophy, this is where emails are sent from.

Two situations have been handled - sending of an invoice from the Finance system and informing managers of Sales when leads have automatically been unassigned (in this case because the consultant has left).

In the Finance example, the user clicks a button on the UI which sends a command to the ItOps service. The service than calls WCF service to get all the information it needs for the email. It then composes the email and sends it (sending has been stubbed out here).

The Sales situation is different - when the leads are unassigned, the Sales service raises an event. The ItOps service subscribes to this event and reacts to it by sending an email informing the manager of the unassigned leads.

In the real world, I am not absolutely happy with having one service handle all the emails for the enterprise - I tend to favour one email handling service per service. This takes the form of a message handler assembly separate from the main message handler.

On issue I have had with this is if the email sending fails, the WCF client is disposed and any retries can't access it. Ths will be ammended in a future version.

Enterprise Example Part 4

The code for this is available on Github, commit 8a00a5f248.

In this commit, I have added the ability to book visits in advance in the Sales system (and assign them to a consultant), and to book holidays in advance in the Human Resources system.

I have also included a Calendar system. This includes core data about appointments (visits/holidays). Those visits and holidays have a foreign key to their appointment. Holidays and visits could have used the same ID as their corresponding appointment, but I have seen confusion arise from this sort of design before.

Both the Sales and Human Resources systems validate a booking against the Calendar system, to ensure a booking does not clash with any other appointment. The UI does this using request/response over WCF because it is effectively a query and therefore not suitable for NServiceBus.

The Human Resources also validates locally against some holiday specific logic (whether the employee has enough holiday left). This is a useful demonstration of validating against 2 sources. Now I no longer have an application layer, I wasn't quite sure where to put the validation code, as it does not really belong in the UI. I decided to place it in its own assembly.

Future plans include retunring details of clashing appointments and the ability to move appointments.

Enterprise Example Part 3

The code for this post is available on <a href="https://github.com/lucidcoding/EnterpriseExample">Github</a>. This post relates to commit 4225b569c2.

Another issue I had with the original design was that the value of the Deal was being stored in Client Services, for no reason other than that it could be passed on to Finance when the Agreement is activated. I have decided this is another candidate for a saga.

This is more of a saga in the traditional sense - a long running process. Here is how it happens:

1. When the user completes the form to register a deal, and the RegisterDeal command is sent, when this is processed, a DealRegistered event is raised.
2. This is subscribed to by the Finance service, and it starts of an OpenAccountSaga in Finance.
3. When a user of the Client Services system actives an Agreement, this raises an AgreementActivated event.
4. This is also handled by the OpenAccountSaga.
5. When the saga has received both these events, it will have all the information it needs to open the Account.

I found it was not possible to use a specific correlation ID here, and I'm still not sure this is the correct way to go. Instead I have used the Deal ID. The Agreement ID is now different from the Deal ID and the Deal ID is just a property of the Agreement .

I have now been able to remove Value from Agreement.

Enterprise Example Part 2

The code for this post is available on Github. This post relates to commit 217cdad9f5. One thing I was not happy about in the last post was the way when a client had been initialised, it then had to use WCF from th UI to query the Sales service for information about the client. The new design uses a Saga.

Sagas are designed for long running processes, but they can also be used for orchestrating services. Here is what now happens:

1. When the user completes the form to register a deal in the Sales system, this sends a RegisterDeal command to the Sales message handlers containing the information about the Deal.
2. This also sends an InitializeClient command to Client Services, which contains information about the Agreement (This command is possibly named incorrectly).
3. When the RegisterDeal command is handled in Sales, it raises a LeadSignedUp event, containing all information about the lead.
4. Bot the InitializeClient command and LeadSignedUp event are handled by the InitializeClient saga. Once the saga has received both of these messages, it has all the information to properly initialize the client.

I had at first used the client/lead ID as the correlation Id for the saga, but decided against this, as two users could register a deal for the same client at the same time. Instead I have used a specific CorrelationId. I'm not sure if this is common practice in NServiceBus?

Now I have been able to remove the service references to Sales from the CLient Services UI.

Enterprise Example Part 1

I have recently completed a cut down example of an entire enterprise, based on Event Driven Architecture:

Click here to get the code form GitHub. This blog post is based around commit 97d6ab2ab0.

To get this up and running, you should first have the NServiceBus examples downloaded and running succesfully. Then in download the code for this example, run the SQL Scripts in the root on your local databases, change all hibernate.cfg.xml files in the projects to point at your local SQL Server and run!

Overview

The enterprise consists of four systems:

• Human Resources
• Sales
• Client Services
• Finance

The term "Service" in this example relates to anything below the UI. The following rules are being followed for inter-service and ui-service communication:

• UIs may only query external services using WCF.
• UIs may send commands to services via NServiceBus messages. (This is currently synchronous but I will explore asynchronous communication is later posts.)
• Services may only communicate with other services using events. (I may exploye asynchronous commands in later posts.)

The workflow for each system and how they interact are described below.

Sales

• A sales consultant is presented with a list of leads. When a consultant returns from visiting a lead, they log this by clicking "Show Visits" and clicking "Log New".
• The consultant fills in information about the visit.
• When the consultant click "Create", a command is sent to the ‘Log Visit’ Message Handler in Sales, which adds a record about the visit.
• If "Resulted in Deal" was not checked, the consultant is returned to the list of visits. If it was checked, the consultant is forwarded to the form to register a deal.
• The consultant fills in this form and clicks "Create".
• This sends a command to the "Register Deal" Message Handler in Sales, and also to "Initialize Client" Message Handler in Client Services.
• The "Register Deal" handler adds a record about the deal, recording that consultant against it and calculating his or her commission.
• The consultant is forwarded to the list of deals.
• The consultant, Sales system and department are no longer concerned with this lead.
• At any point, the system can receive an "Employee Left" event from Human Resources. In this situation, the relevant handler unassigns any leads assigned to that consultant.

Client Services

• The "Initialize Client" command is received by the relevant Message Handler in the Client Services system.
• This adds a record to the user’s list of clients to be activated, and adds an agreement with the start and end dates of the agreement, together with its value, as entered by the sales consultant.
• The user clicks on "Activate" for the new client.
• The user is presented with a form containing all information from the Sales system (name, address, phone number) which he or she must confirm. The user must also enter a reference for the client and a liaison whom they will deal with.
• The user clicks "Activate".
• A command is sent to the "Activate Client" Message Handler. This handler updates the information about the client and activates the current agreement. It also publishes an "Agreement Activated" event that the agreement has been activated.
• The Client now moves from the list of ‘Clients Requiring Activation’ to the list of "Active Clients".
• The user can then click on "Agreements" for any active clients to see the agreements (at this time it only show the current one).
• The user can then click "Cancel" to signify that the client has cancelled the agreement. This sends a "Cancel Agreement" command. The handler for this marks the specified agreement as cancelled and raises an event to signal that this has happened.
• At any point the system can receive a "Account Suspended" event from the Finance system. This signifies that the client’s account has been suspended (because they have fallen behind in their payments). The relevant handler picks this up and marks the corresponding agreement as suspended.
• At any point, the system can receive an "Employee Left" event from Human Resources. In this situation, the relevant handler unassigns any clients assigned to that employee.

Finance

• The "Agreement Activated" event is received by the relevant message handler in the Finance system. This opens a new Account and calculates all the monthly Installments that will need to be paid during the lifetime of the account.
• The list of Accounts is displayed to the user. The user may click on "View Installments" to for any particular Account.
• For each Installment, if the due date is not reached, it is marked as "Pending", if the due date has passed it is "Due", if it is ten days passed the due date it is marked as "Overdue". It the Account is closed it is marked as "NotRequired".
• At any point, the user can click on "Mark As Paid" for an installment. This sends a command to the "Mark Installment As Paid" handler, which updates the domain accordingly. The Installment will then go to the "Paid" state.
• Back in the list of Accounts, the user can click "Suspend" (if a client has fallen too far behind on their payments). This sends a command to the "Suspend Account" handler, which updates the domain and raises an "Account Suspended" event. (This is subscribed to by Client Services).
• At any point, the system can receive and "Agreement Cancelled" event (raised by Client Services) and updates closes the corresponding Account.

Human Resources

• Users of Human Resources can at any point mark an employee as left. This sends a "Mark Employee as Left" command to the Human Resources Message Handler, which updates the domain and raises the corresponding event (subsribed to by Sales and Client Services).

Shortcomings

It is important to not that many aspects of this code are not ideal. I am exploring EDA here, and do to limited time I have skimped on other areas. These areas include validation (both in the UI and the domain), user friendlines, and performance considerations. For example, currently Sales could fail if the user tries to register a Deal for a Lead that is unassigned. At some point it needs validation around this. It has not been done because it is not relevant to the concepts being explored here.

Error handling has not been implemented particularly well - I will be exploring this more in another post. This will probably involve returning ReturnCode.Error from MessageModule.HandleError() but I'm not sure yet.

Finally, The IOC implementation in my WCF services is a bit clumsy - I am still searching for a good solution to this.

In the event of an error, the system seems to retry forever. I'm not sure why it does this because I don't think this is default behviour in NServiceBus.

Upcoming Improvements

• I would like to be able to book holidays for Employees in Human Resources. I would also like to be able to pre-book visits in the Sales system. These bookings should not be allowed to clash, so possibly some sort of central booking service is required.
• When an employee is marked as left, as well as Leads being unassigned in Sales, and Clients being unassigned in Client Services, the manager of those departments should also be informed which Leads/Clients have been affected by email.
• I would like to be able to issue an invoice from Finance. This would possibly be in the form of an email that would compose data form multiple services.
• When a Deal is registered, the value is passed to Client Services and saved in the Agreement. Value has no real place in the Client Services domain – it is of no interest to them (under current specification). The only reason it is recorded is so that it can pass it to Finance when the agreement is activated, so Finance know how much to bill the client. There should be another way of getting this information to accounts – possibly using a Saga? Start and End dates of the Account could be passed through straight from the form rather than Client Services.
• When a Client is initialised, no information is brought through (such as address lines, phone number, even name) except the Id. When the user activates the account, the UI queries Sales for the information so the user can confirm it, and it is then saved. It would be better if the information was passed straight into the initialised Account, but this information is not available on the register deal form. I will look into the possibility of using a Saga here to bring through this information.
• In Client Services, Agreement has a possible status of Expired. This is not derived from the expiry date but it is a persistent value. The only way to ensure this is set correctly is to have a continuous polling service that would set it when the expiry date is reached. This could also be useful for raising an event for other domains.

NHibernate: Filtering on Properties of Subtypes

Here's a useful tip I learned about NHibernate today. Say we have a task list, displaying details about entities of type Task. Various task types inherit from this. Once such type is 'ReviewDocumentTask'. This extends Task in that it has a reference to a Document entity. The base Task does not. The requirement is to filter out all tasks where the document has been deleted. How do we filter on this field, yet still treat tasks polmorphically? Two useful feature for this purpose are the ability to create an alias with a left outer join, and the ability to restrict on the 'class' property. The resulting code looks like this:

  
var result = SessionManager.Session
    .CreateCriteria<Task>();
    .CreateAlias("Document", "document", JoinType.LeftOuterJoin)
    .Add(Restrictions.Or(
        Restrictions.Not( 
            Restrictions.Eq("class", typeof(ReviewDocumentTask))
        ),
        Restrictions.Eq("document.Deleted", false))
    );

return result.List<Task>();

Event Driven Architecture - Problem Scenario

I am quite new to Event Driven Architecture, and while many scenarios are quite straightforward, one has recently posed a problem.

A central login portal gives employees within an enterprise access to various online systems - a bit like how Stack Exchange gives users access to Stack Overflow, Programmers etc.

Each system maintains a list of all employees within the enterprise (denormalised data within its own database, as is standard in Event Driven Architecture). However, each system only maintains a list of users that are registered for that system. There is a one to one relationship between employees and users. Each system may want to store information specific to that system for a user.

The requirement is to be able to register an employee as a user for a particular system from within that system. Lets call it System X.

At first this seemed simple:

System X would be able to get the list of unregistered employees from within its own store (1). The operator would select the employee to register, and the UI would be able to retrieve further information from the System X service layer (2). The operator would then fill in any System X specific information and click 'register'.

The UI would then generate a GUID, call the System X service layer to register that employee as a user (3), and send a command the portal (4). The commands contain the new user ID and the employee ID. If the portal already has the user (because it has already been registered by another system), it doesn't create it but just flags that it can now use System X.

The portal can raise an event (5) to say that the employee has been registered but System X isn't really interested - it may be interested however if there is an error.

The employee is now registered in both System X and the portal as a user with the same ID. The UI then displays the list or users from the store in System X - it does not have to wait for the portal to do its work.

But there is a problem. If as mentioned, another system has already registered the employee, the user will already have an ID in the portal. System X will create the user with a different ID.

Possible Solution 1

Again, the UI gets a list of employees from System X (1), but when the operator selects an employee, as well as getting information from the service layer (2), it would also have to make a call to the portal to see whether the employee is already registered in there (3), and if so, get the corresponding user ID. When the operator clicks 'register', if a user ID is present it would use it, but if not it would not generate one.

It would then call the System X service layer (4) and send the command to the portal (5) and the and the portal would raise a corresponding event (6).

This follows the principle of composite UIs but it feels a bit synchronous. The UI is now reliant on the portal being online, and the performance is dependant on the performance of the portal.

Possible Solution 2

This time, System X and all other systems abandon the rule that they only maintain a record of users that are registered for that particular system - so System X will hold a record of all users, but will have a flag against those that are registered to use it.

This way when the operator selects an employee not registered in System X, it will know that it has been registered before in another system, and also know its user ID. If not it can generate one. The UI can then send the command to both the portal and call the System X service layer.

System X would have to subscribe to 'employee registered' events in order to keep its record of users updated.

This does mean however, that System X retains a lot of data it doesn't really need, and is not really relevant to it.

Possible Solution 3

The UI can get a list of unregistered employees from the service layer (1), when the operator selects an employee it gets further information from the service layer (2).

This time the UI does not need to generate or know a user ID. When the operator clicks 'register', the UI calls the service layer to tentatively register the employee as a user putting it in a PendingUserRegistration table, including the employee ID and all System X specific data (3) and also sends a command to the portal to register the user (4).

When the portal has completed this, it raises an event (5) which conatins the user ID. When System X receives this it can remove the record from the PendingUserRegistration table and add a record and create a user locally.

The problem with this is that it makes it difficult to give instant feedback to the operator. Because it is not immediately updating a local user table, there may be a delay in the user being present in System X (for example, if the portal is down). This could be a problem if after registering the user, the operator expects to see a list of all users.