Thursday 30 May 2013

Instant Feedback With NServiceBus

The majority of content for this post comes from a thread I started on the NServiceBus Yahoo Group, and this and subsequent posts is really just a summary of that. You can view the original thread here.

One of the problems I have encountered with NServiceBus is the issue of the UI reacting instantly to a user's request. The asynchronous nature of NServiceBus is somewhat in conflict with it.

Take the following example: A grid shows a list of records, each with a delete 'X' on them. The user clicks the X, which sets a 'Deleted' flag and publishes an NServiceBus event, so other systems are informed about the deleted record. (The will possibly be other actions, like adding to an audit trail, updating other entities that were dependant on that record etc).

Conventional architecture in NServiceBus dictates that when the 'X' is clicked, a command is sent from the controller, and the handler for this command performs all the actions, including publishing the event.

But how do we update the grid? We can't just requery the data as we can't be certain the command has been processed. Common practice in NServiceBus is to do one of the following:

  1. We forward the user to a new view which says something like 'You request is being processed, it make take a moment for the grid to be updated' and a link back to the list of records.
  2. We manually remove the record from the existing grid using javascript.
The first option is fine if that is acceptable to the client, but often it is not. Udi says that we should move away from grids, but the fact is that they are often part of the specification. The second option could possibly lead to inconsistency between the business state and the displayed data, and can cause serious headaches when combined with paged & sorted grids.


Option 1 - The 'In Progress' Flag


This involves immediately setting a 'deleting in progress' flag, and then sending the command to carry out the rest of the work:
 
public ActionResult DeleteRecord(Guid recordId)
{
    using(var transactionScope = new TransactionScope())  
    {
        var record = _recordRepository.GetById(recordId);
        record.MarkAsDeletingInProgress();
        _recordRepository.Save(record);
        _bus.Send(new DeleteRecord { RecordId = recordId });
        transactionScope.Complete();
    }

    return RedirectToAction("Index");
}

And the message handler would look like this:
 
public void Handle(DeleteRecord message)
{
    var record = _recordRepository.GetById(recordId);
    record.Delete();
    _recordRepository.Save(record);
    _bus.Publish(new RecordDeleted{ RecordId = recordId });
}  

This way, we can return the grid and the record will either not be present or will be displayed as 'deleting in progress', so the user will have some definite feedback.

It is important that the flag is set and the command is sent within the same transaction to avoid inconsistencies creeping in. The 'using' statement above may not be needed if the request is within a transaction.

Option 2 - Request/Response


Generally frowned upon by the NServiceBus community, synchronous communication is an included feature and can be a useful option. If the command is sent, the message handler can update the database and publish the event. If the command is handled synchronously, by the time it has returned, we can be sure the data has been updated and we can therefore query it.
 
public void DeleteRecordAsync(Guid recordId)
{
    _bus.Send(new DeleteRecord { RecordId = recordId })
        .Register<ReturnCode>(returnCode => AsyncManager.Parameters["returnCode"] = returnCode);
}

public ActionResult DeleteRecordCompleted(ReturnCode returnCode)
{
    return RedirectToAction("Index");
}

And the message handler would look like this:
 
public void Handle(DeleteRecord message)
{
    var record = _recordRepository.GetById(recordId);
    record.Delete();
    _recordRepository.Save(record);
    _bus.Publish(new RecordDeleted{ RecordId = recordId });
    _bus.Return(ReturnCode.OK);
}  

This way, everything in our local domain is handled synchronously, while everything in other services/domains is handled asynchronously. There is even the option that the event can be handled in the local domain, and work can be done asynchronously there.

This may lead to some inconsistencies if the UI is gathering some of that asynchronously handled data, so this technique should be used with caution. However, in the right circumstances, this can be a good way of separating things that NEED to be synchronous from those that CAN be asynchronous.

There is no need for the using transaction statement in this case as NServiceBus message handlers are always run within a transaction by default.


Option 3 - Continuous Polling


Poll for completion and update the UI when the command has been completed. Don't do it.


Option 4 - SignalR


A technology I have not yet investigated. This could be interesting but without knowing more about it I can't comment further.

Option 5 - Publish Events from the Web Application


Another suggestion that raises eyebrows. The main reason for sending the command in the first place was so we can raise the event, so why not just do all the database work in the web application (or other assembly directly referenced) and raise the event from there? I won't cover this here because I intend to cover this and its problems in a future post. However, for now I will just list it as an option.


Thank you to Udi Dahan, Andreas Öhlund and Jimmy Bogard for posting on the thread, as well as the many other contributors. My particular favourite is the interaction Jerdrosenberg described here. I think there are a lot of us who have been through this scenario and it is the kind of thing that prompted me to start the thread and write this post.

7 comments:

  1. Given a grid UI, I'd question whether there is really an Event to be published.

    For instance: The user clicks X to delete. 3 seconds later they click undo. Now we we have two "Events" in flight. Now we have to account for this in our message design with an AsOf datetime stamp perhaps and to account for this in message handlers.

    In such a scenario, I begin to wonder if there is really an Event or is this a user just playing around with data. Maybe at some point they need to inform the system of a firm decision and there is a specific UI path that does this. At that, point fire an event.

    For a typical grid/crud screen, I wouldn't bring messaging into play on each action without a strong overriding reason.

    Ha. Perfect example. As I was about to submit my comment, I see a Preview button. Does that need an Event? Not really. Does the Publish button? Maybe.

    ReplyDelete
  2. :-D

    And the Publish button shows me "Your comment will be available after approval". Refreshing the page shows no comments.

    Great example.

    ReplyDelete
  3. Hi Kijana,

    I really does depend on what the record is, the fact that it is displayed in a grid is irrelevant - that's just how it is displayed.

    The event is not 'someone clicked delete' or 'a user tried to delete a record' - the event is 'a record was deleted'. The nature of the record really decides whether it is worth raising an event.

    The undo issue to common to many EDA scenarios and I believe outside the scope of this discussion.

    What if this grid is a list of employees in the company, and this is the central HR system? There may be other systems in the company such as a task list, and that system will need to know they have been deleted, so it can reassign all their work. OK, so in this example you may want to mark the employee as left rather than delete them, but the scenario still applies - the delete scenario was just a simple example.

    If for example, you are just deleting a post from a blog, there probably won't be other systems that are that interested, so you proabably wouldn't raise an event.

    The criteria of this situation was that an event had to be raised.

    ReplyDelete
  4. Yes. But the UI is what leads to the expectations of instant feedback. People look at a grid and feel they are working Excel. They think they should be able to manipulate it and save it and it is completely consistent, like Excel.

    Varying the UI can ease the transition to a system that must, for some reason, be eventually consistent. The user should be aware, at some level, that their actions have larger ramifications that "editing a grid".

    If it has to be a grid, some sort of, "ok I'm all done with my changes" button can help simplify the work _and_ gives the user the sense that they can manipulate the data, but make a firm decision later.

    ReplyDelete
  5. The scenario I'm describing is one where the grid and the instant feedback are part of the client's specification. These are specifications I am quite often tasked with. If it sounds like Excel then that may be no coincidence - user's are very familiar with Excel and so that may lead them to expect all UI's to be like that. If a grid with instant feedback is not the ideal solution to represent process flow then all I can do is advise the client of that, but at the end of the day, the client is always right.

    ReplyDelete
  6. I don't think you have to send an event when the item is deleted as the first user has commented. You may use a MVVM framework like Knockout to allow the user to make changes to the entity collection on the client (browser) and when the user clicks save, send a save command which can be handled by the NServiceBus.

    ReplyDelete
  7. You don't have to send an event, but it's possible that you may want to. As I have described, depending on the item, other services may well need to know it has been deleted, and a publish/subscribe event would be an appropriate way to do this. This scenario is about sending a message to be handled by NService bus, but dealing with the issue of how to give instant feedback in asynchronous situations.

    ReplyDelete