Stakeholder Management Integration
- ivan.neganov (Unlicensed)
Introduction
Stakeholder Management service will serve as a “book of record” for information about Marine customers (aka “stakeholders”). It will store, validate and manage contact information of individuals and organizations, and expose it to other Marine business systems. While Stakeholder Management service is responsible for keeping this information in due order, it delegates its collection to other business applications, which the end users interact with directly. These applications will pass collected contact information to the Stakeholder Management Service, and get back a confirmation token with a globally unique stakeholder Id for each new and valid contact record. When retrieving contact information, the applications would issue queries to the Stakeholder Management Service, getting back records of contact information.
Vessel Registry Dependency on Stakeholder Management Service
Vessel Registry application is an example of a consumer of Stakeholder Management Service, as it needs to store and retrieve contact information associated with work items and vessel registrations that it manages. The requirements Vessel Registry application imposes on Stakeholder Management Service as its consumer are as follows:
Vessel Registry needs to asynchronously publish work item state change events, including creation or updates to contact information to inform the Stakeholder Management service of the changes. The asynchronous publish-subscribe pattern is essential for maintaining low latency of Vessel Registry application, which is built using a set of micro-services. Saga choreography pattern will be utilized to achieve data consistency between Vessel Registry and Stakeholder Management service.
Vessel Registry needs to execute queries against a low latency Stakeholder Management REST API to retrieve contact information given variable parameter values. For example, in a nowadays common “type-ahead” textbox scenario an end user would be typing text in a text field, and as the text is being entered the Vessel Registry application would be making multiple calls to Stakeholder Management REST API, asking it to filter contact records by parts of first or last name, or by parts of one or several address fields.
This proposed architecture is shown on the diagram below.
In this diagram a change to contact information made through Work Management service triggers a corresponding message published to an Azure Service Bus topic. The Stakeholder Management subscriber service would process the message, and in the “happy path” scenario respond back with a confirmation, which includes globally unique Stakeholder Id value for the contact record.
A Service Bus Processing Service on the “vessel registry end” subscribes to messages received from Stakeholder Management service, and updates vessel registry with received global stakeholder Id, or if transaction fails, rolls back the state of contact creation.
Vessel Registry application, or more precisely - its user interface - issues “GET” queries to the low latency Stakeholder Management REST API endpoint in response to end user activity (entering text). The API returns sets of contact information records matching the input parameters provided.
Phase I Integration with Stakeholder Management
Conceptual architecture proposed above reflects future vision of integration between Vessel Registry and Stakeholder Management. Initially however in order to validate the concepts involved in integrating both systems, a simpler architecture model will be followed:
On the above diagram, the existing Service Bus Processing Service, owned by Vessel Registry team will be modified to subscribe to an event of creation or an update made to a contact information. Once such event is received, the service will issue an HTTP call to existing RESTful Stakeholder Management API endpoint to “inform it of the changes”.
The details of Stakeholder API invocation are provided here.
Improving Observability
Asynchronous communication patterns are notorious for their difficulty in tracing and testing of individual parts separately. An effective diagnostics strategy one could employ is a combination of relying on Azure Application Insights for logging, and agreeing to use correlation Ids by owners of all parts involved in event-driven communication.
A correlation Id is a globally unique number, which should be created once the user initiates an operation on an application. As the call transcends through layers of the application, the promise developers would need to make is to pass this Id along the call stack. This includes passing it as a part of a message sent to service bus. Consumers of the message would promise to “look for presence” of the correlation Id, and if found - then log it.
If diagnostic logging is done to Azure Application Insights, and if the messages include correlation Ids, then once they end up in Application Insights, it is trivial to issue queries for specific correlation Ids and see the full history of a particular request from its initiation by the user to its fulfillment by the backend system. This is a powerful method of quickly identifying integration issues.
Absence of correlation Ids makes it quite difficult to identify diagnostic messages logged by different subsystems or micro-services and relate them to each other in a production diagnostic situation.