Audit Collect interface

Modified on Tue, 20 Feb 2024 at 09:57 PM

This is a detailed topic in our support portal in the Using Hopp series and assumes that you have some prior knowledge or experience using Hopp. 


The Hopp Runtime will call the Audit Collect interface twice during the migration of a Business Object:

  • After the Source Engine has exported the Business Object, the Audit Collect interface is called with 
    • The source data that was extracted from the Staging database for the Business Object
    • The interface data that was produced by the Source Engine to be sent to the Target Engine

  • After the Target Engine has imported a Business Objects, the Audit Collect interface is called with
    • The interface data received from the Source Engine
    • The target data that was produced by the Target Engine


The Runtime will call the interface with Xml Documents representing the migrated data. While it is of course possible to implement the interface directly and thus receive the Xml Documents and expect the data using XPath strings to navigate the migrated data, this is not the recommended practice.


Instead, both the Source Engine generator and the Target Engine generator provide intermediate, abstract implementations of the interface in the generated Source- and Target Engines. The great benefit of this approach is that the generated Source- and Target Engines already contains generated Parsers that can be used inspect the migrated data in a strongly named and strongly typed manner. 


As a consequence, the recommended practice is to implement your own Audit Collection by deriving from these generated, abstract implementations. In this way, you no longer have to write 'magic strings' in order to inspect the migrated data in an Xml Document. Instead, you are provided with a very straight forward way of simply overriding methods and receive the strongly typed parsers for the migrated data.


This flow is illustrated in the figure below.



Let's have a look at how this looks in practice. Since the collection of Audit data is slightly simpler in for the Target Engine, we'll start with that one and deal with audit collection from the Source Engine afterwards.


Target Auditor

When the Runtime receives the Business Object result from the Target Engine, the Runtime will call the Target Auditor (if it exists). As mentioned above, the generated Target Engine contains an abstract implementation of the Target Auditor that you can use.


Public Target Auditor interface

First, this is the abstract base class that is exposed by the Runtime:

public abstract class Auditor : ExtensionBase<IAuditorContext>
{
  public abstract string Collect(bool discarded, string businessEntity, XmlElement interfaceXml, XmlElement targetXml);
}

So, the Runtime calls the Collect method every time a Root Business Object has been processed by the Target Engine. It passes these parameters:


discardedTrue if the root Business Object has been discarded
businessEntityThe name of the Root Business Entity
interfaceXmlAn XmlElement containing the interface data received from the Source Engine for the Business Entity
targetXmlAn XmlElement containing the target data produced by the Target Engine for the Business Entity


Again, this is the way the Runtime calls out to the Auditor passing the 2 XmlElements for the interface- and target data. While it is certainly possible to implement this directly and navigate the xml with magic strings, we recommend you instead make use of the class that is generated for you in the Target Engine. 


Generated Target Auditor base class

This is the generated, abstract AuditorBase class for a Target Engine that can migrate Customer and Account root Business Entities:

public abstract class AuditorBase : MigFx.Director.Server.Import.Audit.Auditor
{
  public override string Collect(bool discarded, string businessEntity, XmlElement interfaceXml, XmlElement targetXml)
  {
    var interfaceItem = InterfaceItem.Parse(interfaceXml);
    var targetItem = TargetItem.Parse(targetXml);

    return businessEntity switch
    {
      "Account" => Account(discarded, interfaceItem.As().Account.Parse(), targetItem?.As().Account.Parse()),
      "Customer" => Customer(discarded, interfaceItem.As().Customer.Parse(), targetItem?.As().Customer.Parse()),

      _ => throw new ApplicationException($"Unknown BusinessEntityName passed to Audit collector: {businessEntity}")
    };
  }

  public virtual string Account(bool discarded, Parsers.Interface.IAccountParserProvider.IParser interfaceItem, Parsers.Target.IAccountParserProvider.IEditable.IParser targetItem) => null;
  public virtual string Customer(bool discarded, Parsers.Interface.ICustomerParserProvider.IParser interfaceItem, Parsers.Target.ICustomerParserProvider.IEditable.IParser targetItem) => null;
}

As you can see, the generated AuditorBase implements the Collect from the abstract interface method and receives the 2 XmlElements from the Runtime. 


However, the generated AuditorBase class then maps the incoming Collect call from the Runtime to a generated, virtual method for each root Business Entity, in this case Customer and Account.


The good thing is, that when it calls for instance the generated method for Account, it also parses the interface- and target xml so the generated, virtual method for Account receives a strongly typed, generated parser it can use to inspect both the interface- and the target data for an Account in a strongly named and strongly typed fashion.


In short, the generated AuditorBase bridges the gap between the XmlElements received via the interface shared between the engine and the Runtime and the strongly typed, generated Parsers known only in the generated Target Engine.


Note that the generated AuditorBase is abstract. This means that the Runtime will not discover it and use it for Audit collection. For this to happen, you must create your own Auditor that derives from the AuditorBase.


The actual Target Auditor

Here's a very basic, sample implementation of a Target Auditor that only implements audit collection for Account by overriding the virtual Account method:

[Extension("Demo Auditor")]
internal class Auditor : AuditorBase
{
	public override string Account(
		bool discarded,
		Parsers.Interface.IAccountParserProvider.IParser interfaceItem,
		Parsers.Target.IAccountParserProvider.IEditable.IParser targetItem
		)
	{
		var xml = new XElement("Account",
			new XElement("balance", interfaceItem.Fields.Balance),
			new XElement("productCode", interfaceItem.Fields.ProductCode)
			);

		return xml.ToString(SaveOptions.DisableFormatting);
	}
}

Note that in the end, the Auditor is an extension to the Hopp Runtime just like Reformatters, Unloaders etc. As such, you can

  • Decorate it with an Extension attribute to decide how the extension should be named in the Extension Usage panel in the Portal Operations
  • Provide parameter properties to enable parameterization of the extension in the Extension Usage panel in the Portal Operations 


The Auditor can use the interface and target parsers it received as parameters to inspect the migrated data. In the very simple sample above, the Target Auditor returns an Xml element containing the ProductCode and the Balance interface fields of the Account. It uses the strongly typed interface item parser to retrieve the values. 


Since it has not overridden the Customer virtual method of the generated base class, it does not implement any audit data collection for the Business Object Customer,


The Auditor returns a string that the Runtime will receive and store next to the migrated root Business Entity. Or null if no data should be stored. In the example above, the Auditor creates an xml element with the audit data it wishes to store and returns is as a string back to the Hopp Runtime.


Important note

In order for your Target Engine to be enumerated by the Runtime extension discovery process, it is vital that the assembly has been decorated with the MigDirectorExtension attribute. This is done by adding the attribute to an ItemGroup element in the TargetEngine.csproj file:


<ItemGroup>
  <AssemblyAttribute Include="MigFx.Director.MigDirectorExtension" />
</ItemGroup>


Source Auditor

The Source Auditor flow is similar to the Target Auditor. The Runtime call a generic interface with the extracted source data and the interface data produced by the Source Engine as Xml documents. And the Source Engine generator creates a generated base class that implements the generic interface and call virtual methods with the strongly typed parsers.


However, given the ability to copy a Business Object in the Source Map in Studio in order to define different extraction maps for the same Business Object (read more about this here) there is an extra layer in the implementation to allow for the fact that ere can be different extraction maps - and thus different source data parsers - for the same Business Entity. 


As an example, let's assume that the Business Object Account has been copied in the Source Map so there is the original, unnamed copy as well as a copy the has been named Loan:



When looking at the extraction map for Account [Loan], the root object in the Extraction Map has been named AccountAsLoan:



In this case, the Runtime will call the Source Auditor when it receives the exported Business Entity from the Source Engine. The Runtime calls the public Source Auditor interface.


Public Source Auditor interface

The Runtime calls the Collect method every time a Root Business Object has been processed by the Target Engine.  

[ExtensionType(ExtensionType.SourceAuditor)]
public abstract class Auditor : ExtensionBase<IAuditorContext>
{
  public abstract string Collect(bool discarded, string businessEntity, string name, XmlElement sourceXml, XmlElement interfaceXml);
}

It passes these parameters:


discardedTrue if the root Business Object has been discarded
businessEntityThe name of the Root Business Entity as it is named in the Target Map
nameThe name given to the Business Entity in the Source Map (Loan in the sample above). Null, if the Business Entity is not named in the Source Map
sourceXmlAn XmlElement containing the source data extracted by the Source Engine from the Staging database for the Business Entity
interfaceXmlAn XmlElement containing the interface data produced by the Source Engine to be sent on to the Target Engine


Generated Source Auditor base class

This is the generated, abstract AuditorBase class for a Source Engine that can migrate Customer and Account root Business Entities. Given that the Account Business Entity in our sample has been copied, it is slightly more evolved than its Target cousin:

public abstract class AuditorBase : MigFx.Director.Server.Export.Audit.Auditor
{
	public override string Collect(bool discarded, string businessEntity, string name, XmlElement sourceXml, XmlElement interfaceXml)
	{
		var sourceItem = SourceItem.Parse(sourceXml);
		var interfaceItem = InterfaceItem.Parse(interfaceXml);

		switch (businessEntity)
		{
			case "Account":
				{
					var interfaceParser = interfaceItem.As().Account.Parse();

					return (name ?? "Unnamed") switch
					{
						"Unnamed" => Account(discarded, sourceItem.As().Account.Unnamed.Parse(), interfaceParser),
						"Loan" => Account(discarded, sourceItem.As().Account.Loan.Parse(), interfaceParser),
						_ => throw new ApplicationException($"Unknown Name passed to the Account Audit collector: {name}")
					};
				}
			case "Customer":
				{
					var interfaceParser = interfaceItem.As().Customer.Parse();

					return (name ?? "Unnamed") switch
					{
						"Unnamed" => Customer(discarded, sourceItem.As().Customer.Unnamed.Parse(), interfaceParser),
						_ => throw new ApplicationException($"Unknown Name passed to the Customer Audit collector: {name}")
					};
				}
			default:
				throw new ApplicationException($"Unknown BusinessEntityName passed to Audit collector: {businessEntity}");
		}
	}

	public virtual string Account(bool discarded, Parsers.Source.IAccountParserProvider.IUnnamed.IParser sourceItem, Parsers.Interface.IAccountParserProvider.IParser interfaceItem) => null;
	public virtual string Account(bool discarded, Parsers.Source.IAccountParserProvider.ILoan.IParser sourceItem, Parsers.Interface.IAccountParserProvider.IParser interfaceItem) => null;

	public virtual string Customer(bool discarded, Parsers.Source.ICustomerParserProvider.IUnnamed.IParser sourceItem, Parsers.Interface.ICustomerParserProvider.IParser interfaceItem) => null;
}

Again, the generated AuditorBase implements the Collect from the abstract interface method and receives the 2 XmlElements from the Runtime and maps to generated, virtual methods that are called with the generated, strongly typed parsers for the source- and interface data.


The difference is that the generated base class contains 2 different virtual method overloads for the Account Business Entity, one overload accepts the typed source Item for Unnamed Account and the other overload accepts the typed source item for the Account named Loan.


The actual Source Auditor

Here's a very basic, sample implementation of a Source Auditor that only implements audit collection for both copies of Account by overriding the virtual Account method overloads. Since is does not override the default overload for the Customer, no Audit data is collected for the Customer Business Entity.

[Extension("Demo Auditor")]
public class Auditor : AuditorBase
{
	// Audit Unnamed Account
	public override string Account(
		bool discarded, 
		Parsers.Source.IAccountParserProvider.IUnnamed.IParser sourceItem, 
		Parsers.Interface.IAccountParserProvider.IParser interfaceItem
		)
	{
		var xml = new XElement("Audit",
			new XAttribute("type", "normal"),
			new XElement("balance", sourceItem.ExtractionMap.Account().Fields.Balance),
			new XElement("ProductType", sourceItem.ExtractionMap.Account().Fields.ProductType)
			);

		return xml.ToString(SaveOptions.DisableFormatting);
	}

	// Audit Loan Account
	public override string Account(
		bool discarded, 
		Parsers.Source.IAccountParserProvider.ILoan.IParser sourceItem, 
		Parsers.Interface.IAccountParserProvider.IParser interfaceItem
		)
	{
		var xml = new XElement("Audit",
			new XAttribute("type", "loan"),
			new XElement("balance", sourceItem.ExtractionMap.AccountAsLoan().Fields.Balance),
			new XElement("ProductType", sourceItem.ExtractionMap.AccountAsLoan().Fields.ProductType)
		);

		return xml.ToString(SaveOptions.DisableFormatting);
	}
}

In both overridden Account methods, the Auditor collects the ProductType and the Balanace from the sourceItem = the data that was extracted from the Staging database for the Account.


The difference between the 2 overrides is that the first one deals with the extraction map of the unnamed Account and the second one deals with the extraction map of the Loan account.


Important note

In order for your Source Engine to be enumerated by the Runtime extension discovery process, it is vital that the assembly has been decorated with the MigDirectorExtension attribute. This is done by adding the attribute to an ItemGroup element in the SourceEngineCustom.csproj file:


<ItemGroup>
  <AssemblyAttribute Include="MigFx.Director.MigDirectorExtension" />
</ItemGroup>


Conclusion

 The Runtime and the generated engine code work in unison to make it easy to implement Source- and Target Auditors and have them called by the Runtime when the engines have processed each Business Entity.


Implement your Auditors by overriding the relevant virtual methods in a class deriving from the generated AuditorBase in each engine.


When you override a method in your Auditor for a  specific Business Entity, the generated infrastructure will pass the migration data to that method a strongly typed parsers allowing you to inspect the migrated data in a name and type safe manner.  


You auditor then return a string representation of the collected audit data to the Runtime. The Runtime will then store audit data next to the Business Entity and handle all housekeeping of the audit data in connection to reiteration of the same Business Entity.


So, this is the collection of audit data. But that is of course only half the story. In the next article we will explore how you can get the audit data out of the Runtime to serve your reconciliation infrastructure.




Was this article helpful?

That’s Great!

Thank you for your feedback

Sorry! We couldn't be helpful

Thank you for your feedback

Let us know how can we improve this article!

Select atleast one of the reasons
CAPTCHA verification is required.

Feedback sent

We appreciate your effort and will try to fix the article