Exercise 4.5 - Execute Migration

Modified on Thu, 18 Jan 2024 at 07:12 PM

In the previous exercises in this section, you have prepared the Portal Operations and Runtime to actually migrate Cards. You have:

Deployed the Source and Target Engine, copying the engine libraries to a common location
Restarted the Portal Operations Track causing the Runtime to load your new engines
Run setup jobs to synchronize the Track and your engine. In this step, the Track discovered the new Business Object Card as well as new Source System metadata, Views and Valuesets
Loaded the staging tables for Source System data, Views and Valusets

So now you are - almost - ready to execute the migration. Well, in fact, you are, there is just a little thing you should note at this point.

Parameters

You probably remember the Constants in Studio and how they came in two flavours:

Static	The value for the Constant is provided in Studio and 'baked into' the resulting engine
Runtime	The value is supplied in the Portal Operations at runtime

Before starting any new migration iteration it is prudent to assert that the current values for the Runtime Constants are set correctly. You can see and correct the values for these by selecting Source/Parameters or Target/Parameters in the Portal Operations menu.

Here as an example are the Target Parameters:

In this case, the values of all the Parameters are correct, so you don't actually have to do anything.

Process

The migration is a two-step process involving Export and Import, entirely corresponding to the separation in Studio of the Source- and Target maps.

The Source Engine is executing the Export, exporting the Business Object instances from the staging database producing the Export result for each instance:

Storing the Source Data extracted from the staging database
Storing the Export Result
Storing all Events fired

The Target Engine is executing the Import on the exported Business Object instances producing the Target result for each instance:

Storing the Target Result
Storing all Events fired

Note that up until now, we have been working with the Business Object mapping in Studio as a blue-print for how the migration should work. Now we are switching to execution mode and applying this blue-print to actual instances of the Business Objects.

Export Business Object Card

Select Source/Objects in the Portal Operations menu to display the Business Objects to Export

Card is present in the list but obviously, no instances have been exported yet.

The meaning of the columns is:

Object	The Object is shown as a parent with one or more children. While this may makes less sense in this training setup, it does become useful if the copy function of the Source Map in Studio is brought into play. This ability to copy a Business Object in the Source Map is not part of the exercises, but it was touched upon at the end of Exercise 2.7 - Map Business Object CardStatus. If the 3 copies of Account given in the sample in this exercise were part of the setup, Object: Account would have 3 children in the Objects list: Object: Account Current Portfolio Guarantee This would allow you to export each separately.
Source	The count of Business Object instances in the staging tables (as defined by the Discriminator on the Card Business Object in the Source Map)
Dropped	The number of Business Object instances dropped during the Export. The Business Object is dropped if a Severity Drop is assigned to a raised Flag when exporting a Business Object
Discarded	The number of Business Object instances that were discarded during the Export by firing an Event with disposition Full discard
Exported	The number of Business Object instances that were Exported successfully
Difference	Any difference between Source and the sum of Dropped, Rejected and Exported. Any difference indicates that not all Business Object instances have been processed

There are no dependencies in the export of Business Objects. If you exported all 3, Account, Card and Customer, the Portal Operations would submit 3 jobs to run in parallel, one for each. In this exercise, however, you will only submit a job to export the Card Business Object

Click the Export... item in the context menu for the Card Business Object to show the Business Object Selection dialog

The Business Object Selection Dialog

With minor differences, the Business Object Selection dialog is used to submit jobs in several places in the Portal Operations interface. Let's have a quick look at it:

Available / Selected	Move Business Objects to the Selected box to include in the job(s) being submitted.
Key	If you know the key of a given instance of a Business Object, you can supply it here to process only this instance. The key is subject to an Sql-like query and should at a minimum be pre- and suffixed with the Sql wildcard %.
Limit	Type a number to limit the instances processed by the job.
Fast Track	Only available when exporting. For each instance, the Portal Operations runtime will: Execute the Source Engine Execute the Target Engine immediately, Only store the Target result. Unless the instance is Rejected, then everything is stored as normal
Auditor	The Auditor extension to call to collect Audit data for the Exported instances (advanced topic, not part of the training)
Target Auditor	For Fast Track: The Auditor extension to call to collect Audit data for the Imported instances (advanced topic, not part of the training)
Target Unloader	For Fast Track: The Auditor extension to call to unload the Imported instances (advanced topic, not part of the training)
Mode	Choose what to include: All: All instances Rejects: Only Rejected instances Difference: Any instances in the Differences column
Profiling Level	The generated code is instrumented and in addition, Hopp provides you with the possibility to instrument manual rule implementations. If a Profiling Level other than None is chosen, this instrumentation is activated and will log elapsed time in the execution tree. Useful to locate potential bottlenecks if the migration is suddenly performing badly.
Wait for Debug	If checked, the job will be submitted but immediately pause until a debugger is attached. Allows you to debug your manual rule implementations.

All you have to do is check that the Card Business Object is the only one in the Selected box and click the Submit button to submit the job.

You can follow the execution of the job in the Job list.

Import Business Object Card

Once the export job has finished it's time to do the import. Select Target/Objects in the Portal Operations menu.

The Objects list looks much the same as the Source/Objects list. The Exported column now shows the number of instances produced by the Export. But there is a fundamental difference between the Export and the Import. While there are no dependencies in the Export between the Business Object instances, there most certainly are dependencies in the Import.

Dependencies

These dependencies are the result of the Relationships between the Business Objects that were defined in the Target Map. After all, if a relationship from the Card Business Object to the Account Business Object is used to pick up a calculated value from an Account instance, this Account instance had better be imported before the dependent Card instance.

The Portal handles the resolution of these dependencies in several, different ways. First of all, there is the Full monty button. By clicking this you simply ask the Portal to import everything automatically resolving the dependencies along the way.

You can also import only selected Business Objects by selecting Import in the Context Menu to show the Business Object Selection dialog.

This selection dialog looks much like the same dialog for the Export above. There is one difference, though. Instead of the Fast Track option, the dialog now shows different options for handling dependencies, depending on whether you are importing multiple Business Objects in One Job or in Separate Jobs.

If you have selected multiple Business Objects for import, then you can decide whether to run all in just one job or to run a separate job for each Business Object. If you choose the latter option (separate jobs), it is up to you to ensure that you respect any dependencies between the Business Objects.

If only selected only one Business Object for import - or multiple in one job - you can then optionally ask the Runtime to include the ancestor chain and/or the descendant chain of the Business Objects selected for import.

Dependency resolution is recursive. In fact, the Portal Operations and Runtime simply split the import up into levels. All instances at a given level are imported before proceeding to the next level:

Level 0: Instances with 0 ancestors
Level 1: Instances with 1 ancestor
Level 2: Instances with 2 ancestors
....
Level n: Instances with n ancestors

Do the import

In the case of importing the instances of Business Object Card, it's quite simple. While Card has relationships to both Account and Customer, we know that there are no Business Objects with any Relationship to Card. So you can just select Card for Import and click the Submit button to submit the Import job. In this case, it makes no difference if you check Include Descendants or not.

You can follow the execution of the job in the Job list.

What happened here?

After much ado leading up to this crucial moment it probably turned out to be quite boring to actually migrate the Card Business Object instances by executing the Export and afterwards the Import. But, hey! That's somehow the entire point of all this, to provide a consistent and complete toolset to deliver complete confidence and a logical approach to break down this highly complex problem domain - so in the end the task really ends up being trivial.

Take a moment to reflect on what happened here

The generated Source- and Target engines - with a bit of help from your manual Rule implementations - have executed exactly what you specified in Studio
All Events fired due to some Flag being raised have been stored together with the actual values from the migration to be merged into the Event text for presentation
The data extracted from the Staging database and used for the export of each Business Object instance has been stored
The Export result delivered by the Source engine for each Business Object instance has been stored
The Target result delivered by the Target engine for each Business Object instance has been stored

Of course, everything remains somewhat under the hood in the Portal. In the next set of exercises, you will start out by publishing the migration results to the Portal Operations.

But there is actually stuff available already in the Portal Operations:

Try to select either Source/Events and/or Target/Events in the Portal Operations menu to see an aggregation of the Events fired in the migration. Notice, that you here as well have an Export or Import button respectively, making it easy to redo the given migration step for all the Business Objects that fired a given event
On the Export or Import job in the Portal Operations job list, you can click the Events button to see a list of the Events fired, even as the job is running