In this and the following exercise you will be loading the data necessary to migrate the new Card Business Object. First, in this exercise, you will load the data received from the Source System in to the staging table Src.DebitCard and then load the View CardStatus into the corresponding staging table for this view. 


Next, in the following exercise, you will load data into the staging tables for the new Valuesets TranslateCardTypes and CardTypes.


In this training setup, data is received from the Source System as comma separated files. These files are placed in a folder on the migration server dedicated to the actual track. In your training setup, this folder is Documents\MigFx\Runtime\Track\01 and the data from the Source System resides in the sub-folder SourceData\Src. In this training setup, we have placed the data files in the correct folder for you. In a real life migration, you would have to place the files in the correct folder yourself. 



Note the sub-folder Src corresponding to the alias of the metadata in Studio. It is possible to create and import multiple metadata collections into Studio, and they will end up a sub-folder, thus avoiding problems in case of name clashes.


Data from the Source System for Src.DebitCard has been delivered in two files, one for each bank being migrated.


Relating back to the Metadata in Studio

In the Studio Source Map you imported the metadata describing the source tables and the entire mapping done in Studio is based on these metadata. Now you are further along and about to load the actual data. It is of course crucial that the actual data conforms to the metadata. If this is not the case, the Director will be unable to load the data. To recap the process:

  1. The metadata is imported into Studio and serve as the basis for the Source Map
  2. When running the Setup of the resulting Source Engine in Director, the Source Engine will assure that the Staging database contains a corresponding table for each structure in the metadata.
  3. Loading data in the Director is done in 2 steps
    • The data are reformatted from one of the know formats to a common load format
    • The reformatted data are loaded into the corresponding table in the Staging database

If the actual data for some reason does not conform to the metadata, there is a good chance that either the reformat step or the load step above will fail. In this training setup up we have assured that the data conforms. But in a real life project, this is not an unusual problem.


In case of problems

From now on, we assume that the preceding exercises in Studio were completed correctly. If at any point you realize that you need to go back to a previous exercise to correct something in Studio, please remember the complete path any correction in Studio needs to go through in order to be operational in the Director

  1. Studio
    1. Validate without any validation errors
    2. Publish to Generator
  2. Visual Studio
    1. Generate Engine
    2. Deploy Engine, verify that deployment succeeds
  3. Director
    1. Restart Track
    2. Run Setup of relevant Engine  
    3. Check that Setup job completes successfully 


If you find yourself scratching you head over something that is not how you expect it to be, a good first thing to make absolutely sure is that the above has been done in correct order. 


Load Source Data

In order to load the data for Src.DebitCard, select Source/Data in the Director menu. The Director will show a list of all the tables in the Staging Database. The Src.DebitCard is there as well, but since it is brand new, it uses the default Reformatter DB2. You need to change the reformatter to Csv in the Reformat drop down.


Most tables are marked green, as they have already been loaded as part of earlier iterations of Business Objects Customer and Account. But Src.DebitCard is red - it has not been loaded yet. 



Apart from the obvious like Alias and Name, the meaning of the columns in the list is:


Files
The number of files to load. For instance, for DebitCard there are 2 files
Reformat
The reformatter to run on the received file to reformat it to the common format used to load data into the Staging database
Received file date
The date of the received file. If this file is newer than the Formatted file, it will be marked orange
Formatted file date
The date of the formatted file. If this file is newer than the loaded file, it will be marked orange
Date of loaded file
The date of the formatted file that was loaded into the Staging database. If this file is newer than the date the file was loaded, it will be marked orange
Loaded date
The date of the formatted file was loaded into the Staging database.
Baseline
A saved baseline of rows loaded into the Staging table
Loaded
The actual number of rows loaded into the Staging table



Now, load the the data in the 2 files into the staging table Src.DebitCard

  • Select the Src.DebitCard row
  • Ensure there is a check in Include Reformat
  • Click the Truncate & Load button
  • Click the Ok button in the confirmation dialog to submit a job to load the table


You can go to the Job List to see the progress of the job. Click the Instances button to see the job log. When the job finishes you can go back to the Tables list and click the Refresh button to update the Src.DebitCard row.


Load View CardStatus

Select Source/Views in the Director menu to see the list of Views:



CardStatus is in the list as expected, but not loaded yet. It has a green dot on the left to show it is ready to load. In other words that the items it depends on (in this case Src.DebitCard as you can see by clicking the Dependencies button) are all loaded.


The meaning of the columns is


Plan
migFx analyzes the view query and builds indexes to help it run. The number indicates how many iterations migFx did, before it decided that the query could not be optimized any further. If interested, you can click the blue link to see the query plan (takes a little while as it launches Sql Server Management Studio to visualize the query plan).
Loaded Date
The exact time the data for the view was loaded into the staging table.
Elapsed
The time it took to load the view.
Baseline
A baseline for the row count of the view. You can update the baseline by clicking the Baseline button.
Row count
The number of rows loaded into the staging table
  • Green if equal to Baseline
  • Red if blank (not loaded)
  • Yellow if different from baseline


Now load the staging table of the View CardStatus by selecting it in the list:

  • Ensure there is a check in Include Descendants. As you remember, in Studio it is possible to uses Views from other Views. By checking Include Descendant, you instruct the Director runtime to resolve these dependencies in a recursive manner. So if any views are using the View CardStatus, they get loaded too. Now, no other views are actually using the View CardStatus, but it is a good idea to leave this check mark be
  • Click the Load button and afterwards the Ok button in te confirmation dialog to submit a job to load the view
  • Check the job in the job list
  • When finished, refresh the View list to update the CardStatus row



What happened here?

Getting ever closer to actually migrating the Cards. In this exercise you have loaded the staging table for the data received from the Source System for Src.DebitCard as well as the staging table for the View CardStatus.


Next up is to load the staging tables for the Dynamic and Translation Valuesets.