Start a new topic

Publishing Maps and Deploying engines

This is a detailed topic in our support portal in the Using migFx series and assumes that you have some prior knowledge or experience using migFx. If you want to know more about hopp tech and migFx, our comprehensive solution for complex data migration, please visit our web site hopp.tech 


Once you reach a point where your mapping in Studio is consistent the natural next step is to apply your new mapping by migrating some data and see how it goes. In migFx, you generate a new engine from your mapping and deploy this engine so the Director runtime can discover it and load the new version of the engine. 


This article explains the steps to take to deploy a new engine and get it loaded and ready to run migrations.


Deploy and setup of a new Engine

The figure below illustrates the necessary steps to take to deploy a new engine to the Director. The process is identical, regardless of whether you are deploying a Source Map or a Target Map.



1. PublishThe validated Map is published from Studio to an xml file

2. GenerateThe Engine Generator reads this file and generates the new Engine code

3. DeployThe Engine code is compiled and the resulting load libraries (dll's) are copied to a location where the Director can find them

4. RestartThe track is restarted in the Director causing it to load the new Engine libraries

5. SetupA setup job is run in the Director to make any updates in the Migration- and Staging databases as required by the new Engine



1. Publish

Once you have validated your Map, you can publish it. Choose the Publish to Generator item in the Project menu in Studio (or press alt+G). Bear in mind the Studio will not let you publish a map before it is validated - and not if it has any validation errors.



Studio will prompt you for the path to an xml file where it is going to save the entire map. Studio and the Engine Generator must agree on this file location. Normally, the file resides within the Visual Studio project for the given engine. 


You only have to provide this path the first time you publish a given Map. For all subsequent publish, Studio will remember this path for you.


Once you click the Publish button, Studio will write the entire map to the specified xml file.


2. Generate

The Engine Generator will read the newly published xml file and generate the new engine code. You launch the generator from the context menu of the project in Visual Studio. For the Source Engine, this is the *.EngineCustom project.



Once you click the Generate... menu item, the Visual Studio will show a dialog with the path to the published xml file it will read to generate the code. This must be the same file that Studio wrote the published Map to in the previous step. As a sanity check, the dialog will tell you the age of the published file - in red, if the file is more than 10 minutes old.



When you click the Ok button, the generator will launch. You can follow the progress in the Visual Studio Output tool window



3. Deploy

The generator will have produced (usually quite a lot) of c# code. This code must now be compiled and deployed to the Director and ultimately be loaded into the relevant track. 


The Deploy step offers you the additional options of restarting the track and also running the setup job in the track. This is for convenience only, you are not required to do so and can launch these activities from the Director as described further on.


To Deploy, simply choose the Deploy... menu item from the same context menu in Visual Studio from where you launched the generator.



First Visual Studio will compile (build) the engine. But when it comes to the actual deploy, Visual Studio needs the help of the Director. 


It is the Director that has an connection to the runtime environment and also the Director that knows in which tracks the engine is actually running. So Visual Studio launches the Director, and the Director will query the runtime environment to show you a list of the tracks where the engine is running.



When you click the Deploy button, the Director will firstly deploy the engine to the runtime environment. It will zip the engine libraries and use the Director http connection to copy this zip file to the runtime environment and unzip the libraries to deployment location for this specific engine (as defined in the Director Administration).


Secondly and optionally, for each of the tracks listed (only one track in the screenshot above), the Director will restart the track to load the newly deployed engine and - also optionally - run the setup job for you.


If the Setup job would result in updates to the databases with potential data loss, the Director will prompt you to accept this before actually running the Setup.


4. Restart

If you choose not to restart a given track in connection to the Deploy above, you must restart the track from the Director. 



It is when the track starts, that the newly deployed engine is loaded into the track.


5. Setup

Again, if you choose not to run the setup in a given track in connection to the deploy above, you may have to run the setup job from the Director. 'May have to' only, because the setup is only necessary if the newly deployed engine requires updates to either the Staging- or the Migration database. 


Don't worry, the engine will refuse and fail any job trying to run anything if it misses any database updates - requiring you to run the setup before proceeding.



Again, if the Setup job would result in updates to the databases with potential data loss, the Director will prompt you to accept this before actually running the Setup. 


Conclusion

We hope that this article has given you a good view of how the Source- and Target maps in Studio are actually turned in to operational engines that are capable of running data migrations in the Director


Login or Signup to post a comment