Up till now you have been working in the Studio, producing the Target- and Source Maps. In this section of exercises you will be leaving Studio behind for a while (don't be sad, you will meet again in later exercises). Now you have to turn the Maps from Studio into something that is one step closer to being able to execute the migration: c# code.
The first step is to move from the Studio to the Microsoft IDE Visual Studio. IDE is an Integrated Development Environment and Visual Studio is just that - a large and capable application with a wealth of features to facilitate application development of almost any kind. For the purposes of these training exercises - and indeed migFx - only a subset of the capabilities of Visual Studio come into play.
At the time of writing the version is Visual Studio 2017, but this may change as we strive to keep the training environment updated. As a general rule, always use the highest version of Visual Studio available.
The move from Studio to Visual Studio is done in steps as illustrated below:
- It is always a good idea to perform a Get Latest Version to ensure that you get hold of any modifications your team may have checked in to the Repository
- Validate the Map before you can publish it. You can only publish if there are no validation errors (warnings are ok)
- Once the validation passes without errors you can publish. This simply means that Studio will assemble all your mapping and write it to a file you specify
- The Engine generator will read the published file and produce (a lot of) c# code for you
- In Visual Studio you complement the generated code with the necessary manual Rule and Bag implementations as specified in Studio
- When ready the complete engine with generated code and manual rules can be deployed to the Director runtime environment, all ready to go
- This is where the migration actually executes, more on that later
migFx comes with an extension to Visual Studio that makes it easy to run the Engine generator and to deploy to the Director Runtime
Now, onwards to new and exciting challenges!
Open the Workshop Solution in Visual Studio
You should have the buttonin the Windows Taskbar. If not, you can easily find it in the Windows Start menu. Remember to open the newest version of Visual Studio:
Once you have opened Visual Studio, you may be lucky that the Workshop solution (sln) is listed in the Recent Projects list (1) and you can just click that one. If not, you must click the Open Project/Solution button (2) and navigate to and open the file Documents\MigFx\VisualStudio\Workshop\Workshop.sln.
Now the Workshop solution is open, and the Solution Explorer you will see the 3 groups that corresponds to the 3 projects delivered by us as part of your training setup: Sandbox, Workshop and WorkshopDemo. For the entire duration of these exercises, you are working on the Workshop project.
The Sandbox is an empty project setup for you to play around in later and the WorkshopDemo is a suggested solution to the training exercises.
Normally the Solution Explorer is docked on the right side of Visual Studio. If it is not visible, you can get hold of it from the Visual Studio Menu: View -> Solution Explorer.
Each of the 3 groups contains 3 c# class library projects. For the Workshop you are working on, they are
Start out by having a look at the Workshop.TargetEngine by expanding it a bit in the Solution Explorer tree view
- The _Generated folder contains all the files created by the Engine code generator. You will have a look in this folder in a short while
- The folders Bag, ConditionRule, EntryRule, ExitRule, FieldValidationRule and MappingRule all contain the manual implementations for the rules and bags
- The file PublishedTarget.xml is the file that is published from Studio and serves as input to the Engine Code Generator
- The MigFx.config file contains information that tells the migFx Visual Studio extensio what to do when generating code and deploying the generated engine to the Director Runtime
- The rest of the files and folder contains various functionality used by the manual code implementations
Generate the Target Engine, step 1
To generate the Target Engine there is a little back and forth between Studio and Visual Studio to agree on the name an location of the published file. Please note that this is only needed the first time for a given Map in Studio. Afterwards Studio will remember the file location.
- Right-click the Workshop.TargetEngine project and select Generate... in the migFx context menu
The migFx Visual Studio extension will now show a dialog with the path to the input file to the Engine generator (as well as a message telling either the age of the file or that the file does not exist)
You need to bring this file path to the Studio, so click the Copy button to copy it to the clipboard. Just leave the dialog open, you will get back to it shortly.
Publish from Studio
By now you should be fairly familiar with the Studio so we are not going to beat the same old horse around the track one more time. Just open the Workshop Target Map in Studio, do a top-level Get Latest Version and validate the Target Map. Before you can proceed to publish you must correct any validation errors (Alt-v if you forgot how to run the validation).
Now publish the Target Map, this time to the Generator: Studio Menu -> Project -> Publish to Generator (the last time you published the Target Interface to be imported into the Source Map).
Publish to Generator will show a dialog asking you for the name of the file to publish to.
Assure that this is the same as the one expected by the Engine generator by pasting in the file name from the clipboard.
Then click the Ok button to publish.
Generate the Engine, step 2
Now go back to the Engine generator dialog in Visual Studio you left waiting above. If you got the file name right, the message should now have changed and show that the file exists and is brand new.
Click the Ok button to run the Engine generator. Output from the generator is shown in the Visual Studio Output window docked at the bottom and any errors in the generation should show up here. Anyways, the generator never fails.
Please note that this 2-step back and forth between Visual Studio and Studio with the file name is only necessary the first time for a given Map in Studio. Next time you can just
- Publish in Studio, the correct file name is already there
- Generate in Visual Studio, the file is brand new
Include new Card related items
When you went through the Target Map exercises in Studio you of course made modifications. These modifications made the Engine generator produce a couple of new items that are currently unknown to the Workshop.TargetEngine project in Visual Studio. These items must now be included.
Please note that this step is only relevant for new Business Objects or when activating new metadata structures in Studio. Furthermore it is only relevant for the Target Engine. The Source Engine generator (more later) figures it out by itself.
To include the items, expand these folders in the Workshop.TargetEngine\_Generated folder
You will notice that for instance there is no ItemImporter for the Card Business Object. Now click the Show All Files button in the Solution Explorer Toolbar
Now the missing files will show up with a faded icon and all you have to do is right-click the 3 files
and select Include in Project from the context menu and then ctrl-shift-s to save everything.
That's it for the Target Engine for now. All the modifications you did in the Studio, Target Map exercises (Business Objects, Constants, Valuesets, Lookup Rules and Manual Rules) are now included in the generated code. Well ok, only part of the Manual Rules - they being manual means that you have do to some manual work in the next exercise for these.
While the Workshop.TargetEngine is present in Visual Studio as only one c# class library project, the Workshop.SourceEngine has two:
- Workshop.SourceEngineCustom. This is where you will implement the manual Rules and Bags
- Workshop.SourceEngineGenerated. This contains the generated code. You rarely need to bother with anything in this project. Unlike in the Target Engine, new items are included automatically. However, if somewhere down the line somethings is deleted in the Studio, you may have to manually delete it in the project too. No worries, if that is the case, the c# compiler will tell you what to do - very loudly!
Apart from the comments just above, the rest is basically the same as what you have just gone through for the Target Engine:
- Right-click any of the 2 Workshop.SourceEngine* library projects in Visual Studio and select MigFx: Generate code... in the context menu
- Click the Copy button to copy the file name to the clipboard
- Leave dialog open
- Open the Workshop Source Map in Studio
- Get Latest and validate
- Correct any validation errors
- Publish to Generator from the Project menu
- Paste in the file name
- Click the Ok button to publish
- Go back to the Visual Studio Generator dialog
- Verify the the file is now new
- Click the Ok button to generate
What happened here?
In this exercise you went through the steps required to get the work you have done on the Target- and Source Maps out of Studio, though the Engine Code generator and into Visual Studio. If you want to, you can now compile the 3 Workshop c# libraries (right-click and choose Build (or Build Selection if you selected more than one)) and they will compile.
Furthermore, you have now entered a very distinguished company as one of the most productive c# coders ever. In the steps above, the Engine generator has generated literally thousands of lines of c# code for you. If you are an OO hack you may find it interesting to peruse the generated code. You will find it in these folders:
Even though there is a lot of it, we are pretty sure that you will find it quite well structured and not that difficult to follow.
In fact there are some important points to be made here:
- You expressed your intention in Studio - albeit in a terse and structured manner - but nevertheless using terms and logic that make sense to everybody, not just a core developer team immersed in code (or ETL flows). In addition you expressed everything in a uniform and consistent manner. So your team will understand your work and you will in turn find it straightforward to understand the work of your team mates
- The Engine code generator generated the code to execute your intention. To the letter! Pitch perfect! There is absolutely no doubt that all the code produced by the Engine Code generator will do exactly what you specified in Studio. Working with migFx you too will experience that you never touch the generated code by hand. You will always go back to Studio, correct and generate again
- The consistency between your intention in Studio and the generated code is guaranteed - now and throughout the lifetime of your migration project. Compare that to the standard approach stating your intention in documents or spreadsheets for others to read, understand and implement in code (or ETL flows). While this approach may waddle through the first time it is inevitably bound for trouble. As the project progresses over numerous iterations, intention in documents and spreadsheets and implementation in code (or ETL flows) will drift apart - leading to wasted effort and loss of control
These core points are what makes migFx an animal so different and - frankly - superior to any other approach to data migration. No more - no less!
Next step is to manually implement any new or modified Rules/Bags