Orchestrating a complete migration

3rd party Orchestration applications like Octopus Deploy, Control-M, Automic Automation, Tivoli Workload Scheduler and others are built to handle the submission an monitoring of complicated flows of individual tasks (or steps/jobs/operations/etc depending on the terminology of the application in question).

The Hopp.Automation PowerShell module provides at set of PowerShell cmdlets that makes it possible to orchestrate the entire migration job flow in the Hopp Runtime using a 3rd party application.

A full end-to-end migration in the Hopp Runtime includes the steps in the figure below. This is of course just the bare bones. In many cases, it is beneficial to add manual stops to the flow for verification to take place before proceeding. 

And a great benefit of orchestration is of course that the execution of the migration in the Hopp Runtime can become part of a much larger flow with both predecessors and successors.

This guide will provide the PowerShell script for each of the single steps in the migration flow above and should serve as a good starting point for potential further customization.

Establish connection

Before any command can be issued, it is necessary to connect to the Hopp Portal. For the purpose of 3rd party applications, it is necessary to configure the Hopp Portal to accept connections from the application (client).

This is done by configuring a Client Id and a Client Secret in to configuration of the Hopp Portal. The application can then connect simply by providing this client id and secret.

Important: You must configure your Orchestration application to safely store and use the client secret, so the secret is not unduly exposed.

Every single step in the flow above must establish a connection, execute the Hopp Automation commands for the step and finally tear down the connection. This means that all the steps will use this template:

$portalUrl = "(the url of the Hopp Portal)" 
$trackID = "(The Guid of the track the step should act on)" 
$clientId = "(the Client Id configured in the Hopp Portal)" 
$clientSecret = "(the Client Secret configured in the Hopp Portal)"

# Connect to the Portal and the Track 
Connect-Hp -portalUrl $portalUrl -trackID $trackID -clientId $clientId -clientSecret $clientSecret

try {
    # (Hopp Automation commands go here)
}
finally {
    # Tear down the connection
    Disconnect-Hp
}

Restart Track

It is a good idea to restart the track at the beginning of the flow to ensure that the latest deployed Source and Target Engines are loaded into the Track.

Stop-HpTrack
Start-HpTrack

exit New-HpSetupSourceJob | Submit-HpJob | Wait-HpJob

exit New-HpSetupTargetJob | Submit-HpJob | Wait-HpJob

Load Source Valuesets

Load all dynamic Valuesets for the Source Engine. 

exit Get-HpValuesetList -engine "Source" | New-HpLoadValuesetJob | Submit-HpJob | Wait-HpJob

• Creates and submits a job to load all valuesets in the Source Engine
• Waits for the job to stop running. If the job completes successfully, the script will exit with code = 0, otherwise 1

Load Source Tables

Load all source tables for the Source Engine. 

exit Get-HpSourceTableList | New-HpLoadSourceTableJob | Submit-HpJob | Wait-HpJob

• Creates and submits a series of jobs to load all source tables  in the Source Engine. Will submit 1 job for each table
• Waits for the jobs to stop running. If all jobs complete successfully, the script will exit with code = 0, otherwise 1

Load Target Valuesets

Load all dynamic Valuesets for the Target Engine. 

exit Get-HpValuesetList -engine "Target" | New-HpLoadValuesetJob | Submit-HpJob | Wait-HpJob

• Creates and submits a job to load all valuesets in the Target Engine
• Waits for the job to stop running. If the job completes successfully, the script will exit with code = 0, otherwise 1

Load Source Views

Load all Valuesets for the Source Engine. 

exit Get-HpSourceViewList | New-HpLoadViewJob | Submit-HpJob | Wait-HpJob

• Creates and submits a job to load all views in the Source Engine
• Waits for the job to stop running. If the job completes successfully, the script will exit with code = 0, otherwise 1

Export

Export all entities in the track 

exit Get-HpEntityList | New-HpExportEntityJob | Submit_HpJob | Wait-HpJob

• Creates and submits a series of jobs to export all entities in the track. Will submit 1 job for each entity
• Waits for the jobs to stop running. If all jobs complete successfully, the script will exit with code = 0, otherwise 1

Import

Import all entities in the track. 

exit New-HpImportFullMontyJob | Submit-HpJob | Wait-HpJob

• Creates and submits a job to run a Full Monty import
• Waits for the job to stop running. If the job completes successfully, the script will exit with code = 0, otherwise 1

Publish

Publish migration result to the Portal. 

exit New-HpPublishToPortalJob | Submit-HpJob | Wait-HpJob

• Creates and submits a job to publish the migration result to the Portal
• Waits for the job to stop running. If the job completes successfully, the script will exit with code = 0, otherwise 1

Unload

Unloads all migrates entities. The Guid of the Unloader extension to use must be specified in the -unloaderID option

Get-HpEntityList | New-HpUnloadTargetJob -unloaderID "(Guid)" | Submit-HpJob | Wait-HpJob

• Creates and submits a job to load all valuesets in the Target Engine
• Waits for the job to stop running. If the job completes successfully, the script will exit with code = 0, otherwise 1

Summary

In many cases, the samples above are exactly what is needed to orchestrate the migration.

If not, the modular nature of the Automation module makes it easy an straght forward to create deeply customized automation to accommodate your requirements.

Check out other articles in this section for useful guidance.