Highlights

With release 2.4 we are proud to bring these major improvements to Hopp:

• History on migrated Business Objects
  You can as always see the result of the latest iteration of the migration of a Business Object, but now you can also go back in time and see the Business Object as it was after previous iterations. And you can compare and see exactly what changed between iterations
  
  
• Performance improvements
  The Hopp Runtime now compares migration results on a hash code, and - if no changes - the runtime keeps the existing result in the migration database instead of stored the new (but identical) result once again. In addition, the compression of the migration result has moved from the database to the running job process. In combination, we have seen significant performance improvements of up to 25% in our internal setup.
  
  
• Multi column sort
  Lists in the Portal can now be sorted on multiple columns instead of just 1 column

Reading Guide

These release notes are organised into three Sections:

• New and Improved Features - which summarise the new and improved functionality provided in this release
• Issues fixed - which identify all issues corrected in this release
• Installation and Refactoring - which includes instructions as to how to apply this release to your installation

New and Improved features

This section includes a summary of the new functionality provided in this release, it is broken down into the three main components of the software:

• Portal
• Studio
• Engines
• Runtime
• Automation

Where available links are included to other articles in the Support Portal that provide more detailed information about the new features and how they can be utilised. If the change relates to a Ticket in the Support Portal the number of the Ticket is included in the header after the has symbol (#).

Portal

This section includes improvements to the browser based Portal used to manage the application and display the results of the migrations to the Users.

History on migrated Business Objects

The Portal has up to now shown the state of every Business Object as it is after the last iteration. And state from previous iterations of the same Business Object were forgotten (overwritten) by the most recent iteration.

With version 2.4, Hopp now provides the option to record the history of changes to Business Objects from iteration to iteration:

Different versions can be compared to see any differences in Source, Interface and Target data.

Read much more about this feature here: History on migrated Business Objects.

Multi column sort

The majority of the lists in the Portal now support multi-column sorting.

Full Monty import decommissioned

The previous Full Monty shortcut option to run the import on everything has been decommissioned:

The broad spectrum of Full Monty could in some instances create problems. Especially if some Business Objects had been run migrated using the Fast Track option (Ticket 2541 - Fast Track versus Full monty (requires access to ticket)).

To achieve identical functionality to the previous Full Monty, simply select Import... from the context menu and select all Business Objects for import:

Null values in Source Data

In previous versions, source fields extracted from the Staging database but containing null values were not displayed at all in the Portal Source Data view for a Business Object. This was different from Interface and Target data, where fields containing null values are displayed with the value given as NULL.

Now the Source Data view is aligned and will display the NULL values similarly to the other data views 

Formatting of Timestamp and Time data

Fields of Hopp data types Time and Timestamp are now formatted correctly in the Portal for Source, Interface and Target Data:

Studio

This section includes improvements to the Studio desktop application.

This release contains several bugfixes for Studio (see below) but no major, new functionality. 

Runtime

This section includes improvements to the Server based components of the software and its deployment.

Performance improvement

Significant work has gone into the Runtime in connection with the new capabilities to retain object history (see above). An added benefit is that the Runtime experiences significant performance improvements in the vicinity of 20%, in some situations even more.

Migration database schema changes

The 3 tables for the migrated Business Object data, [mig_item].[SourceXml], [mig_item].[ExportXml] and [mig_item].[TargetXml] no longer contains a column for the migrated xml, only a column for the binary, compressed xml. 

If you have bespoke functionality depending on the clear-text xml in these tables, please instead use the provided views [mig_valid].[SourceXml], [mig_valid].[ExportXml] and [mig_valid].[TargetXml] where the clear-text xml is available in the [XmlData] column.    

Migration xml changes

Attributes have been added to the generated xml to assist in comparisons. In addition, for the source xml, the data values have moved from inner text to a value attribute (as with interface and target xml) and empty elements are added for field containing null values

Source Xml

Interface Xml

Target Xml

Automation

This section includes improvements to the PowerShell Automation module. Please refer to the help documentation for each function for further information (Get-Help (function name)).

Set Baselines

New functions have been added to set baselines:

• Set-HpBaselineData: Set baseline for Source Data
• Set-HpBaselineViews: Set baseline for Views
• Set-HpbaselineValuesets: Set baseline for Valuesets in Source or Target 
• New-HpBaselinePortal: Creates an new Portal baseline and sets it current

Download Valuesets

A new function Get-HpValuesetDownload to download Valuesets to Excel. The function takes piped input and can be used in connection with the Get-HpValuesetList function.

For instance:

Get-HpValuesetList target -includeStatic | Get-HpValuesetDownload -folder "target path (must exist)"

Skip build step on Deploy-HpEngine

The function to deploy an engine now takes an optional second parameter for the path to the build output. If this parameter is specified, the function will not build the engine, but merely deploy it from the specified path.

Use this option if you require to take steps between building the engine and deploying the build result. For instance if you want to scan the build output before deploy.    

Full monty decommissioned

The function New-HpImportFullMontyJob to create a full monty import job has been decommissioned, If you are using this function, please replace with Get-HpEntityList | New-HpImportEntityJob

Issues fixed

The following bugs have been fixed in this release, where there is a Ticket on the Support Portal its number is included after a hash symbol (#):

• Export job forgets few items in a BO without any indication in Joblist #4534
  In rare cases, an export job meant to export everything may leave a number of items behind
  
  
• Change in Portal update causing issue #4521
  When submitting a Publish to Portal job with all Business Objects selected, the job did a selective delete in the relevant database tables instead of - as previously - the faster truncate
  
  
• EventUser table not cleaned up properly by Garbage collector over time #4517
  The EventUser table (and others) were not properly managed by the Garbage collector, potentially leaving behind a large number of obsolete rows
  
  
• Valueset page pagination issue #4255
  When submitting a job to update Valuesets, the submit dialog would only display the Valuesets present in the valueset list. This was an issue for larger lists of valuesets where the user would have to scroll down to the bottom of the list in order to have all valuesets included in the submit dialog
  
  
• Fast Track versus Full monty #2541
  When some Business Objects have been migrated with the Fast Track option, the Full Monty import would fail on these items
  
  
• Copy of lookup rule is created with wrong field order
  When a lookup rule is copied, the column order of the valuset columns available as lookup values are in the wrong order
  
  
• Sql Server Valueset: Null value for date results in DateTime.MinValue
  Loading a null Date from a Sql Server valueset results in a date value of 0001-01-01 instead of null
  
  
• Wrong # of decimals in Lookup rule select
  When using a standard lookup rule against a static valueset with a num 9(6) as key, the select returns no row even it should
  
  
• Load Translation valueset: Format Float removes decimals
  Values for columns of type Float including decimals when entered in the Portal Translation lose the decimals when subsequently loaded in Portal Operations
  
  
• Binary Translation Valueset columns not downloaded correctly
  When downloading a translation valueset from the Portal, columns of type Binary were not downloaded correctly (contained the text 'System.Byte[]')
  
  
• Lookup in Extraction map does not work on a copied child object
  After copy, the lookup list in a child object in the copied extraction map does not work
  
  
• Source Map: Copy ExitRule dumps
  Studio fails when attempting to copy an Exit Rule in the Source Map
  
  
• Assignments lost in Source Engine on exception
  If a Rule throws an exception in MapFields method the Source Engine, the assignments are never translated into the item xml. This is especially a problem on the root BO because then no interface fields will actually be created.
  
  
• Predicate Parm type Export does not work properly
  When double-click on Content on Predicate Parameters Value Export, the BO dropdown is empty and no selection is possible.
  
  
• Predicate Parm type Source does not work properly
  When double-click on Content on Predicate Parameters Value Source, a list of possible content values are shown in dropdown, but the current value is kept no matter which one I choose
  
  
• Studio crashes when closing static valueset with invalid cell value #4177
  When closing a static valueset that has a cell value that is invalid according to the datatype of the column, Studio crashes
  
  
• Cast is not copied for Export Field
  When copying an Export Entity in the Source Map, Cast on Export Field values are not copied
  
  
• Value change must remove Cast
  When changing value type the Cast indicator must be removed.
  
  
• Studio validation msg 0142 is misleading
  Studio wrongly prefixes the Cast required to validation error 0142: Assigning Key Field ({field1} to {field2}) in Lookup: return value: Cast required - {reason}
  
  
• Add -nobuild switch to PowerShell Deploy
  The PowerShell command to deploy an engine (Deploy-HpEngine) should have an option to skip the build step
  
  
• Remove LineageId from Identity path
  The lineageid attribute in the xpath stored with identities will result in a new identity for the same key if new children are inserted earlier in the sequence
  
  
• Nullable bool from Condition Rule
  A Bag Method that returns a boolean can be used as Filter, Condition, Entry and Exit rule. But if the Bag Method returns a nullable boolean, this results in validation assignment error 0042. Make the validation error clearer
  
  
• Unable to create new Item from Links dialog
  The dialog to add create a new Item and add a link from the Links list is broken. It is not possible to select a type for the new item
  
  
• Remove Bag Property from values available for Relationships in the Target Map
  Relationships have to be defined by using values that are known at the time of export. Bag Property values are not and should be removed from the values available for relationship lookup

Installation and Refactoring

Artefacts can be downloaded from the download page . Additional database refactor scripts are attached to these release notes

Database

• The Sql scripts to run are all attached at the bottom of these release notes
  
  
• Please back up all databases before running refactor scripts
  
  
• Run these refactor scripts
  • MasterDb.09.sql in the MigFx_Master database 
  • ProjectDb.06.0.sql in all Project databases
  • MigrationDb.15.sql in all Migration databases

Object history start up

Once version 2.4 is deployed, the option to start recording object history is set to false for the entire installation:

If you wish to activate recording of Object history, you need to set this option to true. Either for the entire installation in the master configuration or for selected tracks. 

Before doing so, it is strongly recommended to run a complete iteration in every active track for which object history is being activated.

This is because this initial iteration will update the source, interface and target xml to the new format required by the history and comparison functionality. When the history recording is then activated, the first version - and the oldest version of the history - will in the new format.

If this initial iteration without history recording is skipped, the oldest version in the object history will in fact be in the obsolete format of version 2.3. This may cause problems in the Portal.    

Workstations

Studio

• Reinstall Studio from the download page

Visual Studio

• Reinstall the Hopp Visual Studio extension (VsExtension.2022.vsix)
  
  

Visual Studio Engine Projects

• In all *.SourceEngineGenerated.csproj files
  • Update Microsoft.EntityFrameworkCore.SqlServer package reference to 8.0.7
    <PackageReference Include="Microsoft.EntityFrameworkCore.SqlServer" Version="8.0.7" />
    
    
  • Update Microsoft.Data.SqlClient package reference to 5.2.1
    <PackageReference Include="Microsoft.Data.SqlClient" Version="5.2.1" />
    
    
• In all *.TargetEngine.csproj files
  • Update Microsoft.EntityFrameworkCore.SqlServer package reference to 8.0.7
    <PackageReference Include="Microsoft.EntityFrameworkCore.SqlServer" Version="8.0.7" />
    
    
• If using Nuget
  • Update Hopp.SourceEngine package reference to 2.4.* in all *.SourceEngineGenerated.csproj files
    <PackageReference Include="Hopp.SourceEngine" Version="2.4.*"/>
    
    
  • Update Hopp.TargetEngine package reference to 2.4.* in all *.TargetEngine.csproj files
    <PackageReference Include="Hopp.TargetEngine" Version="2.4.*"/>
    
    
  • Update the Hopp.Generator tool
    dotnet tool update hopp.tools.enginegenerator --global
    
    
• If using local libraries
  • Reinstall MigFx.Libraries from download page
  • Reinstall MigFx.EngineGenerator from download page

Powershell Automation

• Re-install the PowerShell Automation module on relevant workstations.

Master Server

• Redeploy the Hopp Portal
  • Preserve the appsettings.json and wwwroot/index.html from previous 2.3 installation
    
    
• Upload the runtime.zip (from the download page) to the Portal 

Execution Servers

• Re-install Director.Agent.msi on all execution servers
  • Preserve master connection string in the config file
  • Remember to re-enter logon for the Hopp Agent Windows Service for the MigFx service account