New and Changed features

External Identity Provider

• The Hopp Portal now supports OpenID Connect providers, such as Microsoft Entra ID - formerly known as Azure Active Directory.
  
  

• By utilizing an external identity provider, you centralize user management. This means fewer passwords for users to remember and less administrative overhead for account creation, deletion, and password resets
  
  

• Important: Please note that the current option to use the Windows NTLM Negotiate authentication flow will be phased out in a future release. If you are using the Windows identity provider, it is highly recommended to transition to the OpenID authentication flow as soon as possible
  
  

• More information on External Identity Providers here: Setting up External Authentication

PowerShell Automation

• We are exited to include our brand-new PowerShell Automation module in this release. With PowerShell Automation, the jobs in the Portal Operations can be submitted and monitored from PowerShell. The PowerShell Automation brings major benefits in these 2 use-cases:
  • Orchestration: If your end-to-end migration is automated in an orchestration application like Tivoli, Automic, Control-M or similar, you can now integrate the Hopp migration flow
      
  • Interactive: Developers can now create their own schedules of jobs to run and the PowerShell Automation will monitor and submit the jobs for them - doing away with the need to have eyes on the job list in the Portal Operations UI in order to manually launch job in succession
    
    
• You can read much more about the PowerShell Automation here:  PowerShell Automation

NuGet libraries and Engine generator

• We have now made both the Hopp base libraries referenced by the Source- and Target Engine Visual Studio projects as well as the Engine Generator available on the public NuGet gallery.
  
  This enables our customers to build DevOps pipelines to generate, build and deploy the Hopp engines. Since the generator and the libraries are available on NuGet, it is not a requirement to install the EngineFramework and Hopp.Libraries specifically on the build server since these can be automatically restored from NuGet
  
  
• Importantly, we continue to support the current local install of the EngineFramework and the Hopp.Libraries specifically for customers deploying the Hopp software in a ringfenced environment without access to internet and the NuGet gallery
  
  
• Some refactoring is required in existing engine projects, see below

Portal

• Dashboard
  • The Dashboard is now included in the default Portal installation
    
  • Existing customers that currently do not have the Dashboard activated, will ned to add the Dashbords database to the Portal. See below under installation and refactoring
    
    
• My Projects filter
  • The Project Board can now be filtered to see only 'My projects' (the Projects of which I am a Team Member)
    
  • Note that the switch is not present if you are a member of all available projects (in which case there is no difference between All and Mine)
    
    
• appsettings.json - Configurable maximum allowed file size in megabytes that can be uploaded or used as an attachment

  "AttachmentConfiguration": {
    "MaximumSizeInMegaBytes": 100
  }

You do not have to add this section to the appsettings. If the setting is not present, the maximum download size will default to 100MB

• Operations
  • Data, Views and Valuesets pages allow database objects display with filtering by single column and column sorting
    
    
    
  • For filtering, a column name should be selected first and then a "CONTAINS" type search can be made or if a NULL search is needed, the NULL checkbox can be used
    
    
    
    
• ItemMgr
  • new app notification messages and email notifications when user is set as owner or assigned to an item
  • changes of the NotificationConfiguration in appsettings.json including existing Email configuration described in Installation and Refactoring section
    
    
  • Linking new item
    • when clicking the Add link button, a contextual menu appears with 2 options: "Existing element" and "New element"
    • users can create directly a link with a new Item (ItemMgr) from Test areas and from Item details dialog
    • when selecting the internal type of an element to be linked, the dropdown displays the internal types grouped by their source extension
      
      

Installation and Refactoring 

Artifacts can be downloaded from the download page. Additional database refactor scripts are attached to these release notes

Migrating to External Identity Provider 

If you are migrating from using Windows or MigFx as your IdentityProvider to an external identity provider, you can reuse existing users and maintain audit trails by following these steps:

• In the Portal: Update or verify that all user emails are accurate and up-to-date. This must be the emails users afterwards will use to authenticate through the external identity provider 
• Once the emails are correct, run the attached PortalDb.ExternalIdp.sql script in the Portal database
• Important: Please ensure that a given email address only appears once. If you have multiple users with the same email address, the script will pick the user that was created first. If this is not what you want, you may have to modify the script manually

Database

• Please back up all databases before running refactor scripts
• Run attached MigrationDb.12.0.sql script in all Migration databases
• Run attached MasterDb.05.0.sql script in in the Master database
  
  
• Customers that have not already included the Dashboards in the Portal
  • Create a new MigFx_Dashboards database (files can be placed in same folders as the MigFx_Portal database)
  • Add the MigFx_Dashboards connection string to the appsettings of the Portal
  • Download and run the MigFx.DashboardsDb.Setup.sql in the new database

Visual Studio

You can now opt in to reference the Hopp libraries and install the Engine Generator from NuGet. Alternatively, you can choose to continue to install the Hopp.Libraries and EngineFramework with the MSI installers provided for download.

In the Visual Studio options, you can now decide which model to use when adding new engine projects:

When the Use local libraries and engine framework option is set to True, new engine projects added to the solution will work as previously, that is the new project will reference the Hopp libraries as installed by the Hopp.Libraries MSI installer and execute the Engine Generator that is installed locally by the EngineFramework MSI installer.

On the other hand, when the option is set to False, new engine projects will contain a NuGet package reference to automatically restore the Hopp libraries required by the engine from NuGet. And when launching the Generate command for the engine, the Visual Studio extension will expect the generator to be installed via NuGet. 

Important: The default for the Use local libraries and engine framework option is False. If you wish to remain with local libraries and engine generator, you must change the value to True.    

Alternative 1: Switching to NuGet

• Edit all *.SourceEngineCustom csproj files
  • Remove all <Reference> elements
    
    
• Edit all *.SourceEngineGenerated csproj files
  • Remove all <Reference> elements 
  • Add a single PackageReference element:
    <PackageReference Include="Hopp.SourceEngine" Version="2.1.*" />
    
    
• Edit all *.TargetEngine csproj files
  • Remove all <Reference> elements 
  • Add a single PackageReference element:
    <PackageReference Include="Hopp.TargetEngine" Version="2.1.*" />
    
    
• Install the Hopp generator from NuGet with this command in PowerShell:
  dotnet tool install hopp.tools.enginegenerator --global
  
  
• Set the Visual Studio Hopp option Use local libraries and engine framework option to False
  
  
• Edit the MigFx.manifest file for all projects
  • For Source EngineCustom projects, change the Cmdelement for the Source generator
    • Change the FileName to hopp
      
      
    • Change the Arguments: 
      • Add generate source at the start of the arguments
      • Remove the --projectName option
      • Rename the --output option to --projectFile and change the value to name of the *.SourceEngineCustom.csproj file
        
        
    • Sample Cmd element for Source Engine:

      <Cmd name ="Hopp Source">
        <FileName>Hopp</FileName>
        <Arguments>generate source --verbose --formatCSharp --input "PublishedSource.xml" --projectFile "Workshop.SourceEngineCustom.csproj"</Arguments>
      </Cmd>

      
      

      
      

  •  For Target Engine projects, change the Cmdelement for the Target Generator
    • Change the FileName to hopp
      
      
    • Change the Arguments: 
      • Add generate target at the start of the arguments
      • Rename the --publishedXml option to --input
      • Rename the --output option to --projectFile and change the value to the name of the *.TargetEngine.csproj file
        
        
    • Sample Cmd element for Target Engine

<Cmd name ="Hopp Target">
  <FileName>Hopp</FileName>
  <Arguments>generate target --verbose --formatCSharp --input "PublishedTarget.xml" --projectFile "Workshop.TargetEngine.csproj"</Arguments>
</Cmd>

Alternative 2: Remaining with local libraries and Engine Framework

• Reinstall EngineFramework and Hopp.Libraries
  
  
• Set the Visual Studio Hopp option Use local libraries and engine framework option to True
  
  
• Edit the MigFx.manifest file for all projects
  • For Source EngineCustom projects, change the Cmdelement for the Source generator
    • Change the Arguments: 
      • Remove the --projectName option
        
        
    • Rename the --output option to --projectFile and change the value to name of the *.SourceEngineCustom.csproj file
      
      
    • Sample Cmd element for Source Engine:

      <Cmd name ="Hopp Source">
        <FileName>SourceEngineGenerator</FileName>
        <Arguments>--verbose --formatCSharp --input "PublishedSource.xml" --projectFile "Workshop.SourceEngineCustom.csproj"</Arguments>
      </Cmd>

      
      

      
      

•  For Target Engine projects, change the Cmdelement for the Target Generator
  • Change the Arguments: 
    • Rename the --publishedXml option to --input
      
      
    • Rename the --output option to --projectFile and change the value to the name of the *.TargetEngine.csproj file
      
      
    • Sample Cmd element for Target Engine

<Cmd name ="Hopp Target">
  <FileName>TargetEngineGenerator</FileName>
  <Arguments>--verbose --formatCSharp --input "PublishedTarget.xml" --projectFile "Workshop.TargetEngine.csproj"</Arguments>
</Cmd>

Security update

Microsoft is reporting a high-level security vulnerability in the System.Data.SqlClient 4.8.5 NuGet package. We recommend that you upgrade this Package reference to 4.8.6 in all SourceEngineGenerated projects 

Master Server  

• Redeploy Portal and all extensions
  • For this deploy, do not preserve your version of wwwroot/index.html but instead deploy the new version and update the base tag after deploy
    
• If you are using the Portal sign in form for authentication, you must update the Portal appsettings 
  • Change the Authentication.IdentityProvider setting from MigFx to Form
    
    
• Update the appsettings.json file of the Portal for the new app/email notifications. The packages for the Portal Server has default templates and the update is optional as follows:
  • Parent section is NotificationConfiguration and it contains: 
    • LogEntryTemplate - HTML text template with placeholders for one log entry; this is used in both app notifications and email notifications for element change
    • MessageTemplates - section for app notification templates
    • EmailTemplates - section for email notification templates
    • EmailSettings - section for email settings such email server, sender address, username, password, etc.
  • Full section description can be found here:  Setting up Notification and Email 
  • Changes from previous NotificationConfiguration manual adapatations required:
    • Rename "EmailConfiguration" section to "EmailSettings"
    • Move "EmailSubject" and "EmailBody" contents from root to "EmailTemplates.ElementChangeTemplate.EmailSubject" and "EmailTemplates.ElementChangeTemplate.EmailBody"
    • Move "DetailTemplate" content from root to "MessageTemplates.ElementChangeTemplate.ContentTemplate"
    • Move "TitleTemplate" content from root to "MessageTemplates.ElementChangeTemplate.TitleTemplate"
    • Remove "EmailSubject", "EmailBody", "DetailTemplate" and "TitleTemplate" sections from root