Ometa Framework Release v5.3.0

Version 5.3.0 of the Ometa Framework has been released with the following new features. With version 6.0.0 we are migrating all XML files to a database. This requires some "minor" changes which will not be treated as breaking. Refer to the Changelog for a list of all bug fixes.

Before Upgrade

Analyze the changes.
Make sure you have installed Ometa Framework 5.2.x. It will not be possible to upgrade to 5.3.0 from a lower version.
Uninstall all .NET Core 2.x and .NET Core 3.x versions. Note that there can be a lot of entries which needs to be uninstalled.
Install .NET 6.0.x Hosting Bundle.
Update the Connectionstrings %OMETA_INSTALL_ROOT%\Ometa Software Suite\appsettings.json and add TrustServerCertificate=True; to the connection string. Another solution is to configure a certificate on your SQL server(s) which is signed by a trusted authority. Consult this article for more information about using/configuring certificates on SQL server.
```
 For example: "OmetaFrameworkDatabase": "Data Source=sqlserver;Initial Catalog=ometadb;Trusted_connection=yes;TrustServerCertificate=True;",
```

During Upgrade

During the upgrade, it is possible that the BCRPatcher gives the following error(s):

Error message: No connection could be made because the target machine actively refused it 127.0.0.1:2005

Error message: Internal Server Error

Open the Application Event Log. You probably find the following entries for BCM:

Maximum number of retries (6) exceeded while executing database operations with 'SqlServerRetryingExecutionStrategy'. See inner exception for the most recent failure.  
A network-related or instance-specific error occurred while establishing a connection to SQL Server. The server was not found or was not accessible. Verify that the instance name is correct and that SQL Server is configured to allow remote connections. (provider: Named Pipes Provider, error: 40 - Could not open a connection to SQL Server)  
The network path was not found

Restart the BCM Service.
Perform an iisreset.

During the setup, it is possible that you get the following error:

A previous version [version] has been detected which cannot be upgraded by this version. Please install version 5.2.x first before upgrading to this version.

Make sure to upgrade the Ometa Framework version 5.2.x.
It is possible that the %OMETA_INSTALL_ROOT%\Ometa Software Suite still has an older reference of the Ometa Framework DLL. This could be because there was a copy made of folder. Remove this backup folder first and try again.

Previous version detected

During the upgrade, it is possible that the BCRPatcher gives the following error:

- The following error(s) can contain more information to help you diagnose the problem:
        - A connection was successfully established with the server, but then an error occurred during the login process. (provider: SSL Provider, error: 0 - The certificate chain was issued by an authority that is not trusted.)
        - The certificate chain was issued by an authority that is not trusted

Patcher error

Update the Connectionstrings %OMETA_INSTALL_ROOT%\Ometa Software Suite\appsettings.json and add TrustServerCertificate=True; to the connection string.

For example: "OmetaFrameworkDatabase": "Data Source=sqlserver;Initial Catalog=ometadb;Trusted_connection=yes;TrustServerCertificate=True;",

After Upgrade

Open the %OMETA_INSTALL_ROOT%\Ometa Software Suite\appsettings.json.
Remove the OmetaFrameworkDatabase key which still refers to the OmetaBusinessConnector or OmetaFramework database in the %OMETA_INSTALL_ROOT%\Ometa Software Suite\appsettings.json.

Rename the OmetaDcsDatabase key to OmetaFrameworkDatabase in the %OMETA_INSTALL_ROOT%\Ometa Software Suite\appsettings.json.

 "ConnectionStrings": {
         "OmetaFrameworkDatabase": "Data Source=sqlserver;Database=OmetaDynamicCaseSystem;Trusted_Connection=yes;TrustServerCertificate=True;",
         "OmetaBamDatabase": "Data Source=sqlserver;Database=OmetaBAMLogging;Trusted_Connection=yes;TrustServerCertificate=True;"
 }

All the appsettings.json files under %OMETA_INSTALL_ROOT%\Ometa Software Suite\Services can be migrated to the global %OMETA_INSTALL_ROOT%\Ometa Software Suite\appsettings.json.

 {
   "BamConfig": {
     "Enabled": true,
     "Destination": "File",
     "LogDirectory": "C:\\Program Files (x86)\\Ometa BVBA\\Log",
     "PortalUrl": "",
     "MaxPropertySize": 100,
     "MinimumLogLevel": "Information",
     "ProcessOverrides": [],
   },
   "LogProcessing": {
     "OperatingMode": "Fixed",
     "FileProcessing": {
       "Enabled": true,
       "IntervalTimeMinutes": 1
     },
     "Cleaning": {
       "Enabled": true,
       "CleaningTime": "23:30:00",
       "DaysToKeep": 15,
       "MaximumLogDataSizeGByte": 10.0
     },
     "Maintenance": {
       "Enabled": false,
       "MaintenanceDay": "Sunday",
       "UpdateStats": false,
       "UpdateIndexes": "None",
       "DatabaseShrinking": false
     }
   },

   "ConnectionStrings": {
     "OmetaFrameworkDatabase": "Data Source=sqlserver;Initial Catalog=OmetaDynamicCaseSystem;Trusted_connection=yes;TrustServerCertificate=True;",
     "OmetaBamDatabase": "Data Source=sqlserver;Initial Catalog=OmetaBAMLogging;Trusted_connection=yes;TrustServerCertificate=True;"
   },

   "FrameworkBCFolder": "C:\\Program Files (x86)\\Ometa BVBA\\Ometa Software Suite\\TopDir\\BC",

   "AllowedHosts": "*",
   "AllowedOrigins": [
     "*"
   ],
   "SigningCertificate": "CN=*.yourdomain.net",

   "AppSettings": {
     "EnableCompression": true,
     "UseUnsafeUserContext": false,
     "CoreServiceUrl": "https://coreservice.domain.net",
     "SharePointAuthentications": [
       {
         "Source": "",
         "IsCloudEnvironment": true,
         "Domain": "",
         "Username": "",
         "Password": "",
         "ClientId": "",
         "ClientSecret": "",
         "AzureKeyVaultUrl": "",
         "SharePointOnlineAzureADCertificate": "",
         "SharePointOnlineTenant": ""
         }
     ],
     "SharePointRewriteUrls": []
   }
 }

Rename all appsettings.json files under %OMETA_INSTALL_ROOT%\Ometa Software Suite\Services, so they can be safely removed after the portal have been validated.
When this is a new clean installation, run the Ometa.Framework.Infra.DbMigration.exe tool: %OMETA_INSTALL_ROOT%\Ometa Software Suite\TopDir\BC\Bin\DbMigration. Refer tot the Post Installation Settings for more information.
Perform an iisreset in CMD or PowerShell under administrative permissions.
Validate the portals.
Remove all appsettings.json files under %OMETA_INSTALL_ROOT%\Ometa Software Suite\Services.
We no longer need the Ometa Framework database, because we have migrated all the data to the Ometa Dynamic Case System database, which is the main database for our solution. Removing the Ometa Framework database will free up some disk space and improve performance. However, please make sure you have a backup of the Ometa Framework database before you delete it, in case you need to restore it later.

Changes

Database Merge

In the upgrade, we'll be merging the tables from [Ometa Framework] database into the [Ometa Dynamic Case System] database. This only affects tables maintained by the Ometa Framework, custom tables are not moved. There are no changes to the column names of any table. The only change is that AuditTrail and AuditTrail were moved to the audit. scheme (from dbo.). During the setup, the Config entry 'Database Schema Version' will be checked. If this value is not set or smaller than 3, the setup will merge the databases.

By default, we'll use connection strings from the previous version of the framework. To migrate the data from the [Ometa Framework] to the [Ometa Dynamic Case System] database, the [Ometa Dynamic Case System] connection string is used to access both databases through a SQL script. Thus, for this to work, this user should have access to both databases for this migration.

This should only take a few minutes. Data on the old [Ometa Framework] database is not removed. This database can be dropped in future, but we recommend keeping it in case of issues. The BAM database remains unaffected.

The connection string for the [Ometa Framework] can be removed from the appsettings.json files, but is not necessary. The framework will prefer the OmetaDcsDatabase connection string over OmetaFrameworkDatabase to remain backwards compatible.

Important

Profiles that use the [Ometa Framework] database should now also use the [Ometa Dynamic Case System] database. Any interface scripts that referred to the old Ometa Framework in their queries, should also be changed to the new [Ometa Dynamic Case System] database.

Manual Database Merge

This section is only relevant if your server infrastructure can not allow the user of the [Ometa Dynamic Case System] connection string to access the [Ometa Framework] database, or if the databases are on different SQL servers.

The recommended action is to let the setup run its regular course until the data migration will show an error in the console. At this point it will offer to retry, or the 'manual' option. Empty tables will be already created at this point on the [Ometa Dynamic Case System].

The following tables do not need to be migrated as they were deprecated in previous versions:

[dbo].[__EFMigrationsHistory]
[dbo].Combiners
[dbo].Config
[dbo].Operators
[retry].[Settings]
[retry].[Actions]
[opcua].[DataExtensionProfiles]

When finished, set the Database Schema Version value in the Config table (in [Ometa Dynamic Case System]) to 3.

Installation And Configuration

SQL Server Certificate

Since the update to Microsoft.Data.SqlClient v4, encryption is used for the connections to the SQL databases.

In many cases (e.g.: local SQL servers) the encrypted connection will fail with this error:

A connection was successfully established with the server, but then an error occurred during the login process. (provider: SSL Provider, error: 0 - The certificate chain was issued by an authority that is not trusted.)

It means that the certificate, used by the SQL server, is not trusted by the client. Local SQL servers use self-signed certificates by default.

Solution 1: TrustServerCertificate=True

The first and easiest solution is to force the client into trusting the SQL server certificate. This can be easily configured by adding TrustServerCertificate=True to the connection strings in the appsettings.json file(s).

"ConnectionStrings": {
    "OmetaFrameworkDatabase": "Data Source=SQL\\MYSQLSERVER;database=OmetaFramework.Master;trusted_connection=yes;",
    "OmetaDcsDatabase": "Data Source=SQL\\MYSQLSERVER;database=OmetaDynamicCaseSystem.Master;trusted_connection=yes;",
    "OmetaBamDatabase": "Data Source=SQL\\MYSQLSERVER;database=OmetaBAMLogging;trusted_connection=yes;"
}

Becomes

"ConnectionStrings": {
    "OmetaFrameworkDatabase": "Data Source=SQL\\MYSQLSERVER;database=OmetaFramework.Master;trusted_connection=yes;TrustServerCertificate=True;",
    "OmetaDcsDatabase": "Data Source=SQL\\MYSQLSERVER;database=OmetaDynamicCaseSystem.Master;trusted_connection=yes;TrustServerCertificate=True;",
    "OmetaBamDatabase": "Data Source=SQL\\MYSQLSERVER;database=OmetaBAMLogging;trusted_connection=yes;TrustServerCertificate=True;"
}

Solution 2: Use Trusted Certificate

Another solution is to configure a certificate on your SQL server(s) which is signed by a trusted authority.

Consult this article for more information about using/configuring certificates on SQL server.

Generic REST

The configuration value AuthorityUrl is fully removed from the appsettings.json file. Make sure the correct URL is defined in the item with the name Authority Service Url in the dbo.Config table, as that will be the only source of the setting.

See Database Config Table for more information on how to set the correct value.

BCA / BCJS / BCM / BCSL / BCSP Framework Services

Previously most configuration settings were stored in the Windows Registry. But due to a consolidation of configuration sources, from registry, appsettings.json files and dbo.Config table to only a single appsettings.json file and dbo.Config table, all settings from the Windows Registry are moved to the dbo.Config table.

During an upgrade, the framework patcher will copy over most settings from the Windows Registry to the dbo.Config table.

With a fresh framework install, several settings will be seeded into the dbo.Config table and will need to be adapted right after. See Database Config Table for more information on those specific settings. On top of that, not all settings may be present yet. It is not expected to become an issue as the framework services provide default values for those settings.

Performance Impact

If you have not migrated your custom DLL4 code, it is strongly recommended to do so. We have updated this recommendation because it can have a severe performance impact.
If DLL4 methods are inexplicably slower than expected, verify this setting on the profile.

Refer to the migration guide released with Ometa Framework version 5.0.0 article for more information.

Deprecations

Refer to 6.0.0 for upcoming deprecations.

Deprecation of Azure Key Vault Authentication

Authentication using Azure Key Vaults is now made deprecated since it was not fully implemented in all components. It will remain as is, but will be removed in v6. We recommend using certificate authentication instead. See this article for a guide on configuring authentication using certificates throughout the framework.

What's New

.NET 6

Ometa Framework Release 5.3.0 is fully .NET 6 compatible.

Data Extensions

It is now possible to configure conditions on extensions. This makes it possible depending on the context, to include or exclude certain fields.

Extension rules

Refer to the output data extensions for more information.

Transfer (Export / Import)

The Ometa Export / Import tool has the following optimizations:

Added a progressbar for validations.
Option for limited validation - previously importing a single small object would cause all objects to be loaded and make the import take too long for just a small import. Now, the import is checked if it has any changes that can break any other objects. Examples of this would be: changes to data extensions can mess with the extension profiles on view functions, changes to method templates can break the profile selection in functions, changing view type can break data sources, etc..
Option to retry, or skip import steps on error: this currently isn't practical for large imports that have validated for a while, only to break at the slightest issue. Instead a popup is shown to continue, retry or abort. At the end of the import, a list of ignored errors is shown and this can be validated manually, or imported again through manual mode.
Worked on improving the performance.

ADM

Save Filters

When a user sets filtering and/or sorting options in the multi record view, these filtering and sorting settings will be remembered as long as the browser tab is not closed.

Save filters

Does Not Contain Filters

In the ADM it is now possible to use the Does not contain text filter in the multi record views.

Does not contain text filter

Event Parameter: KeyUp, OnChange, ValueChanged

The OnChange and ValueChanged have a new parameter value which gives access to the current value.

The KeyUp gives access to the event parameter so you can check for instance the key code which was pressed.

Refer to the view field events article for more information.

Prevent Realtime Actions

It is now possible to hook into the OnRealtimeEvent and return false to not process the realtime event.

Refer to the view events article for more information.

Interfaces

File Interface

Introduce a new flag for the file interface, Use Windows User Impersonation. This flag indicates if the username and password should be used to impersonate a Windows user to perform the file action. This can be helpful when working with folders that are restricted to certain users only, or when the username and password do not correspond with a Windows user. When disabled, the BCSL user is used instead. By default, it is enabled.

SharePoint Interface

The SharePoint Interface has a new optional setting IgnoredFieldsOnUpdate to ignore certain fields on the update action.

Refer to the SharePoint interface article for more information.

Web Service Interface

It is now possible to configure the OAuth Client Credentials flow for the Web Service Interface.

Document Approval

Resolved issues with special characters in the document ADM:

Single quotes are now allowed for new documents.
Special characters blocked by SP will also be blocked by the document ADM with a CSL field validation and validation script when ensuring a file or updating the properties.

Added support for checking in and discarding changes:

Added an indicator at the end of the file to show if the file is checked out by you or someone else.
Added a checkout message in the document details.
Added check in and discard check out button at the end of the icon and in the document details.
Updating or deleting documents is now blocked if the file is checked out by someone else.
Added a new 'Undo Check Out File' building block
Added a new option in the SP interface: ActionForCheckedOutDocument: this specifies the behaviour when the file is checked out, by default the changes will be discarded. The document ADM will checkin the changes of the checkout.

Ometa Framework Release v5.3.0

Before Upgrade

During Upgrade

Important

After Upgrade

Changes

Database Merge

Important

Manual Database Merge

Installation And Configuration

SQL Server Certificate

Solution 1: TrustServerCertificate=True

Solution 2: Use Trusted Certificate

Generic REST

BCA / BCJS / BCM / BCSL / BCSP Framework Services

Performance Impact

Deprecations

Deprecation of Azure Key Vault Authentication

What's New

.NET 6

Data Extensions

Transfer (Export / Import)

ADM

Save Filters

Does Not Contain Filters

Event Parameter: KeyUp, OnChange, ValueChanged

Prevent Realtime Actions

Interfaces

File Interface

SharePoint Interface

Web Service Interface

Document Approval

Object Manager

BAM Reports