Reload fails in QMC even though script part is successfull in Qlik Sense Enterprise on Windows November 2023 and above.

When you are using a NetApp based storage you might see an error when trying to publish and replace or reloading a published app.

In the QMC you will see that the script load itself finished successfully, but the task failed after that.

ERROR QlikServer1 System.Engine.Engine 228 43384f67-ce24-47b1-8d12-810fca589657
 Domain\serviceuser QF: CopyRename exception:
 Rename from \\fileserver\share\Apps\e8d5b2d8-cf7d-4406-903e-a249528b160c.new 
to \\fileserver\share\Apps\ae763791-8131-4118-b8df-35650f29e6f6 
failed: RenameFile failed in CopyRename

ExtendedException: Type '9010' thrown in file
 'C:\Jws\engine-common-ws\src\ServerPlugin\Plugins\PluginApiSupport\PluginHelpers.cpp'
 in function 'ServerPlugin::PluginHelpers::ConvertAndThrow'
 on line '149'. Message: 'Unknown error' and additional debug info:
 'Could not replace collection 
\\fileserver\share\Apps\8fa5536b-f45f-4262-842a-884936cf119c] with
[\\fileserver\share\Apps\Transactions\Qlikserver1\829A26D1-49D2-413B-AFB1-739261AA1A5E],
 (genericException)'

<<< {"jsonrpc":"2.0","id":1578431,"error":{"code":9010,"parameter":
"Object move failed.","message":"Unknown error"}}

ERROR  Qlikserver1  06c3ab76-226a-4e25-990f-6655a965c8f3
  20240218T040613.891-0500  12.1581.19.0  
Command=Doc::DoSave;Result=9010;ResultText=Error: Unknown error
  0  0  298317  INTERNAL&
emsp; sa_scheduler  b3712cae-ff20-4443-b15b-c3e4d33ec7b4
  9c1f1450-3341-4deb-bc9b-92bf9b6861cf  Taskname  Engine  Not available
  Doc::DoSave  Doc::DoSave  9010  Object move failed.
  06c3ab76-226a-4e25-990f-6655a965c8f3

Resolution

Potential workarounds

• Roll back your upgrade to a version earlier than November 2023
• Change the storage to a file share on a Windows server

Cause 

The most plausible cause currently is that the specific engine version has issues releasing File Lock operations. We are actively investigating the root cause, but there is no fix available yet.

An update will be provided as soon as there is more information to share.

Internal Investigation ID(s) 

QB-25096
QB-26125

Environment

• Qlik Sense Enterprise on Windows November 2023 and above

When attempting to add data files, the preview does not utilize the entire modal to display data in Microsoft Edge or Chrome browsers (latest releases) on both Qlik Sense Enterprise on Windows and Qlik Cloud.

When selecting a source data file, the preview of the data is limited to showing only one row, and the whole modal space is not being utilized.

Qlik Sense Enterprise preview:

Qlik Cloud preview:

Environment

Affected versions are Qlik Sense Enterprise on Windows:

• May 2023
• August 2023
• November 2023
• February 2024

Resolution

This behavior is caused by defect QB-25217.

Fix Version:

Fixed in Qlik Cloud Services as of the 28th of February 2024.

Fixed for Qlik Sense Enterprise on Windows on the following versions:

• May 2023 Patch 15
• August 2023 Patch 13
• November 2023 Patch 7
• February 2024 Patch 2

Workaround

Use non-Chromium-based browsers such as Firefox or Safari.

Internal Investigation ID(s)

QB-25217

Information provided on this defect is given as is at the time of documenting. For up-to-date information, please review the most recent Release Notes, or contact support with the QB-25217 for reference.

Related Content

No scroll bar in the data files preview in Microsoft Edge or Chrome Browsers 
Table is not rendered properly in Chrome/Edge while selecting data from the Data Source 

Content

• Background
• Qlik Licensing Service (QLS)
• Signed License Key (JWT)
• What is a JSON Web Token?
• What’s in this for me as a Customer?
• Activation Using the Signed License Key
• Frequency in which license information is updated
• Data Transmitted by an Activation Request
• Activation Reply
• Analyzer Capacity License Report Process
• Data Transmitted
• Sync Reply
• User Assignment Sync Process
• Data Transmitted
• Reply fromLicense Backend

Background

Qlik Products have generally utilized license keys to enforce license entitlements and use rights.  During activation, the licensed entitlement is downloaded to the product in the format of a Licensing Enabler File (LEF). Activation requires internet connectivity to the deployment and is triggered by entering a 16-digit serial number and a corresponding control number. Offline activation is also supported using a manual LEF by copying and pasting a text file into the activation user dialog. Communication is through an http protocol.

Qlik Licensing Service (QLS)

Introduced with the February 2019 release[GM1]  of Qlik Sense Enterprise, Qlik has developed an alternative process for product activation. There have been several drivers for this change, including a move to an https protocol for a more secure connection between the Customer deployment and Qlik infrastructure. More information follows below. 
 
To allow for Customers to make the decision when to move, Qlik has introduced the use of a Signed License Key to determine which activation method to use. Over time Qlik Licensing Service will replace the current activation process, but for now both methods of activations will work.

Signed License Key (JWT)

As mentioned above, Qlik has added one additional way to activate Qlik products. 

What is a JSON Web Token?

JSON Web Token (JWT) is an open standard (RFC 7519) that defines a compact and self-contained way for securely transmitting information between parties as a JSON object. This information can be verified and trusted because it is digitally signed. JWT’s can be signed using a secret (with the HMAC algorithm) or a public/private key pair using RSA or ECDSA.
 
Although JWT’s can be encrypted to also provide secrecy between parties, we will focus on signed tokens. Signed tokens can verify the integrity of the claims contained within it, while encrypted tokens hide those claims from other parties. When tokens are signed using public/private key pairs, the signature also certifies that only the party holding the private key is the one that signed it. That is why we refer to this as the Signed License Key.

What’s in this for me as a Customer?

With the use of a Signed License Key, there are more Product and deployment offers to use.

• Use of multi-site deployments, such as connecting a Qlik Sense Enterprise site with a QlikView Server deployment using Qlik Sense Enterprise license options. This is enabled by accepting a Dual-Use offer for each QlikView deployment that should be enabled for a Unified License scenario.
• Use of multi-geo deployments, such as having Qlik Sense Enterprise sites in different locations using the same list of users.
• Use of the consumption-based license, Qlik Sense Enterprise Analyzer Capacity. This additional user license is possible to use in a single or multi-cloud scenario.

All of the above is enabled by the use of the Signed License Key. This made possible as the local deployment will sync entitlement data with all deployment’s using the same Signed License Key through an online database, License Backend, hosted by Qlik within Qlik Cloud.

Activation Using the Signed License Key

This is initiated by entering a Signed License Key to the Control Panel. The request is performed by the service Licenses using port 443 (https protocol procedures applies).

Frequency in which license information is updated

Assignment information (what user has what type of access assigned) is synchronized from the license service to license.qlikcloud.com every 10 minutes. 
Changes to a license (such as adding additional analyzer capacity) can take up to 24 hours to be retrieved.

Data Transmitted by an Activation Request

Activation Reply

Analyzer Capacity License Report Process

(Time schedule is not disclosed and includes grace time to support outages in the internet connection, a/k/a Optimistic Delegation.)

Data Transmitted

Sync Reply

User Assignment Sync Process

Data Transmitted

Reply from License Backend 

 
This Reference Guide is intended solely for general informational purposes and its contents do not form part of the product documentation.  The information in this guide is subject to change without notice.  ALL INFORMATION IN THIS GUIDE IS BELIEVED TO BE ACCURATE BUT IS PRESENTED WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED.  Qlik makes no commitment to deliver any future functionality and purchasing decisions should not be based upon any future expectation.

comparing the number of rows between the backend SAP ODB environment and the Qlik Replicate Full Load task, Qlik Replicate does not show the correct value.

Example:

Qlik Replicate shows 357,462 rows loaded, while the SAP ODP (ODQMON) job log shows 1.5 million rows.

Resolution

Qlik is investigating this as RECOB-8294. No workaround is currently available. 

Information provided on this defect is given as is at the time of documenting. For up-to-date information, please review the most recent Release Notes, or contact support with the   RECOB-8294  for reference.

Cause

Product Defect ID: RECOB-8294  

Environment

Qlik Replicate 

When patching Studio from the Feature manager, the Clean up libraries option is greyed out:

Resolution

1. Navigate to <Talend Studio Home>\studio\
2. Backup and edit Talend-Studio-win-x86_64.ini (Win OS) or Talend-Studio-macosx-cocoa.ini (Mac OS)
3. Remove the following parameter -Dtalend.studio.m2.clean=true
4. Save and restart Studio

Cause 

The -Dtalend.studio.m2.clean=true property ensures Studio removes obsolete jars during patching.

Related Content 

For more information, please review the Studio installation guide.

Environment

Talend Studio 8.0.1

After upgrading Qlik Enterprise Manager to 2022.11, increased connection errors to the Qlik Replicate server can be observed, with the connection terminating approximately every 30 minutes.

The following error is logged: 

3 2023-12-19 06:09:06 [ServerDto ] [ERROR] Failed to get monitoring data for server: 172.29.73.29. Message:'SYS-E-SESS_LOST, The client is not connected to the Server. Please connect and then try again.'''.

Resolution

This is a known defect. The fix will be included in the next possible Qlik Enterprise Manager release (tentatively 2023.11 SP03).

Fix Description: Added auto-reconnect flag where it was missing to allow QEM to better manage the connection drops and attempt to re-establish the EM<-->QR communication, especially in high-load situations.

Workaround:

The following are test patches that address this issue. Please contact support to obtain the test patch.

• QEM: QlikEnterpriseManager_2022.11.0.919_X64
• QEM: QlikEnterpriseManager_2023.5.0.689_X64
• QEM: QlikEnterpriseManager_2023.11.0.368_X64

Environment

Qlik Enterprise Manager 2022.11.0.634

A Qlik Replicate Task reading from Log Stream SAP ODP Source endpoint task fails with:

Failed during unload
SDK endpoint encountered a recoverable error [1024719]

Resolution

This error is currently under investigation as QB-26198, however, no resolution is available as the SAP ODP Source Endpoint does not support the Log Stream Staging Task setup.

Information provided on this defect is given as is at the time of documenting. For up-to-date information, please review the most recent Release Notes, or contact support with the QB-26198 for reference.

Cause

Product Defect ID: QB-26198 

Environment

Qlik Replicate 

The user is unable to open the remote project. A pop-up error is displayed, reading:

An error occurred (java.langException: Can't find the update properties file at: C:\ProgramFiles (x86)\Talend-Studio\studio\p2\org.eclipse.equinox.p2.repository\cache\-xxxx)

Resolution

1. Run studio in debug mode:
   
   "Talend-Studio-win-x86_64.exe --talendDebug > debugLog.txt"
   
   
2. Review the resulting debugLog.txt for the root cause of the error, such as:
   
   !SUBENTRY 1 org.eclipse.equinox.p2.transport.ecf 4 1002 2023-12-27 11:20:24.857
   !MESSAGE HTTP Proxy Authentication Required: https://update.talend.com/Studio/8/base/content.xml
   !STACK 1
   org.eclipse.ecf.filetransfer.BrowseFileTransferException: Proxy Authentication Required

In this example, the password of the proxy was changed and was then not updated synchronously in the Proxy setting of Talend Studio. The issue could be fixed by manually updating the user credentials of the proxy in studio -> preferences -> proxy setting.

Environment

Talend Cloud 
Talend Data Integration 8.0.1
 

The QlikSenseUtil comes bundled with Qlik Sense Enterprise on Windows. It's executable is by default stored in:

%Program Files%\Qlik\Sense\Repository\Util\QlikSenseUtil\QlikSenseUtil.exe

Qlik Sense Util is no longer supported as a backup and restore tool. For the officially supported backup and Restore Process, see Backup and Restore.

It serves as a: 

• Qlik Sense Port Checker: Pings every active port Qlik sense is using for every node. Note that some ports can be for internal use and might not be exposed to the node you are pinging from.
• Data structure checker: Mostly applicable for a synchronized persistence deployment. Checks that the data references are not broken.
• Duplicate server nodes: Mostly applicable for a synchronized persistence deployment. Checks there are no duplicate server nodes in the database.
• Remove old app binaries: Remove deleted app binaries from the selected folder.
• Get system configuration: Returns the system configuration from the database.
• Connection string editor: A tool for editing the connection string.
• Service Cluster: Functionality for updating the service cluster in the database.
• Log Fetcher: A tool for fetching logs from the system. If the time span is shorter than 24h all logs will be summarized in one single file. This will make trouble shooting easier if you are looking for events what occurred within a short time frame.
  • Fetching logs with a time span larger than 24h will copy the logs and paste them in their logical structure. The filter functionality selects every row with the specified keyword and creates log files based on rows selected.
    Note : Qlik Support prefers using the Log Collector included in How To Collect Qlik Sense Log Files
• Service type checker: Mostly applicable for a synchronized persistence deployment. Checking for duplicated service types in the database.

The native Qlik PostgreSql connector (ODBC package) does not import strings with more of 255 characters. Such strings are just cut off, the following characters are not shown in the applications. No warnings are thrown during the script execution.

The problem affects the Qlik connector but not all the DNS drivers.

Environment

Qlik Sense February 2023 and higher versions
Qlik Cloud

Resolution

When setting up the PostgreSQL connection, set the parameter TextAsLongVarchar  with value 1 in the connector Advanced settings.

When TextAsLongVarchar is set, and the Max String Length is set to 4096, 4096 characters are loaded. 

Notice that there still are limitations to this functionality related to the datatype used in the database. Data types like text[] are currently not supported by Simba and they are affected by the 255 characters limitation even when the TextAsLongVarchar parameter is applied.

Qlik has opened an improvement request to Simba to support them.

As workaround, it is possible to test a custom connector using a DSN driver to deal with these data types.

Internal Investigation ID

QB-21497

DB2 Z/OS Source Endpoint loading a table with date columns leads to the data coming in as blank. The source data has valid dates but is not loading to the target in the Qlik Replicate Task

Resolution

The ODBC Driver for DB2 Z/OS requires the 11.5.6 or the 11.5.8 ODBC driver to be installed on the Qlik Replicate Server.

Cause 

 ODBC driver 11.5.9 or other installed.

Related Content

z/OS Prerequisites | Qlik Replicate Help

Environment

Qlik Replicate 2023.5

Upgrade installation or fresh installation of Qlik Replicate 2023.11 (includes builds GA, PR01 & PR02), Qlik Replicate reports errors for MySQL or MariaDB source endpoints. The task attempts over and over for the source capture process but fail, Resume and Startup from timestamp leads to the same results:

[SOURCE_CAPTURE  ]T:  Read next binary log event failed; mariadb_rpl_fetch error 0 () [1020403]  (mysql_endpoint_capture.c:1060)

[SOURCE_CAPTURE  ]T:  Error reading binary log. [1020414]  (mysql_endpoint_capture.c:3998)

Environment:

• Replicate 2023.11 (GA, PR01, PR02)
• MySQL source database , any version
• MariaDB source database , any version

Fix Version & Resolution:

Upgrade to Replicate 2023.11 PR03 (coming soon)

Workaround:

If you are running 2022.11, then keep run it.

No workaround for 2023.11 (GA, or PR01/PR02) .

Cause & Internal Investigation ID(s):

Jira: RECOB-8090 , Description: MySQL source fails after upgrade from 2022.11 to 2023.11

There is a bug in the MariaDB library version 3.3.5 that we started using in Replicate in 2023.11.
The bug was fixed in the new version of MariaDB library 3.3.8 which be shipped with Qlik Replicate 2023.11 PR03 and upper version(s).

Related Content:

support case #00139940, #00156611

Replicate - MySQL source defect and fix (2022.5 & 2022.11)

Qlik Replicate 

Steps

1. Retrieve the SFTP server fingerprint (this example uses port 22)
   
   
   1. Open a Windows command prompt
   2. Run the following command: 
      
      sftp -P 22  username@ftptest.test.com 
      
      Where username is 'username' and 'ftptest.test.com' is the sftp hostname.
      
      
   3. The response received will be similar to:
      

      C:\Windows\System32>sftp -P 22 username@ftptest.test.com
      The authenticity of host 'ftptest.test.com (18x.6x.15x.21x)' can't be established.
      RSA key fingerprint is SHA256:KijFUxxxxxxxxxxxxxx/xxxxxxxxxxxxxxxxxapWjo

      if the above does not retrieve your fingerprint, contact your SFTP server administrator who can provide fingerprint or public key details for you.

   4. Copy the RSA fingerprint 
      
      
2. Create a Qlik Cloud Data Connection
   
   
   1. Open your Qlik Cloud Application and navigate to the data load editor
   2. Under Data Connection choose Create new connection 
   3. Search and select SFTP
   4. Enter your credentials and fingerprint information
      • Hostname
        • (ie: myftpserver.domain.com. Do not pre-pend the address with ftp:// nor sftp://)
      • Username
      • Password
      • Insert fingerprint
   5. Click Test connection
      
      
      

      If there are connectivity issues, to request data from your data connections through a firewall, you need to add the underlying (all three) IP addresses for your region to your firewall's allow list. These addresses are static and will not change. See Allowlisting domain names and IP addresses.

Other Diagnostics:

Error in SFTP connector

• Error shows public key mismatch error in Qlik SFTP connector

Diagnostic

• Open Command Prompt and enter "ssh-keyscan -t rsa <host ip or name) | ssh-keygen -lf -"

Example: (insert a valid SFTP server host IP)

• C:\Windows\System32>ssh-keyscan -t rsa 192.168.xxx.255 | ssh-keygen -lf -

Result:

• "(stdin) is not a public key file"

In case of this or other SFTP connection errors, please work with the SFTP server administrator or SFTP server vendor in order in order to enable the generation of a publicly available fingerprint.

Related Information:

SFTP

The information in this article is provided as-is and to be used at own discretion. Depending on tool(s) used, customization(s), and/or other factors ongoing support on the solution below may not be provided by Qlik Support.

The dedicated tFTPxxx components do not support reading files directly from FTP servers. Usually the file is obtained from the FTP server to the local system first and then the file is read. However, this can cause performance issues when many files or large files need to be read.

Resolution

One solution is to use the JSch library, which allows the creation of an InputStream object from a file via SFTP. Then, use the tFileInputDelimited component to read data from the InputStream object. This avoids downloading the file to the local system.

For example, in the following Job, the Java code iterates over the FTP files one by one, using a JSch library file and a tFileInputDelimited component to read the file directly.

The tLibraryLoad component loads the jsch-0.1.55.jar JSch library file, then the tJava component uses the following code to create the InputStream object for the current FTP file.

JSch jsch = new JSch();
        Session session = null;
        try {
            session = jsch.getSession("username", "localhost", 22);
            session.setConfig("StrictHostKeyChecking", "no");
            session.setPassword("password");
            session.connect();

            Channel channel = session.openChannel("sftp");
            channel.connect();
            ChannelSftp sftpChannel = (ChannelSftp) channel;
            java.io.InputStream in=sftpChannel.get( ((String)globalMap.get("tFTPFileList_1_CURRENT_FILE")));
            globalMap.put("sftp_inputStream",in);
        
        } catch (JSchException e) {
            e.printStackTrace();  
        } catch (SftpException e) {
            e.printStackTrace();
        }

The tFileInputDelimited component reads the data from the InputStream object:

Environment

Talend Data Integration 

From R2024-05, Java 17 will become the only supported version to start most Talend modules, enforcing the improved security of Java 17 and eliminating concerns about Java's end-of-support for older versions. In 2025, Java 17 will become the only supported version for all operations in Talend modules.

Starting from v2.13, Talend Remote Engine requires Java 17 to run. If some of your artifacts, such as Big Data Jobs, require other Java versions, see Specifying a Java version to run Jobs or Microservices.

Content

• Prerequisites
• Procedure
• Windows
• Linux
• MAC OS
• Multiple JDK versions
• Studio
• Remote Engine
• ESB - Runtime
• Studio
• Talend Administration Center (TAC)
• CICD
• Windows Users
• Linux Users
• Jenkins Users
• Additional Notes
• Specifying a Java version to run Jobs or Microservices

Prerequisites

Procedure

Windows

For Windows users, please follow the JDK installation guide (docs.oracle.com).

Linux

For Linux users, please follow the JDK installation guide (docs.oracle.com).

MAC OS

For MAC OS users, please follow the JDK installation guide (docs.oracle.com).

Multiple JDK versions

When working with software that supports multiple versions of Java, it's important to be able to specify the exact Java version you want to use. This ensures compatibility and consistent behavior across your applications. Here is how you can specify a specific Java version on the following products (such as build servers, shared application server, and similar):

Studio

For Studio users who are using multiple JDKs, please follow the appropriate instructions listed above and follow the proceeding additional steps:

1. Backup and edit the <Studio Home>\Talend-Studio-win-x86_64.ini file
2. Prepend:
   

   -vm
   <JDK17 HOME>\bin\server\jvm.dll

Remote Engine

For Remote Engine (RE) users who are using multiple JDKs, please follow the appropriate instructions listed above and follow the proceeding additional steps.

1. Backup and edit the <RE HOME>/etc/talend-remote-engine-wrapper.conf file
2. Modify the set.default.JAVA_HOME= property to point to the <JDK 17 HOME> path.
   
   Note 1: If Remote Engine is not installed as a service, the JDK file will be set in the <RE HOME>/bin/setenv file.
   
   Note 2: When it comes to running Jobs or Microservices, you retain the flexibility to either use the default Java 17 version or choose older Java versions, through straightforward configuration of the engine.
   
   How to modify?
   
   Check the following etc folder based configuration and change it to installed jdk/jre path: 
   
   {
   org.talend.ipaas.rt.dsrunner.cfg--> ms.custom.jre.path
   org.talend.remote.jobserver.server.cfg--> org.talend.remote.jobserver.commons.config.JobServerConfiguration.JOB_LAUNCHER_PATH
   }

ESB - Runtime

For Runtime users who are using multiple JDKs, please follow the appropriate instructions listed above and follow the proceeding additional steps.

1. Backup and edit the <Runtime home>/etc/<TALEND-8-CONTAINER service>-wrapper.conf
2. Modify the set.default.JAVA_HOME=C:\<JDK 17 HOME> path
   

If Runtime is not running as a service:

1. Backup and edit the <Runtime home>/bin/setenv.sh
2. Modify the SET JAVA_HOME= <JDK 17 HOME> path
   
   

Studio

• Data Integration (DI):  After installing the 8.0 R2023-10 Talend Studio monthly update or a later one, if you switch the Java version to 17 and relaunch your Talend Studio with Java 17, you must enable your project settings for Java 17 compatibility.
  
  
  1. Go to Studio
  2. Go to File
  3. Edit Project properties
  4. Go to Build
  5. Go to Java Version
  6. Activate "Enable Java 17 compatibility"
     

     With the Enable Java 17 compatibility option activated, any Job built by Talend Studio cannot be executed with Java 8. For this reason, verify the Java environment on your Job execution servers before activating the option.

• Big Data Users:  Do not enable Java 17 compatibility unless your Spark Cluster supports Java 17.  

Talend Administration Center (TAC)

To use Talend Administration Center with Java 17, you need to open the <tac_installation_folder>/apache-tomcat/bin/setenv.sh file and add the following commands:

# export modules 
export JAVA_OPTS="$JAVA_OPTS --add-opens=java.base/sun.security.x509=ALL-UNNAMED --add-opens=java.base/sun.security.pkcs=ALL-UNNAMED"

Windows users use <tac_installation_folder>\apache-tomcat\bin\setenv.bat

CICD

Windows Users

For Java 17 users, Talend CICD process requires the following Maven options:

1. Backup and edit <Maven_home>\bin\mvn.cmd
2. Modify to:

   set "MAVEN_OPTS=%MAVEN_OPTS% --add-opens=java.base/java.net=ALL-UNNAMED --add-opens=java.base/sun.security.x509=ALL-UNNAMED --add-opens=java.base/sun.security.pkcs=ALL-UNNAMED"

Linux Users

For Java 17 users, Talend CICD process requires the following Maven options:

1. Backup and edit <Maven_home>/bin/mvn
2. Modify to:

   export MAVEN_OPTS="$MAVEN_OPTS \ 
   --add-opens=java.base/java.net=ALL-UNNAMED \ 
   --add-opens=java.base/sun.security.x509=ALL-UNNAMED \ 
   --add-opens=java.base/sun.security.pkcs=ALL-UNNAMED"

Jenkins Users

1. Backup and edit the jenkins_pipeline_simple.xml 
2. Include the following in the Talend_CI_RUN_CONFIG parameter:
   

   <name>TALEND_CI_RUN_CONFIG</name>
   <description>Define the Maven parameters to be used by the product execution, such as:
       - Studio location
       - debug flags
   These parameters will be put to maven 'mavenOpts'.
   If Jenkins is using Java 17, add:
   --add-opens=java.base/java.net=ALL-UNNAMED --add-opens=java.base/sun.security.x509=ALL-UNNAMED --add-opens=java.base/sun.security.pkcs=ALL-UNNAMED
   </description>

Additional Notes

Specifying a Java version to run Jobs or Microservices

Overview

Enable your Remote Engine to run Jobs or Microservices using a specific Java version.

By default, a Remote Engine uses the Java version of its environment to execute Jobs or Microservices. With Remote Engine v2.13 and onwards, Java 17 is mandatory for engine startup. However, when it comes to running Jobs or Microservices, you can specify a different Java version. This feature allows you to use a newer engine version to run the artifacts designed with older Java versions, without the need to rebuild these artifacts, such as the Big Data Jobs, which reply on Java 8 only.

When developing new Jobs or Microservices that do not exclusively rely on Java 8, that is to say, they are not Big Data Jobs, consider building them with the add-opens option to ensure compatibility with Java 17. This option opens the necessary packages for Java 17 compatibility, making your Jobs or Microservices directly runnable on the newer Remote Engine version, without having to go through the procedure explained in this section for defining a specific Java version. For further information about how to use this add-opens option and its limitation, see Setting up Java in Talend Studio.

Procedure

1. Stop the engine.
   
   
2. Browse to the <RemoteEngineInstallationDirectory>/etc directory.
   
   
3. Depending on the type of the artifacts you need to run with a specific Java version, do the following:
   
   For both artifact types, use backslashes to escape characters specific to a Windows path, such as colons, whitespace, and directory separators, while keeping in mind that directory separators are also backslashes on Windows.
   
   Example:
   

   c:\\Program\ Files\\Java\\jdk11.0.18_10\\bin\\java.exe

   • For Jobs, in the <RemoteEngineInstallationDirectory>/etc/org.talend.remote.jobserver.server.cfg file, add the path to the Java executable file.
     
     Example:
     

     org.talend.remote.jobserver.commons.config.JobServerConfiguration.JOB_LAUNCHER_PATH=c:\\jdks\\jdk11.0.18_10\\bin\\java.exe

   • For Microservices, in the <RemoteEngineInstallationDirectory>/etc/org.talend.ipaas.rt.dsrunner.cfg, add the path to the Java executable file.
     
     Example:
     

     ms.custom.jre.path=C\:/Java/jdk/bin

     Make this modification before deploying your Microservices to ensure that these changes are correctly taken into account.
     
     
4. Restart the engine.

The Qlik Replicate task log (with source_capture and trace logging enabled) shows LSN information similar to:

00023576: 2024-03-08T17:46:03 [SOURCE_CAPTURE ]T: MS-CDC capture loop. SQL Server start time '2024-03-06 22:41:29.170', start LSN '000A9B8100000050000D', end time '2024-03-08 19:46:04', end LSN '' (sqlserver_mscdc.c:3684)
00023576: 2024-03-08T17:46:08 [SOURCE_CAPTURE ]T: MS-CDC capture loop. SQL Server start time '2024-03-06 22:41:29.170', start LSN '000A9B8100000050000D', end time '2024-03-08 19:46:09', end LSN '' (sqlserver_mscdc.c:3684)

This indicates the LSN is looping and that the LSN was purged.

Resolution

• If you can restore the LSN, the task can be resumed.
• If the LSN cannot be restored, the task will require a full reload.

Environment

Qlik Replicate 

A Qlik Replicate task with an Oracle target fails with the following in the task log:

[TARGET_LOAD ]T: ORA-03156: OCI call timed out

This can be expected with long-running executions during a full load. 

Resolution

To disable the timeout, set the Internal Parameter executeTimeout on the Oracle target endpoint.

1. Go to the Oracle Endpoint connection
2. Switch to the Advanced tab
3. Click Internal Parameters
4. Add executeTimeout with a value of 0
   
   
   
   For more information about Internal Parameters, see Qlik Replicate: How to set Internal Parameters and what are they for?

Environment

Qlik Replicate 

This article briefly explains how to find what tables (articles) are used in a Publication. This method is useful to identify issues when a table does not capture updates. 

Resolution

Run the following query from Microsoft Studio Management or any Database Utility (such as DBeaver):

SELECT 
  msp.publication AS PublicationName,
  msa.publisher_db AS DatabaseName,
  msa.article AS ArticleName,
  msa.source_owner AS SchemaName,
  msa.source_object AS TableName
FROM distribution.dbo.MSarticles msa
JOIN distribution.dbo.MSpublications msp ON msa.publication_id = msp.publication_id
ORDER BY 
  msp.publication, 
  msa.article;

Environment

Qlik Replicate 

Qlik’s Data Integration Platform provides functionality to support near real-time data delivery from traditional sources, files, and some SaaS environments via Qlik Replicate. Qlik Compose for Data Warehouses is integrated with Qlik Replicate to automate the incremental load process of the data warehouse based on change data capture (CDC) from the source system.

However, there are certain scenarios where the automated incremental load processes cannot be leveraged. (For example, SAP Hana Log based replication, third party data ingestion or cloud data sharing programs).   

In those instances there are extract transform and load (ETL) patterns that can be implemented in Qlik Compose to support incremental data loads. This paper will describe these scenarios and show how to implement custom incremental load patterns in Qlik Compose and how those patterns can be applied to query or view based mappings.

HSTS (HTTP Strict-Transport-Security response header) security check failed.

HTTP Strict Transport Security (HSTS) is a policy mechanism that helps to protect websites against man-in-the-middle attacks such as protocol downgrade attacks and cookie hijacking. It allows web servers to declare that web browsers (or other complying user agents) should automatically interact with it using only HTTPS connections, which provide Transport Layer Security (TLS/SSL), unlike the insecure HTTP used alone.

Resolution

Before adding HSTS to either the QlikView AccessPoint or the QlikView Management Console (QMC), set both up to use HTTPS. See for QlikView AccessPoint and QMC with HTTPS and a custom SSL certificate instructions.

HSTS for the QlikView AccessPoint

Custom response headers can be set in both the QlikView WebServer (beginning with 12.30) and Microsoft IIS (all QlikView versions).

The custom header needed for HSTS is: Strict-Transport-Security

1. Run text editor (e.g. Notepad) as Administrator 
   
   
2. Edit QlikView WebServer configurations file. The default path is C:\ProgramData\QlikTech\WebServer\config.xml
   
   
3. Locate CustomHeaders element within the config file. For more information, see QlikView WebServer: Custom HTTP Header.
   
   
4. Add custom response header as <Header> element(s) with sub-elements defining Strict-Transport-Security as the name and your desired max-age= as value.
   
   Example:
   

   <Config>
     ...
     <Web>
      ...
       <CustomHeaders>
         <Header>    
           <Name>Strict-Transport-Security</Name>
           <Value>max-age=31536000</Value>
          </Header>
       </CustomHeaders>
     </Web>
   </Config>​

5. Restart QlikView WebServer service

For information on how to configure custom headers with Microsoft IIS, see Setting Custom HTTP Headers in IIS for QlikView. The site https://https.cio.gov/hsts/ gives information on how to setup the webserver to enable HSTS.

Testing can be achieved using any number of third party sites, such as: 

• https://tools.geekflare.com/hsts-test
• https://hstspreload.org/
• https://www.ssllabs.com/

HSTS for the QlikView Management Console (QMC)

This setting was introduced with QlikView 12.70 (May 2022) SR1.

QVManagementService.exe.Config Changes:

1. Stop the QlikView Management Services
   
   
2. Go to ProgramFiles => qliktech => management service => open QVManagementService.exe.config using an administrator notepad
   
   
3. Update this value to true =>
   
   <add key="UseHSTS" value="true" />
   
   
4. To enable HSTS to header this value has to be set to true
   
   <add key="UseHTTPS" value="true" />

Environment:

QlikView