Talend Remote Engine Heart Beat is failing hourly with 'GOAWAY received' error and the karaf.log shows below:

2024-08-01T10:16:58,638 | ERROR | pool-28-thread-1 | JAXRSUtils | 469 - org.apache.cxf.cxf-rt-frontend-jaxrs - 3.6.2 | | Problem with writing the data, class org.talend.ipaas.rt.engine.model.HeartbeatInfo, ContentType: application/json
2024-08-01T10:16:58,638 | WARN | pool-28-thread-1 | PhaseInterceptorChain | 467 - org.apache.cxf.cxf-core - 3.6.2 | | Interceptor for {http://pairing.rt.ipaas.talend.org/}PairingService has thrown exception, unwinding now
org.apache.cxf.interceptor.Fault: Problem with writing the data, class org.talend.ipaas.rt.engine.model.HeartbeatInfo, ContentType: application/json
Caused by: java.io.IOException: /<IP>:<port>: GOAWAY received

This could also be a possible solution when the following error is also showing in the log:

Caused by: java.io.IOException: HTTP/1.1 header parser received no bytes

Resolution

1. Stop Remote Engine.
2. Edit the <RemoteEngineInstallation>/etc/org.talend.ipaas.rt.pairing.agent.cfg file and replace the line 

   heartbeat.interval=60 (by default) 

   with 

   heartbeat.interval=65

3. In the file <RemoteEngineInstallation>/etc/system.properties, remove or comment out the line below (if it exists):
   

   org.apache.cxf.transport.http.forceVersion=1.1​

4. Restart Remote Engine.

Do not set heart interval to more than 180 seconds(3 mins). If Talend Cloud does not receive a heartbeat from the Remote Engine for more than 3 mins, it will show the Remote Engine's status as unavailable.

Cause

The connection to the server pair.xx.cloud.talend.com is kept during 60 seconds and Heartbeat.interval > 60 seconds allows to close this connection and prevent "GOAWAY received" error message.

Related Content

For more information about heartbeat concept and heartbeat interval, please refer to documentation:

Monitoring-remote-engine-health

Internal Investigation ID(s)

 Internal Jira Case ID: TMC-4122

Environment

Talend Cloud 

To disable all current triggers in a QlikView distribution environment, modify the Trigger.xml. 

Disabling all triggers

1. Open Windows Explorer and browse to the location of the QVPR folder
2. Edit the file Trigger.xml
3. Replace Enabled="true" with Enabled="false"
4. Save the file
5. Resend the Work Order
   1. Open the QlikView Management Console
   2. Go to System
   3. Go to Setup
   4. Go to your Distribution Service
   5. Switch to the Advanced tab
   6. Click Send Work Order
6. Restart the QlikView Management Service
7. You should now see that all triggers are disabled.

Enabling all triggers

1. Open Windows Explorer and browse to the location of the QVPR folder
2. Edit the file Trigger.xml
3. Replace Enabled="false" with Enabled="true"
4. Save the file
5. Resend the Work Order
   1. Open the QlikView Management Console
   2. Go to System
   3. Go to Setup
   4. Go to your Distribution Service
   5. Switch to the Advanced tab
   6. Click Send Work Order
6. Restart the QlikView Management Service
7. You should now see that all triggers are disabled.

Environment

• QlikView

The information in this article is provided as-is and to be used at own discretion. Depending on tool(s) used, customization(s), and/or other factors ongoing support on the solution below may not be provided by Qlik Support.

Support has collected a number of resources helpful for the implementation of Qlik Sense Version Control functionality.

• QS event driven cross site app promoter:  https://github.com/eapowertools/qs-event-driven-cross-site-app-promoter
• EAPowertools - QMCU- SCLite:  https://github.com/eapowertools/qmcu-sclite
• Qlik Sense Developers Help                 
• C# .NET Example: https://github.com/StevenJDH/Qlik-Sense-.NET-Example
• QRS API Endpoints (subset)  
• Qlik Branch (Garden)  https://developer.qlik.com/garden      

Qlik Partner Solutions:

• Gitoqlok: A Chrome Extension that adds source versioning control to Qlik, Gitoqlok 
  For more information from the vendor, see How to use git with Qlik Sense 
• EBIExperts Solution:  WIP: https://www.ebiexperts.com/products/
• In4BI Solution:  Platform Manager: https://in4bi

The tFtpGet component is unable to filter and download the necessary file according to the designated filemask. In the FTP UI tool, the file is listed and displayed with uppercase characters. However, when using the same uppercase-based filemask to filter the file, no file is downloaded.

Cause

The file was saved in lowercase, but it appears in uppercase when viewed through the FTP client UI. This inconsistency between the saved filename and its displayed version in the UI indicates a case-mismatch issue in the FTP system.

Resolution

To resolve this issue, adjust the character case in the Filemask settings. Alternatively, enable the "Use Perl5 Regex Expression as Filemask" option to create a new filter that disregards case sensitivity. (This can be achieved by using "/i" in the Perl5 Regex Expression, which enables case-insensitive pattern matching, allowing "A" to match "a".)

Related Content

tFTPGet Standard properties

Perl Regular-Expressions

Environment

• Talend Data Integration 8.0.1

Some Qlik Replicate tasks intermittently show a high amount of false latency. Latency may be observed at random times and will then immediately be gone again. Stopping and resuming the task may resolve the issue.

Resolution

This is often observed when the Qlik Replicate server is out of sync with the time on the source and target endpoint servers.

Resolve the time sync issue.

Cause

Time is out of sync between servers.

Environment

• Qlik Replicate

Your jobs maybe failing in 30s with an error below from Talend Management Console.

org.talend.remote.commons.exception.ClientServerException: TimeOutInterruptor has interrupted the socket stream, timeout has been reached
org.talend.remote.commons.exception.ClientServerException: org.talend.remote.commons.exception.ClientServerException: TimeOutInterruptor has interrupted the socket stream, timeout has been reached

All jobs can be impacted by that and happen at any time.

Resolution

As this issue is caused by timeout issue during file transferring in Qlik Talend Remote Engine, it can be resolved by excluding the Remote Engine folder from Antivirus Scanning or Firewall. Please double check your Antivirus Software and Windows defender.

Or if above solution does not help, please try to increase the timeout value by following below steps:

1. Go into the folder <RE_FOLDER>\system\org\talend\jobserver\org.talend.remote.jobserver.client\8.0.x.yyyymmdd_xxxx_patch

   cd <RE_FOLDER>\system\org\talend\jobserver\org.talend.remote.jobserver.client\8.0.x.yyyymmdd_xxxx_patch

2. Extract the file job_server_client.properties from the jar file
   

   jar xvf org.talend.remote.jobserver.client-8.0.2.20240904_0703_patch.jar job_server_client.properties
   inflated: job_server_client.properties

3. Edit the file job_server_client.properties and change the value of JobServerClient.conf.timeout

   JobServerClient.conf.timeout=300000 -→ increase this to the time desired in milliseconds.

4. Replace the file job_server_client.properties in the jar file by the one modified
   

   jar uf org.talend.remote.jobserver.client-8.0.2.20240904_0703_patch.jar 
   job_server_client.properties

5. Restart the Remote Engine Server to apply the change.
    

    

Cause

From Karaf logs, the issue occurs due to timeout issue during file transferring in Talend Remote Engine, where the default timeout is set to 30 seconds in the existing system.properties

The socket related errors in context of ClientServerException + TimeOutInterruptor indicate there is a networking related congestion that prevents opening socket connections, which could also hint to an exhaustion of the network.

Internal Investigation ID(s) 

Internal Jira ID: TMC-1915

Environment

Talend Cloud 

Creating a tenant in a specific region using either Automation or the available API fails. 

The API call returns:

"Forbidden" error (status 403)

The Automation block for Application Automation - Platform operations → Create Tenant is restricted to a different region.

Resolution

Create a region-level OAuth client for the specific region. For instructions, see Create a region level M2M OAuth2 client.

Cause

It is necessary to have a dedicated OAuth client for each region.

Environment

• Qlik Cloud Analytics
• Qlik Application Automation

Saving an edited task in Qlik Cloud may lead to the error:

Unable to save task

Analyzing the issue with the browser's developer tools, we see that the error 500 Internal Server Error is returned. For tips on how to use developer tools, see Save network web traffic (HAR/XML file) and console logs from the browser's developer tools.

Resolution

To resolve the issue, we remove the connection to the corrupt cron job, then set the task to its correct schedule:

1. Edit the task again, set the schedule to When data is refreshed and Save. This will remove the connection to a corrupt cron job.
   
   
   
   
2. Edit the task again, this time with the correct schedule and the desired change.

Cause

There was a problem with the task's cron job in Qlik Cloud.

Environment

• Qlik Cloud

This article outlines the steps for CICD P2 zero install for pipeline using Maven and provides command examples.

1. Install the necessary jar files.  
   

   mvn org.talend.ci:builder-maven-plugin:8.0.19:install  -Dtalend.studio.p2.update= https://update.talend.com/Studio/8/updates/R2024-12 -Dlicense.path='/home/use1/cicd/license8    -s C:\app\apache-maven-3.9.0\conf\maven_settings.xml

2. Prepare poms for Job building.  
   

   mvn org.talend.ci:builder-maven-plugin:8.0.18:generateAllPoms  -Dtalend.studio.p2.update='https://update.talend.com/Studio/8/updates/R2024-12'   -Dfilter.include.dependencies=true  -Dgeneration.type=local -e -X -s /home/wguo/cicd/maven_settings.xml -Dcodes.compile.failOnError=false   -Dlicense.path='/home/use1/cicd/license8' -Dinstall.org.eclipse.equinox.p2.transport.ecf.retry=10  -Dstudio.error.on.component.missing=false

3. Build/Compile, package and publish project artifacts.
   

   mvn -X clean deploy -U -f /home/user1/ws/project1/poms/pom.xml -Dlicense.path='/home/use1/cicd/license8'  -Dtalend.studio.p2.update= https://update.talend.com/Studio/8/updates/R2024-12 -Dpackaging=zip -Dgeneration.type=local -Dinstall.org.eclipse.equinox.p2.transport.ecf.retry=10 -s /home/wguo/cicd/maven_settings.xml -DaltDeploymentRepository=dmt-releases::default:: http://s.ap.abc.net:8081/repository/dmt-releases/ -DaltSnapshotDeploymentRepository=dmt-snapshots::default:: http://s.ap.abc.net:8081/repository/dmt-snapshots/ -DaltReleaseDeploymentRepository=dmt-releases::default:: http://s.abc.net:8081/repository/dmt-releases/

For Windows-based CICD host machines, use the --define "xxx=yyy" option instead of -Dxxx=yyy to circumvent parameter passing failures.

Additionally, you can filter and specify which project items to generate in order to make the POM generation faster. For further details, please refer to Filtering project items | Talend Software Development Life Cycle Best Practices Guide Help.

Environment

• Talend Data Integration 8.0.1

In Qlik Sense, when there is a commented out portion of the script, if "//" is deleted and the script is saved, all the text will turn black and all syntax highlighting will be lost. If pages of the script are toggled between, a message that says "we have disabled syntax highlight to maintain performance" appears and the highlighting will come back but if previous steps are followed again,commenting will also be disabled again.  

This is working as designed.

It is the current solution for dealing with deeply nested if statements. In many cases these deeply nested if blocks will cause real performance issues (crashing the browser). The best practice is to not use nested ifs when possible. The commenting/uncommenting works because we count the levels. Pretty sure 10 levels deep we trigger the perf warning. So if they comment out one and it's only 9 it won't complain.

Environment: 

Internal Investigation ID(s):

Qlik Answers is Qlik's personalized AI assistant that generates on-point answers from your own trusted business content. See this article to learn how to get started with Qlik Answers

This connector unlocks new powerful use cases like indexing new data ad-hoc and taking your action oriented automations to the next level by including unstructured insights. This article explains how the Qlik Answers connector in Qlik Application Automation can be used.

Article contents

• Authentication
• Use cases
• 1. Re-indexing a knowledge base
• 2. Ask questions
• Notes
• Related Content

Authentication

This connector does not require additional configuration to authenticate, it will automatically connect to the automation owner's Qlik account. Whenever blocks of this connector are executed, they will use that account.

Use cases

For this connector's first release, we will introduce blocks that can re-index knowledge bases and ask questions to an assistant. We will deliver more blocks in future releases, do not hesitate to request new blocks through Ideation.

1. Re-indexing a knowledge base

This use case focuses on ad-hoc updates to the knowledge base. The creation and initial configuration of a knowledge base should happen through the Qlik Answers UI as described here:

Once your knowledge base is configured and you've set up a data connection to a cloud storage location, you can then use Qlik Application Automation to add specific files to the Cloud Storage location and trigger a re-indexing. This could be useful in scenarios when you only want to index specific records that might depend on a context.

For this use case, you can use the blocks Add Data Source To A Knowledge Base and Sync Knowledge Base Data Source. The below example will fetch incidents from ServiceNow, store them in a Dropbox folder and then add the Dropbox folder as a data source to a Knowledge Base in Qlik Answers. After this, the data source will be synced to the knowledge base index.

2. Ask questions

By using the blocks Create Thread and Execute Synchronous Prompt, you can ask questions from an assistant to enrich other messages that are sent through automations with unstructured insights. In the below example, these blocks are added to the existing "Notify your team on Microsoft Teams based on a measure" template that can be found in the template picker:

Notes

Don't ask a question immediately after indexing. Asking a question should be done after the knowledge base has completed indexing successfully.

Related Content

• Introducing Qlik Answers
• Qlik.com product page for Qlik Answers

Environment

• Qlik Cloud Analytics
• Qlik Application Automation
• Qlik Answers

The information in this article is provided as-is and will be used at your discretion. Depending on the tool(s) used, customization(s), and/or other factors, ongoing support on the solution below may not be provided by Qlik Support.

After applying the Oracle July 2024 patch, the following error will be seen when working with compressed tables:

[ASSERTION       ]W:  A compressed row cannot be parsed for table  (oradcdc_parse.c:2640)

The error leads to missing records.

Only Oracle instances with compressed tables are affected.

Resolution

R&D addressed the compression warning and also another missing insert events.

Fix versions

• 2023.11 and later
• 2024.5 and later

Before rolling out the fix in production, always first test in UAT or other pre-release environments.

If you are uncertain whether or not you are using compressed tables, run the following query:

select owner, table_name, COMPRESSION,COMPRESS_FOR from dba_tables where compression='ENABLED'

Cause

Oracle July 2024 Patch with compressed tables. 
RECOB-8914 by Qlik.

Environment

• Qlik Replicate 
• Oracle with July 2024 patch
  
  

The QlikSenseUtil comes bundled with Qlik Sense Enterprise on Windows. It's executable is by default stored in:

%Program Files%\Qlik\Sense\Repository\Util\QlikSenseUtil\QlikSenseUtil.exe

Qlik Sense Util is no longer supported as a backup and restore tool. For the officially supported backup and Restore Process, see Backup and Restore.

It serves as a: 

• Qlik Sense Port Checker: Pings every active port Qlik sense is using for every node. Note that some ports can be for internal use and might not be exposed to the node you are pinging from.
• Data structure checker: Mostly applicable for a synchronized persistence deployment. Checks that the data references are not broken.
• Duplicate server nodes: Mostly applicable for a synchronized persistence deployment. Checks there are no duplicate server nodes in the database.
• Remove old app binaries: Remove deleted app binaries from the selected folder.
• Get system configuration: Returns the system configuration from the database.
• Connection string editor: A tool for editing the connection string.
• Service Cluster: Functionality for updating the service cluster in the database.
• Log Fetcher: A tool for fetching logs from the system. If the time span is shorter than 24h all logs will be summarized in one single file. This will make trouble shooting easier if you are looking for events what occurred within a short time frame.
  • Fetching logs with a time span larger than 24h will copy the logs and paste them in their logical structure. The filter functionality selects every row with the specified keyword and creates log files based on rows selected.
    Note : Qlik Support prefers using the Log Collector included in How To Collect Qlik Sense Log Files
• Service type checker: Mostly applicable for a synchronized persistence deployment. Checking for duplicated service types in the database.

If an issue occurs with the existing Data Gateway and only the Gateway of the source connection is changed, the following error will be thrown when the task is resumed:

Failed to send proxy request

REPLICATE-AGENT-SERVICE-22: Gateway coordinates are modified. The source connection was changed to use a different gateway resource, and therefore your data asset cannot continue running. You can either revert the connection to the previous gateway resource or recreate the data asset.

Resolution

For replication projects

1. Scenario One
   
   Target with no Gateway
   
   Solution
   
   Stop the task, run a prepare, then resume the task.
   
   
2. Scenario two
   
   Target with Gateway
   
   Solution
   
   
   1. Stop the task
   2. Change the target gateway to match the source connection
   3. Run a prepare
   4. Resume the task
      

For pipeline projects

Stop the task, run a prepare, then resume the task.

Cause

The data asset has been modified and cannot continue running on a different gateway.

Environment

• Qlik Talend Data Integration

The problem occurs when applying a valid SLK to QlikView. When clicking Apply License, the following error is shown:

Error when applying license: Post "https://license.qlikcloud.com/v1/definitions/signed": net/http: invalid header field value for "Authorization"

Resolution

This error occurs when the license code is copied with line break characters. Those are considered as part of the license, invalidating the code.

To resolve the problem, copy the license code in a text file editor and manually remove the line break characters. Then copy the correctly formatted string to activate the license. 

When removing the line breaks, verify that no other characters are deleted.

Example of an incorrectly and correctly formatted key:

Environment

• QlikView all versions

Qlik Replicate offers a powerful logging feature when its level is set to "Trace" or "Verbose". Instead of directly writing to log files, detailed information is first stored in memory. This is particularly useful for troubleshooting unpredictable errors.

In case of a problem, Qlik Replicate captures the complete picture by writing the in-memory log to disk, including events leading up to and following the error. This strategy proves effective when dealing with issues that lack obvious causes, allowing for deeper analysis and faster resolution.

This article provides step-by-step instructions for Storing trace and verbose logging in memory.

1. Open the Designer tab
    
2. Click Task settings
   
   
   
   
3. Click Logging
   
   
   
   
4. Select Logging Level and tick the Store trace/verbose logging in memory, but if any error occurs write to the logs checkbox.
   
   
5. Confirm with OK
   
   
   
   
6. Click Save
   
   

Environment

• Qlik Replicate

Extracting data from SAP using the OData connector mail fail with error:

Non-https url is not valid

Even if creating the OData connection string succeeds, depending on the volume of data to extract, the error 'Non-https url is not' valid may occur when using tables like GetData, JsonV4GetData, etc.

Example scenario and steps to reproduce:

1. After creating an OData connection to SAP, we used Select Data and correctly listed entities using ListResources table.
   
   
   
   
2. Then, we used GetData table to generate and insert scripts.
3. However, when loading scripts the error Non-https url is not valid occurred
   
   
   
   
4. Repeating the same steps using the parameter Max Results= 100 (as an example) allowed reloading the same scripts without error and returned the expected data.
   
   

Environment

• Qlik Sense Enterprise on Windows, all versions
• Qlik ODBC Connector Package

Resolution

Workaround

Update generated script to manually page through using OData query options $skip and $top

Example:

LIB CONNECT TO 'OData_sap-s20';

let page = 0;
let pageSize = 5;

for i=1 to 5

    let skip = page * pageSize;
    let options = '$skip=$(skip)&$top=$(pageSize)'; 
  trace 'page=$(page), options=$(options)';

  SELECT id,
    title,
    updated
  FROM GetData
  WITH PROPERTIES (
  oDataResourcePath='AttrOfIPRODUCT',
  oDataQueryOptions='$(options)',
  acceptContentType='',
  maxResults=''
  );

  page = page+1;
next;

Cause

'Non-https url is not valid' is caused by the OData nextlink being returned as a relative URL, rather than an absolute URL:

<link xmlns="http://www.w3.org/2005/Atom" href="AttrOfIPRODUCT?$skiptoken=F20250228100118_000252000_0000000002" rel="next"/>

When the connector then performs a check on the URL before making the next (page) request, this fails, because the URL does not start with the expected HTTPS:// and instead with HTTP://.

By defining Max Results= 100, the check is not required, meaning the connector does not perform it. 

Internal Investigation ID(s)

QCB-30986, QB-30986

Question

Is it possible to use the feature "use or register a shared DB Connection" for a subjob or child job with tSnowflakeConnection like we can with other database connections?

Answer

This a new feature for tSnowflakeConnection Component. So far, it is not possible to share a SnowflakeConnection with a subjob/joblet in Talend Studio 8.0 R2025-03 or prior. This new feature will be available in Talend Studio 8.0 R2025-04 and above.

Internal Investigation ID(s)

AJira request (Internal) has been raised for this feature: QTDI-1376

Environment

Talend Data Integration 

All candidates can review their exam history and score reports by logging into Pearson VUE and utilizing the “My Account” menu.

Qlik is partnering with Credly to provide digital badges and certificates for our Qlik Product Certifications. 24-48 hours after you pass your exam, you will receive an email from Credly. Follow the instructions in the email to accept your digital badge and access your certificate.

If you experience issues while accepting your digital badge, please contact Credly support.

More information, see: Certifications and Qualificiations