Updating an Excel Worksheet using Qlik Application Automation leads to Microsoft Excel reporting a 503 error.

Resolution

A 503 (Service not Available) error indicates the target system, in this case Microsoft Excel, cannot handle the calls due to high frequency. To overcome this, a retry mechanism would need to be implemented in your automation, adding robustness in front of those errors.

Attached to this article is a sample automation demonstrating how to handle these errors in the Excel block and implement retry logic when it fails.

Cause

Third-party bottleneck.

Environment

• Qlik Application Automation
• Qlik Cloud

TAC tasks are hanging on "Running" status for a long time in Job Server, and it does not end. 

Analysis

This issue can be identified by long duration of task execution time and it can be seen from the large time gap between the job end time and the task end time.

Usually, the time gap between the job end time and the task end time are small if there are no problems occurring.

Resolution

Please check and try the following points.

1. Check that the Talend Administration Center application (or service) is up and running.
2. Check that the connection can be established to the Job Server.
3. Check that there are no differences in the version of the Talend Administration Center and the Job Server, since the Talend Administration Center and the Job Server affect one another. If the version is old on either side, it may cause problems.
4. In order to reconnect Talend Administration Center and the Job Server, restart Talend Administration Center to see if it fixes the problem.

Cause

It is most likely that there are network connection issues (because of network infrastructure, VPN, etc)between the Talend Administration Center and the execution server (Job Server).
One possible situation may be the following: After the task is run by the Job Server, the execution result should be sent to the Talend Administration Center normally. If there is a connectivity problem between the Talend Administration Center and the Job Server, then the execution result will not be sent to the Talend Administration Center, which causes this problem.

Environment

Talend Administration Center 

You can access Qlik Sense demo apps from https://demos.qlik.com/.

If you are looking for real-life business examples, our Qlik Gallery hosts a platform meant for sharing apps, outcomes, and ideas. It is not restricted to Qlik-provided demos and mostly consists of customer examples.

If you are looking to download demo apps:

1. Go to https://demos.qlik.com/ 
2. Select Downloadable from the available Tags
   
3. Open the App
4. A Download button will be available (if logged in)
   
   

After upgrading to Qlik Sense Enterprise on Windows May 2023 (or later), the Qlik Sense Repository Service may cause CPU usage spikes on the central node. In addition, the central Engine node may show an increased average RAM consumption while a high volume of reloads is being executed. 

The Qlik Sense System_Repository log file will read:

"API call to Engine service was successful: The retrieved 'StaticByteSize' value for app 'App-GUID' is: 'Size in bytes' bytes." 

Default location of log: C:\ProgramData\Qlik\Sense\Log\Repository\Trace\[Server_Name]_System_Repository.txt

This activity is associated with the ability to see the Base memory size in the Qlik Sense Enterprise Management Console. See Show base memory size of apps in QMC.

Resolution

The feature to see Base memory size can be disabled. This may require an upgrade and will require downtime as configuration files need to be changed.

Take a backup of your environment before proceeding.

1. Verify the version you are using and verify that you are at least on one of the versions (and patches) listed: 
   
   
   • May 2023 Patch 13
   • August 2023 Patch 10
   • November 2023 Patch 4 
   • February 2024 IR
     
2. Back up the following files:
   
   
   • C:\Program Files\Qlik\Sense\CapabilityService\capabilities.json
   • C:\Program Files\Qlik\Sense\Repository\Repository.exe.config
     
     *where C:\Program Files can be replaced by your custom installation path.
     
     
3. Stop the Qlik Sense Repository Service and the Qlik Sense Dispatcher Service
   
4. Open and edit the capabilities.json with a text editor (using elevated permissions)
   1. Open C:\Program Files\Qlik\Sense\CapabilityService\capabilities.json
   2. Locate: "enabled": true,"flag": "QMCShowAppSize"
   3. Change this to: "enabled": false,"flag": "QMCShowAppSize"
   4. Save the file  
5. Open and edit the repository.exe.config with a text editor (using elevated permissions)
   1. Open C:\Program Files\Qlik\Sense\Repository\Repository.exe.config
   2. Locate: </appSettings>
   3. Before the </appSettings> closing statement, add:
      <add key="AppStaticByteSizeUpdate.Enable" value="false"/>
   4. Save
6. Start all Qlik Sense services
7. Verify you can access the Management Console

If any issues occur, revert the changes by restoring the backed up files and open a ticket with Support providing the changed versions of repository.exe.config, capabilities.json mentioning this article. 

Related Content

Show base memory size of apps in QMC

Internal Investigation:

QB-22795
QB-24301

Environment

Qlik Sense Enterprise on Windows  May 2023, August 2023, November 2023, February 2024. 

When using Qlik Sense behind a reverse proxy, environment.user.ip is always the IP of the reverse proxy instead of the end user client.

Environments:

• Qlik Sense Enterprise for Windows

This is a product limitation.

A feature request (SHEND-225) has been logged but there is no information if and when the feature will be implemented yet.

Unable to Execute Talend job query after Talend Remote Engine and JAVA version 17 upgrade in PROD.

The Same query was running fine before on Java 11 version but after Java upgrade to 17,  it is unable to run the particular query and same jobs are failing due to below error

The original query inside hive tDBInput component has no grammar issue.

Execution failed with code 1 and error Job stopped with errors or unable to run. Exception in component tDBInput_4 (Outbound_File_Creation_Type_1) org.talend.sdk.component.api.exception.ComponentException: (java.sql.SQLException) [Amazon][HiveJDBCDriver](500051) ERROR processing query/statement. Error Code: 40000, SQL state: TStatus(statusCode:ERROR_STATUS, infoMessages:[*org.apache.hive.service.cli.HiveSQLException:Error while compiling statement: FAILED: ParseException line 1:3328 missing EOF at 'JOIN' near ')':17:16, org.apache.hive.service.cli.operation.Operation:toSQLException:Operation.java:335, org.apache.hadoop.hive.ql.reexec.ReExecDriver:compileAndRespond:ReExecDriver.java:126,

Resolution

Try the setting "UseNativeQuery=1" in JDBC connection URL property to enable the use of Native query option.

For example:
jdbc:hive2://localhost:10000/foundation;AuthMech=2;UseNativeQuery=1

This parameter specifies whether the driver transforms the queries emitted by applications. 

1: The driver does not transform the queries emitted by applications, so the native query is used. 

0: The driver transforms the queries emitted by applications and converts them into an equivalent form in HiveQL.

If the application is Hive-aware and already emits HiveQL, then enable this option to avoid the extra overhead of query transformation.

Cause

As HiveQL is using parameter "UseNativeQuery=0" by default, which means it will transform query inside JDBC before sending to HS2 , and the introduced false optimization query will cause the problem. 

Environment

Talend Big Data 

Qlik Replicate tasks using DB2 LUW Source Endpoint experience latency when consuming data.

Resolution

Verify if the DB2 LUW configuration for the DB2 instance of the Source endpoint has its configuration parameters LOGMETHOD1 and LOGMETHOD2 set to a backup device location.

This would indicate the DB2 Log Reader Agent is loading the transaction logs from a backup device which will add overhead to the task using this DB2 Source Endpoint.

To correct the condition you can set the DB2 parameter by the DBA Overflow Log path from NULL to archive_log_path. See overflowlogpath - Overflow log path configuration parameter for more information.

This path can point to a local disk on the DB2 server for optimal access to the DB2 transaction logs.

First log archive method:

(LOGARCHMETH1) = VENDOR:/opt/simpana/Base64/libDb2Sbt.so

Second log archive method:

(LOGARCHMETH2) = DISK:/db2/P10/alternate_logpath/

Overflow log path:

(OVERFLOWLOGPATH) = archive_log_path

Cause

This is the result of the DB2 Log Reader Agent, which Qlik Replicate calls using the DB2 API and from which we access the transaction logs from DB2. In a setup in which LOGMETHOD1 is pointing to a backup device, the DB2 Log Reader Agent loading the transaction logs will lead to overhead and ultimately latency. 

Related Content

• overflowlogpath - Overflow log path configuration parameter

Environment

• Qlik Replicate

When Qlik Data Gateway Direct Access is implemented, sometimes as the workload increases post-implementation, customers may experience general stability issues. We suggest a basic health check to address the more well-known problems and stabilize your gateway implementation. 

This article outlines three 

1. Verify the number of disconnects
2. Verify the number of ODBC connector crashes
3. Verify long-running metadata requests

Resolution

(1) Verify the number of disconnects

A high number of disconnects can cause reload disruptions. Some important disconnects will be recorded in the DirectAccessAgent logs. While a low volume of disconnects is expected and will typically recover within a few seconds, a high number of entries in the logs are indicative of a problem.

We suggest initiating an internal investigation if more than ten disconnects are found per day. 

Search for “[ERROR] Connection to”, sample:

[Service ] [ERROR] Connection to <mytenant>.us.qlikcloud.com:443 failed

If the amount of disconnections is high, we suggest to refer to articles: 

• Qlik Data Gateway - Direct Access. Reloads failing intermittently
• Data Gateway Direct Access disconnections (0x80004005): The remote party closed the WebSocket connection without completing the close handshake. 

(2) Verify the number of ODBC connector crashes

ODBC connector crashes can impact concurrent reloads if Process Isolation is not enabled. We suggest customers who experience crashes in their logs enable Process Isolation and configure Dump file creation. 

Process isolation for ODBC connector can be confirmed in the C:\ProgramData\Qlik\Gateway\configuration.properties. ODBC_PROCESS_ISOLATION_MODE, will be set to None if disabled, and set to Load if enabled.

It can also be verified using the logs. The ODBC logs named odbc-connector_logs.txt, odbc-connector_logs.txt.1 from 1 to n. If isolation is enabled, the files includes a port number, for example odbc-connector_logs_49156.txt where 49156 would represent a port number.

To identify whether there are crashes or not in your DAG (Direct Access Gateway) and proceed to enable Process Isolation we suggest reviewing Qlik Data Gateway - Direct Access - ODBC Crashes.

(3) Verify long-running metadata requests 

Other cause of issues would be due to misconfiguration of third-party drivers, the details are outlined in Qlik Data Gateway Direct Access Data Preview long metadata requests.

Keep in mind that these recommendations are additional validations to the Best Practices and System Prerequisites documented in our Help Site:

• Best practices when using Qlik Data Gateway - Direct Access
• System prerequisites

Cause

Best Practices and System Requirements may not have been fully met. 

Environment

• Qlik Data Gateway Direct Access

Advanced options is not visible when editing a sheet. This can happen on a specific app even if the option was present before.

Resolution

You can activate the "Show Sheet Header" option in the app settings to make the "Advanced option" button visible again.

1. Open the App settings.
   
2. Activate "Show Sheet Header".
   

Cause 

For some reasons, the "Show Sheet Header" could be deactivated in an app. "Advanced Options" is invisible when this happens, because it is located in the sheet header that is removed. The app looks like this in editing mode:

Environment

• Qlik Cloud

This document is a general guide and is provided as is. Modifications to the process may be necessary depending on your individual database setup.

If you have installed a standalone PostgreSQL database, or if you have used the Qlik PostgreSQL Installer (QPI) to upgrade and decouple your previously bundled database, then you can upgrade PostgreSQL at any time. This means you control maintenance and can immediately react to potential PostgreSQL security concerns by upgrading to a later service release or a later major version.

Content

• Requirements:
• Preparation
• Upgrading to a later service release
• Upgrading to a new major PostgreSQL release

Requirements:

• Only upgrade to a supported version of PostgreSQL. See the Qlik Sense Enterprise on Windows System Requirements for details. 
• This guide covers deployments where PostgreSQL has been installed separately or previously unbundled using the Qlik PostgreSQL installer. 

This document covers the following scenario:

• Windows Server OS. The use of Linux, in managed cloud deployments or on-prem, is not covered. 
• Generic PostgreSQL deployment. Stream replication, SSL, or any other advanced configuration are not covered.

Preparation

Run a complete backup of Qlik Sense Enterprise on Windows site as described in Backup and restore Qlik Sense Enterprise on Windows.

• Qlik Sense Repository Database (from PostgreSQL database server)
• Qlik Sense share folder
• Qlik Sense CA-signed certificates

Upgrading to a later service release

These steps apply if you are upgrading within a major PostgreSQL release (example: 14.5 to 14.8). 

1. Download the PostgreSQL installer from them PostgreSQL site
2. Stop the Qlik Sense Enterprise on Windows services. Leave your standalone PostgreSQL database running.
3. Run the PostgreSQL installer and follow the instructions.
4. Start the Qlik Sense Enterprise on Windows services.

No further steps are required.

Upgrading to a new major PostgreSQL release

If you are moving to a higher major version, an in-place upgrade will not be possible. Instead, we will install the the new version in parallel, then then migrate the old database and eventually uninstall the old version. Our example is written using PostgreSQL 12 to 14. 

1. Download the PostgreSQL installer from them PostgreSQL site
   
   
2. Run the installer
   
   During install:
   
   
   • Install the database on the standard 5432 port. 
     
     This allows us to have 12 running on the Qlik Sense default 4432 port and the 14 instance on 5432.
     
     
   • Set the same password as the one currently in use with the version 12 PostgreSQL instance. 
     
     
3. Open a Windows File explorer and create a backup folder, such as C:\backup 
   
   
4. Backup your version 12 database 
   
   This step is crucial. You may have already backed up your site, but in this step we will create the specific backup we will use to migrate your database, using a modified command.
   
   To do so:
   
   
   1. Stop all Qlik Sense Enterprise on Windows services on all nodes. Leave postgresql-x64-12 service (your current repository) running
      
      
   2. Stop postgresql-x64-14 (your newly installed PostgreSQL instance)
      
      
   3. Open a Windows Command Line (CMD) as administrator
      
      
   4. Run:
      SET PGPASSWORD=postgrespassword (replace this with the actual password)
      
      
   5. Run:
      cd “C:\Program Files\PostgreSQL\12\bin”
      
      
   6. Run
      pg_dumpall.exe -h localhost -U postgres -p 4432 -f c:\backup\backup_db.sql
      
      
   7. Follow the onscreen instructions and wait until the backup is finished
      
      Note: If you are prompted to input the password (even if it was previously entered correctly), you will need to modify pg_hba.conf. 
      
      Location: C:\Program Files\PostgreSQL\12\data\pg_hba.conf
      
      Change md5 to trust and restart the postgresql-x64-12 service. See Changing the Database Superuser Password without Qlik Sense Installed for details.
      
      Once the backup has completed, revert back to md5.
      
      
5. Migrate from 12 to 14
   
   We are now moving our Qlik Sense repository from the earlier release of PostgreSQL into the new one.
   
   
   1. Start the postgresql-x64-14 service
      
      
   2. Stop postgresql-x64-12
      
      
   3. Open a Windows Command Line (CMD) as administrator
      
      
   4. Run:
      cd “C:\Program Files\PostgreSQL\14\bin”
      
      
   5. Run
      psql -h localhost -U postgres -p 5432 -f c:\backup\backup_db.sql
      
      
   6. Follow the onscreen instructions.
      
      Note: If you receive the error password authentication failed for user "postgres", (even if it is correct), you will need to modify pg_hba.conf. 
      
      Location: C:\Program Files\PostgreSQL\14\data\pg_hba.conf
      
      Change scram-sha-256 to md5 and restart the postgresql-x64-14 service  and redo step e). See Changing the Database Superuser Password without Qlik Sense Installed for details.
      
      
6. With the import finished, copy the following two files from your old 12 data folder into your new 14 data folder:
   
   
   • pg_hba.conf 
   • postgresql.conf 
     
     
7. Open postgresql.conf and verify that the port is set to 4432 (the standard Qlik Sense port)
   
   
8. (If not done already) stop the postgres-x64-12 service
   
   
9. Restart the postgres-x64-14 service
   
   
10. Start all Qlik Sense services
    
    
11. Once you have confirmed all functionality and tested, uninstall PostgreSQL 12. This can be done directly from the Windows Control Panel and Programs and Features

Related Content:

Upgrading and unbundling the Qlik Sense Repository Database using the Qlik PostgreSQL Installer 
How to manually upgrade the bundled Qlik Sense PostgreSQL version to 12.5 version 
Changing the Database Superuser Password without Qlik Sense Installed 

It is common to override Talend Studio jars with external/new jars, as demonstrated in the documentation page titled "Customizing the Maven URI for External Module Deployment".

However when trying to override a jar with another jar that is already installed in Studio, you will get in the error below. In the example below, the goal is to replace all references to Snowflake JDBC 3.13.30 with Snowflake 3.13.29. Both drivers are already installed in Studio/exist in the .m2 folder.

To get to the window above, navigate to Window | Show View in Talend Studio and choose Modules:

Open the Module view in the bottom, click on the Maven URI/component you want to customize and click the "..." button.

To point all references of Snowflake JDBC from version 3.13.30 to 3.13.29, remove the value of the Module File, check the Custom MVN URI, and enter the path of the existing module into the URI field. Below is an example that demonstrates the addition of the complete maven path for the Snowflake 3.13.29 driver.

NOTE: This change will affect all components that reference the jar. In the example below, 11 components utilize Snowflake JDBC 3.13.30. If you confirm the aforementioned selection, all 11 components will switch to using JDBC 3.13.29. It is not possible to selectively update specific components that rely on 3.13.30 to use 3.13.29, all 11 references will change.

Once you click the "OK" button, all references will change. As demonstrated below, all modules that were previously pointing to 3.13.30.jar will now use the Maven URI for 3.13.29.jar.

If the module needs to be reverted, simply return to the same window and uncheck "Custom Maven URI".

You are experiencing with the following error when connecting to Talend Administration Center

org.talend.administrator.common.exception.DBException: The database "{database URL}" is currently unreachable

Resolution

Investigating for problems that makes the TAC application not connect to the database properly.
Please check the following points:

• Make sure that the connection between TAC and the database is configured correctly on TAC (for example: parameter settings).
• The database server is running well.
• The database is reachable from an application or tool other outside of Talend Administration Center.
• The database is not facing any issues.

Cause

When the error occurs, it means that there is a problem in the connection between TAC application and the database.

Environment

Talend Administration Center 

After merging feature branches into the main branch in Talend Studio, several Jobs are not displayed correctly. Some Jobs are missing, while others contain incorrect files. Additionally, when importing Jobs with the same ID into Talend Studio, the following error message is displayed.

The 2 following items have the same ID, please unselect one of them:

Cause

Upon comparing the properties files of two Jobs, it was discovered that both Jobs contain the same ID. Talend Studio is unable to import Jobs with the same ID.

Resolution

Please do not attempt to manually change the ID as it may lead to other dependency issues. Instead, follow the steps below to import these two Jobs that have the same ID.

1. Import the first Job to avoid the duplicate error.
2. Then, right-click on the Job and select "Duplicate" to create a duplicate Job.
3. Delete the original Job.
4. Empty the Recycle bin.
5. Import the second Job.
6. Rename the duplicated Job back to its correct name.
7. Now, the two Jobs have different ID.

Environment

• Talend Studio 7.3.1, 8.0.1

Are you looking to download your purchased Qlik Products or download a Trial? All supported on-premise Qlik Products can be downloaded from Qlik's Product Download Site.

To access the Download Site, you need an active QlikID. You will be able to see all products your account is eligible for.

You can access the Download Page directly here, or navigate to it from the Community Home page:

1. Open the Qlik Community
2. Click on Support
3. Select Product Downloads
   
   
   
   

This shows the download page with the Latest release and patch preselected.

1. Choose a different filter in (1) Show Releases if you are looking for older product versions. By default, only the Latest release and patch will be displayed. 
   
   
   
   
2. From here:
   
   
   1. choose your Product Category (Qlik Data Analytics, Qlik Data Integration, Search All),
   2. your Product,
   3. your Release,
   4. and Release Number.
   5. Then click the Download Link
      
      

Unsupported versions are not available for download. See Product Lifecycle for details on what versions have reached end of support. 

Alternatively, instead of selecting your product directly, you can also search all available columns:

1. Click the Magnifying Glass
2. Begin typing your search
3. Select the result
4. Confirm the result

If you encounter issues with the download site, start a chat with us and we will be able to help you right away.

Click here for video transcript

When an ODBC connector crash occurs in Qlik Data Gateway — Direct Access, other concurrent reloads may be impacted following the original crash. This will specifically be the case if Process Isolation is not enabled. 

To verify if there are ODBC connector crashes:

1. Go to the DAG (Direct Access Gateway) folder: C:\Program Files\Qlik\ConnectorAgent\data\logs\
2. From there, use any file search tool and search for “has exited” entries in the DirectAccessAgent log files, an example:
   
   [Manager ] [WARN ] ODBC process (2172) has exited (exit code: 0), going to restart it...

Multiple connector crashes can have a big impact on the stability of the Direct Access Gateway. 

Resolution

Enable Process Isolation as documented in Mitigating connector crashes during reload. 

Cause

Some crashes can occur due to malformed queries, refer to the Data Gateway - Direct Access. Initial validation for ODBC crashes article.

If there are other crashes without traces that could explain their nature, enable dump file creation, collect the dump files and necessary information for DAG cases and contact Qlik Support. 

Environment

• Qlik Cloud
• Qlik Data Gateway - Direct Access

When embedding a sheet in an iFrame, the "full screen" toggle button for Qlik Sense native chart objects is missing. This feature is present at the sheet level in the hub

Full Screen toggle button is available at the sheet level in the hub

The same feature is  missing, once a sheet is added in an iFrame (Single Configurator)

Environment

• Qlik Sense Enterprise (all versions)

Resolution

This is a known limitation.

An improvement task has been created (Jira task VIZ-103) and an idea is available on our Ideation Platform.

To check the current status or to add your vote, head over to the idea: full screen for iframe and embed 

Internal Investigation ID(s):

QB-4967
VIZ-103

When publishing a Job from Talend Studio, configured with a Nexus Artifact Repository, the following error is shown:

java.io.IOException: org.eclipse.aether.deployment.DeploymentException: Failed to deploy artifacts: Could not transfer artifact xxxx from/to xxxx (xxxx): transfer failed for xxxx, status: 503 Repository offline

Resolution

Please check your Talend Artifact Repository and set the status to online.

Cause

The Status of 503 Repository offline means that the Talend artifact repository is offline.

Environment

Talend Data Integration 

Question

Can Talend products run on a virtualization system?

The answer is Yes. In order to run Talend products on a virtualization system (for example: Hyper-V), please check if the virtualization system meets the following requirements.

1. Software requirements
2. Hardware requirements
3. Compatible Operating Systems

For further information, please check the “Statement regarding virtualization and Docker deployments” section in the following URL: Compatible Operating Systems.

Environment

• Talend Studio
• Talend Data Integration
• Talend Big Data 

If a tenant previously had an incomplete SMTP configuration with their Qlik Cloud, an error message will now be shown to the Tenant Administrator:

• Your email providers is not working – Go to Settings 

Resolution

To resolve this error a Tenant Admin can enter valid SMTP credentials. 

At the moment, it is not possible to delete/clear the previous credential entry in the authentication.  An option to clear the credentials is being prepared to support a return to a default (non-configured/empty) state and is expected to be available in the coming weeks.

We will update this article when the ability to clear becomes available. 

Example SMTP Error

Cause 

With the release of the SMTP service connectivity for Microsoft O365 from the Management Console, more stringent error checking was added to the basic authentication configuration. 

Related Content 

If interested, Admins can still successfully connect to Microsoft 0365 SMTP with this error showing. More details on the new available 0Auth2 authentication can be found here: Qlik Cloud: Introducing OAuth2 authentication for ... - Qlik Community - 2444243

Internal Investigation ID

• QB-26792

Environment

• Qlik Cloud

When connecting to a Microsoft SQL database from an untrusted domain, the result is a The login is from an untrusted domain and cannot be used with Windows authentication error.

Resolution

A possible two solutions exist. Choose one of the below:

Option One

If using Windows Authentication.

1. Open the Windows Services.msc
2. Right-click on SQL Services
3. Open Properties
4. Change Log On As from from Local System Account to an account with Local Administrator permissions which is set up with the correct permissions on both Qlik Sense and the SQL server. This account must have Log on locally security permission.

Option Two

Use SQL authentication.

Rather than using using Windows Authentication connecting to the SQL server, use SQL Authentication. 

For more information, see Choose an authentication mode.

Environment

• Qlik Sense Enterprise on Windows, any