Unlock a world of possibilities! Login now and discover the exclusive benefits awaiting you.
In Talend tDBInput component, CLOB data types can not be handled properly when you are attempting to ingest data from source DB2 to Target one.
This article briefly introduces how to properly handle CLOB data types when ingesting from DB2 to Target using Talend.
The code below is a scratch version that needs testing and rewriting.
Use this as an example to study:
==Example code: === package routines; import java.sql.Clob;
import java.sql.SQLException;
import java.util.logging.Level;
import java.util.logging.Logger;
import org.apache.commons.io.IOUtils; package routines; import java.sql.Clob;
import java.sql.SQLException;
import java.util.logging.Level;
import java.util.logging.Logger;
import org.apache.commons.io.IOUtils; import java.io.IOException;
import java.io.InputStream;
import java.io.StringWriter;
import java.lang.String; public class ClobUtils { public static String getClobAsString(Object obj) {
String out = null;
if (obj != null && obj instanceof Clob) {
out=getClobAsString((Clob) obj);
} else {
Logger.getLogger("ClobUtils").log(Level.FINE, "null value");
}
return out;
} public static String getClobAsString(Clob clobObject) {
String clobAsString = null;
if (clobObject != null) {
long clobLength;
try {
clobLength = clobObject.length(); if (clobLength <= Integer.MAX_VALUE) {
clobAsString = clobObject.getSubString(1, (int) clobLength);
} else {
InputStream in = clobObject.getAsciiStream();
StringWriter w = new StringWriter();
IOUtils.copy(in, w, "UTF-8");
clobAsString = w.toString();
}
} catch (SQLException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
}
return clobAsString;
}
}
A ESB route is failing to deploy with error message" xxx is likely to be incompatible with this Talend Runtime version, try recompiling the route with a newer Talend Studio version".
ERROR | Blueprint Event Dispatcher: 1 | b.CompatibilityBlueprintListener 73 | 332 - deployer-interceptor - 8.0.1.<Monthly Patch>-RT | Bundle cifmessagequeue.<Route Name> id : <Karaf> is likely to be incompatible with this Talend Runtime version, try recompiling the route with a newer Talend Studio version
Ensure Runtime and Studio are at the same version and patch level.
As this issue may be occurring due to Camel 3 is out of support, please try to move up to Camel 4 to see if it get resolved.
In order to do so, please upgrade to the following versions:
If you have built ESB route jobs using Talend Studio R2025-01 or previous versions which use Camel 3, please rebuild them in Talend Studio R2025-02 or later versions which use Camel 4.
This error indicates the Runtime Server and Talend Studio version are incompatible. The recent Camel upgrade for 2025-02 and beyond may be the issue.
Talend Runtime 8.0.1-R2025-02 (Camel 4) is available as a complete installation package, this version supports only Routes and Data Services artifacts built with Talend Studio R2025-02 or later. Artifacts built with previous Studio versions are incompatible. This is due to the Camel 4 and Java 17 compiler changes that have been updated to both Runtime and Studio.
Please check if both Studio and Runtime server are patched to the same version.
For more information about upgrade, please refer to below Qlik Talend Release Notes
https://help.qlik.com/talend/en-US/release-notes/8.0/r2025-02-studio
https://help.qlik.com/talend/en-US/release-notes/8.0/r2025-02-talend-runtime
Due to tHttpRequest, tFileFetch and tRest components have been removed from Talend Studio 8.0 R2025-04 onwards in favor of tHTTPClient, some adjustments are required to transition to tHTTPClient if your current Talend jobs actively use the deprecated tHttpRequest/tFileFetch/tREST/component.
This article briefly introduces how to find all jobs rely on the deprecated components for replacement purpose.
QTDI-783 - Deprecate tREST component in favor of tHTTPClient
QTDI-784 - Deprecate tFileFetch component in favor of tHTTPClient
QTDI-785 - Deprecate tHTTPRequest component in favor of tHTTPClient
You might be experiencing an issue in data formatting where input source values with leading zeros are not being reflected in the destination.
For example, a value like "00000001“ in the source may appear as ”1“ in the target
From above example, the leading zeros are not retained in the target column( unique ID in this case) when transferring data from a BigDecimal source type to a String type
Use the following expression in the tMap Component of Talend Studio while mapping from source to destination
"String.format("%08d", row1.InputColumn.toBigInteger())"
A BigDecimal's standard canonical string representation does not include leading zeros.
Therefore, if you need to preserve leading zeros, you must explicitly format the value accordingly.
For more information about BigDecimal math, please refer to Oracle Official Document as below
https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/math/BigDecimal.html
A Talend ESB Route failed with below error when executing in studio
org.apache.camel.processor.errorhandler.DefaultErrorHandler- Failed delivery for (MessageId: xx on ExchangeId: xx).
Exhausted after delivery attempt: 1 caught: org.apache.camel.TypeConversionException: Error during type conversion from type: org.apache.camel.component.file.GenericFile to the required type: java.lang.String with value GenericFile[myfile] due to java.nio.charset.MalformedInputException: Input length = 1
The Route Design looks like:
cFile →cProcessor (with the code exchange.getIn().getBody(String.class) in it)
From above error message, you can see: "The java.nio.charset.MalformedInputException: Input length = 1” error in Java arises when the program attempts to decode a character sequence with a character encoding that is incompatible with the actual encoding of the input data. Specifically, "Input length = 1" indicates that a single byte or character is causing the decoding issue.
When attempting to deploy a Route in Talend Runtime, encountered the following error:
java.lang.NoClassDefFoundError: Could not initialize class org.talend.components.common.oauth.Jwt
Caused by: java.lang.NoClassDefFoundError: org/codehaus/jackson/map/ObjectMapper
Missing bundles for Jackson library.
Launch the Karaf console (./trun) located in the <Runtime-Home>\bin directory, and then execute the following command:
list | grep Jackson
If the following bundles are not installed please install them using the command below:
bundle:install -s mvn:org.codehaus.jackson/jackson-mapper-asl/1.9.16-TALEND
bundle:install -s mvn:org.codehaus.jackson/jackson-core-asl/1.9.16-TALEND
Furthermore, you have the option to install the feature that encompasses all necessary bundles by running:
feature:install camel-jackson-avro
When transitioning from password-based authentication to key pair authentication for Snowflake, you may encounter issues during the configuration of import models, especially when using encrypted private key files generated with OpenSSL v3. A common error observed during connection attempts is:
Fatal: MITI.MIRException: Connection to the database with URL 'jdbc❄️//.snowflakecomputing.com' failed: JWT token is invalid.
java.security.NoSuchAlgorithmException: 1.2.840.113549.1.5.13 SecretKeyFactory not available
java.security.InvalidKeyException: IOException : DER input, Integer tag error
The Java process fails to complete due to an invalid JWT token, commonly associated with unsupported key formats or missing runtime parameters.
The error typically occurs due to one or more of the following factors:
When using an unencrypted private key file:
User: Set to the Snowflake username.
Password: Enter the password
Private Key File: Provide the path to the unencrypted .pem file.
Miscellaneous Parameters: No additional Java options are required.
When using an encrypted private key file (e.g., OpenSSL v3-generated):
User: Set to the Snowflake username.
Password: Enter the passphrase used to encrypt the private key.
Private Key File: Provide the full path to the encrypted .pem file.
Miscellaneous Parameters: You must enable BouncyCastle support for decryption:
-Dnet.snowflake.jdbc.enableBouncyCastle=true
For further details, please refer to the official Snowflake documentation: Snowflake Key-Pair Authentication Guide.
Qlik Talend Product: How to set up Key Pair Authentication for Snowflake in Talend Studio
After upgrading from Talend Version 7 to Version 8, you may encounter an issue where a Custom SQL function (Mixed read/write SQL) previously invoked via tDBInput component in Talend Version 7 and containing both INSERT/UPDATE and SELECT statements no longer performs the INSERT/UPDATE operations in Talend V8.
Even the function still returns data, though the changes to the target tables are not applied well as they did in Talend Version 7 before.
Please use tDBRow and tParseRecordSet components to support the mixed read/write SQL function call
Mixed read/write SQL function execution is no longer supported via tDBInput component in the latest Studio Version.
combining-two-flows-for-selective-output-standard-component-in-this
The primary error is displayed as follows in Studio/Remote Engine/Job Server Logs:
Error: LinkageError occurred while loading main class <job>_<version>.<job> java.lang.UnsupportedClassVersionError: <project>/<job>_<version>/<job> has been compiled by a more recent version of the Java Runtime (class file version 61.0), this version of the Java Runtime only recognizes class file versions up to 55.0
A review of the error shows
“…compiled by a more recent version of the Java Runtime (class file version 61.0)”
A review of the Java Runtime class table will show what version of the Job was compiled. In this case, it's Java 17.
JAVA SE | Major Version |
8 | 52 |
11 | 55 |
17 | 61 |
Configure the Job execution on the Remote Engine/Job Server to utilize Java 17. For further details, please refer to the documentation page Remote Engine configuration guide's section on setting the JAVA execution path.
You may experience intermittent issues when accessing the Talend Administration Center (TAC), along with the following warning message if you are using MySQL as the database
Caused by: com.mysql.cj.exceptions.CJCommunicationsException: Communications link failure
Loading class `com.mysql.jdbc.Driver'. This is deprecated. The new driver class is `com.mysql.cj.jdbc.Driver'. The driver is automatically registered via the SPI and manual loading of the driver class is generally unnecessary.
And this issue is random and sometimes can be temporarily resolved by refreshing the Talend Administration Center page.
(ConnectionImpl.java:456) at com.mysql.cj.jdbc.ConnectionImpl.getInstance(ConnectionImpl.java:246) at com.mysql.cj.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:199) at java.sql.DriverManager.getConnection(DriverManager.java:664) at java.sql.DriverManager.getConnection(DriverManager.java:247)
MySQL JDBC drivers are backward compatible.
As per MySQL documentation, the driver class name has been updated from com.mysql.jdbc.Driver (deprecated) to com.mysql.cj.jdbc.Driver and using the deprecated driver may cause compatibility or communication issues.
For more information, please refer to MySql official document:
https://dev.mysql.com/doc/connector-j/en/connector-j-api-changes.html
installing-external-modules-manually-for-tac
You may encounter the following issue while trying to push code to the Bitbucket repositories from Talend Studio Application
Error: Command org.eclipse.jgit.api.PushCommand was called in the wrong state
Caused by: org.eclipse.jgit.errors.TransportException: ********.git: push not permitted
As Write access is required to push code to the repository, please kindly check with the Bitbucket admin to obtain write access for the repository/project and attempt to push again.
This error typically occurs when there is no proper write access for the repository/project. For example, you may have read-only access to the repository on Bitbucket.
When using the CICD process, you may point the P2 to patch from your Artifact Repository instead of the Studio update site. Whether using a remote/proxy repository or have uploaded the patch contents to a local repository, you will encounter some error when attempting to deploy, redeploy, or patch your existing P2 deployment.
If you have setup basic authentication for your studio, Eclipse (within Studio) will require the keyring (secure storage) file and master password generated by Studio to gather the authentication credentials for the Artifact Repository and the P2. However, since the defined "secure_storage" file has references to Windows Security, if you attempt to use it on a Linux-based instance for CICD, it will fail due to the missing items. The root reason is the current existing keyring(secure storage) of Talend studio can not be used on another computer. By default, master key is generated by OS related API.
So if basic authentication is required for the P2 to reach out for patch items and there is a Linux-based CICD instance, you can try to re-generate keyring with provided master key in this way the newly generated keyring (secure storage) and master key can be shared to another computer.
Please note even this will not delete your existing secure_storage, though it is recommended to back them up for in case.
cat mymaster.key
mysecretmasterkey
-Declipse.keyring=/opt/talend-801/keyring/keyring.txt
-Declipse.password=/opt/talend-801/keyring/masterpassword.txt
Additionally, those files can be stored on the scheduler or an external location, and the eclipse flags can point there.-Dinstaller.clean=trueor the specified p2 install directory cleared out.
basic-authentication-for-update-repositories-in-studio
Trying to connect SQL server with Azure MFA Authentication, It works fine using metadata connection. However, when the metadata connection is added to a Job, the Job fails with below error:
Failed to load MSAL4J Java library for performing ActiveDirectoryInteractive authentication.
com.microsoft.sqlserver.jdbc.SQLServerException: Failed to load MSAL4J Java library for performing ActiveDirectoryInteractive authentication.
Upon checking the modules view in Talend Studio, the MSAL4J jar is already installed.
In the Advanced settings of the tDBConnection for the SQL Server connection, enable the option labeled "Authenticate using Azure Active Directory", and subsequently attempt to run the Job once more. Upon enabling this option, the msal4j jar will be automatically packaged during the Job build process.
When attempting to create a datasource that enables routes/services to access an Azure-based SQL Server with AD Authentication, the datasource may fail to connect. Despite providing the necessary ms__auth.dll to Runtime (either through the service or etc folder) and windows, the tesb log may still display the following error message:
"Failed": "com.microsoft.sqlserver.jdbc.SQLServerException:Failed to load MSAL4J Java library for performing ActiveDirectoryPassword authentication.
In many cases, the reason for this error is due to Runtime not having the specific library installed within it. Although they are very similar, Azure-based SQL Instances do require specific dependencies that may not be found with the regular Microsoft SQL JDBC drivers. This may affect those users who have a Runtime instance at or newer than the R2024-06 Patch.
To address this issue, please follow these steps:
If the datasource or task is still encountering connection issues, kindly contact Qlik Support.
Running a Talend Job using a key pair authentication for Snowflake fails with the exception:
Starting job Snowflake_CreateTable at 09:21 19/07/2021. [statistics] connecting to socket on port 3725 [statistics] connected Exception in component tDBConnection_2 (Snowflake_CreateTable) java.lang.RuntimeException: java.io.IOException: Missing Keystore location at edw_demo.snowflake_createtable_0_1.Snowflake_CreateTable.tDBConnection_2Process(Snowflake_CreateTable.java:619) at edw_demo.snowflake_createtable_0_1.Snowflake_CreateTable.runJobInTOS(Snowflake_CreateTable.java:3881) at edw_demo.snowflake_createtable_0_1.Snowflake_CreateTable.main(Snowflake_CreateTable.java:3651) [FATAL] 09:21:38 edw_demo.snowflake_createtable_0_1.Snowflake_CreateTable- tDBConnection_2 java.io.IOException: Missing Keystore location java.lang.RuntimeException: java.io.IOException: Missing Keystore location at edw_demo.snowflake_createtable_0_1.Snowflake_CreateTable.tDBConnection_2Process(Snowflake_CreateTable.java:619) [classes/:?] at edw_demo.snowflake_createtable_0_1.Snowflake_CreateTable.runJobInTOS(Snowflake_CreateTable.java:3881) [classes/:?] at edw_demo.snowflake_createtable_0_1.Snowflake_CreateTable.main(Snowflake_CreateTable.java:3651) [classes/:?]
The Keystore path is not configured correctly at the Job or Studio level before connecting to Snowflake DB on the metadata and using the same metadata connection in the Jobs.
To use key pair authentication for Snowflake, the Keystore settings must be configured in Talend Studio before connecting to Snowflake.
Perform one of the following options.
Update the appropriate Studio initialization file (Talend-Studio-win-x86_64.ini,Talend-Studio-linux-gtk-x86_64.ini,or Talend-Studio-macosx-cocoa.ini depending on your operating system), with the following settings:
-Djavax.net.ssl.keyStore={yourPathToKeyStore} -Djavax.net.ssl.keyStoreType={PKCS12}/{JKS} -Djavax.net.ssl.keyStorePassword={keyStorePassword}
Update the Keystore configuration in Studio SSL preferences with the required Path, Password, and Keystore Type.
Add the Key Alias to the Snowflake metadata.
Key Alias will be created with your keystore .p12 file
Update the tSetKeystore components in your Job, if you plan to run the Job when the target execution is local, Remote Engine, or JobServer (the versions do not matter). Before selecting the Key Pair option for the tSnowflakeConnection component, configure the key pair authentication on the Basic settings tab of the tSetKeystore component:
Select JKS from the TrustStore type pull-down list.
Enter " " in the TrustStore file field.
Clear the TrustStore password field.
Select the Need Client authentication check box.
Enter the path to the Keystore file in double quotation marks in the KeyStore file field.
Enter the Keystore password in the KeyStore password field.
If you are looking for how to set up key-pair authentication in Snowflake using Qlik Talend Studio step by step, please refer to this article:
Qlik-Talend-Product-How-to-set-up-Key-Pair-Authentication-in
The tFtpGet component is unable to filter and download the necessary file according to the designated filemask. In the FTP UI tool, the file is listed and displayed with uppercase characters. However, when using the same uppercase-based filemask to filter the file, no file is downloaded.
The file was saved in lowercase, but it appears in uppercase when viewed through the FTP client UI. This inconsistency between the saved filename and its displayed version in the UI indicates a case-mismatch issue in the FTP system.
To resolve this issue, adjust the character case in the Filemask settings. Alternatively, enable the "Use Perl5 Regex Expression as Filemask" option to create a new filter that disregards case sensitivity. (This can be achieved by using "/i" in the Perl5 Regex Expression, which enables case-insensitive pattern matching, allowing "A" to match "a".)
Question
Is it possible to use the feature "use or register a shared DB Connection" for a subjob or child job with tSnowflakeConnection like we can with other database connections?
Answer
This a new feature for tSnowflakeConnection Component. So far, it is not possible to share a SnowflakeConnection with a subjob/joblet in Talend Studio 8.0 R2025-03 or prior. This new feature will be available in Talend Studio 8.0 R2025-04 and above.
AJira request (Internal) has been raised for this feature: QTDI-1376
After upgrading to v8-R2025-01/02, a Talend Job containing both tRest and tRestClient components encounters an error as shown below:
- javax.ws.rs.client cannot be resolved to a type
- The type javax.ws.rs.client.ClientBuilder cannot be resolved. It is indirectly referenced from required type org.glassfish.jersey.client.JerseyClientBuilder
- javax.ws.rs.client.WebTarget cannot be resolved to a type
- javax.ws.rs.core.Response cannot be resolved to a type
- javax.ws.rs.client.Invocation cannot be resolved to a type
- javax.ws.rs.WebApplicationException cannot be resolved to a type
When tRest and tRestClient components are used together within a Job, a conflict arises in the implementation of the Rest API.
To resolve this issue, a temporary workaround as shown below is to use tLibraryLoad to load external jar files in the beginning of Job:
tPrejob --onComponentOK-> tLibrary (javax.ws.rs-api-2.1.jar) --onComponentOK-> tLibrary (javax.annotation-api-1.3.2.jar)
Note: If the aforementioned workaround proves effective, kindly remember to remove the tLibraryLoad components after installing the Studio v8-R2025-03 patch.
Alternatively, apply the Studio v8-R2025-03 patch.
QTDI-1208
QTDI-1300
A talend Job is experiencing a tGoogleDriveGet permission issue during credential file initialization in Talend Administration Center Application. It works well in side of Talend Studio.
Here is the error log below:
[ERROR] 15:55:50 org.talend.components.google.drive.runtime.GoogleDriveGetRuntime- D:\GDCredential\InterflourGDrive
[FATAL] 15:55:50 id_distributor_sellout.copy_of_getfile_v3__manohara_0_3.Copy_of_GetFile_V3__Manohara- tGoogleDriveGet_6 UNEXPECTED_EXCEPTION
org.talend.components.api.exception.ComponentException: UNEXPECTED_EXCEPTION
at org.talend.components.google.drive.runtime.GoogleDriveGetRuntime.getFile(GoogleDriveGetRuntime.java:87) ~[components-googledrive-runtime-0.37.42.jar:?]
at org.talend.components.google.drive.runtime.GoogleDriveGetRuntime.runAtDriver(GoogleDriveGetRuntime.java:60) ~[components-googledrive-runtime-0.37.42.jar:?]
at id_distributor_sellout.copy_of_getfile_v3__manohara_0_3.Copy_of_GetFile_V3__Manohara.tGoogleDriveGet_6Process(Copy_of_GetFile_V3__Manohara.java:673) [copy_of_getfile_v3__manohara_0_3.jar:?]
at id_distributor_sellout.copy_of_getfile_v3__manohara_0_3.Copy_of_GetFile_V3__Manohara$5.run(Copy_of_GetFile_V3__Manohara.java:2429) [copy_of_getfile_v3__manohara_0_3.jar:?]
Caused by: java.nio.file.AccessDeniedException: D:\Simon_Folder\GDCredential\InterflourGDrive
at sun.nio.fs.WindowsAclFileAttributeView.getFileSecurity(WindowsAclFileAttributeView.java:89) ~[?:?]
at sun.nio.fs.WindowsAclFileAttributeView.getOwner(WindowsAclFileAttributeView.java:122) ~[?:?]
at sun.nio.fs.FileOwnerAttributeViewImpl.getOwner(FileOwnerAttributeViewImpl.java:91) ~[?:?]
at com.google.api.client.util.store.FileDataStoreFactory.setPermissionsToOwnerOnlyWindows(FileDataStoreFactory.java:160) ~[google-http-client-1.38.0.jar:1.38.0]
at com.google.api.client.util.store.FileDataStoreFactory.<init>(FileDataStoreFactory.java:80) ~[google-http-client-1.38.0.jar:1.38.0]
at org.talend.components.google.drive.runtime.client.GoogleDriveCredentialWithInstalledApplication$Builder.<init>(GoogleDriveCredentialWithInstalledApplication.java:91) ~[components-googledrive-runtime-0.37.42.jar:?]
at org.talend.components.google.drive.runtime.client.GoogleDriveCredentialWithInstalledApplication$BuilderWithIdAndSecret.<init>(GoogleDriveCredentialWithInstalledApplication.java:128) ~[components-googledrive-runtime-0.37.42.jar:?]
at org.talend.components.google.drive.runtime.client.GoogleDriveCredentialWithInstalledApplication.builderWithIdAndSecret(GoogleDriveCredentialWithInstalledApplication.java:42) ~[components-googledrive-runtime-0.37.42.jar:?]
at org.talend.components.google.drive.runtime.GoogleDriveRuntime.getCredential(GoogleDriveRuntime.java:164) ~[components-googledrive-runtime-0.37.42.jar:?]
at org.talend.components.google.drive.runtime.GoogleDriveRuntime.getDriveService(GoogleDriveRuntime.java:178) ~[components-googledrive-runtime-0.37.42.jar:?]
at org.talend.components.google.drive.runtime.GoogleDriveRuntime.getDriveUtils(GoogleDriveRuntime.java:186) ~[components-googledrive-runtime-0.37.42.jar:?]
at org.talend.components.google.drive.runtime.GoogleDriveGetRuntime.getFile(GoogleDriveGetRuntime.java:66) ~[components-googledrive-runtime-0.37.42.jar:?]
... 3 more
Ensure that the credential file owner matches the user who are running Jobserver to prevent permission access exceptions.
Please use the following command to set owner in Windows
icacls D:\GDCredential /setowner "SYSTEM" /t /l
The Error logs indicates that when tGoogleDriveGet component initializes the credential file, the new API design will verify the owner and modify ACL permissions. Since the credential file was configured using a non-Jobserver-running user, a permission exception occurs.
com.google.api.client.util.store.FileDataStoreFactory.setPermissionsToOwnerOnlyWindows
FileOwnerAttributeViewImpl.getOwner
A Talend Job includes a tRest component that attempts to post 100kb of JSON/XML data to an API, but fails with the following error:
tREST component fails with the error "400 , {
"failures": [
{
"field": "name",
"constraint": "Must be a string value, at least 1 character and at most 100 characters long."
},
{
"field": "fields",
"constraint": "Is required."
}
]
}"
When a Job includes both tRest and tRestClient components, tRest will use CXF implement auto to make HTTP requests. This will cause some random issues, as the CXF framework forces the use of the transfer-encoding header when the body size exceeds 4096. This will result in the post data being chunked (Transfer-Encoding: chunked), and then the server side cannot interpret the data fully as a whole. This was a previously identified defect.
To resolve this issue, a temporary workaround is to select the "Disable chunked encoding" checkbox on tRestClient.
Alternatively, apply the Studio v8-R2025-03 patch.
QTDI-1208