Unlock a world of possibilities! Login now and discover the exclusive benefits awaiting you.
Search our knowledge base, curated by global Support, for answers ranging from account questions to troubleshooting error messages.
The following error message appears repeatedly in the logs.
2024-01-20 08:50:10 ERROR RepoProjectRefresher -
2024-01-10 16:58:47 ERROR GC - D:\Talend\7.3.1\tac\apache-tomcat\temp\_git\
The main reason for the log type is that the RepoProjectRefresher faces out of memory issues. By default, TAC automatically caches/checks out the project source code into the tac\apache-tomcat\temp folder. When the source code accumulation is large, the RepoProjectRefresher module will consume a lot of memory. When maximum value is reached, a GC error will occur and a GC error log will be generated.
Note: The RepoRefresher cache functionality has been deprecated in the latest version of TAC v8.0.1
Disable the Git caching mechanism for the TAC project by following these steps:
update t731.configuration set value='true' where configuration.key='git.whiteListBranches.enable';
update t731.configuration set value='\"Technical labels of project\",\"Active branch name\"' where configuration.key='git.whiteListBranches.list';
Our Customer has over existing 200 big data spark jobs migrated from 7.3.1 to 8.0.1 and it gives "IndexOutOfBoundsException" after "use Dataset API in migrated components" checkbox is enabled in Spark Configuration.
Error Log
!ENTRY org.eclipse.ui.workbench 4 2 2024-03-28 18:01:50.038 !MESSAGE Problems occurred when invoking code from plug-in: "org.eclipse.ui.workbench". !STACK 0 java.lang.IndexOutOfBoundsException: Index 0 out of bounds for length 0 at java.base/jdk.internal.util.Preconditions.outOfBounds(Preconditions.java:64) at java.base/jdk.internal.util.Preconditions.outOfBoundsCheckIndex(Preconditions.java:70) at java.base/jdk.internal.util.Preconditions.checkIndex(Preconditions.java:248) at java.base/java.util.Objects.checkIndex(Objects.java:372) at java.base/java.util.ArrayList.get(ArrayList.java:459) at org.talend.designer.bigdata.avro.AvroRecordStructGenerator.firstSchemaFieldHasAnAsterisk(AvroRecordStructGenerator.java:256) at org.talend.designer.bigdata.avro.AvroRecordStructGenerator.generateRecordStructForDataset(AvroRecordStructGenerator.java:242) at org.talend.repository.spark.editor.SparkDataProcess.generateRecordStructForConnection(SparkDataProcess.java:207) at org.talend.designer.bigdata.common.BigDataDataProcess.buildRecordStructsForConnections(BigDataDataProcess.java:612) at org.talend.designer.bigdata.common.BigDataDataProcess.buildRecordStructsForConnections(BigDataDataProcess.java:588) at org.talend.repository.spark.editor.SparkDataProcess.buildFromGraphicalProcess(SparkDataProcess.java:156) at org.talend.designer.bigdata.common.BigDataProcess.getGeneratingNodes(BigDataProcess.java:93) at org.talend.designer.core.ui.editor.process.Process.getNodesOfType(Process.java:3949) at org.talend.designer.core.ui.editor.nodes.Node.checkParameters(Node.java:3029) at org.talend.designer.core.ui.editor.nodes.Node.checkNode(Node.java:4406) at org.talend.designer.core.ui.editor.process.Process.checkProblems(Process.java:3801) at org.talend.designer.core.ui.editor.process.Process.checkProcess(Process.java:3792) at org.talend.designer.core.ui.views.problems.Problems.initCurrentProblems(Problems.java:265) at org.talend.designer.core.ui.views.problems.Problems.addProcess(Problems.java:257) at org.talend.designer.core.ui.ActiveProcessTracker.addJobInProblemView(ActiveProcessTracker.java:169) at org.talend.designer.core.ui.ActiveProcessTracker.partOpened(ActiveProcessTracker.java:294)
After checked with R&D and made some investigation and analysis on it, this is a Plugin code generation logic bug from: plugins/org.talend.designer.bigdata/src/org/talend/designer/bigdata/avro/AvroRecordStructGenerator.java
Fixed patch: V8.0.1-R2024-04 patch
Internal defect ID, TBD-16238
Changing the Admin User in Talend Academy requires following specific steps and ensuring accurate details are provided.
For Customer Domain Administrators needing assistance, please refer to the templates provided below.
If encountering difficulties, use the support template to contact us for assistance.
Subject: Request to Change Admin User in Talend Academy
Dear Customer Domain Administrator,
I hope this message finds you well.
I am writing to request a change in the Admin User for our organization's Talend Academy account. Below are the necessary details:
Note: Please refer to the document attached to check if you have the authority to make the requested changes. If you are unable to proceed, kindly contact Talend Support for further assistance.
Your prompt assistance in changing the Admin User would be greatly appreciated.
Thank you for your support and consideration.
Best regards,
[Your Name]
Subject: Request to Change Admin User in Talend Academy
Dear Academy Team,
I hope this message finds you well.
I am writing to request a change in the Admin User for our organization's Talend Academy account. Below are the necessary details:
Your prompt assistance in changing the Admin User would be greatly appreciated.
Thank you for your support and consideration.
Best regards,
[Your Name]
If you need to resend a license email along with license installer links and the license key to a customer/user, follow the steps outlined below.
This guide will help you understand the process and the necessary details you need to provide.
Gather Necessary Information:
Send an Email Request:
Use the following template to structure your email request:
Subject: Resend License Email Request
Dear Support Team,
I hope this email finds you well.
I am writing to request that a license email be resent. Below are the details:
I would greatly appreciate your assistance in resending the license email. Thank you for your support and consideration.
Best regards,
[Your Name]
Subject: Resend License Email Request for Talend Data Integration
Dear Support Team,
I hope this email finds you well.
I am writing to request that the license email for Talend Data Integration be resent. Below are the details:
I would greatly appreciate your assistance in resending the license email. Thank you for your support and consideration.
Best regards,
Sam
Following these steps will help you effectively resend a license email to your customer.
If you have any further questions, feel free to reach out to the respective teams.
When launching Talend Studio the following message appears
A Java Runtime Environment (JRE) or Java Development Kit (JDK) must be available in order to run Talend_Sign_On_Tool_win-x86_64. No Java virtual machine
was found after searching the following locations:
<Studio HOME>\configuration\studio_SS o_client\jre\bin\javaw.exe
javaw.exe in your current PATH
JAVA_HOME or Java dll path may be incorrectly set in environmental variables or ini file.
In the realm of software integration and data management, ensuring compatibility and functionality across different versions is crucial.
Talend, a leading provider of data integration and integrity solutions, offers a straightforward process for users to test migration from existing active licenses to newer versions.
This guide outlines how to request and effectively utilize a Talend Test Migration License, whether for cloud or on-premises deployments.
A Talend Test Migration License allows users to transition from their current active license to a newer version temporarily for testing purposes.
This enables organizations to assess the new features, functionalities, and compatibility with their existing systems before committing to a full migration.
Before requesting a Talend test migration license, ensure you have the following details ready:
Send your request to customercare@qlik.com with the following email template.
Subject: Request for Talend Test Migration License
Dear Support Team,
I hope this message finds you well.
I am writing to request a Talend test migration license for our cloud deployment. Below are the required details:
Your prompt assistance in providing this test migration license would be greatly appreciated. Thank you for your support and consideration.
Best regards, [Your Name]
Subject: Request for Talend Test Migration License (Cloud)
Dear Support Team,
I hope this message finds you well.
I am writing to request a Talend test migration license for our cloud deployment. Below are the required details:
Your prompt assistance in providing this test migration license would be greatly appreciated. Thank you for your support and consideration.
Best regards,
John Doe
For On-Premises Deployments:
Send your request to customercare@qlik.com with the following email template.
Subject: Request for Talend Test Migration License
Dear Support Team,
I hope this message finds you well.
I am writing to request a Talend test migration license for our on-premises deployment. Below are the required details:
Your prompt assistance in providing this test migration license would be greatly appreciated. Thank you for your support and consideration.
Best regards, [Your Name]
Subject: Request for Talend Test Migration License (On-Premises)
Dear Support Team,
I hope this message finds you well.
I am writing to request a Talend test migration license for our on-premises deployment. Below are the required details:
Your prompt assistance in providing this test migration license would be greatly appreciated. Thank you for your support and consideration.
Best regards, John Doe
Effectively utilizing a Talend test migration license is essential for organizations looking to evaluate new Talend software versions without disrupting existing operations.
By following the outlined steps and using the provided templates, users can streamline the process of requesting and deploying a test migration license tailored to their specific deployment needs.
If you are looking to extend a Test Migration License, see How to extend Talend Test Migration License
Assigning a Talend course to a user involves specific steps to ensure the process is carried out accurately. For Customer Domain Administrators needing assistance, please refer to the templates provided below. If you encounter any challenges, utilize the support template to contact us for further assistance.
Subject: Request to Assign Talend Course to User
Dear Customer Domain Administrator,
I hope this message finds you well.
I am writing to request the assignment of a Talend course to a user within our organization. Below are the necessary details:
Note: Please refer to the document attached to verify if you have the authority to assign courses to users. If you encounter any difficulties, please contact Talend Support for further assistance.
Your prompt attention to this request would be greatly appreciated. Thank you for your support and consideration.
Best regards,
[Your Name]
Subject: Request to Assign Talend Course to User
Dear Academy Team,
I hope this message finds you well.
I am writing to request the assignment of a Talend course to a user within our organization. Below are the necessary details:
Your prompt assistance in assigning the Talend course to the specified user would be greatly appreciated. Thank you for your support and consideration.
Best regards,
[Your Name]
Requesting a Talend migration token involves several specific steps to ensure a smooth transition from one version of Talend Studio to another.
If you encounter any challenges, utilize the support template to contact us for further assistance.
Subject: Request for Talend Migration Token
Dear Support Team,
I hope this message finds you well.
I am writing to request a migration token for our organization's Talend Studio. Below are the necessary details:
Your prompt attention to this request would be greatly appreciated.
Thank you for your support and consideration.
Best regards,
[Your Name]
Subject: Request for Talend Migration Token
Dear Support Team,
I hope this message finds you well.
I am writing to request a migration token for our organization's Talend Studio. Below are the necessary details:
Your prompt assistance in providing the migration token would be greatly appreciated.
Thank you for your support and consideration.
Best regards,
John Doe
Why tLogcatcher / tStatCatcher components are not available for big data Spark batch?
tLogcatcher / tStatCatcher components are not available that is because :
Elasticsearch logs are generated in the Logserver/elasticsearch-1.5.2/log directory. If the log files are not moved or deleted, disk space can fill up. Can Elasticsearch log files be purged automatically?
Update the Elasticsearch log configuration, and use the MaxBackupIndex option to determine how many backup files are kept before the oldest is erased.
The Elasticsearch logs configuration can be found in Logserver/elasticsearch-1.5.2/config/logging.yml.
The default configuration is:
"
file:
type: dailyRollingFile
file: ${path.logs}/${cluster.name}.log
datePattern: "'.'yyyy-MM-dd"
layout:
type: pattern
conversionPattern: "[%d{ISO8601}][%-5p][%-25c] %m%n"
"
With this configuration a file will be created every day in order to save the previous day's log. As a result, the number of log files will increase, and it can lead to the disk full problem.
To restrict the amount of backup files, set MaxBackupIndex:
Stop these Talend services:
Talend Log Server search engine
Talend Log Server analytics and visualization platform
"
file:
type: rollingFile
file: ${path.logs}/${cluster.name}.log
maxFileSize : 200KB
maxBackupIndex: 4
layout:
type: pattern
conversionPattern: "[%d{ISO8601}][%-5p][%-25c] %m%n"
"
Qlik Talend Product: Reducing the logging threshold for Elasticsearch in Talend Log Server
A Job fails to run on a remote JobServer with the following error message:
Unrecognized option: --add-opens=java.base/java.lang=ALL-UNNAMED
Error: Could not create the Java Virtual Machine.
Error: A fatal exception has occurred. Program will exit.
The issue occurs with Talend Remote Engine or Talend JobServer using Java 8, but Talend Studio has JDK 17 enabled. The --add-opens option is only supported in java 11 and above, Java 8 does not support it.
If the 'Enable Java 17 compatibility' option is enabled, the Jobs built by Talend Studio will have the --add-opens parameters in the job.sh or job.bat script files, such as:
JAVA_OPTS=-Dfile.encoding=UTF-8 -Dsun.stdout.encoding=UTF-8 -Dsun.stderr.encoding=UTF-8 -Dconsole.encoding=UTF-8 --add-opens=java.base/java.lang=ALL-UNNAMED
The Jobs built this way cannot be executed with Java 8.
Option 1:
Option 2:
From R2024-05, Java 17 will become the only supported version to start most Talend modules, If you want to keep the "Enable JAVA 17 Compatibility" option enabled, make sure that Talend Remote Engine or Talend JobServer also uses JDK 17 to execute jobs.
Setting-compiler-compliance-level
Opening Talend Studio when using the remote connection setup
Repository = AWS - USA East
Token=XXXXXX
Web-app URL = https://tmc.us.cloud.talend.com/studio_cloud_connection
Whether it is clicking on the 'Check url' or clicking OK, an error always appears in the Talend Studio log, the initial lines is:
javax.net.ssl.SSLException: java.lang.RuntimeException: Unexpected error: java.security.InvalidAlgorithmParameterException: the trustAnchors parameter must be non-empty
The exception indicates that it is a SSL handshake error. The previous JDK keystore contains wrong certificate of the Talend Cloud or the JDK path was wrongly set.
Reinstall Java and Talend Studio.
The IBM DB2 for iSeries source endpoint occasionally encounters an error during the CDC stage. This issue appears to be linked to the presence of the IBM i Access ODBC Driver versions 7.1.26 and 7.1.27.
The error message in task log file:
[SOURCE_CAPTURE ]E: Error parsing [1020109] (db2i_endpoint_capture.c:652)
The issue specifically arises during the CDC stage; however, the Full Load stage operates smoothly without any complications.
As a workaround please downgrade IBM i Access ODBC client from versions '07.01.027'/'07.01.026' to '07.01.025'
The most recent version of IBM i Access ODBC Client is '07.01.027' as of today. For compatibility reasons, it's advisable to revert to version '07.01.025', as '07.01.026' exhibits the same issue.
Various factors can contribute to encountering the 'Error parsing' message, including:
• DB2i ODBC Version '07.01.027' (as described in this article)
• In a single task, the total number of captured tables exceeds 300
• The source table is created by DDS
• Garbage data in table
• Special characters in table object identifier (table name, or column name)
If you continue to encounter the error after switching to '07.01.025', please reach out to Qlik Support for further assistance.
The behavior of the IBM DB2i ODBC Versions '07.01.026' & '07.01.027' differ slightly from that of '07.01.025'. In certain scenarios, it may return incorrect column lengths
#00158029, #00160002, QB-26413
Azure DevOps pipelines fails with the following error when building Talend Jobs.
[ERROR] Failed to execute goal on project job_AladdinToBusiness_000020_profitAndLossFileChild: Could not resolve dependencies for project org.example.tcorp_talend_ims.job:job_AladdinToBusiness_000020_profitAndLossFileChild:jar:0.2.0: Failed to collect dependencies at org.talend.components.lib:talend-proxy:jar:1.0.0:
Maven blocks external HTTP repositories by default since version 3.8.1, for more details, please read Release Notes – Maven 3.8.1.
Open the Maven settings file settings.xml located in ${maven.home}/conf/ or ${user.home}/.m2/, add a dummy mirror with the id maven-default-http-blocker to override the existing one. This disables HTTP blocking for all repositories.
<mirrors>
<mirror>
<id>maven-default-http-blocker</id>
<mirrorOf>dummy</mirrorOf>
<name>Dummy mirror to override default blocking mirror that blocks http</name>
<url>http://0.0.0.0/</url>
</mirror>
</mirrors>
</settings>
Disable maven blocking external HTTP repositories
Upgrade installation or fresh installation of Qlik Replicate 2023.11 (includes builds GA, PR01 & PR02), Qlik Replicate reports errors for MySQL or MariaDB source endpoints. The task attempts over and over for the source capture process but fail, Resume and Startup from timestamp leads to the same results:
[SOURCE_CAPTURE ]T: Read next binary log event failed; mariadb_rpl_fetch error 0 () [1020403] (mysql_endpoint_capture.c:1060)
[SOURCE_CAPTURE ]T: Error reading binary log. [1020414] (mysql_endpoint_capture.c:3998)
Upgrade to Replicate 2023.11 PR03 (expires 8/31/2024).
The fix is included in Replicate 2024.05 GA.
If you are running 2022.11, then keep run it.
No workaround for 2023.11 (GA, or PR01/PR02) .
Jira: RECOB-8090 , Description: MySQL source fails after upgrade from 2022.11 to 2023.11
There is a bug in the MariaDB library version 3.3.5 that we started using in Replicate in 2023.11.
The bug was fixed in the new version of MariaDB library 3.3.8 which be shipped with Qlik Replicate 2023.11 PR03 and upper version(s).
support case #00139940, #00156611
Replicate - MySQL source defect and fix (2022.5 & 2022.11)
The dedicated tFTPxxx components do not support reading files directly from FTP servers. Usually the file is obtained from the FTP server to the local system first and then the file is read. However, this can cause performance issues when many files or large files need to be read.
One solution is to use the JSch library, which allows the creation of an InputStream object from a file via SFTP. Then, use the tFileInputDelimited component to read data from the InputStream object. This avoids downloading the file to the local system.
For example, in the following Job, the Java code iterates over the FTP files one by one, using a JSch library file and a tFileInputDelimited component to read the file directly.
The tLibraryLoad component loads the jsch-0.1.55.jar JSch library file, then the tJava component uses the following code to create the InputStream object for the current FTP file.
JSch jsch = new JSch();
Session session = null;
try {
session = jsch.getSession("username", "localhost", 22);
session.setConfig("StrictHostKeyChecking", "no");
session.setPassword("password");
session.connect();
Channel channel = session.openChannel("sftp");
channel.connect();
ChannelSftp sftpChannel = (ChannelSftp) channel;
java.io.InputStream in=sftpChannel.get( ((String)globalMap.get("tFTPFileList_1_CURRENT_FILE")));
globalMap.put("sftp_inputStream",in);
} catch (JSchException e) {
e.printStackTrace();
} catch (SftpException e) {
e.printStackTrace();
}
The tFileInputDelimited component reads the data from the InputStream object:
When using tFileInputRaw to load file contents as a string, the Job fails with an OutOfMemoryError if the file size exceeds 2GB. This happens even if there is enough memory and the JVM parameter "-Xmx" value is increased to a higher value.
A Java String internally uses a char array (example: char[]) and the indices of an array is an integer. The maximum value of an integer is Integer.MAX_VALUE, which is 2^31 – 1 (or approximately 2 billion). So, you can store a file up to 2 GB in size as a String, and for which you need at least 4 GB memory to store as each char is 2 bytes in Java, plus additional ~4 GB memory for creating the String object, so, in total around 8GB of heap space.
Java String & Array limitations and OutOfMemoryError
Reading ~2 GB file into memory is not a good design. Avoid loading ~2GB large files into strings at once, split the file into smaller files, and then use tFileList to iterate over each file.