Unlock a world of possibilities! Login now and discover the exclusive benefits awaiting you.
Hi all,
we are experiencing an issue with a Qlik Replicate Full Load from MariaDB to SQL Server.
The source table is very large, containing approximately 7.6 billion rows.
The load fails while Replicate tries to create the primary key/unique index on the target table:
Stream component 'st_1_TargetEndpoint' terminated
Stream component failed at subtask 1, component st_1_TargetEndpoint
Error executing data handler
Endpoint is disconnected
Execute create Primary Key/Unique Index failed, statement:
CREATE UNIQUE INDEX [LARGE_DATA_TABLE_PK]
ON [TARGET_SCHEMA].[LARGE_DATA_TABLE]
(
[KEY_COLUMN_1],
[TIMESTAMP_COLUMN],
[SEQUENCE_COLUMN]
)
RetCode: SQL_ERROR
SqlState: HYT00
NativeError: 0
Message: [Microsoft][ODBC Driver 18 for SQL Server]
Query timeout expired
Failed (retcode -1) to execute statement:
CREATE UNIQUE INDEX [LARGE_DATA_TABLE_PK]
ON [TARGET_SCHEMA].[LARGE_DATA_TABLE]
(
[KEY_COLUMN_1],
[TIMESTAMP_COLUMN],
[SEQUENCE_COLUMN]
)The Full Load is segmented using KEY_CULUMN_1. We also checked the source-side performance and investigated the SQL Server load and connection behavior.
The actual failure seems to occur when Replicate creates the unique index after loading the table.
We are currently looking into the timeout settings in Replicate/ODBC.
Has anyone experienced this behavior with very large tables before?
In particular:
Any hints or recommended settings would be appreciated.
Thanks in advance!
Are there any videos, demos, or demonstrated use case for using Talend Data Catalog and showing its use and capabilities instead of the learning paths on learning portal?
Hello everyone,
I’ve been thinking about the possibility of converting some of the scripts I use in various apps into semantic models. It is possible to convert a data-loading script that results in a data model into a semantic model?
I do'nt know if it is possible (yet?), at least I haven't found it...but it would be amazing.
Thanks in advance
Hi,
Apologies for the slightly vague question but is there anything changed in the last few weeks that will have broken some old visualisation extensions - specifically in cloud - on prem and even technical preview QS desktop seem fine?
more detail:
probably 7 or 8 years ago I hacked together my first extension following the very few tutorials available and it worked reasonably well - i'm no web developer but got it working and seemed performative if almost certainly no where near any best practice standards or anything. Has had a few tweaks to the internal logic over the years but in terms of structure its not really changed, to my great surprise it even worked when copied to the cloud. As far as i'm aware they were still working a few weeks ago but as of yesterday (possibly earlier) the don't render, nothing is jumping out to me in the console log other than this (there are a bunch of launchdarkly errors but they are there extension or no and more of a whitelisting thing from what i read)
which doesnt appear in the QS desktop (Nov 26 technical preview) dev tools so am wondering if its being enforced maybe?
If anyone has any ideas or if i've missed something, would greatly appreciate it
This Techspert Talks session covers:
Chapters:
Resources:
Q&A:
Q: Is the best practice to install QTCMT on the same TAC server or on the dedicated server?
A: The best is a dedicated server (or VM) to use the migration features. QTCMT is a standalone application with its own bundled Java runtime. It needs to reach TAC remotely: you register the TAC source server and its database in QTCMT. Its migration pipelines run heavy Maven builds; the memory used by the build is configurable, with several GB as the default. On the TAC server they would compete with TAC and job execution. QTCMT only needs network access to TAC and its database, to Git, and to Nexus or Talend Cloud.
Q: I currently use Talend Studio 8 running on-premises. Is there any tool that can migrate Talend jobs (with context variables and Excel references) to tasks on Talend Cloud?
A: Yes, the Qlik Talend Cloud Migration Toolkit (QTCMT). There are two cases:
Jobs scheduled in TAC: the TAC to TMC migration moves your TAC tasks, with their context parameters, to Talend Management Console tasks.
Studio only, no TAC: QTCMT takes the project from Git, builds it and publishes the artifacts to Talend Cloud, where tasks are then created.
Two points to plan for:
Context values and passwords: passwords encrypted by Studio don't carry over, because they're tied to the source install. In Talend Cloud, values are stored as connections or resources and injected at run time on the Remote Engine. They're re-entered once (QTCMT and our tooling help inventory them), and after that rotation needs no rebuild.
Excel files: the job must find the file where the Remote Engine runs. Make the file path a context variable, and put the files on storage the engine can reach (a shared folder, S3…), set per environment in Talend Cloud.
Q: If you have a client with Talend Open Studio, can you still use the QTC Migration Toolkit to migrate them to QTC when they do not have TAC? Also, can it migrate from Talend OS?
A: Yes. QTCMT does not need TAC. Without a TAC server, it works with external projects, taken from Git or a local folder.
The flow:
Put the Open Studio project in Git. Open Studio has no built-in Git integration, so it's a simple export and push.
QTCMT migrates and builds it for Studio 8 / Java 17. The amount of work depends on the source version: an older version gets the automatic fixes; a recent one mostly just builds.
QTCMT publishes to Qlik Talend Cloud.
QTCMT's working environment needs a license, and a Qlik Talend Cloud license is accepted, which the client will have when moving to QTC.
Two points to plan for:
Scheduling: Without TAC there are no tasks to migrate. Schedules that ran from cron or exported scripts are recreated as tasks and plans in Talend Cloud.
Passwords and custom components: same password point as above. In a real Open Studio case we analyzed, about 20,000 encrypted values came down to roughly 10–20 distinct secrets, which became Talend Cloud context parameters. Any custom or community components need to be checked.
On last thing, a migration token is needed to be able to convert your open studio project into the commercial version of Talend Studio, QTCMT currently doesn’t support it out of box, but we have workaround in place until it's supported on our December release.
Onecatch with open Studio, usually custom Java routines are heavy uses because of the lack of native components. We are working on a feature that would translate java routines into native Studio components for 2027.
Click here to see the video transcript
Hello team,
I am writing to inquire about the limitation on using Filters in bidirectional replication tasks.
According to the user guide below, it states that Filters should not be used in bidirectional replication:
https://help.qlik.com/en-US/replicate/May2026/Content/Global_Common/Content/SharedEMReplicate/Design%20Tasks/Limitations_bidirectional_tasks.htm#ar_tasks_1150461142_1273508
Our customer has specific records within the target tables that they do not want to synchronize (for example, keeping records with IDs 1 to 10 isolated within their respective databases only).
Therefore, they would like to apply a Filter.
Theoretically, it seems possible to achieve this by configuring symmetrical filters on both tasks in the bidirectional setup.
Could you please clarify the specific technical reason why using Filters is restricted/not recommended in bidirectional replication?
Additionally, if using Filters is strictly unsupported or carries technical risks (such as infinite looping or data divergence), could you advise on any recommended workarounds to achieve this requirement?
Best Regards.
Dear Qlik Replicate Team,
We would like to request technical clarification regarding an Oracle CDC limitation in Qlik Replicate when replicating UPDATE operations on tables containing column names longer than 30 bytes.
1. Environment
Source Database: Oracle Database 19c
Qlik Replicate Version: 2025.11 SP06
CDC Method: Oracle Log Reader (Direct Redo Log Access)
Task Architecture: LogStream Staging Task → Replication Task
Oracle Parameter: ENABLE_GOLDENGATE_REPLICATION=FALSE
Supplemental Logging: ALL COLUMN
Table Characteristics: Contains one or more column names exceeding 30 bytes
2. Observed Behavior
When an UPDATE transaction is executed against the affected table, we observe the following behavior:
ALL COLUMN Supplemental Logging is enabled on the Oracle source database.
However, the Primary Key values are missing from the UPDATE event in the SOURCE_CAPTURE TRACE of the LogStream Staging Task.
As a result, the downstream Replication Task cannot properly construct the WHERE clause for the UPDATE statement in the TARGET_APPLY stage.
Consequently, the UPDATE operation cannot be applied successfully to the target database.
We have reproduced this issue with Qlik Replicate 2025.11 SP06.
3. Qlik Documentation
According to the Qlik Replicate Oracle Source Limitations documentation, Full Supplemental Logging is not available for tables containing column names longer than 30 characters when ENABLE_GOLDENGATE_REPLICATION=FALSE.
We also reviewed the Qlik Replicate May 2026 documentation and found that the same limitation still appears to be documented.
Therefore, it seems that this limitation has not been addressed in the currently documented product versions.
4. Comparison with Other CDC Solutions
We understand that other Oracle CDC products, such as Quest SharePlex and XLog, can support UPDATE replication for tables containing long column names even when ENABLE_GOLDENGATE_REPLICATION=FALSE.
Both products use direct Oracle Redo Log reading rather than Oracle LogMiner.
In particular, the Quest SharePlex Administration Guide documents a related Oracle key logging issue involving long identifiers and describes the SP_OCT_USE_SUPP_KEYS=0 setting as a workaround.
This suggests that different CDC implementations may use alternative approaches to selecting or processing key information for UPDATE operations.
5. Technical Questions
We would appreciate clarification on the following points.
Q1. Root Cause of the Limitation
Is the missing Primary Key information caused by Oracle not recording the required key values in the Redo Log, or by a limitation in Qlik Replicate's Oracle Log Reader when extracting or interpreting the Redo records?
Q2. Difference from Other CDC Products
Since other direct Redo Log Reader-based CDC solutions appear to support this scenario, could you explain the specific technical limitation that prevents Qlik Replicate from supporting it?
Q3. Alternative Key Handling Mechanism
Does Qlik Replicate provide an equivalent mechanism to SharePlex's SP_OCT_USE_SUPP_KEYS=0, allowing UPDATE row identification without relying exclusively on Oracle's supplemental key logging?
Q4. Workaround or Internal Parameter
Is there any supported configuration, Internal Parameter, Hotfix, or alternative capture mechanism that would allow UPDATE and DELETE replication for tables containing column names longer than 30 bytes while keeping ENABLE_GOLDENGATE_REPLICATION=FALSE?
Q5. Future Product Support
Is there any plan to address this limitation in Qlik Replicate 2026.5 SR1 or a subsequent release?
If an existing defect, enhancement request, or support case is associated with this limitation, could you provide the relevant reference number?
6. Request
Due to operational policies and potential Oracle licensing implications, enabling ENABLE_GOLDENGATE_REPLICATION=TRUE may not be feasible in our environment.
We would therefore like to understand whether Qlik Replicate can support this CDC scenario without changing the Oracle parameter.
If no supported workaround currently exists, we would appreciate an official explanation of the technical limitation and any potential plans for future enhancement.
Thank you for your assistance.
Best regards.
I'm trying to import apps via QMC and receiving the error message "The REST API call timed out between services."
It doesn't have to do with the app size. It occurs with different size apps.
Does anyone know how to solve this situation?
Hi everyone,
We are experiencing an OAuth 2.0 authentication issue when trying to connect Qlik Cloud Analytics to Salesforce using the Salesforce connector from Add Data.
A new Salesforce External Client App was created following Salesforce's recent security changes.
According to Salesforce's updated security requirements, PKCE is mandatory for newly created applications, and the option to disable it is locked.
However, the Qlik documentation instructs us to disable PKCE as part of the connection setup.
We attempted to configure the Salesforce Callback URL using both:
https://connector.qlik.com/auth/oauth/v2.htm
https://connector.qlik.com/auth/oauth/v3.htm
Both attempts failed during authentication, resulting in the following errors:
redirect_uri_mismatch
OAUTH_APPROVAL_ERROR_GENERIC
We would appreciate hearing from anyone who has encountered a similar issue or has successfully established a Salesforce OAuth 2.0 connection in Qlik Cloud Analytics under the new PKCE requirements.
Thanks in advance for sharing your experience!
Qlik is a Leader in the 2026 Gartner® Magic Quadrant™ for Data Integration Tools
See how agentic AI is changing business intelligence and analytics teams, and why trusted data and shared context matter most.
Here’s your agentic AI readiness check. (Got Qlik? You're closer than you think.)
Built for the builders: Modern data engineering is AI powered — and engineer led.
The pilot-to-production gap is real. From fragmented data to governance gaps Qlik and AWS break down what’s holding your enterprise AI back — and how to solve them.
Your journey awaits! Join us by Logging in and let the adventure begin.