Skip to main content
Announcements
UPGRADE ADVISORY for Qlik Replicate 2024.5: Read More
cancel
Showing results for 
Search instead for 
Did you mean: 
iti-attunity-sup
Partner - Creator III
Partner - Creator III

Is it possible to enable Store Changes when using Logstream ?

Hello support team.

[Info]
Source : Oracle(Logstream)
Target : Snowflake

[Question]
Is it possible to enable Store Changes when using Logstream ?(Is there any limitation ?)

When executed task (enable Store Changes and using Logstream), task was crashed.

If there is no limitation, would you modify task won't crash.

Best Regards.

Labels (2)
10 Replies
john_wang
Support
Support

Hello @iti-attunity-sup ,

I don't believe this is an appropriate usage, as it introduces unnecessary overhead to storage and resources without providing any benefit to downstream tasks.

Hope this helps.

John.

Help users find answers! Do not forget to mark a solution that worked for you! If already marked, give it a thumbs up!
aarun_arasu
Support
Support

Hello @iti-attunity-sup ,

Please refer to the below user guide for the limitation 

https://help.qlik.com/en-US/replicate/May2024/Content/Replicate/Main/Log%20Stream%20Staging/ls3.htm

 

Note : The Full Load and Store Changes replication options are not relevant.

Regards

Arun

iti-attunity-sup
Partner - Creator III
Partner - Creator III
Author

Hello Arun.
Thank you for your reply.

This is Log Stream Staging task limitation.
But replication task to transfer the staged data to the target also has same limitation ?

Best regards.

sureshkumar
Support
Support

Hello @iti-attunity-sup 

Replication (Child) Task is like normal task, there is no such limitation.

You can able "Store Changes/Apply Changes" once the Full Load completes.

 

Regards,
Suresh

iti-attunity-sup
Partner - Creator III
Partner - Creator III
Author

Hello Suresh.

What I want to do is following
-task configuration
Logstream task : Oracle → Logsteam
Repcation task : Oracle(logstream) → Snowflake ※Replication task is enable Store Changes

But I recognized first answer is "this task configuration is not appropriate usage".
After all this task configuration is possible? (But when task executed, task crashed)

Best Regards.

SushilKumar
Support
Support

Hello @iti-attunity-sup 

Logstream task and replicate task are there to serve Different purpose.  

Logstream task mainly configure were you want to reduce no of process/query to get same kind of data and put less load on Connected Source Db. So only few processes engage Source db. to read the changes from Db or the logs and put them in a format where Replicate task extract changes and push them to intended targets.

So, in nutshell the Staging folder where logstream put the files to further read by replicate task provide kind of fault tolerant as well. And generated stored files only have the data related to participating tables while if you configure normal task which direct read from db. or redo logs.

when we choose LOGSTREAM task it will scan and push data work in this manner ( extract change and put into files for further read) 

When we choose Normal REPLICATE task then it will scan and push data work in this manner ( extract change , Transform and push data to Target end) 

Change table is created on the target side. hence, it's not possible for logstream to work with store changes

hope it helps.

if you like the explanation then request you to mark is as a solution.

Regards,

Sushil Kumar

SachinB
Support
Support

Hello @iti-attunity-sup ,

Log Stream enables a dedicated Replicate task to save data changes from the transaction log of a single source database and apply them to multiple targets thereby eliminating the overhead of reading the logs for each target separately.

When you create task with Log stream you will store source data in stage and from stage you can consume the data as part of CDC it could be apply or store changes

 

SachinB_0-1719559609211.png

If your task is crashing, I would request you create a case and will help you with screen sharing.

 

Hope this helps you.

 

Regards,

Sachin B

 

 

iti-attunity-sup
Partner - Creator III
Partner - Creator III
Author

Hello Sushil

Thank you for your reply.

To sum up,
there is limitation that Replication task with Logstream cannot use Store Changes ?

So following is impossible?
-task configuration
Logstream task : Oracle → Logsteam
Repcation task : Oracle(logstream) → Snowflake ※Replication task is enable Store Changes

Best Regards.

kng
Support
Support

Hi @iti-attunity-sup 

Good Day

 

Its possible to have a task configuration like this

 

logstream task: from source Oracle to Logstream

Replication task: Oracle(logstream) → Snowflake ※Replication task is enable Store Changes

 

In Replication task Yes we can enable store changes or apply apply changes also

 

Regards,

shivananda