Is it possible to enable Store Changes when using Logstream ?

[iti-attunity-sup]

Hello support team.

[Info]
Source : Oracle(Logstream)
Target : Snowflake

[Question]
Is it possible to enable Store Changes when using Logstream ?(Is there any limitation ?)

When executed task (enable Store Changes and using Logstream), task was crashed.

If there is no limitation, would you modify task won't crash.

Best Regards.

[john_wang]

Hello @iti-attunity-sup ,

I don't believe this is an appropriate usage, as it introduces unnecessary overhead to storage and resources without providing any benefit to downstream tasks.

Hope this helps.

John.

[aarun_arasu]

Hello @iti-attunity-sup ,

Please refer to the below user guide for the limitation 

https://help.qlik.com/en-US/replicate/May2024/Content/Replicate/Main/Log%20Stream%20Staging/ls3.htm

Note : The Full Load and Store Changes replication options are not relevant.

Regards

Arun

[iti-attunity-sup]

Hello Arun.
Thank you for your reply.

This is Log Stream Staging task limitation.
But replication task to transfer the staged data to the target also has same limitation ?

Best regards.

[sureshkumar]

Hello @iti-attunity-sup 

Replication (Child) Task is like normal task, there is no such limitation.

You can able "Store Changes/Apply Changes" once the Full Load completes.

Regards,
Suresh

[iti-attunity-sup]

Hello Suresh.

What I want to do is following
-task configuration
Logstream task : Oracle → Logsteam
Repcation task : Oracle(logstream) → Snowflake ※Replication task is enable Store Changes

But I recognized first answer is "this task configuration is not appropriate usage".
After all this task configuration is possible? (But when task executed, task crashed)

Best Regards.

[SushilKumar]

Hello @iti-attunity-sup 

Logstream task and replicate task are there to serve Different purpose.  

Logstream task mainly configure were you want to reduce no of process/query to get same kind of data and put less load on Connected Source Db. So only few processes engage Source db. to read the changes from Db or the logs and put them in a format where Replicate task extract changes and push them to intended targets.

So, in nutshell the Staging folder where logstream put the files to further read by replicate task provide kind of fault tolerant as well. And generated stored files only have the data related to participating tables while if you configure normal task which direct read from db. or redo logs.

when we choose LOGSTREAM task it will scan and push data work in this manner ( extract change and put into files for further read) 

When we choose Normal REPLICATE task then it will scan and push data work in this manner ( extract change , Transform and push data to Target end) 

Change table is created on the target side. hence, it's not possible for logstream to work with store changes

hope it helps.

if you like the explanation then request you to mark is as a solution.

Regards,

Sushil Kumar

[SachinB]

Hello @iti-attunity-sup ,

Log Stream enables a dedicated Replicate task to save data changes from the transaction log of a single source database and apply them to multiple targets thereby eliminating the overhead of reading the logs for each target separately.

When you create task with Log stream you will store source data in stage and from stage you can consume the data as part of CDC it could be apply or store changes

If your task is crashing, I would request you create a case and will help you with screen sharing.

Hope this helps you.

Regards,

Sachin B

[iti-attunity-sup]

Hello Sushil

Thank you for your reply.

To sum up,
there is limitation that Replication task with Logstream cannot use Store Changes ?

So following is impossible?
-task configuration
Logstream task : Oracle → Logsteam
Repcation task : Oracle(logstream) → Snowflake ※Replication task is enable Store Changes

Best Regards.

[kng]

Hi @iti-attunity-sup 

Good Day

Its possible to have a task configuration like this

logstream task: from source Oracle to Logstream

Replication task: Oracle(logstream) → Snowflake ※Replication task is enable Store Changes

In Replication task Yes we can enable store changes or apply apply changes also

Regards,

shivananda