Unlock a world of possibilities! Login now and discover the exclusive benefits awaiting you.
I would like the QLIK replicate full load to be transparent to end-users querying the table.
Can I reload to a temp table and rename it to minimize any disruption to end-user queries?
How can this be accomplished within the tool?
The full load is required.
The source object is an Oracle VIEW (with parallel hint) over a materialized view snapshot table.
The target object is a Snowflake table.
-Dave
@david_lange ,, out of the box Replicate can not automate the process.
however, you can use QEM API to start the task full load, check on task status when complete and have your script do the two options after full load is complete .
[Edit: meant to ask - is the full-load only or CDC as well. CDC makes rename even harder if not impossible]
I understand the desire to load to temp tables and switch when done.
This is not a new request and many years ago an attempt was made to implement this for SQLserver target based on a specific customer request. I suspect the code is still there controlled by an advance, internal, parameter loadToTempTables.
The basic required actions are clear: on successful table full-load, drop the original table, make sure the temp table protections are as they should be for the real table, rename.
The error handling and timing challenges are less clear, similar to a direct full load but just less clear. While it worked, the customer at that time decided NOT to use it in production and with that the availability remained limited to (5) SQLserver family targets and was never 'advertised', only visible by figuring out how to see internal parameters, guess at their usage and verify with reptasks and target db SQL LOGS.
You could (should?) consider an feature request. Why not. Just don't hold your breath while waiting.
Instead figure out how to use a Replicate API (preferably the Enterprise Manager, but REPCTL command will work also and make yourself a script looking at "full_load_completed","full_load_start","full_load_end" (and possibly full_load_counters) from GetTaskDetails or more likely GetTableStatuses looking for per-table "table_full_load_info" and status. A good bit of work, but only done every minute or so and computers don't mind.
Or one could scan the reptask log(s) for table load completed messages every so often remembering where one left off and mindful of logs being rolled.
Details you depend on whether managing a full load for all tables, for some, and whether some table have foreign keys and are best renamed when all all done (something Replicate could/would not know).
Hope this helps some!
Hein.
I'm assuming that you already have the Target table in place with the current set of data and in need to do a Full Load without truncating or recreating the existing table.
I would accomplish this by changing the target table to <table_name>_test in the Replicate table settings and run the Full Load:
And once the Full Load is complete, have two options:
1) Drop the original table and rename _test table to original name
or
2) Sync the data from _test to the original table and drop the _test table.
Thank you,
Can these steps be automated via the QLIK replicate tool?
"And once the Full Load is complete, have two options:
1) Drop the original table and rename _test table to original name
or
2) Sync the data from _test to the original table and drop the _test table."
@david_lange ,, out of the box Replicate can not automate the process.
however, you can use QEM API to start the task full load, check on task status when complete and have your script do the two options after full load is complete .
Hello @david_lange ,
Agree with @Heinvandenheuvel , @Steve_Nguyen and @shashi_holla .
Not sure if your source table/view contains PK or Unique Index, or the view has a de facto 'logical' PK (for example we can make a column as PK in target table). If yes then we may make it automated; Otherwise we've no chance to trigger the 'renaming' operation. Let us know and we will confirm for you.
Regards,
John.