Skip to main content
Announcements
Introducing Qlik Answers: A plug-and-play, Generative AI powered RAG solution. READ ALL ABOUT IT!
cancel
Showing results for 
Search instead for 
Did you mean: 
edward221096
Partner - Contributor
Partner - Contributor

TARGET HEADER SETTINGS

Hi Everyone,

 

We are currently stuck with the configuration of Header to our task. So our requirements is that within the task there will be a 2 columns that will be additionally created in the target table (1st is the inserted timestamp where it will be reflected a timestamp when it was full loaded in the target, 2nd is the updated timestamp where it shows whenever the record change in the source, then it will record the timestamp in this 2nd column)

the problem is that, whenever there is a change in the source the 1st column in the target will also update the timestamp value. The requirment is that Only the 2nd column MUST be updated. 

The headers we used in the columns are

CommitDate = $AR_H_COMMIT_TIMESTAMP
UpdateDate = $AR_H_CHANGE_SEQ

Labels (2)
1 Solution

Accepted Solutions
Heinvandenheuvel
Specialist III
Specialist III

It is pretty common to desire both initial insert and update time columns added to a target table.

Of those two, the insert time is tricky as,  as you observed, if you add that as a timestamp in Replicate it will also be updated during a subsequent update or delete. 

The reason is that there is no option to NOT update a column once it is in the output list. There is a complex  and costly workaround by using TARGET_LOOKUP and OPERATION_INDICATOR to set the value to itself on update and deletes. Obviously this will slow down the target actions.

The simple solution for the insert time is to NOT have it be added by Replicate but to use a target database provision to apply a default value. The only tricky part there is that you must switch the task to truncate for full-load and not drop+create as that would create without the insert time. Mind you, for most database you could also just add the insert time column with default value soon after the full-load and have a good enough result.

I urge you to go practice some with a test table in a test task. Add a bunch of columns loaded with all the available header variables and see for yourself when they are filled in or not (full-load vs insert vs update). Also add some columns with values from expressions based on the 'operation_indicator' function as per user guide and help. Trust but verify!

hth,

Hein.

View solution in original post

3 Replies
Heinvandenheuvel
Specialist III
Specialist III

It is pretty common to desire both initial insert and update time columns added to a target table.

Of those two, the insert time is tricky as,  as you observed, if you add that as a timestamp in Replicate it will also be updated during a subsequent update or delete. 

The reason is that there is no option to NOT update a column once it is in the output list. There is a complex  and costly workaround by using TARGET_LOOKUP and OPERATION_INDICATOR to set the value to itself on update and deletes. Obviously this will slow down the target actions.

The simple solution for the insert time is to NOT have it be added by Replicate but to use a target database provision to apply a default value. The only tricky part there is that you must switch the task to truncate for full-load and not drop+create as that would create without the insert time. Mind you, for most database you could also just add the insert time column with default value soon after the full-load and have a good enough result.

I urge you to go practice some with a test table in a test task. Add a bunch of columns loaded with all the available header variables and see for yourself when they are filled in or not (full-load vs insert vs update). Also add some columns with values from expressions based on the 'operation_indicator' function as per user guide and help. Trust but verify!

hth,

Hein.

Steve_Nguyen
Support
Support

@edward221096 did any of the reply help ?

Help users find answers! Don't forget to mark a solution that worked for you! If already marked, give it a thumbs up!