Redshift post-processing

[simonB2020]

I need to execute some post processing script on my Redshift target once the full load has completed.

i.e. call a stored procedure

Is this possible to trigger from Replicate 

Thanks

[SwathiPulagam]

Hi @simonB2020 ,

For the Redshift target, there is no option to run a post-processing script or stored procedures from Qlik Replicate.

Thanks,

Swathi

[john_wang · Accepted solution]

Hello @simonB2020 , copy @SwathiPulagam ,

Maybe you can STOP the task automatically after the Full Load completion by enabling option "Before cached changes have been applied". Then run your custom scripts and RESUME the task again after the scripts execution.

Hope this helps.

Regards,

John.

[Heinvandenheuvel · Accepted solution]

Replicate Build-in post processing command are limited to CSV type file processing.

I think your best bet is to create a polling script either using the REPCTL commandline gettaskstatus on the repiicate server, or the Enterprise    Manager gettaskstatus Api call from anywhere to find the current task status.

You would look for state =STOPPED 

When state=running, just wait and check again. Optionally report current time, full_load_completed_tables_count / full_load_loading_tables_count / full_load_queued_tables_count

When state=error - report via Email or similar optionally indicating start_time, and various table counts

When state=stopped verify you further details for 'reasonable values' to decide to call it good and run post-processing of report failure after all.

• full_load_completed=1
• full_load_finish_time > last recorded starting time  or 'yesterday' or whatever is appropriate.
• full_load_completed_tables_count = expected count
• full_load_total_record_transferred >= some expected minimum (yesterday's number?)
• full_load_error_tables_count = 0

Hein