topic Re: Dynamic target folder creation in Qlik Replicate

Dynamic target folder creation

eksmirnova — Tue, 21 Mar 2023 18:52:17 GMT

I have a Qlik task with ADLS storage as a target endpoint. Is there a way to configure QLIK task to create target subfolder depending on source schema name?

source schema name	source table name	target folder	target file name
A1	TABLE1	/mypath/A1/TABLE1/	LOAD00001.csv
B2	TABLE1	/mypath/B2/TABLE1/	LOAD00001.csv
C3	TABLE1	/mypath/C3/TABLE1/	LOAD00001.csv

Re: Dynamic target folder creation

pedrobergo — Wed, 21 Feb 2024 18:39:51 GMT

Hi @eksmirnova

Considering that you using Full Load only task, you can try following info. It works from May2022 version, but I´m tested on November2022.

On target Endpoint input '/mypath' on Target folder field.
On task create a Global Rule Transformation to Rename tables adding the Prefix '/' char. ADLS will interpret it with as a sub-folder.

With this structure you will have

/EndpoinTargetFolder/SchemaName/TableName/LOAD000X.csv

NOTE: This won't work for CDC as expected, Please refer to ADLS target limitations.

[],

Pedro

Re: Dynamic target folder creation

eksmirnova — Mon, 27 Mar 2023 12:57:58 GMT

Thank you! It works

Re: Dynamic target folder creation

MoeE — Fri, 28 Jul 2023 00:43:07 GMT

Hi Pedro,

Is there a way to rename the files themselves from LOAD000X to for example <tablename>?

Kind regards,

Mohammed

Re: Dynamic target folder creation

SushilKumar — Fri, 28 Jul 2023 01:07:14 GMT

Hello Team,

Regarding changing the name of the file: You can achieve this through the "Run command after upload" option for the target endpoint.

You can process the final target files using a custom command. The command will be run whenever a data file is created.

Kindly refer to the user guide in Setting Advanced connection properties for the S3 target point. In the advanced tab, you can use the "Post Upload Processing - Run command after upload."

Setting advanced connection properties #Setting advanced connection properties | Qlik Replicate Help

Let us know in case if you need any additional information for the same.

If our response has been helpful, please consider clicking "Accept as Solution". This will assist other users in easily finding the answer.

Regards,

Sushil Kumar

Re: Dynamic target folder creation

MoeE — Fri, 28 Jul 2023 01:17:32 GMT

Hi Sushil,

Thanks for the quick response, I really appreciate it. I am unable to accept this as a solution maybe because I am not the owner of the original post? Anyways, thank you.

Regards,

Mohammed