topic Moving certain columns from source to target in Qlik Replicate

Moving certain columns from source to target

MdFazil — Thu, 29 Feb 2024 05:43:40 GMT

Hi, I'm new to Qlik replicate. I always wonder that can replicate move only certain or required columns alone to target table. I have gone through the internet but found nothing. Can anyone please clarify me this?

Re: Moving certain columns from source to target

john_wang — Thu, 29 Feb 2024 05:53:03 GMT

Hello @MdFazil ,

Thanks for reaching out to Qlik Community!

Yes, we may replicate only interested columns rather than all the columns. By default Qlik Replicate include all columns in the task design. You may simply to remove some columns from the table transformation for example I'd like to remove the column "NOTES" from the replication:

After the column "NOTES" is removed from the design, the table has 2 columns in target side.

Certainly in some scenarios you may want to add some additional columns eg the transaction datetime, and other computed columns etc. see User Guide Overview of the Expression Builder.

Hope this helps.

John.

Re: Moving certain columns from source to target

Yadunandan — Thu, 29 Feb 2024 05:51:20 GMT

Hello @MdFazil

Yes it is possible, you need to select the tables which you require to replicate in the table selection window while setting up a task. please see selecting tables for more information.

Regards,"
Yadunandan

Re: Moving certain columns from source to target

MdFazil — Thu, 29 Feb 2024 05:52:58 GMT

Hello @john_wang, thanks for the reply.

Yes, this clarify my doubt

Regards
Fazil

Re: Moving certain columns from source to target

john_wang — Thu, 29 Feb 2024 05:55:17 GMT

Glad to hear that, Thank you for the support! @MdFazil

Re: Moving certain columns from source to target

MdFazil — Thu, 29 Feb 2024 05:55:28 GMT

Hello @Yadunandan , Thank you for the reply.

Thanks for sharing some documentation. I'll look into that for more details

Regards
Fazil

Re: Moving certain columns from source to target

dommiiii — Tue, 06 May 2025 11:47:05 GMT

Hello,

Wanted to ask if we unmark on table setting By default include all columns and explicitly choose some columns to create schema and send to Kafka target. And after some , e.g. a week DB2 table (source) is modified - additional columns are added. Will new schema will be sent to Kafka, or schema will remain same, as we explicitly set needed columns? We are using November 2023 version.

Best Regards
Domile

Re: Moving certain columns from source to target

john_wang — Tue, 06 May 2025 14:09:24 GMT

Hello @dommiiii ,

If DDL replication is enabled in the Apply Changes Settings, refer to the highlighted section below:

When additional columns are added on the source side, the Kafka messages will reflect these metadata changes and will include the values for the newly added columns.

Regards,

John.

Re: Moving certain columns from source to target

dommiiii — Wed, 07 May 2025 12:22:34 GMT

Hello,

Thanks for Your reply. So what would happen if we turn off ALTER target table our schema will not change for a while. But if later on will do Full load for our task and new columns were added to table? I think still new schema will be pushed in that case and messages with more columns would go to Kafka topic?

Best Regards,

Domile

Re: Moving certain columns from source to target

john_wang — Wed, 07 May 2025 13:39:36 GMT

Hello Domile, @dommiiii

>> So what would happen if we turn off ALTER target table our schema will not change for a while.

The DDL will be ignored so no new columns and values will be published to Kafka.

>> But if later on will do Full load for our task and new columns were added to table? I think still new schema will be pushed in that case and messages with more columns would go to Kafka topic?

Correction: Yes. If you reload the table, or task, the new columns will be loaded to Kafka, in both Full Load stage and Change Processing stage.

Let me know if you have any doubts.

Regards,

John.

Re: Moving certain columns from source to target

dommiiii — Wed, 07 May 2025 19:31:48 GMT

Thanks again John.

So assume DDL is set to "Alter target table" and we unmark "By default include all columns"
and choose just particular columns which we need use. And task runs for a while, and later on additional columns are added to DB2/zos. So then new schema will include all columns, "By default include all columns" ticked again, and particularly chosen columns will be reset to default (all columns selected and used)?

Is only reason for button "By default include all columns", to remove all columns from target table faster?

Best Regards,

Domile

Re: Moving certain columns from source to target

john_wang — Sat, 10 May 2025 03:40:55 GMT

Hello @dommiiii ,

Button "By default include all columns" is used to remove all columns (Except the PK columns) from target table faster.

Regards,

John.