Qlik Cloud Data Integration: How to enable Full Load Passthru filter on Landing Tasks

shashi_holla — Mon, 28 Oct 2024 09:26:42 GMT

The purpose of this article is to provide details about enabling Full Load Passthru filter in Qlik Cloud Data Integration (QCDI) and get the selected data from the source during the initial load of the Landing or Replication Tasks.

Enable Passthrough filter in the Qlik Data Movement Gateway server configuration

Navigate to the path /opt/qlik/gateway/movement/bin on the Qlik Data Movement Gateway Linux server
Edit the file repctl.cfg
Set the parameter enable_passthrough_filter to true and save the file
Restart the Gateway server using the service repagent restart command

Build the filter in the Qlik Cloud Data Integration tenant

Log in to our QCDI tenant
Open the Landing Task in the project.
Click on the filter button (top right corner above the table details)

Syntax:

--filter: filter_condition

Example One:

--filter: id > 2

Example Two:

--filter: id between 1 and 2
We can add any valid condition using the syntax supported by the source Database and the tool internally constructs the select statement using the condition to filter the data from the source.
Once the filter is added, prepare the landing task and then do a Reload of the table to validate the results
Please note that the filter is at the table level and every table can have its own filter condition for a column specific to that table.

The information in this article is provided as-is and will be used at your discretion. Depending on the tool(s) used, customization(s), and/or other factors, ongoing support on the solution below may not be provided by Qlik Support.

Environment

Qlik Cloud Data Integration

Re: Qlik Cloud Data Integration: How to enable Full Load Passthru filter on Landing Tasks

qlikplaut — Mon, 19 Aug 2024 08:24:26 GMT

Hi,

Can this be done using SAP Extractors as a source? I have tried your approach but regardless of the filter i am trying to use, nothing is happening, i am still getting the full set of data even if i use a proper filtering value or a bogus one. Seems filtering just does not work.

Best regards,

Bogdan

Re: Qlik Cloud Data Integration: How to enable Full Load Passthru filter on Landing Tasks

mtreubig — Thu, 17 Oct 2024 19:37:19 GMT

Should service regagent restartbe service repagent restart?

Re: Qlik Cloud Data Integration: How to enable Full Load Passthru filter on Landing Tasks

Sonja_Bauernfeind — Mon, 28 Oct 2024 09:27:03 GMT

@mtreubig That's correct, thank you! I've corrected the typo.

article Qlik Cloud Data Integration: How to enable Full Load Passthru filter on Landing Tasks in Official Support Articles

Qlik Cloud Data Integration: How to enable Full Load Passthru filter on Landing Tasks

Enable Passthrough filter in the Qlik Data Movement Gateway server configuration

Build the filter in the Qlik Cloud Data Integration tenant

Environment

Re: Qlik Cloud Data Integration: How to enable Full Load Passthru filter on Landing Tasks

Re: Qlik Cloud Data Integration: How to enable Full Load Passthru filter on Landing Tasks

Re: Qlik Cloud Data Integration: How to enable Full Load Passthru filter on Landing Tasks