Incremental load from PostgreSQL

martinholm — Fri, 10 Dec 2021 20:23:41 GMT

Hi, I'm trying to do an incremental load from PostgreSQL. Currently I have this code

let ThisExecTime = Date(Today(), 'YYYY-MM-DD');

[measurement]:

LOAD Date(meas_date) as %DateKey,
Time(meas_time) as Måletidspunkt,
meas_time,
tag_id,
meas_value;

Select date("meas_time") as meas_date,
"meas_time",
"tag_id",
"meas_value"
from measurement
where tag_id in(348, 341, 694, 695, 698, 699, 349, 1020)
and meas_time >= '2021-02-07' //'$(PreviousExecTime )' Using static date for test purpose
AND meas_time < '$(ThisExecTime)';

concatenate LOAD %DateKey, Måletidspunkt, meas_time, tag_id, meas_value
FROM 'lib://01. Data repository Prod/2_Transform/Biologisk/Measurement.qvd';
//WHERE NOT Exists(tag_id, meas_time);

STORE measurement INTO 'lib://01. Data repository Prod/2_Transform/Biologisk/Measurement.qvd';

let PreviousExecTime = ThisExecTime;

On "concatenate LOAD %DateKey..." i get the error: "Field '%DateKey' not found". However, both in my preceding load statement and in the QVD file (picture of QVD file attached) I have %DateKey.

Additionally when "WHERE NOT Exists(tag_id, meas_time)" is uncommented I get the error:

"Unexpected token: 'tag_id', expected one of: 'txt', 'Table', 'tab', 'biff', 'dif', 'fix', 'html', ...: concatenate LOAD %DateKey, Måletidspunkt, meas_time, tag_id, meas_value FROM 'lib://01. Data repository Prod/2_Transform/Biologisk/Measurement.qvd' WHERE NOT Exists(>>>>>>tag_id<<<<<<, meas_time)"

From the documentation: https://help.qlik.com/en-US/sense/November2020/Subsystems/Hub/Content/Sense_Hub/LoadData/use-QVD-files-incremental-load.htm, the primary key should be in the not exist statement, in my case, I have a composite PK. Is that a problem?
I don't quite get why it would expect a table or html (and some of the others which I don't know what is). Using the measurement table, i get similar error message.

How can I fix these errors?

Thanks in advance for any help

topic Incremental load from PostgreSQL in Connectivity & Data Prep

Incremental load from PostgreSQL