topic Re: Incremental Load with Replace - unexpected behavior in QlikView

Incremental Load with Replace - unexpected behavior

exfosteve — Wed, 25 Nov 2020 16:16:04 GMT

I have some behavior I'm not expecting.

I'm doing an incremental load, and the common field is timestamp which I have stored as a number (only 1 entry exists '43149').

If I don't use the where clause, the concatenation works fine.

Concatenate(INCDATA)

LOAD * from '$(outputFile)' (qvd) ;

This works.

Concatenate(INCDATA)

LOAD * from '$(outputFile)' (qvd)

where text(timestamp) <> '43159';

This does not.

Concatenate(INCDATA)

LOAD * from '$(outputFile)' (qvd)

where not Exists(timestamp);

Any Ideas?

Re: Incremental Load with Replace - unexpected behavior

krishnacbe — Fri, 02 Mar 2018 16:36:00 GMT

Check the format of the timestamp in both the tables.

Re: Incremental Load with Replace - unexpected behavior

exfosteve — Fri, 02 Mar 2018 17:02:41 GMT

I did floor(timestamp,1) prior to the incremental load.

Re: Incremental Load with Replace - unexpected behavior

YoussefBelloum — Fri, 02 Mar 2018 17:15:31 GMT

Hi,

can you show the script, especially when you first load your field timestamp ?

Re: Incremental Load with Replace - unexpected behavior

exfosteve — Fri, 02 Mar 2018 17:57:12 GMT

Here you go. I also included the 2 QVD Files

Re: Incremental Load with Replace - unexpected behavior

exfosteve — Mon, 05 Mar 2018 13:21:03 GMT

I ended up getting working based on the the following post:

Concatenate Load Where New ID (Not Exists)

by adding a temporary variable, this fixed it.

Let baseFile = 'skinnyBase.qvd';

Let updateFile = 'skinnyUpdate.qvd';

UDATA:

Load timestamp, Result, timestamp as tsTMP

from '$(updateFile)' (qvd);

Concatenate(UDATA)

Load timestamp, Result from '$(baseFile)' (qvd)

where not exists(tsTMP, timestamp);

DROP Field tsTMP;

Re: Incremental Load with Replace - unexpected behavior

YoussefBelloum — Mon, 05 Mar 2018 14:10:35 GMT

Hi Steve, it is working for me

with the where not exists, it is concatenating and removing correctly the value 43159 read from the first table.

Re: Incremental Load with Replace - unexpected behavior

exfosteve — Mon, 05 Mar 2018 14:12:11 GMT

what version are you using? I'm running 11 SR17.