topic Re: Inconsistent number of rows loaded from text file in QlikView

Inconsistent number of rows loaded from text file

bhorneber — Tue, 20 Jun 2017 17:21:44 GMT

Hi,

I discovered an extremely weird loading behavior in QlikView (11.2 SR17):

I load data from three different text files with the same structure, data is separated with '|' (no quotes). The script is running for years now. But after making some coding improvements I wanted to make sure that the resulting file has not changed.

During testing, I found that the QVD generated by the new script read in less rows. I made various tests and couldn't find any error in the script.

So I started storing the intermediate results and exiting the script after. That's when I realized, the number of read in rows strangely enough changes by simply moving the STORE/EXIT SCRIPT commands around!

These are three different loads with only changing the position of the two lines, but nothing else.

Also, the store and exit always come after the three files are read in:

I have no explanation for this behavior and no idea how this can happen in the first place ... anyone having an idea?

Addendum:

It seems like this behavior only occurs when reloading within the desktop version of QlikView, but not when starting the reload job via QMC. Could it be a resource issue?

Re: Inconsistent number of rows loaded from text file

Sergey_Shuklin — Tue, 20 Jun 2017 17:58:13 GMT

Hello, Benedikt!

I had faced one time an issue when num of rows after load was lesser than in original txt-file.

When I removed "(txt, codepage is 932, no labels, delimiter is ' ', msq);"<this one in the load script everything had become ok.

Maybe in your case it will help too.

Re: Inconsistent number of rows loaded from text file

bhorneber — Tue, 20 Jun 2017 18:13:12 GMT

Hi Sergey,

Thanks for the reply. Unfortunately, that's not it.

This part of the load command is

(txt, utf8, no labels, delimiter is '|', no quotes)

Re: Inconsistent number of rows loaded from text file

Sergey_Shuklin — Wed, 21 Jun 2017 04:35:14 GMT

You can do one more test:

Copy your txt data to excel and run your script with it. If num of rows will be the same - it's mean that issue exactly in the load-options-line ((txt, utf8, no labels...) and you'll have to try different variations of it.

If it's not - then take a look closely to your data, maybe there is some delimiter cases.

There was a discussion with a tip to transform data a little before load:

MSQ format truncates data while loading from de... | Qlik Community

Not much, but maybe you'll find something usefull!

Re: Inconsistent number of rows loaded from text file

bhorneber — Wed, 21 Jun 2017 20:48:50 GMT

Thanks for following up.

I tested what you were recommending: Excel shows the same amount of rows as the last load (position 3).

I played with various combinations of the load options, without any success.

As mentioned in my original post (added it yesterday before your 2nd post):

It seems like this behavior only occurs when reloading within the desktop version of QlikView, but not when starting the reload job via QMC. Could it be a resource issue?

After extensive testing, it seems to me either a bug in the current version/SR or an issue with the available resources when using the desktop edition.

I also wouldn't understand why sometimes the whole file gets loaded and sometimes not, purely based on the position of a STORE command. That is independent from any data load. Just doesn't seem right.

Thanks for your suggestions though! Really appreciate it.