How much lines QlikView can read from txt file? - Qlik Community

Report Inappropriate Content · ‎2013-10-25

Hi all

Do importing text table with 1300000 lines with 47 coloumns. Stopped on 684000 line without explanation.

After that exporting give me error:

Then i've do close this "qvw" file and open it again.

Run of reading script give me error:

Pressing OK will go further but again stopped with error

How much lines Qlik from txt file can read?

Jason_Michaelides · ‎2013-10-25

Your issue is not a QV one but a server problem. QV will read billions of lines I believe (although there is a limit on distinct values). I have read many millions of rows from a text file.

I think you might be paging too much - check the paging file settings.

Hope this helps,

Jason

flipside · ‎2013-10-25

1.3 million rows isn't massive in Qlikview terms, so it could be the 47 columns. Try loading fewer columns (remove long text description fields first) and see if performance improves.

flipside

Report Inappropriate Content · ‎2013-10-25

What version are you on? Make sure you run the x64 and not the x86 version on QV.

Report Inappropriate Content · ‎2013-10-25

Yes. I'm trying x32 free personal version 11.20.12018 SR3. Without server, of course.

The text file size is 650 Mb. It's look for me that there is enough room in my 3Gb to be in memory.

Report Inappropriate Content · ‎2013-10-25

Its hard to estimate what kind of resources needed expand that data in memory. The size on disk and the size in memory can differentiate allot depending on the data. My recommendation is to try with the x64 version instead and see if that gives you any different result.

christian77 · ‎2013-10-25

Hi:

32 bits QlikView desktop version only supports 2GB. More than that forget it. It won't take it.

For a 64 bit version.

Find out how many other processes are being executed at the same time in that server.

Do you have SQL, or other DB engine in the same machine? That’s probably it.

Try to control or reduce other applications or processes cache, or the space they have reserved for their operations.

I believe you’re working in 64 bits.

Luck.

flipside · ‎2013-10-25

Your screenshot seems to indicate a single core processor so I am guessing you are still on 32-bit - is that correct? If that's the case then moving to 64-bit won't work. The single core processor will also be an issue as it will be used for other system processes.

I suggest you only load in the minimum numer of fields from the file, say 10 numeric type fields, and if that works increase the fields and see how it is affected. Does the file have any large text fields?

Report Inappropriate Content · ‎2013-10-25

Please note. I'm trying x32 free personal version 11.20.12018 SR3 on WinXP SP3!

As services on my computer running only Antivirus Kaspersky. Nothing more essential.

As is.

My experiments today shows that Qlikview can't read from text file more than 682564 lines. Not depends on number of fields i've choosed. All 47or only 10. The text file was verified using special text editor. Lines 682564 and 682565 all OK. Stopped Qlikview.

Let's see ase with 10 fields:

Qlikview with imported table occupied 500Mb.

If i want to send the table with 10 fields and 682564 rows to excel- Qlinkview aborted.

If i want to find someting in the table- Qlinkview aborted.

Josh_Good · ‎2013-10-25

You can download the 64 bit version of Personal Edition. That will make more RAM available to you (assuming you are running a 64bit OS). The limitations you are experiencing are purely due to hardware and/or OS architecture (32 vs 64 bit).