Qlik Community

New to QlikView

Discussion board where members can get started with QlikView.

Announcements
The #1 reason QlikView customers adopt Qlik Sense is a desire for a modern BI experience. Read More
cancel
Showing results for 
Search instead for 
Did you mean: 
Highlighted
Not applicable

How much lines QlikView can read from txt file?

Hi all

Do importing text table with 1300000 lines with 47 coloumns. Stopped on 684000 line without explanation.

After that exporting give me error:

Image 1.jpg

Then i've do close this "qvw" file and open it again.

Run of reading script give me error:

Image 2.jpg

Pressing OK will go further but again stopped  with error

Image 3.jpg

How much lines Qlik from txt file can read?

11 Replies
Highlighted
Partner
Partner

Re: How much lines QlikView can read from txt file?

Your issue is not a QV one but a server problem. QV will read billions of lines I believe (although there is a limit on distinct values).  I have read many millions of rows from a text file.

I think you might be paging too much - check the paging file settings.

Hope this helps,

Jason

Highlighted
Specialist II
Specialist II

Re: How much lines QlikView can read from txt file?

1.3 million rows isn't massive in Qlikview terms, so it could be the 47 columns. Try loading fewer columns (remove long text description fields first) and see if performance improves.

flipside

Highlighted
Not applicable

Re: How much lines QlikView can read from txt file?

What version are you on? Make sure you run the x64 and not the x86 version on QV.

Highlighted
Not applicable

Re: How much lines QlikView can read from txt file?

Yes. I'm trying x32 free personal version 11.20.12018 SR3. Without server, of course.

The text file size is 650 Mb. It's look for me that there is enough room in my 3Gb to be in memory.

Highlighted
Not applicable

Re: How much lines QlikView can read from txt file?

Its hard to estimate what kind of resources needed expand that data in memory. The size on disk and the size in memory can differentiate allot depending on the data. My recommendation is to try with the x64 version instead and see if that gives you any different result. 

Highlighted
Partner
Partner

Re: How much lines QlikView can read from txt file?

Hi:

32 bits QlikView desktop version only supports 2GB. More than that forget it. It won't take it.

For a 64 bit version.

Find out how many other processes are being executed at the same time in that server.

Do you have SQL, or other DB engine in the same machine? That’s probably it.

Try to control or reduce other applications or processes cache, or the space they have reserved for their operations.

I believe you’re working in 64 bits.

Luck.

Highlighted
Specialist II
Specialist II

Re: How much lines QlikView can read from txt file?

Your screenshot seems to indicate a single core processor so I am guessing you are still on 32-bit - is that correct? If that's the case then moving to 64-bit won't work. The single core processor will also be an issue as it will be used for other system processes.

I suggest you only load in the minimum numer of fields from the file, say 10 numeric type fields, and if that works increase the fields and see how it is affected. Does the file have any large text fields? 

Highlighted
Not applicable

Re: How much lines QlikView can read from txt file?

Please note. I'm trying x32 free personal version 11.20.12018 SR3 on WinXP SP3!

As services on my computer running only Antivirus Kaspersky. Nothing more essential.

As is.

My experiments today shows that Qlikview can't read from text file more than 682564 lines. Not depends on number of fields i've choosed. All 47or only 10. The text file was verified using special text editor. Lines 682564 and 682565 all OK. Stopped Qlikview.

Let's see ase with 10 fields:

Image 5.jpg

Qlikview with imported table occupied 500Mb.

If i want to send the table with 10 fields and 682564 rows to excel- Qlinkview aborted.

Image 6.jpg

If i want to find someting in the table- Qlinkview aborted.

Image 7.jpg

Highlighted
Employee
Employee

Re: How much lines QlikView can read from txt file?

You can download the 64 bit version of Personal Edition.  That will make more RAM available to you (assuming you are running a 64bit OS).  The limitations you are experiencing are purely due to hardware and/or OS architecture (32 vs 64 bit).