topic File size limit in QlikView

File size limit

Anonymous — Sun, 18 Oct 2009 02:47:52 GMT

Simple question.
is there it a limit of txt file size used as a data source?
There is a file ~1GB. The expected usefull information is about 1/1000 of all data. I'm wondering if QV can be used to exctract it.
(At this moment I don't know anything yet about the file structure, what the separators are, if any, and how to recognize the useful data. Concerned only about file size.)

File size limit

prieper — Sun, 18 Oct 2009 08:57:32 GMT

Hi Michael,

have loaded also quite bit .txt-files. Is there really a difference between data coming from an ODBC-connection or a text-file?

edit: Give it a try!

Peter

File size limit

Anonymous — Mon, 19 Oct 2009 08:46:19 GMT

Thanks for reply Peter, I've loaded my share of txt, xml, config, csv, (you name it) files as well, but never that's large. The file just can't be opened by notepad. But it turned out there's no reason for concern. I received the file today, and loaded in QV with no hint of a problem, worked like a charm. QV never used more than 70 MB of RAM on reload. Thanks.

File size limit

biester — Mon, 19 Oct 2009 17:52:19 GMT

Just for info: in expectation of having to handle big log files I once tested (in QV 9 SR1) with a programmatically created text file with ca 3 GB size and 80 million rows with random data. It took about two hours to load tha data on my relatively small server, but it worked without any problems.

Rgds,
Joachim

File size limit

Anonymous — Mon, 19 Oct 2009 21:31:13 GMT

Yes Joachim,
This was exactly my concern. Now I know from the result that it does work. In my case, I loaded my ~0.1% of this 1 GB file in just 5 min.

Re: File size limit

francoiscave — Fri, 12 Dec 2014 09:01:23 GMT

Hi Michael,

I just discovered this issue. That it would be a solution for you :

"A common issue with text files is special (and non-visible) characters causing Qlikview to think it has reached the end of the file. If the load is always ending at the same record, then this is probably the reason. Does the last record end part-written? Try making a copy of the text file, delete the final record it is loading and see if it loads more records."

François