File size limit

[Anonymous]

Simple question.
is there it a limit of txt file size used as a data source?
There is a file ~1GB. The expected usefull information is about 1/1000 of all data. I'm wondering if QV can be used to exctract it.
(At this moment I don't know anything yet about the file structure, what the separators are, if any, and how to recognize the useful data. Concerned only about file size.)

[prieper · Accepted solution]

Hi Michael,

have loaded also quite bit .txt-files. Is there really a difference between data coming from an ODBC-connection or a text-file?

edit: Give it a try!

Peter

[Anonymous]

Thanks for reply Peter, I've loaded my share of txt, xml, config, csv, (you name it) files as well, but never that's large. The file just can't be opened by notepad. But it turned out there's no reason for concern. I received the file today, and loaded in QV with no hint of a problem, worked like a charm. QV never used more than 70 MB of RAM on reload. Thanks.

[biester]

Just for info: in expectation of having to handle big log files I once tested (in QV 9 SR1) with a programmatically created text file with ca 3 GB size and 80 million rows with random data. It took about two hours to load tha data on my relatively small server, but it worked without any problems.

Rgds,
Joachim

[Anonymous]

Yes Joachim,
This was exactly my concern. Now I know from the result that it does work. In my case, I loaded my ~0.1% of this 1 GB file in just 5 min.

[francoiscave]

Hi Michael,

I just discovered this issue. That it would be a solution for you :

"A common issue with text files is special (and non-visible) characters causing Qlikview to think it has reached the end of the file. If the load is always ending at the same record, then this is probably the reason. Does the last record end part-written? Try making a copy of the text file, delete the final record it is loading and see if it loads more records."

François