Skip to main content
Announcements
Introducing Qlik Answers: A plug-and-play, Generative AI powered RAG solution. READ ALL ABOUT IT!
cancel
Showing results for 
Search instead for 
Did you mean: 
Anonymous
Not applicable

File size limit

Simple question.
is there it a limit of txt file size used as a data source?
There is a file ~1GB. The expected usefull information is about 1/1000 of all data. I'm wondering if QV can be used to exctract it.
(At this moment I don't know anything yet about the file structure, what the separators are, if any, and how to recognize the useful data. Concerned only about file size.)

1 Solution

Accepted Solutions
prieper
Master II
Master II

Hi Michael,

have loaded also quite bit .txt-files. Is there really a difference between data coming from an ODBC-connection or a text-file?

edit: Give it a try!

Peter

View solution in original post

5 Replies
prieper
Master II
Master II

Hi Michael,

have loaded also quite bit .txt-files. Is there really a difference between data coming from an ODBC-connection or a text-file?

edit: Give it a try!

Peter

Anonymous
Not applicable
Author

Thanks for reply Peter, I've loaded my share of txt, xml, config, csv, (you name it) files as well, but never that's large. The file just can't be opened by notepad. But it turned out there's no reason for concern. I received the file today, and loaded in QV with no hint of a problem, worked like a charm. QV never used more than 70 MB of RAM on reload. Thanks.
biester
Specialist
Specialist

Just for info: in expectation of having to handle big log files I once tested (in QV 9 SR1) with a programmatically created text file with ca 3 GB size and 80 million rows with random data. It took about two hours to load tha data on my relatively small server, but it worked without any problems.

Rgds,
Joachim

Anonymous
Not applicable
Author

Yes Joachim,
This was exactly my concern. Now I know from the result that it does work. In my case, I loaded my ~0.1% of this 1 GB file in just 5 min.

francoiscave
Partner - Creator III
Partner - Creator III

Hi Michael,

I just discovered this issue. That it would be a solution for you :

"A common issue with text files is special (and non-visible) characters causing Qlikview to think it has reached the end of the file. If the load is always ending at the same record, then this is probably the reason. Does the last record end part-written? Try making a copy of the text file, delete the final record it is loading and see if it loads more records."

François