Skip to main content
Woohoo! Qlik Community has won “Best in Class Community” in the 2024 Khoros Kudos awards!
Announcements
Nov. 20th, Qlik Insider - Lakehouses: Driving the Future of Data & AI - PICK A SESSION
cancel
Showing results for 
Search instead for 
Did you mean: 
Not applicable

running out of memory loading 2gb text file

Trying to load a 2gb text file (17 columns, roughly 28M rows) and my 4gb computer (32 bit) is running out of memory about half way through the load. I'm looking for options without having to move to a 64 bit machine. Please advise.

Thanks,

Kevin

1 Reply
Not applicable
Author

Hello,

on a 32-bit Windows system you can have 100GB RAM, a single process can still address max. 2GB (3 GBs on Win. servers with a tweak).

Still I can think of 2GB text to fit into QV in a binary form.
So I would try to split the large file in several smaller files. Probably you can split them by date or any other content criteria, otherwise split by rows ... 1M rows per file ...

Honestly I don't know which text editor I would use for that task.
If you created the text file by DB export, the split could be done there ... ?

hth,
Thilo