Skip to main content
Announcements
Qlik Connect 2024! Seize endless possibilities! LEARN MORE
cancel
Showing results for 
Search instead for 
Did you mean: 
eduardo_dimperio
Specialist II
Specialist II

How to break a csv into small csv's

Hi, i have a massive csv and i can't process this without crash my QS. 

So, do exist a way to divide my csv file (85 GB) into several qvd files withou need to read all csv first?

I thinked to do something like this, but after first 1000, run again and get more 1000 etc etc. 

 

Data:
First 1000 
LOAD
@1
FROM [lib://Temp(bi_bi.desenv)/F.K032001K.txt]
(txt, codepage is 28591, no labels, delimiter is ';')

 

 

Labels (1)
3 Replies
JustinDallas
Specialist III
Specialist III

You would have to do this dataprep with another tool.  You can split a CSV of that size either with a tool, or with a  Powershell/Python script.  I'm not sure how Qlik stores it's data internally, especially during reads, but wouldn't your system still choke even if you managed to split your data into multiple files?

eduardo_dimperio
Specialist II
Specialist II
Author

Hi Vegar, yes this will work as using grep in Linux or Chuck in Python. But i expected some QS solution