Do not input private or sensitive data. View Qlik Privacy & Cookie Policy.
Skip to main content

Announcements
Qlik Open Lakehouse is Now Generally Available! Discover the key highlights and partner resources here.
cancel
Showing results for 
Search instead for 
Did you mean: 
arulsettu
Master III
Master III

load issue

hi

i am having 20 gb of data. when i load only three fields from a table it taking 1+ hour to load. is there any way to make it fast?

what is the issue?

26 Replies
arulsettu
Master III
Master III
Author

within two min

Not applicable

I guess .

But qvd extractor is good solution, because there you can create tables one by one and this will save a lot of RAM.

arulsettu
Master III
Master III
Author

Capture.JPG.jpg

arulsettu
Master III
Master III
Author

any suggestions?

simenkg
Specialist
Specialist

Hi, Arul.

I see that you are using QlikView desktop with 2 gb RAM, as you say further down. This is most certainly the reason for the long reload time.

There are a few other things to consider here:

First of all you have to consider the rate of transfer from the SQL database. If the network connection is slow then reading from the DB will be slow. Transferring 20 gb og data over a 10 mbit line will take 4,5 hours no matter how much RAM you have.

Secondly: QlikView stores data by only keeping one instance of every distinct value. eg. if you have a date field with time stamps you can have hundreds of thousands of distinct values for the Date Field. This will explode the memory/Disk usage. I do not know whether QlikView reduces the size of the tables on the fly or whether it waits until the load is complete. If it is the first case then you might have some luck reducing the number of distinct values in your query. If you have a datefield with time stamps - consider splitting it up into one DateField and one TimeField while rounding the time to some accuracy.

But no matter how you look at it 2GB of RAM is not enough to load 20 GB of data quickly, because as soon as the 2 GB is full - the application starts swapping to disk. I will not explain it more than to say that this is extremely slow and can only be fixed with more RAM or smaller data sets.

Regards
Simen Kind Gulbrandsen

arulsettu
Master III
Master III
Author

thanks for reply

arulsettu
Master III
Master III
Author

any suggestion please help....