Unlock a world of possibilities! Login now and discover the exclusive benefits awaiting you.
Hi all
I got a problem which really confuse me.
I have two .qvw files, one is of size 6M and another one is of size 25M, it's a wonder that the data model (in memory) of these two files are exactlly the same ! (same tables with same fileds and records)
difference is , file with 25M was loading from excel and doing some data transfer while file with 6M was loading from a qvd file by samplly "load * "
but I really don't think this would make any difference on qvw document size
experts , please help to explain this phenomenon . thanks in advance
Timestamps could require a lot of resources and any change of the numeric value and/or their formatting will have an impact of how may RAM/Storage will be consumed. Here some background to this topic: The Importance Of Being Distinct.
- Marcus
Hi
That is the difference. Excel loading will take more memory than loading QVD. QVD loading is Compressed format.
Hope it helps!!
QVD is a native Qlik format and can only be written to and read by QlikView. The file format is optimized for
speed when reading data from a script but it is still very compact. Reading data from a QVD file is
typically 10-100 times faster than reading from other data sources.
Hi Anil
thx for your reply
Excel loading will take more memory than loading QVD, are you sure about this ?
I did a test just now , firstly load from Excel with about 800k records and generate a qvd, save qvw , it takes 15M,
then create a new qvw loading from qvd file , it takes 15M too...
I believe loading from qvd faster than reading from Excel . but they should take same memory when loading finished I suppose.
Yes, Loading from QVD is faster than Excel.
Hi Dave Lau,
I recommend to read some of the discussions and reply by some experts on the topic below and it will be very helpful.
You need some patience in reading the below threads, but all of them are from individuals experience and it will answer most of your questions.
Large amount of data (120 million rows) ... anyone got experience?
How to Handel Large data set while loading in qlikview.
*** 6 Weeks in to QV Development, 30 Million Records QV Document and Help Needed!!! ****
How much data can QlikView handle?
Handling Big Fact table. | Qlik Community
BI Review: A few tips for dealing with large QlikView applications
-Siva
Things you can check in both documents:
The fact that your experiment with an Excel source file and a QVD produced from this Excel results in documents with the exact same size, leads me to think that you were initially comparing different data sets.
Yes..there is something different in time values, but rows ,columns and tables are totally the same.
Do you meant that timestamps would take more memory?
Timestamps could require a lot of resources and any change of the numeric value and/or their formatting will have an impact of how may RAM/Storage will be consumed. Here some background to this topic: The Importance Of Being Distinct.
- Marcus
Yes, Time Stamp consumes more memory