When the data is stored in QVD files either through Buffer or the Store Statement. Does the data is stored in RAM or it will be there in the hard disk?
As per the documentation it is given as it will be used to store the slow changing data to reduce the load time and for incremental upload . But if the data is stored in RAM. Then the same cane be done through load statement.
Please correct me if i am wrong and tell me the advantage of QVD
QVD files store all of the data in a file. They are not in RAM as they are essentially permanent. In your application, you load the data into RAM just like you would from an Excel file or database. In some cases the QVD can be an optimized load, which greatly decreases load times.
QVDs have many advantages, but once they are loaded into your application, the results shouldn't be much different than data loaded from other sources.
Since the QVD can be created using the Buffer and the Store statements. While using the Buffer keyword to create the QVD the data will be there in both RAM and QVD or only in QVD. I think if we are using the Store statement to create the QVD the data will not be there in RAM.
We use separate applications to build our QVDs. These applications load data from the databases and store it in a QVD, but don't do anything else (no sheet objects, charts, etc.). When these applications are reloaded, the data is in the QVD and in RAM. Once the QVDs are built, these applications can then be closed, freeing up the RAM.
We then go to our front end applications. These applications don't access the databases, but instead load all data from the QVDs. When these applications are reloaded (from QVD), the data is once again back in RAM and it is here where we are building our charts and such.
When I build a QVD building application, I make sure that it has no data stored in memory, to keep the file sizes down. I'll build the load script until it is working how I want and save the file empty. Then I will reload the application, which writes the data to QVD, but I do not save afterwards. This ensures that the QVD building application is empty and the filesize is small.
Using the Store statement or the Buffer statement will create a QVD file on the hard disk. The difference between the two statements is where QlikView will write the file to. With the Store statement you can control where the QVD file is written to. With the buffer statement, Qlikview decides where the QVD file is written to. You can check where this is written by going to User Preferences -> Locations and view the QVD Buffers location. This is where the QVD's are stored. These are stored using an encrypted file name.
Normally, your using your QVD for your final QVW. So you want to build your QVD's before getting to that point. Usually what I would do is like NMiller said. Use a seperate QVW to build the QVD's that your going to use in your final QVW. So I would have a QVW to build the data model that I'm going to use in my final user presentation QVW.
As far as automating you can setup a batch job to do that, or you could use Publisher to schedule it for you.