I often choose to split my QlikView development into three stages, Extract, Transform and Load (ETL).
Extract: Read and store source data without modification into a QVD Extract folder.
Transform: Read the QVD Extract folder and do all my manipulation and transformation of data and store it into a QVD Transform folder.
Load: Read the transform qvd files without any modification
LOAD * FROM my.qvd (qvd);
Load application are my end user application. The focus in my load applications are user interface and layouts, not data transformation.
One benefit from splitting the data loads into several applications is that you can reuse data created in E and T. You may want to load certain data frequently every 10 minutes, but other data less frequently every night. That you can do by splitting up your data loads.
Examplefield1 as [Field 1],
Date(Exampledate) as [Date],
Monthname(Exampledate) as [Month name]
ExampleValue as [Value]
FROM C:\ExtractFolder\Example.qvd (qvd);
Store Example into c:\TransformFolder\Transaction.qvd (qvd);
Hi Steven and Vegar.
I completely agree with Vegar and that is how we do for our company. The only thing I would add to that, is that we create different extractions and set of QVDs for each business area of the company.
We created a QVF to extract sales that from different sources every hour. In this QVF we combined all extracts for sales in different tabs to organize.
Then we created another QVF extract inventory data every day.
Separating your extracts by creating different QVFs, you don't have to run all extracts every time if it is not necessary, avoiding over loading the server memory.
I hope that helps.