Loading QVW from multiple QVDs on the basis of entities
Parallel loading of QVD and QVR.
We have a scenario wherein data needs to be loaded on the basis of entities(companies)
The architecture to follow is as below :
AU DB ---- QVD Generator for AU --- QVDs for AU
SG DB ---- QVD Generator for SG ---- QVDs for SG Final QVW file holding data for all the companies
MX DB---- QVD Generator for MX ------ QVDs for MX
Can you help me as how to achieve loading of data from multiple QVDs holding data for various companies in a single qvw file.
There would be different tasks created for loading QVDs for different companies. During the non-business hours of companies tasks would be executed which will load the latest data for a particular company in QVD and QVW. But the data for other companies should not be removed from QVW.
Re: Loading QVW from multiple QVDs on the basis of entities
I work with a data-source that sounds very similar to yours, we use a single extract to loop over each database then each company within it, and then each table of interest and produce a QVD per each Database.Company.Table.
We can then do optimised QVD loads of that data as needed into a model, and do any modelling work etc as needed.
During the extract we use incremental loading so the first reload gets all data, and subsequent loads just get the latest data per company, based on a time stamp field in the table.
The help inside QV developer has some examples on incremental loading, and I'm sure there's lots here on the community too.