Experts ,
We developed sales report for phase1 which covers the data from Aug 2016 to July 2017 from Aug onwards they have added the new columns to few fact tables in the data base side. now we have to report them as well.
we are using 2 tier architecture one is extractor to generate the qvds and other is the final app with UI.
2 methods we have right now :
1. add the new columns to the Phase1 extractor and convert them into QVDs - but here we need to apply some conditions for the data before the August 2017 new columns are not available. so we need a conditional load for this seanario and also some join logic is changing with the addition of new columns.
2. we have old data in qvds now ( Aug 2016 to July 2017), use a different extractor to extract the tables with new columns from august-17 to last month and concatinate both of them and generate the final Qvds for the final app.
But here the big concern is data is very huge each QVD is of 1 to 2 GB's like that we have 2 fact tables and total of 25 QVDds. so we may face the server over load if we app for the
Please suggest the best method, here am worrying about load time it will take so want to go for the best method so that can avoid delays in delivery. Thanks in advance.
please let me know if any more information required.
