Hello All,
I have my Source data in the following structure.
| ID | part Info |
| 1 | /year=2019/month=09/day=17/hour=16/ |
| 2 | /year=2019/month=09/day=17/hour=16/ |
| 3 | /year=2019/month=09/day=17/hour=16/ |
| 4 | /year=2019/month=09/day=17/hour=17/ |
| 2 | /year=2019/month=09/day=17/hour=17/ |
| 3 | /year=2019/month=09/day=17/hour=17/ |
| 1 | /year=2019/month=09/day=18/hour=12/ |
| 2 | /year=2019/month=09/day=18/hour=12/ |
| 1 | /year=2019/month=09/day=19/hour=17/ |
I would like to perform a Delta load and save each partinfo as a QVD file. The QVD files should have the data like in the below table .
| ID | part Info | QVD Name |
| 1 | /year=2019/month=09/day=17/hour=16/ | data_2019091716.qvd |
| 2 | /year=2019/month=09/day=17/hour=16/ | data_2019091716.qvd |
| 3 | /year=2019/month=09/day=17/hour=16/ | data_2019091716.qvd |
| 4 | /year=2019/month=09/day=17/hour=17/ | data_2019091717.qvd |
| 2 | /year=2019/month=09/day=17/hour=17/ | data_2019091717.qvd |
| 3 | /year=2019/month=09/day=17/hour=17/ | data_2019091717.qvd |
| 1 | /year=2019/month=09/day=18/hour=12/ | data_2019091812.qvd |
| 2 | /year=2019/month=09/day=18/hour=12/ | data_2019091812.qvd |
| 1 | /year=2019/month=09/day=19/hour=17/ | data_2019091917.qvd |
Basically, i have so far generated Date Time (Ex: 20190917-16) from the part Info field and tried to paas it on as a Variable in the select statement and also in the Store qvd Statement.
Also, each time the Delta Load should only fetch records which are not already stored in any of the QVDs.
Let's say the baove table is already saved as QVDs with names as mentioned in the QVD Name column. And there is a new entry in the source table
| ID | part Info |
| 1 | /year=2019/month=09/day=19/hour=23/ |
| 2 | /year=2019/month=09/day=20/hour=10/ |
Now the QVD Generator should only read both these records and generate two new QVD files.
| ID | part Info | QVD Name |
| 1 | /year=2019/month=09/day=19/hour=23/ | data_2019091923.qvd |
| 2 | /year=2019/month=09/day=20/hour=10/ | data_2019092010.qvd |
My QVD Generator has to do two things here: Chekc for part Info in Source Table which is greater than the existing QVDs and save the data with similar part Info. in a single QVD file.
Please find attached Sample data and Sample Application. Any ideas would be really helpful.
Thanks and Best Regards
Sai.
