This content has been marked as final. Show 2 replies
I have a requirement to take a group of monthly files, say:
They are of different types and contain different data, but I merge them in the scripts and have a final output into a QVD file.
The next month I would get a new group of files in the same directory, say:
I need to be able to load the new files and check if the data already exists in the QVD, I would do this by loading the QVD with a date column made up from the last 8 characters of the filename.
I want to be able to write the solution to do an incremental load, or a full reload using all the files in the directory, but one group at a time.
So, I was thinking, and would welcome some help please:
If qvd does not exist then a full load can take place by looping through the group files
If qvd exists then select out of the qvd count of records for the date being processed
If no records then incremental load
Has anyone got example scripts that processes files in groups (or collections) in an iterative manner please ?