@GuptaN this question is about 'Using a file as a source' endpoint, chapter 8.4 in the Replicate user guise correct?
I suspect you'll want to generate and use 'reference files'.
For some application, and it feels like this is a candidate, you may want to use the CDC process, even for full-load.
Look at all the optional elements in the chance source:
Each row in a Change File consists of the following delimited columns: l (Optional) The change operation e.g. DELETE. If the operation field is absent, INSERT is assumed. l The name of the target table to which to apply the change (only required if the Change File contains changes for multiple tables) l (Optional) The timestamp of the change i.e. when the change occurred l (Optional) The user who applied the change l The data to change (one or more columns)
Therefor is you use a reference file with the tablename and CDC source file name, that source file only needs to have raw data rows. No metadata required.
In the past, for other solutions, I have used a pre-processing command to take data as provided and provide CDC metadata as needed based on file and directory naming conventions. Much more work obviously.
@GuptaN this question is about 'Using a file as a source' endpoint, chapter 8.4 in the Replicate user guise correct?
I suspect you'll want to generate and use 'reference files'.
For some application, and it feels like this is a candidate, you may want to use the CDC process, even for full-load.
Look at all the optional elements in the chance source:
Each row in a Change File consists of the following delimited columns: l (Optional) The change operation e.g. DELETE. If the operation field is absent, INSERT is assumed. l The name of the target table to which to apply the change (only required if the Change File contains changes for multiple tables) l (Optional) The timestamp of the change i.e. when the change occurred l (Optional) The user who applied the change l The data to change (one or more columns)
Therefor is you use a reference file with the tablename and CDC source file name, that source file only needs to have raw data rows. No metadata required.
In the past, for other solutions, I have used a pre-processing command to take data as provided and provide CDC metadata as needed based on file and directory naming conventions. Much more work obviously.
