topic Re: Incremental Load By String, NVARCHAR, Text Field in Data Movement & Streaming

Incremental Load By String, NVARCHAR, Text Field

Shako — Wed, 19 Mar 2025 23:06:21 GMT

Hi,

Is there way to implement incremental load without having Date field and Key column is NVARCHAR. Database is SQL

I want to Load only New Rows. No update or Delete is Required

The field example:

DocNo

PSI16_0000001

PSI16_0000002

PSI16_0001003

Re: Incremental Load By String, NVARCHAR, Text Field

rubenmarin — Wed, 05 Oct 2022 10:51:29 GMT

Hi, that field looks like a number, if it's always increasing that number you can use it to do the incremental load.

Loading from qvd the max value will be easy, using Right() or Subfield() to retrieve the higuer value loaded.

A Where condition should be applied on sql using that value, syntax will depend on the language used by the database, it could be something like: Where Right(DocNo,7)>=$(vHigherValueFromQvd)

Re: Incremental Load By String, NVARCHAR, Text Field

Shako — Thu, 06 Oct 2022 11:07:43 GMT

I Checked key field and it may change not only numerically , for example:

PMI16_0000001

PSI16_0000001

Re: Incremental Load By String, NVARCHAR, Text Field

rubenmarin — Thu, 06 Oct 2022 13:38:48 GMT

At the end the question is, can you identify in some way which records are new based on the data previosly loaded?. If you can do it manually then there is an option to do it automatic,if there is no way you can do it only with the last DocNo then there is not possible.

If the initial character have a limited combimnations you can do a bucle to load each prefix separatedly. It it will be more dinamic, you can also recognize them while loading the data, do a list, a bucle based ont hat list... but maybe all this extra process makes the incremental load take more time than anormal load.