If you want to get IDs from "1%" to "99%" add a tLoop before tTeradataInput and change the select like this: "Select ID from schema_name.table_name where ID like '"+((Integer)globalMap.get("tLoop_1_CURRENT_VALUE"))+"%'"
You may also explain the reason why you want to proceed incrementaly for better proposal.
Your 1st question wasn't the same or wasn't clear. Try this: Select ID from myTable where myDate > current_date Check the syntax for teradata SQL query.
Thanks , but first time it will be full load then it will be incremental load , will this logic work in this scenario? will i have to store the max date and then compare it with current date or something like that ? can you let me know steps for the job ?
Sure in this case you need to have a logic as you suggest, store the last queried date and if it doesn't exists start with a date such as 1900-01-01 to retrieve all the records.
One more time, try to explain your case completly at the 1st question to avoid bad answers.
On daily basis that query will run and will pull the records incrementally , 1st run will pull entire data set and from next day it will pull records incrementally based on date ,
Please let me know steps how shall implement this CDC logic
To load records created since the last load, you need to store the last run time somewhere. You can write it to a file or to a database. At the start of your load job, read in the value from where you stored it, assign it to a variable (replacing a null value as TRF suggests), then use that variable in your query. After the load is complete, you write the current timestamp to your data store for the next run.
The first time it runs, you won't have a value, so you'll replace it with 1900-01-01 which should retrieve all records. Subsequent runs will use the timestamp you store instead of 1900-01-01.
