I have a requirement. In my source we have 1000(example) records and performing logic and loading into database.
The job failed after loading 450 records. Now I want to start loading/ reading the source file from 451 to 1000 and It should not read 1-450(which already loaded). Please suggest how can we achieve this using talend.
If you just want to ignore the n first records from your input file (that's your request), you may have a context variable set to 1 by default ignore the header if any (set to 0 in case of no header), and set the number of records to ignore (450 in your example) + 1 if you have a header.
Then, use this variable to populate the tFileInputDelimited "Header" field.
If you just want to ignore the n first records from your input file (that's your request), you may have a context variable set to 1 by default ignore the header if any (set to 0 in case of no header), and set the number of records to ignore (450 in your example) + 1 if you have a header.
Then, use this variable to populate the tFileInputDelimited "Header" field.
1) If you have a unique key in your delimited file, then you can apply a join (inner join) between the file and the database table in which you are inserting the records. That can be done through tMap. In this way it will escape all the records that are already inserted into the table.
2) Its not the ideal way, but you can set the header value to 451 (If you have headers) OR 450 (If you don't have headers). In this way it will escape the first 450 records.
From the information you've provided, and assuming this is a one-off exercise, you just need to temporarily change the Header property in your tFileInputDelimited to 450, as suggested by @TRF ad @abhishek.
If there are actual header lines in the source file, then you'll also need to add the number of header lines, so e.g. if you have a two line header, the Header value should be 452.
