Question on tFileList and tMSSQLInput

Sanjay5 — Sat, 16 Nov 2024 09:52:00 GMT

Hi,

I am working on a job where I need:

1. Read list of distinct files from SQL database. (Say 10 files are already processed)

2. Read the list of files from Shared directory. (2 new files are arrived, so total 12 files, 10 Old + 2 new files)

3. Process the files which are newly added to the directory. (Process only 2 newly added files)

Note: I can't delete the existing files in share directory.

I am using tMSSQLInput to read list of file names as below

"select DISTINCT FileName FROM MBAMDetail"

This query returns all 10 existing files, now I am not able to exclude these 10 files while using tFileList and process only 2 newly added files

Can someone help on this?

Re: Question on tFileList and tMSSQLInput

cterenzi — Wed, 26 Apr 2017 00:07:05 GMT

Connecting tFileList to tIterateToFlow will let you treat the file list as a flow and compare it to the query results with a tMap. If you create a tMap output and set it to catch inner join rejects, you can then iterate over that list to process only new files.

topic Question on tFileList and tMSSQLInput in Talend Studio

Question on tFileList and tMSSQLInput

Re: Question on tFileList and tMSSQLInput