topic Issue with TFileProperties in Talend Studio

Issue with TFileProperties

Jeff__FR_ — Mon, 03 Feb 2020 12:33:22 GMT

Hello,

I'm new with Talend.

I have the following issue ; I want to store files with their properties (specially modification date) in a database table. For that, I use

TFileList ---> Iterate ----> TFileProperties -->main ---> TDBOutPut

The file propertie of my TFileList is ((String)globalMap.get("tFileList_1_CURRENT_FILEPATH"))

In my TDBOutput, f I parameter the "table action" to default, it works but I want to delete table before each run so I parameter the 'table action" to "delete the table" and then, after running I only get information for the last file treated.

Someone can explain it to me why ?

Thanks by advance for your help.

Re: Issue with TFileProperties

Anonymous — Mon, 03 Feb 2020 12:40:12 GMT

This is because your tFileList is iterating over your files. When an iterator link is used, everything after it is run from component start to component complete for every iteration. So, you are deleting and creating a new table for every file that is found.

To get around this, simply add a tHashOutput where your DB component is, then in a subjob below your current subjob, add a tHashInput joining to your DB component. What this does is fill your Hash component with your file details, then it sends them to the DB component all in one go after the files have been analysed. You must set the tHashOutput component to "append" for this to work.

Re: Issue with TFileProperties

Jeff__FR_ — Mon, 03 Feb 2020 22:28:28 GMT

Thanks a lot ; it works.

In parallel, I tried another solution. I added a TDBRow which delete the table before my job (TDBRow - TFileList -- TFileProperties -- TDBOutput). It works too.

What do you think about this alternative way of doing (I see a difference on stats: with your solution we easily can see the number of files inserted in the table) ?

Thanks for your attention.

Re: Issue with TFileProperties

Anonymous — Mon, 03 Feb 2020 22:52:52 GMT

Whichever way suits you will be fine. This pointer will hopefully help you when dealing with similar situations in the future. Getting used to how Iteration links work can give you a great deal of power when working with Talend. Understanding the flow is key to unlocking the full power of a Talend job 🙂