topic Re: tFileProperties as Lookup in tMap how to match the fields when reading from tFileList in Talend Studio

tFileProperties as Lookup in tMap how to match the fields when reading from tFileList

mobmsc2 — Tue, 02 Oct 2018 17:26:30 GMT

I have a tFileList which iterates through a directory of excel files and a tMap to append the CurrentPath, Directory to the flow.

For each file I read the contents but I want to include the modified date as well so I have the tFileProperties as lookup into a 2nd tmap and I use ((String)globalMap.get("tFileList_1_CURRENT_FILEPATH")) in the tFileInputExcel and tFileProperties

I tried matching the CurrentPath [set in tMap1 using ((String)globalMap.get("tFileList_1_CURRENT_FILEPATH")) ] to the tFileProperties basepath and abspath but my output doesn't match for any,

I also changed the tMap lookupModel to reload at each row.

I guess in a nutshell how do I match the tFileProperties to the flow of data out of the same File?

Re: tFileProperties as Lookup in tMap how to match the fields when reading from tFileList

fdenis — Wed, 03 Oct 2018 09:13:40 GMT

did you use reload for each row for lockup on properties?
I do not know if you can retrieve properties on an open file!

Re: tFileProperties as Lookup in tMap how to match the fields when reading from tFileList

mobmsc2 — Wed, 03 Oct 2018 09:47:25 GMT

I did use reload for each row for lookup.

My job is outlined below. If its not possible to get properties on an open file how do I re-engineer the job so I don't end up getting the properties but not the data rows (assuming I can't ReadExcel if the file is open for tFileProperties

ReadFolder -> ReadExcel->tMap(addSourceFileDataToRow) ->tMap(GetModifiedDate) ->

tFileProperties

Re: tFileProperties as Lookup in tMap how to match the fields when reading from tFileList

Anonymous — Mon, 08 Oct 2018 03:08:28 GMT

Hi mobmsc2
Read the file properties and cache them into memory for used later, change the job as below:
ReadFolder -> tfileProperties--tHashoutput--oncomponentok-
ReadExcel->tMap(addSourceFileDataToRow) ->tMap(GetModifiedDate) ->

|

|

tHashInput
Let me know if it works.

Regards
Shong

Re: tFileProperties as Lookup in tMap how to match the fields when reading from tFileList

mobmsc2 — Mon, 08 Oct 2018 12:28:13 GMT

That didn't work. If I run in debug mode there are no rows coming in from the lookup hash flow so nothing to match against the flow from the tExcel. Thats even though there there is content read into the hashOutput

Re: tFileProperties as Lookup in tMap how to match the fields when reading from tFileList

Anonymous — Wed, 10 Oct 2018 07:46:52 GMT

Change the job design as below:
main job:
tfilelist--iterate--tRunJob

on tRunjob: call the child job and pass the current file path to child job, refer to this page to know how to pass a value from main job to child job.
https://help.talend.com/reader/Wi~~2Q0v6A43uBlbET9gQg/MHs41sIsaV91cFGXj4DVMQ

child job:
tfileProperties--tHashoutput
|
onsubjobok
|
ReadExcel->tMap(addSourceFileDataToRow) ->tMap(GetModifiedDate) ->

|

|

tHashInput
This way should work.

Regards
Shong