topic Re: Do lookup after file read in Talend Studio

Do lookup after file read

tale103108 — Thu, 01 Oct 2015 18:32:37 GMT

Talend 6.x
Fedora Linux 21
Sybase 12.x
+++++++++

Can a lookup be delayed to only work after data is read from an input flat file? Seems my lookup works before my input file is read.
I can see why this is necessary in most instances but can Talend be "scheduled" such that:

- a flat file is read
- then a lookup is executed
- then flat file data is passed to the lookup

thx.

Re: Do lookup after file read

Anonymous — Fri, 02 Oct 2015 16:04:04 GMT

hi,

Seems my lookup works before my input file is read.

you're right
in most cases (default behavior) , all datas on a lookup are stored in memory, waiting for data (id) coming from main flow.

ps : I don't understand your workflow

have a look at different way you can manage tmap & flow& memory :
https://help.talend.com/search/all?query=tMap+operation

regard
laurent

Re: Do lookup after file read

Anonymous — Fri, 02 Oct 2015 16:32:06 GMT

Like Laurent, I don't understand why you would want to do this. But you can.
1) Read the file into a tHashOutput component in subjob that is fired first.
2) Use a tHashInput to start the next subjob (set the Input to return the rows read into the Output)
3) Connect the tHashInput to your tMap which has the lookup linked

The lookup will still fire before the main input is loaded into the tMap, but the file will have been read in the previous subjob.

Re: Do lookup after file read

tale103108 — Mon, 05 Oct 2015 20:45:30 GMT

Hi rhall_2.0
Thanks for the input. I want to do this so that (using your example) what is read from the file into the tHashOutput are context variables or global variables that can be used in the tMap lookup. The tMap lookup is dynamic and requires the data from the same file that is being read and uses the tMap lookup later on in the processing.
To put it another way:
1.) read file A to populate lookup SQL with context or global variables that completes the SQL for the lookup
2.) then read file A again and use tMap lookup (which is executed before file A is read the second time) to get some data to pass along to the next step
3.) iterate steps 1.) and 2.) for each row in file A until EOF