Surely what you are suggesting would mean me reading the whole of the required table into the tjoin or tmap, and not just selecting the required record from the database ?
I have a file being read that contains a single column (row_id), for each record in the flow I want to query a database for something like;
select rating_id from rating where row_id = the row_id from the input flow
The output flow will look like, row_id, rating_id
You initially suggested a join or map, but this would mean reading all the records from the database ? If the table has 3.5 million records but the flow only 100 then this would not be very efficient.
You're right, tMap load all your lookup table in memory or on disk.
I perfectly understand your request. The solution to resolve your problem is to use the difference between iterate and flow links :
To anwser to your request, you should design a job like
tFileInputDelimited -> tFlowToIterate -> (iterate) -> tMySQLinput -> ...
HTH,
