topic Re: Equivalent to Ab Initio Join with DB Component in Talend Studio

Equivalent to Ab Initio Join with DB Component

Anonymous — Sat, 16 Nov 2024 14:08:30 GMT

Hi,

I have a flow of data that I need to join to a database, selecting * where database.field = input.field

I am using informix

Any ideas please,

John.

Re: Equivalent to Ab Initio Join with DB Component

Anonymous — Fri, 28 Nov 2008 14:08:08 GMT

This can be done very easily with the tMap or with the tJoin.
Regards,

Re: Equivalent to Ab Initio Join with DB Component

Anonymous — Fri, 28 Nov 2008 14:19:06 GMT

Thanks for the response

Surely what you are suggesting would mean me reading the whole of the required table into the tjoin or tmap, and not just selecting the required record from the database ?

Re: Equivalent to Ab Initio Join with DB Component

Anonymous — Fri, 28 Nov 2008 18:55:18 GMT

Can you give precise details about what you really want to do ?

Re: Equivalent to Ab Initio Join with DB Component

Anonymous — Sun, 30 Nov 2008 14:13:11 GMT

Thanks for the response,

I have a file being read that contains a single column (row_id), for each record in the flow I want to query a database for something like;

select rating_id from rating where row_id = the row_id from the input flow

The output flow will look like, row_id, rating_id

You initially suggested a join or map, but this would mean reading all the records from the database ? If the table has 3.5 million records but the flow only 100 then this would not be very efficient.

I hope this makes sense, appreciate your help.

Re: Equivalent to Ab Initio Join with DB Component

Anonymous — Sun, 30 Nov 2008 22:46:46 GMT

You're right, tMap load all your lookup table in memory or on disk.
I perfectly understand your request. The solution to resolve your problem is to use the difference between iterate and flow links :
To anwser to your request, you should design a job like
tFileInputDelimited -> tFlowToIterate -> (iterate) -> tMySQLinput -> ...
HTH,

Re: Equivalent to Ab Initio Join with DB Component

Anonymous — Thu, 11 Dec 2008 08:54:56 GMT

yep, this works fine

thanks for your help