topic Re: Merging Data from conditional REST call into flow in Talend Studio

Merging Data from conditional REST call into flow

Anonymous — Wed, 03 Apr 2019 22:37:31 GMT

Hi, I'm new at Talend (and ETL), so apologies if I'm trying to do something that doesn't make sense.

I'm trying to conditionally add data from a REST call for some of the records I'm processing.

This is what I'm trying to achieve, with the red line being the lookup I'd like to add, but doesn't seem to be possible:

I'm conditionally making a REST call for some of the records from the first iterate, and then I want to create a combine flow of the original data plus the results from the rest call (with those columns being null for the records which didn't make the REST call) - effectively making the output from the REST call a lookup on the main flow, without having to store the results to a table or file.

Given that the output from the tExtractXML is a flow per iteration (where I really need to add some logic to retrieve the first match from the XML), I think it makes sense that I wouldn't just be able to combine that with a flow for the entire result set, but I'm stuck trying to figure out how to do this via the Talend components rather than just making the REST call part of a Java function and hiding the details away in something like tJavaRow.

Thanks,

Andrew

Re: Merging Data from conditional REST call into flow

DataTeam1 — Wed, 03 Apr 2019 23:48:55 GMT

Hi @AndrewMay you can store the data from tJavaRow_1 in tHashOutput component and then you can use tHashInput component with your tMap_3

Here is info showing where to find tHash... component https://community.talend.com/t5/Migration-Configuration-and/Where-are-the-tHashInput-and-tHashOutput-components/ta-p/309

and here you can see how to use it: https://www.youtube.com/watch?v=V5zqUtzQbBk

Don't forget to give kudos when a reply is helpful and when your query is answered, please mark the topic as resolved.

Re: Merging Data from conditional REST call into flow

Anonymous — Thu, 04 Apr 2019 14:03:51 GMT

Thanks, I think that puts me on the right track (and I wouldn't have found where those components were without the link).

I hit a few snags when attempting to implement this, but I think that's all due to the order things are initialized within a subjob (i.e. the lookups are done early) - I think I can work around that by doing my REST calls in a subjob before I use the results as a lookup.

Re: Merging Data from conditional REST call into flow

DataTeam1 — Thu, 04 Apr 2019 23:30:39 GMT

@AndrewMay do you mean subjob or child job?

Subjob is within a job (a few components joined together).

Child job is a separated job you can run within another job by tRun component.

Sometimes we create complex separated job and we run it within another job (parent job)

If you want to use child job you will need to use tBufferInput and tBufferOutput components to get data from child job instead of tHash... components which you can use within a job to get data from sub job..