topic Re: How to join data from multiple tExtractJson ? in Talend Studio

How to join data from multiple tExtractJson ?

Anonymous — Sat, 16 Nov 2024 08:43:44 GMT

I have 2 tExtractJson and want to join both result set together before importing into a database.

I tried this method:

tExtractJson1 _____>>

tMap ( trying to join both tExtractJson output )

tExtractJson2 -------->>

but it doesn't allow becasue tExtractJson is not an input datasource. Any suggestion/workaround ? Thanks.

Re: How to join data from multiple tExtractJson ?

TRF — Thu, 08 Feb 2018 20:15:14 GMT

Have look to tUnite if both tExtractJSONFields have the same schema.
If they don't, rearrange them using tMap or tFilterColumn.

Re: How to join data from multiple tExtractJson ?

Anonymous — Thu, 08 Feb 2018 21:41:17 GMT

I should have be more clear. What I want to do is not to union the data. I wanted to do a left join from tExtractJson1 to tExtractJson2 on the id field. Is there any way ?

I tried using tMap to join but it doesn't allow because tExtractJson is not an input source.

Thanks.

Re: How to join data from multiple tExtractJson ?

TRF — Thu, 08 Feb 2018 21:53:11 GMT

Are you speaking about tExtractJSONFields?
Can you share your job design?

Re: How to join data from multiple tExtractJson ?

Anonymous — Thu, 08 Feb 2018 22:16:28 GMT

Yes. I am trying to do something like that setting multiple tExtractJsonFields as an input to tMap so I can do a left join from 1sr tExtractJsonFields to the 2nd tExtractJsonFields. Is there a workaround ? Thanks.

Capture.JPG

Re: How to join data from multiple tExtractJson ?

TRF — Thu, 08 Feb 2018 22:29:48 GMT

It seems both tExtractJSONFields are issued from the same input (tXMLMap?)
If so, you cannot join them in the same subjob where they have been created.
Store the result of each in a dedicated tHashOutput then on the next subjob get the rows from the corresponding tHashInput and make join in a tMap.

Re: How to join data from multiple tExtractJson ?

TRF — Thu, 08 Feb 2018 22:34:13 GMT

This topic may also help https://community.talend.com/t5/Design-and-Development/Multiple-input-links-merged/m-p/34854#M8939

Re: How to join data from multiple tExtractJson ?

vapukov — Fri, 09 Feb 2018 00:34:48 GMT

@TRF wrote:
It seems both tExtractJSONFields are issued from the same input (tXMLMap?)
If so, you cannot join them in the same subjob where they have been created.
Store the result of each in a dedicated tHashOutput then on the next subjob get the rows from the corresponding tHashInput and make join in a tMap.

as variant also - store to csv for avoid "out of memory" problems with huge json

Re: How to join data from multiple tExtractJson ?

Anonymous — Fri, 09 Feb 2018 04:32:04 GMT

I tried using tHashOut / tHashInput, but somehow still wouldn't allow me to connect, so I used the file method. If I have to export a HUGE file and then read the file right after, will the "SubJobOK" step make sure that the file is "done" exporting before the process reads it ? Below is a sample design, please feel free to comment if there is a better way of doing things. Thanks.