topic Best approach to import relational data into complex JSON in Talend Studio

Best approach to import relational data into complex JSON

Anonymous — Sat, 16 Nov 2024 10:53:42 GMT

Hi,

I would like to know the general approach to import relational data into mongoDB. I want to create a very complex document which uses all the one to many relationships in my relational database, so the document will be very complex and multiple levels deep.

I am not sure of the approach, as the closest article I have come across seems to indicate you need to interface with the mongoDB API manually, but then fails to elaborate on how to add complexity to the document.

http://open-bi.blogspot.co.uk/2013/10/loading-mongodb-with-talend-easy-way.html

I managed to join 2 tables in a query, and create a tWriteJSONFields component to create an array for the many side of the join, but I seem able to only add one loop element. I am likely going to need to bring together 20 tables into a complex document.

As a side matter, my log component created beautiful JSON, but ,my downstream input into mongoDB...all the documents have the double quotes escaped with a backslash....but that's a separate issue.

Any advice, links appreciated.

Stephen

Re: Best approach to import relational data into complex JSON

Anonymous — Wed, 23 Dec 2015 09:08:21 GMT

The approach I hear going around is two fold.
1) De-Normalize to the lowest level
2) If you have two or more loop at the lowest level do multiple passes mapping & merging

Re: Best approach to import relational data into complex JSON

Anonymous — Wed, 23 Dec 2015 09:15:23 GMT

Hi,

Thanks - but I probably need more detail. What would be great is a reverse of this article:

http://community.jaspersoft.com/blog/dealing-nested-documents-mongodb-and-talend-aka-baking-cake

Not asking you to write it of course 😉 but I am new to Talend, and as this is a complicated task, mapping and merging is not enough detail for me - I wish it was ;-(.

Regards
Stephen