topic Re: Handling Complex JSON Strcutures in Talend Studio

Handling Complex JSON Strcutures

devanpeebles — Sat, 16 Nov 2024 02:56:16 GMT

We're starting to work quite a bit with more complex JSON structures, IE multiple nested arrays etc... typically we're creating these from relational data sources and shredding them back into that.

I was just using an approach of chaining tWriteJsonField or tExtractJSONField to create them, is there a better way to do so?

Which gets me what I want,

[
{
"KEY": "1",
"COL_TWO": "ROOT_RECORD",
"CHILD_JSON": [
{
"COL_ONE": "dfgsdfg"
},
{
"COL_ONE": "xcvbxcvb"
}
]
}
]

Re: Handling Complex JSON Strcutures

Anonymous — Thu, 19 Mar 2020 15:20:19 GMT

If you have the subscription version you can use Talend Data Mapper. This is a better way of dealing with complex JSON structures, but can be quite a learning curve if you've never used it before. If you are using the Open Source product, then the way you are doing this is fine. Some people might be inclined to build more complex JSON using Java (since this is an option for people with Java skills), but from what I have seen you are approaching this in a reasonable way.

Re: Handling Complex JSON Strcutures

devanpeebles — Thu, 19 Mar 2020 15:44:26 GMT

Thank you for the reply.

We do have real time big data version so we do have access to the data mapper. What would be the advantages of the data mapper in this case? I was looking at trying to use it but it didn't quite seem to work in this case where we're trying to map in multiple data flows that are a relational structure(or inversely shred json out into a relational structure).

Re: Handling Complex JSON Strcutures

manodwhb — Thu, 19 Mar 2020 17:02:30 GMT

Using Talend data mapper you can convert XML to Json, structure to Jason and many things. It would give more flexible to convert into hirachial mappings.

Re: Handling Complex JSON Strcutures

Anonymous — Thu, 19 Mar 2020 17:16:37 GMT

The Data Mapper gives you better functionality for multiple loops, arrays, etc. As I said, it is a tricky tool to get to grips with, but once you understand it is very useful. I would recommend taking a Talend Data Mapper course to get to grips with it.

Re: Handling Complex JSON Strcutures

devanpeebles — Thu, 19 Mar 2020 17:36:17 GMT

Thanks for the feedback both of you, we'll be looking at the datamapper in the future.

For now the solution we have seems to work for what we need.

Re: Handling Complex JSON Strcutures

Anonymous — Tue, 09 Jun 2020 05:31:01 GMT

@rhall

Currently I have 2 csv files and I expect the first is the json file according to the image I sent. Currently I have not thought of a solution. Can you help me

Re: Handling Complex JSON Strcutures

devanpeebles — Tue, 09 Jun 2020 13:36:14 GMT

What are you trying to do? Merge the two csv's into a json structure?

Re: Handling Complex JSON Strcutures

Anonymous — Wed, 10 Jun 2020 09:30:52 GMT

Yes, I merged 2 csv files into json files in an integrated structure. But I have found a solution. I have followed tJSONDoc
https://jan-lolling.de/talend/components/help/tJSONDoc.pdf

Thank you.