topic Re: Dynamic json schema in Talend Studio

Dynamic json schema

DVasiljevic1698407385 — Fri, 15 Nov 2024 21:17:44 GMT

I am currently trying to transfer data from an Oracle database to c4c via a rest client component. I use tdbinpout component to read the data, then tflowtoiterate, twritejson file, tmap and finally trestclient.

This is only possible if I hardcode the columns in the json file, but the number of columns in the source table can change and I want to read them dynamically. Since the twritejson component does not support a dynamic data type, I would like to know if there is a different approach I can use?

Re: Dynamic json schema

Anonymous — Mon, 30 Oct 2023 01:34:03 GMT

Hello,

So far, tWriteJSONField component only accepts a hard coded schema and does not support the dynamic datatype. This feature would make a lot of sense, however there's no easy way to transform JSON into a dynamic schema. JSON can have no or multiple loops. For this reason it's really hard to transform it to Dynamic schema.

It could be possible to create a routine that would:

public static Dynamic jsonToDynamic(String myJson)

But then you'd likely require a way to split the json into multiple jsons (i.e handle the loop) which could be achieved via tJavaFlex

Here is also a tWriteDynamicFields component which creates an output dynamic column from input columns.

https://help.talend.com/r/en-US/8.0/processing/twritedynamicfields

Hope it could help.

Best regards

Sabrina

Re: Dynamic json schema

jlolling — Mon, 30 Oct 2023 20:26:30 GMT

It is possible using Java code. The idea is to get the Dynamic-Object and iterate through the columns and get the name and the value. Talend Studio ships with a JSON library and this can be used to create the json object per record.

Re: Dynamic json schema

DVasiljevic1698407385 — Wed, 01 Nov 2023 10:25:16 GMT

Thanks for your answer. Can you give me more details or an example if you have done this before?

I don't have enough experience in java.