topic JSON Dynamic Schema processing in Talend Studio

JSON Dynamic Schema processing

ankushdeshpande — Fri, 01 Nov 2019 19:17:41 GMT

HI All,

Do we have any capability to process JSON schema dynamically? The input JSON file will be evolved with new elements over a period of time and Talend job needs to accommodate the schema changes with least code changes. any hints are appreciated. file needs to moved from one S3 bucket to another S3 bucket.

Re: JSON Dynamic Schema processing

Anonymous — Wed, 06 Nov 2019 01:45:17 GMT

Do you need some transformation? Why don't you use tS3Copy to copy the file from one bucket to another bucket?

Re: JSON Dynamic Schema processing

ankushdeshpande — Fri, 22 Nov 2019 16:24:37 GMT

Hi Shong,

We need basic transformation. but our main requirement is to dynamically extracts JSON key/value pair with minimum changes in the job. so for e.g as of today we have to extract 50 elements from JSON...later it will change to 70..100...150..and so on...so that will need change in the Job to map additional/new JSON elements.

Re: JSON Dynamic Schema processing

Anonymous — Tue, 26 Nov 2019 06:10:07 GMT

It's impossible, because we have to define the mapping between columns and the query elements when using tFileInputJson to extract data from an input json file.