How to separate json array elements in separate json outputs?

adbdkb — Wed, 18 Dec 2019 00:36:03 GMT

I have an input json file that contains array elements for an object. The number of elements in the array varies from 0 to a max of 15.

I need to create as many output jsons as there are elements in the array. Each output will also contain common elements from the input json.

Is it possible to create multiple outputs - each containing one element of the array?

Input JSON

{
  "id" : "c064e784-5d3b-4d78-9451-eae608ad66a9",
  "format" : "AVRO",
  "EventMessage" : {
    "SourceSystem" : {
      "eventOccurredTime" : "10-04-0009",
      "id" : 1,
      "name" : "FAST",
      "sourceCorrelationId" : "bac5de63-ba46-4aaf-a508-101f41a22d1e"
    },
    "Domain" : {
      "id" : 1,
      "name" : "Policy",
      "type" : "Raw"
    },
    "EventAttributes" : {
      "id" : "522533ce-fe82-4e73-bfbc-52cacb2c7903",
      "msgType" : "Raw",
      "correlationId" : "bac5de63-ba46-4aaf-a508-101f41a22d1e",
      "eventCreatedTime" : "10-06-0016",
      "eventName" : "PolicyFinancialTrx",
      "parentEventName" : "FastFinancialTrx",
      "parentEventOccurredTime" : "10-04-0009",
      "DataElements" : {
        "TrxHdrDisburse" : {
          "TaxDtl" : [ {
            "ID" : "d42f00f2-2f70-4fed-bcb3-035b2d1febbd",
            "DisbHdrID" : "f3b1ee06-fe01-470d-ae59-e1372788d11b",
            "DeductionType" : "FederalTax",
            "TaxDedPct" : "0",
            "TaxDedAmt" : "0",
            "TaxableGain" : "0",
            "WithholdAmt" : "0",
            "TaxDtlParty" : {
              "TaxDtlID" : "d42f00f2-2f70-4fed-bcb3-035b2d1febbd",
              "PartyID" : "260bebec-d70f-4d69-b702-817e12acffbe",
              "CustomerProfileGUID" : "260bebec-d70f-4d69-b702-817e12acffbe",
              "BeneficiaryGUID" : "260bebec-d70f-4d69-b702-817e12acffbe"
            }
          }, {
            "ID" : "eb333766-51fa-4b0c-99a8-a30c0379d454",
            "DisbHdrID" : "f3b1ee06-fe01-470d-ae59-e1372788d11b",
            "DeductionType" : "StateTax",
            "TaxDedPct" : "0",
            "TaxDedAmt" : "0",
            "TaxableGain" : "0",
            "WithholdAmt" : "0",
            "TaxDtlParty" : {
              "TaxDtlID" : "eb333766-51fa-4b0c-99a8-a30c0379d454",
              "PartyID" : "260bebec-d70f-4d69-b702-817e12acffbe",
              "CustomerProfileGUID" : "260bebec-d70f-4d69-b702-817e12acffbe",
              "BeneficiaryGUID" : "260bebec-d70f-4d69-b702-817e12acffbe"
            }
          } ]
      }
    }
  }
}

Expected Outputs - One per array element

{
  "id" : "c064e784-5d3b-4d78-9451-eae608ad66a9",
  "format" : "AVRO",
  "EventMessage" : {
    "SourceSystem" : {
      "eventOccurredTime" : "10-04-0009",
      "id" : 1,
      "name" : "FAST",
      "sourceCorrelationId" : "bac5de63-ba46-4aaf-a508-101f41a22d1e"
    },
    "Domain" : {
      "id" : 1,
      "name" : "Policy",
      "type" : "Raw"
    },
    "EventAttributes" : {
      "id" : "522533ce-fe82-4e73-bfbc-52cacb2c7903",
      "msgType" : "Raw",
      "correlationId" : "bac5de63-ba46-4aaf-a508-101f41a22d1e",
      "eventCreatedTime" : "10-06-0016",
      "eventName" : "PolicyFinancialTrx",
      "parentEventName" : "FastFinancialTrx",
      "parentEventOccurredTime" : "10-04-0009",
      "DataElements" : {
        "TrxHdrDisburse" : {
          "TaxDtl" : {
            "ID" : "d42f00f2-2f70-4fed-bcb3-035b2d1febbd",
            "DisbHdrID" : "f3b1ee06-fe01-470d-ae59-e1372788d11b",
            "DeductionType" : "FederalTax",
            "TaxDedPct" : "0",
            "TaxDedAmt" : "0",
            "TaxableGain" : "0",
            "WithholdAmt" : "0",
            "TaxDtlParty" : {
              "TaxDtlID" : "d42f00f2-2f70-4fed-bcb3-035b2d1febbd",
              "PartyID" : "260bebec-d70f-4d69-b702-817e12acffbe",
              "CustomerProfileGUID" : "260bebec-d70f-4d69-b702-817e12acffbe",
              "BeneficiaryGUID" : "260bebec-d70f-4d69-b702-817e12acffbe"
            }
          }
        }
      }
    }
  }
}

AND

{
  "id" : "c064e784-5d3b-4d78-9451-eae608ad66a9",
  "format" : "AVRO",
  "EventMessage" : {
    "SourceSystem" : {
      "eventOccurredTime" : "10-04-0009",
      "id" : 1,
      "name" : "FAST",
      "sourceCorrelationId" : "bac5de63-ba46-4aaf-a508-101f41a22d1e"
    },
    "Domain" : {
      "id" : 1,
      "name" : "Policy",
      "type" : "Raw"
    },
    "EventAttributes" : {
      "id" : "522533ce-fe82-4e73-bfbc-52cacb2c7903",
      "msgType" : "Raw",
      "correlationId" : "bac5de63-ba46-4aaf-a508-101f41a22d1e",
      "eventCreatedTime" : "10-06-0016",
      "eventName" : "PolicyFinancialTrx",
      "parentEventName" : "FastFinancialTrx",
      "parentEventOccurredTime" : "10-04-0009",
      "DataElements" : {
        "TrxHdrDisburse" : {
          "TaxDtl" :  {
            "ID" : "eb333766-51fa-4b0c-99a8-a30c0379d454",
            "DisbHdrID" : "f3b1ee06-fe01-470d-ae59-e1372788d11b",
            "DeductionType" : "StateTax",
            "TaxDedPct" : "0",
            "TaxDedAmt" : "0",
            "TaxableGain" : "0",
            "WithholdAmt" : "0",
            "TaxDtlParty" : {
              "TaxDtlID" : "eb333766-51fa-4b0c-99a8-a30c0379d454",
              "PartyID" : "260bebec-d70f-4d69-b702-817e12acffbe",
              "CustomerProfileGUID" : "260bebec-d70f-4d69-b702-817e12acffbe",
              "BeneficiaryGUID" : "260bebec-d70f-4d69-b702-817e12acffbe"
            }
          } 
        }
      }
    }
  }
}

Thanks

Re: How to separate json array elements in separate json outputs?

Anonymous — Tue, 24 Dec 2019 04:32:38 GMT

Hi
It is easy to extract all elements from the array, and then iterate each record and create a separate json file for each record.

Regards
Shong

Re: How to separate json array elements in separate json outputs?

adbdkb — Tue, 24 Dec 2019 11:34:03 GMT

Hi Shong,

I am very new to Talend. Can you please give an example I could use to learn from? An example would help in learning the flow I should have.

Thanks

Re: How to separate json array elements in separate json outputs?

Anonymous — Fri, 27 Dec 2019 03:29:35 GMT

The job design looks like:
tFileInputJson--tLogRow--main--tFlowToIterate--iterate--tIterateToFlow-main-tWriteJsonField--main--tFileOutputDelimited.

Please learn these components and test each step. Let me know if you have any questions.

Regards
Shong

topic Re: How to separate json array elements in separate json outputs? in Talend Studio

How to separate json array elements in separate json outputs?

Re: How to separate json array elements in separate json outputs?

Re: How to separate json array elements in separate json outputs?

Re: How to separate json array elements in separate json outputs?