topic Re: Salesforce loading related objects with single output in Talend Studio

Salesforce loading related objects with single output

mcomstobe — Sat, 16 Nov 2024 07:48:02 GMT

I have talend job that works as designed, except that I need to create a parent-child relationship between the two custom objects and to do so i need the first custom object to load before the second custom object.

I have read a bunch of documentation, regarding the tFixedInputFlow, but because I need the output of the tmap for both tsalesforceoutput components I don't believe this will work. I have also tried to use tSleep to pause the second/related custom object, so the first custom object can load completely and this did not work either.

What am I missing?

I have attached a simple representation of the real job, and all I want to do is load the patient output completely, before the patient_encounter output.

Your help is greatly appreciated.

Re: Salesforce loading related objects with single output

mcomstobe — Sun, 19 Aug 2018 15:37:08 GMT

It's always the last thing you try that works.

I got it working by using the tSleep and most importantly I unchecked the "Extend Insert" on the Patient tsalesforceoutput. I believe by unchecking the "Extend Insert" each patient record is committed one by one, thus allowing the Patient_Encounter to find the related patient record on insert.

Re: Salesforce loading related objects with single output

TRF — Sun, 19 Aug 2018 16:30:45 GMT

This is really not a good design option.
Consider to have 2 separate subjobs, 1 for each object and complete the parent object injection before to go with child object. Both subjobs must be linked by an onSubjobOk link.

Re: Salesforce loading related objects with single output

mcomstobe — Sun, 19 Aug 2018 17:15:13 GMT

Thanks for the reply.

But how do I utilize the data coming out of the tmap into the two subjobs?

Re: Salesforce loading related objects with single output

TRF — Sun, 19 Aug 2018 18:12:45 GMT

For the parent object, you can let as it is.
For the child object, store the records in memory using a tHashOutput.
On next subjob, get the memorized records using a tHashInput linked with the previous tHashOutput.
If you don't find these components in the palette, you need to add them manually (see related posts about this subject).

Re: Salesforce loading related objects with single output

mcomstobe — Sun, 19 Aug 2018 18:33:42 GMT

One clarifications. The tHashout component only has "On Component Ok" trigger, I assume I use that correct? The tHashInput has both "On Component Ok" and "On Subjob Ok". So I use the "On Subjob Ok" trigger to connect to child object flow, correct?

Thanks for your help on this.

Re: Salesforce loading related objects with single output

TRF — Sun, 19 Aug 2018 18:43:31 GMT

Not really.
Connect your tMSSQLInput to the tHashInput using the onSubjobOk trigger.
This the 1rst subjob will inject records into the parent object and store records for the child object into the tHashOutput.
Then, the 2nd subjob will get child records from the tHashInput and inject them into Salesforce.

Re: Salesforce loading related objects with single output

mcomstobe — Sun, 19 Aug 2018 19:17:11 GMT

I don't think it is exactly what I am after. I am processing a set of database records and splitting the data via a tmap into two outputs. Output 1 is used to create a patient record in a salesforce custom object, and Output 2 is used to create a patient encounter in a salesforce custom object (and relates it to the previously created patient records). When I create the patient encounter I am using the relations lookup and need to make sure all the patient records have all been created before the patient encounter does the lookup.

The challenge is I have to process the source record all the same time, so I am trying to make sure the patients are created before the related patient encounters.

Hope this helps. Thanks again for the help.

Re: Salesforce loading related objects with single output

TRF — Sun, 19 Aug 2018 20:27:53 GMT

This is a very common use case and the solution I proposed is exactly what you need to do in such a case.
Imagine you have 1,000,000 records to inject for parent object, then to child object. Do you think you can insert records 1 at a time?
No, you will use bulk components and you will cut the job in 2 subjob, exactly as I proposed.
Believe me, I did it for hundreds of jobs.

Re: Salesforce loading related objects with single output

mcomstobe — Sun, 19 Aug 2018 20:40:52 GMT

Ok, you have me sold.

I'm still confused on your suggestion, if I need to trigger the subjobs using the two outputs from the tMap. Your suggestions was the single output of the TMssqlInput, this won't work in my scenario (or I am missing something). I have attached an image of the flow. Where would i insert the thashout and thashinput to get the Add_EhrEncounters to run second?

Capture_subjobflow.PNG

Re: Salesforce loading related objects with single output

TRF — Sun, 19 Aug 2018 21:30:51 GMT

Remove the tSleep and replace the Add_EhrEncounters by a tHashOutput.
Add a tHashInput connected to the tMSSQLInput by an onSubjobOk trigger.
Connect the Add_EhrEncounters to this tHashInput. Don't forget to refer the tHashOutput from the tHashInput and to to set the schema.