topic Re: tfixedflowinput and trunjob in Talend Studio

tfixedflowinput and trunjob

Anonymous — Sat, 16 Nov 2024 12:25:14 GMT

Hi,
I am trying to create job with a child job as follows:
Parent job
======
tDBInput1 (read a table)
- iterate (to each of the following)
- tDBInput2 (perform a count based on each record from tDBInput1 and output count)
- tDBInput3 (if tDBInput2 count = 0, perform a different count based on each record from tDBInput1 and output count)
if tDBInput2 count > 0 then fetch records from table1 -> tRunJob (child job)
if tDBInput3 count > 0 then fetch records from table2 -> tRunJob (child job)
the schema of table1 and table2 fetch is the same
records from table1/table2 is then fed into the child job which performs some aggregation
Child job
=====
tFixedFlowInput -> tAggregateRow -> tLogRow
To achieve the above, I created a child job with tFixedFlowInput which takes in the context parameter
The parent job then has tRunJob which passes the rows from tDBInput2/tDBInput3 to tRunJob.
The context parameter is string/double/long. Perhaps object will work but I am not sure how to set it up.
My job seems to work but then I realised that the child job is executed for each row which gave me the wrong answer. The aggregation in child job needs to take in entire records from tDBInput2/tDBInput3, otherwise the aggregation is incorrect.
My question is:
Is there a way to configure/pass into the child job all the records before the child job runs?
Is there an example to show how to setup object context (at the moment, I have to create context parameters for each of the field)?
Thank in advance.

Re: tfixedflowinput and trunjob

Anonymous — Mon, 30 Jan 2012 16:14:25 GMT

Hi,
I don't know if i have rightly understood your problem but here is what I suggest. Instead of using a subjob to do your agregation, you may be shall use the tjavarow component with something like this. output_row.newColumn+=input_row.newColumn;
Could you give a screenshot of your job?

Re: tfixedflowinput and trunjob

Anonymous — Mon, 30 Jan 2012 23:38:48 GMT

My simplified example is attached. I used tFlowToIterate_1 and tFileInputDelimited_3 to emulate the iterate that I have on database. Note: Table1 and Table2 have the same out schema but the real table has different schemas.
tFileInputDelimited_1 input is
ID,Type
A1,T1
A1,T2
Table1 input is
ID,Number,Data
A1,1,10
A1,2,20
A1,3,30
A1,4,40
A1,5,50
Table2 input is
ID,Number,Data
A1,1,11
A1,2,21
A1,3,31
A1,4,41
A1,5,51
Based on tFileInputDelimited_1, if Type = 'T1', then I load data from Table1, otherwise load data from Table2.
The results is then fed into the JobChild.
The results that I am expecting are:
- if Type=T1, the tLogRow_1 (JobChild) output to be A1,150
- if Type=T2, the tLogRow_1 (JobChild) output to be A1,155
However, if I run, I am getting a results for each row, hence
connecting to socket on port 4001
connected
A1|10.0
A1|20.0
A1|30.0
A1|40.0
A1|50.0
A1|11.0
A1|21.0
A1|31.0
A1|41.0
A1|51.0