Talend duplicating records

Anonymous — Thu, 12 Apr 2018 14:20:34 GMT

Hi guys.

My talend job is pulling a file in and breaking down the data into records, however on the input_row the data is coming in fine but the output_row is duplicating the data.

the following is my tjavarow code:

int recordcount = ((int)globalMap.get("readrecords"));

int validcounter = ((int)globalMap.get("validcounter"));
int invalidcounter = ((int)globalMap.get("invalidcounter"));

System.out.println(input_row.toString());
if (input_row.Broken_Name_Flag==1) { // 1 = the row is fine

globalMap.put("validcounter", validcounter + 1);

output_row.Source = input_row.Source;
output_row.UCN = input_row.UCN;
output_row.Broken_Name_Flag = input_row.Broken_Name_Flag;
output_row.Customer_Title = input_row.Customer_Title;
output_row.Customer_Name_s = input_row.Customer_Name_s;
output_row.Customer_Middle_Names = input_row.Customer_Middle_Names;
output_row.Customer_Surname = input_row.Customer_Surname;
output_row.Customer_Full_Name = input_row.Customer_Full_Name;
output_row.Gender = input_row.Gender;
output_row.ID_Number = input_row.ID_Number;
output_row.Date_of_Birth = input_row.Date_of_Birth;
output_row.Citizenship = input_row.Citizenship;
output_row.Nationality = input_row.Nationality;
output_row.Country_of_Residence = input_row.Country_of_Residence;
output_row.Address_Line_1 = input_row.Address_Line_1;

}
else {
globalMap.put("invalidcounter", invalidcounter + 1);
}

output_row.validcount= validcounter + 1;
System.out.println(output_row.toString());
output_row.invalidcount= invalidcounter + 1;

the following is my output:

Starting job RiskRatingFileUpload at 15:17 12/04/2018.

[statistics] connecting to socket on port 3677
[statistics] connected
[trace] connecting to socket on port 4614
[trace] connected
riskratingfileupload.riskratingfileupload_0_1.RiskRatingFileUpload$row1Struct@effeb7e5[Source=Ashburton Principal Funds,UCN=,Broken_Name_Flag=1,Customer_Title=Dr,Customer_Name_s=Martin,Customer_Middle_Names=Johannes Ulrich Thomas,Customer_Surname=van Wijngaarden,Customer_Full_Name=,Gender=M,ID_Number=,Date_of_Birth=Thu Dec 01 00:00:00 CAT 1960,Citizenship=South Africa,Nationality=South Africa,Country_of_Residence=South Africa,Address_Line_1="5 Grieg Street, Vanderbijlpark, 1911"]
riskratingfileupload.riskratingfileupload_0_1.RiskRatingFileUpload$row3Struct@effeb7e5[Source=Ashburton Principal Funds,UCN=,Broken_Name_Flag=1,Customer_Title=Dr,Customer_Name_s=Martin,Customer_Middle_Names=Johannes Ulrich Thomas,Customer_Surname=van Wijngaarden,Customer_Full_Name=,Gender=M,ID_Number=,Date_of_Birth=Thu Dec 01 00:00:00 CAT 1960,Citizenship=South Africa,Nationality=South Africa,Country_of_Residence=South Africa,Address_Line_1="5 Grieg Street, Vanderbijlpark, 1911",validcount=1,invalidcount=null]
Load jars loaded successfully!
Connected to TIBCO !
session not null !
riskratingfileupload.riskratingfileupload_0_1.RiskRatingFileUpload$row1Struct@effeb7e5[Source=Ashburton Principal Funds,UCN=,Broken_Name_Flag=0,Customer_Title=Dr,Customer_Name_s=Adriaan,Customer_Middle_Names=,Customer_Surname=,Customer_Full_Name=,Gender=M,ID_Number=,Date_of_Birth=Thu Mar 02 00:00:00 CAT 1961,Citizenship=South Africa,Nationality=South Africa,Country_of_Residence=South Africa,Address_Line_1="Plot 56, 121 Swallow Street, Doringpoort, 0159"]
riskratingfileupload.riskratingfileupload_0_1.RiskRatingFileUpload$row3Struct@effeb7e5[Source=Ashburton Principal Funds,UCN=,Broken_Name_Flag=1,Customer_Title=Dr,Customer_Name_s=Martin,Customer_Middle_Names=Johannes Ulrich Thomas,Customer_Surname=van Wijngaarden,Customer_Full_Name=,Gender=M,ID_Number=,Date_of_Birth=Thu Dec 01 00:00:00 CAT 1960,Citizenship=South Africa,Nationality=South Africa,Country_of_Residence=South Africa,Address_Line_1="5 Grieg Street, Vanderbijlpark, 1911",validcount=1,invalidcount=1]
Load jars loaded successfully!
Connected to TIBCO !
session not null !
riskratingfileupload.riskratingfileupload_0_1.RiskRatingFileUpload$row1Struct@effeb7e5[Source=Ashburton Principal Funds,UCN=,Broken_Name_Flag=0,Customer_Title=Mr,Customer_Name_s=Craig,Customer_Middle_Names=Warrick,Customer_Surname=Lyall,Customer_Full_Name=,Gender=M,ID_Number=,Date_of_Birth=Thu Jan 03 00:00:00 CAT 1985,Citizenship=South Africa,Nationality=South Africa,Country_of_Residence=South Africa,Address_Line_1="Unit 10, 44 ON Bantry, 3 Antrim Road| Bryanston, 2191"]
riskratingfileupload.riskratingfileupload_0_1.RiskRatingFileUpload$row3Struct@effeb7e5[Source=Ashburton Principal Funds,UCN=,Broken_Name_Flag=1,Customer_Title=Dr,Customer_Name_s=Martin,Customer_Middle_Names=Johannes Ulrich Thomas,Customer_Surname=van Wijngaarden,Customer_Full_Name=,Gender=M,ID_Number=,Date_of_Birth=Thu Dec 01 00:00:00 CAT 1960,Citizenship=South Africa,Nationality=South Africa,Country_of_Residence=South Africa,Address_Line_1="5 Grieg Street, Vanderbijlpark, 1911",validcount=1,invalidcount=1]
Load jars loaded successfully!
Connected to TIBCO !
session not null !
riskratingfileupload.riskratingfileupload_0_1.RiskRatingFileUpload$row1Struct@effeb7e5[Source=Ashburton Principal Funds,UCN=,Broken_Name_Flag=1,Customer_Title=Mr,Customer_Name_s=Ashay,Customer_Middle_Names=,Customer_Surname=Deochand,Customer_Full_Name=,Gender=M,ID_Number=,Date_of_Birth=Tue May 03 00:00:00 CAT 1983,Citizenship=South Africa,Nationality=South Africa,Country_of_Residence=South Africa,Address_Line_1="Unit 1 Umuzi, 136 Daisy Street, Sandown, 2196"]
riskratingfileupload.riskratingfileupload_0_1.RiskRatingFileUpload$row3Struct@effeb7e5[Source=Ashburton Principal Funds,UCN=,Broken_Name_Flag=1,Customer_Title=Mr,Customer_Name_s=Ashay,Customer_Middle_Names=,Customer_Surname=Deochand,Customer_Full_Name=,Gender=M,ID_Number=,Date_of_Birth=Tue May 03 00:00:00 CAT 1983,Citizenship=South Africa,Nationality=South Africa,Country_of_Residence=South Africa,Address_Line_1="Unit 1 Umuzi, 136 Daisy Street, Sandown, 2196",validcount=1,invalidcount=1]
Load jars loaded successfully!
Connected to TIBCO !
session not null !
riskratingfileupload.riskratingfileupload_0_1.RiskRatingFileUpload$row1Struct@effeb7e5[Source=Ashburton Principal Funds,UCN=,Broken_Name_Flag=1,Customer_Title=Mr,Customer_Name_s=Zeyn,Customer_Middle_Names=,Customer_Surname=Mahomed,Customer_Full_Name=,Gender=M,ID_Number=,Date_of_Birth=Sun Jan 08 00:00:00 CAT 1984,Citizenship=South Africa,Nationality=South Africa,Country_of_Residence=South Africa,Address_Line_1="26A Central Street, Houghton Estate"]
riskratingfileupload.riskratingfileupload_0_1.RiskRatingFileUpload$row3Struct@effeb7e5[Source=Ashburton Principal Funds,UCN=,Broken_Name_Flag=1,Customer_Title=Mr,Customer_Name_s=Zeyn,Customer_Middle_Names=,Customer_Surname=Mahomed,Customer_Full_Name=,Gender=M,ID_Number=,Date_of_Birth=Sun Jan 08 00:00:00 CAT 1984,Citizenship=South Africa,Nationality=South Africa,Country_of_Residence=South Africa,Address_Line_1="26A Central Street, Houghton Estate",validcount=1,invalidcount=1]
Load jars loaded successfully!
Connected to TIBCO !
session not null !
[statistics] disconnected
[trace] disconnected
Job RiskRatingFileUpload ended at 15:17 12/04/2018. [exit code=0].

any help will be appreciated

Re: Talend duplicating records

vboppudi — Fri, 13 Apr 2018 17:11:26 GMT

If possible please provide screen shot of job.

Regards,

Re: Talend duplicating records

Anonymous — Mon, 16 Apr 2018 07:52:19 GMT

As requested

Talend.PNG

topic Re: Talend duplicating records in Talend Studio

Talend duplicating records

Re: Talend duplicating records

Re: Talend duplicating records