topic Re: load csv file data into mysql output table( dynamic table name) in Talend Studio

load csv file data into mysql output table( dynamic table name)

bhagyarekha — Thu, 03 Nov 2016 04:17:27 GMT

Hi TalendTeam,

Hi i have 61 files each having 80,000 (depends quite changing on daily basies) records and schema of all these files are same.
so i wanted to keep all these files in 61 different tables instead into one as per my company requirments.
so i have given dynamic table name in mysql output component taken from filelist as
((String)globalMap.get("tFileList_1_CURRENT_FILE"))
but here the problem is when am running with out batch mode and kept line limit as 10000 on run tab,all the records are inserting and comitted

but when am trying to increase the " Number of rows to insert" to 100000 then none of the records are inserting in to the table which is having less number of records.

and tried in auto commit mode also but no result

Appreciate for your quick response

Regards,
Rekha

Re: load csv file data into mysql output table( dynamic table name)

Anonymous — Thu, 03 Nov 2016 05:21:23 GMT

Hello, can you please share your job design once?

Re: load csv file data into mysql output table( dynamic table name)

bhagyarekha — Thu, 03 Nov 2016 06:44:07 GMT

Hi sankalp,
please find the below screen shots

Re: load csv file data into mysql output table( dynamic table name)

avinashbasetty — Thu, 03 Nov 2016 12:18:55 GMT

Hi Bhagya,
You could not get reject option when you check the Extend Input in advance settings.
Try to check it and provide the commit size.

Regards,
Avinash.B
8885453419

Re: load csv file data into mysql output table( dynamic table name)

Anonymous — Thu, 03 Nov 2016 13:05:06 GMT

can you please share screenshot of tMYSQL_Output5 component also?

Re: load csv file data into mysql output table( dynamic table name)

avinashbasetty — Thu, 03 Nov 2016 13:08:27 GMT

Hi Bhagya,
I am able to run it, Also i checked the Enable Parallel Execution option.

Regards,
Avinash.B
8885453419

Re: load csv file data into mysql output table( dynamic table name)

Anonymous — Thu, 03 Nov 2016 13:22:36 GMT

This should not be the case. I have done it in past where batch size was more than the number of records coming from source. There is something else that is missing here.

Re: load csv file data into mysql output table( dynamic table name)

talendtester — Thu, 03 Nov 2016 16:43:35 GMT

I think the size of the data which is trying to be committed at a single time is too much at 100K.
Trying putting the insert size to 20K and increasing by 10K till you find where it fails.