[resolved] load data from amazon redshift to mysql

[Anonymous]

Hi all,
I can load 1 million data from tfileinputdelimited to tmysqloutput and also I can load data from tredshiftinput to tfileoutputdelimited via tmap transformation in talend.
But, I cannot able to load data from tredshiftinput to tmysqloutput via tmap in Talend. I am getting error as No operations allowed after statement closed.
I do not know why I am getting this error.
I am using windows 8.1 64 bit OS.
Mysql -Heidisql
Please tell me how to load data??
Please give me a solution.

[Anonymous]

Great !!!
- First step of optimization you can try is to read all the data from the source and put into the flat file
- Read flat file and then insert data into the Tartget database
- Experiment with the batch size parameter
- There is also a concept of bulk load from flat file check with the bulk or ELT component, this will give you much improved results.
What was the problem earlier?
>> Can you show the properties of your earlier output component? there must be some connection issue...
Keep posting your performance improvement progress.. this will help others as well..
Vaibhav

[Anonymous]

Hi,
Can you please tell me more about batch size parameter and bulk load??
Regards,
Vinuta

[Anonymous]

- Batch size is number of rows sent to database for commit... this could be 1k,10k etc
- Bulk load is loading data from flat file directly into the database, this is usually a single component job, where you provide metadata, file path and db details.. this is much faster as compared to other methods.
Vaibhav

[Anonymous]

Hi,
Thank you so much. 
That´s true. If we load data from CSV file to database it takes very less time to load.
Batch size is commit size right which you are explaining previously that I can set those values in output db components in Advanced settings´???
Vinuta

[Anonymous]

You are correct.
Vaibhav

[Anonymous]

Hi,
I loaded data from tRedshiftinput to tmysqloutput using tprejob component. The data loaded successfully. But, when I checked the data in MySql database(Heidisql) has only 999700 rows instead of 1 million rows. totallz 3 rows are missing.
I do not what is the problem??I have not got any errors.
can you please tell me why the rows are missing??
The job design:

Regards,
Vinuta

[Anonymous]

Hi Vinuta,
Those records might be rejected from database side... connect reject link to tlogrow/flat file from database output component, you will get the reason for rejection...
Vaibhav

[Anonymous]

Hi, 
I try to connect tfileoutputdelimited component from tmysqloutput component. But, It showing main connection not reject link.
talend version:5.6
What to do??
Regards,
Vinuta

[Anonymous]

Hi,
Sorry for bothering you.
I have got the reject link.
I checked out extend insert in advanced settings. then I got it.
Regards,
Vinuta

[Anonymous]

Ok... Good.
So whether the problem is solved or not?
Vaibhav