topic Re: Need batch wise data pull from source (tFileInputDelimited) in TOS 6.0 in Talend Studio

Need batch wise data pull from source (tFileInputDelimited) in TOS 6.0

Anonymous — Tue, 06 Oct 2015 05:57:57 GMT

My requirement is I need to fetch every 1000 records from CSV file/ tNetsuite ERP and move to elastic search as batches in specific interval times. I didn't find any component to solve my requirement. In TOS 6.0 we have a connector tMySqlInput. In this connector in advance settings we have "Enable Stream" with check box. If we select the checkbox, we can utilize the batch wise data to output file or tMysqlOutput.
1. How can I see the code for tMysqlInput?
2. Is there any other component that will solve my requirement? If yes, please share the link.
3. Is there a way to design the Custom component?
Please let me know if need more information.

Re: Need batch wise data pull from source (tFileInputDelimited) in TOS 6.0

Anonymous — Fri, 09 Oct 2015 02:44:43 GMT

Hi
1. Click on the Code tab aside the Designer, you will see the java code of job.
2. If the data is stored in a CSV file, you can use tFileInputDelimited, I don't understand you use a tMysqlInput component, can you please explain your requirement a bit?
3. Yes, you can develop custom components, refer to the following article:
https://help.talend.com/pages/viewpage.action?pageId=189661214

Best regards
Shong

Re: Need batch wise data pull from source (tFileInputDelimited) in TOS 6.0

Anonymous — Sat, 10 Oct 2015 04:39:50 GMT

Thanks a lot Shong. I got solution from your reply on custom component.
My requirement is I need to fetch only 10,000 records per day from Netsuite ERP using tNetsuiteInput connector and store it in Elasticsearch using tElasticsearchIndex component in TOS 6.0 as Json format. How can I do that, please suggest.

Re: Need batch wise data pull from source (tFileInputDelimited) in TOS 6.0

Anonymous — Sat, 10 Oct 2015 09:44:02 GMT

Sorry, I am not family with Elasticsearch, the tElasticsearchIndex component is not documented yet, so I can't give you some suggestion right now.;(

Re: Need batch wise data pull from source (tFileInputDelimited) in TOS 6.0

Anonymous — Wed, 14 Oct 2015 06:00:14 GMT

For TFileInputDelimited we have limit option in advance settings. we can provide the input as 1000 or required number of records to fetch. It got resolved