Do not input private or sensitive data. View Qlik Privacy & Cookie Policy.
Skip to main content

Announcements
Qlik Open Lakehouse is Now Generally Available! Discover the key highlights and partner resources here.
cancel
Showing results for 
Search instead for 
Did you mean: 
Anonymous
Not applicable

Need batch wise data pull from source (tFileInputDelimited) in TOS 6.0

My requirement is I need to fetch every 1000 records from CSV file/ tNetsuite ERP and move to elastic search as batches in specific interval times. I didn't find any component to solve my requirement. In TOS 6.0 we have a connector tMySqlInput. In this connector in advance settings we have "Enable Stream" with check box. If we select the checkbox, we can utilize the batch wise data to output file or tMysqlOutput. 
1. How can I see the code for tMysqlInput?
2. Is there any other component that will solve my requirement? If yes, please share the link.
3. Is there a way to design the Custom component?
Please let me know if need more information.

Labels (2)
4 Replies
Anonymous
Not applicable
Author

Hi 
1. Click on the Code tab aside the Designer, you will see the java code of job.
2. If the data is stored in a CSV file, you can use tFileInputDelimited, I don't understand you use a tMysqlInput component, can you please explain your requirement a bit?
3. Yes, you can develop custom components, refer to the following article:
https://help.talend.com/pages/viewpage.action?pageId=189661214

Best regards
Shong
Anonymous
Not applicable
Author

Thanks a lot Shong. I got solution from your reply on custom component.
My requirement is I need to fetch only 10,000 records per day from Netsuite ERP using tNetsuiteInput connector and store it in Elasticsearch using tElasticsearchIndex component in TOS 6.0 as Json format. How can I do that, please suggest.
Anonymous
Not applicable
Author

Sorry,  I am not family with Elasticsearch, the tElasticsearchIndex component is not documented yet, so I can't give you some suggestion right now.;(
Anonymous
Not applicable
Author

For TFileInputDelimited we have limit option in advance settings. we can provide the input as 1000 or required number of records to fetch. It got resolved