Hi Guys,
I was wondering about toracleinput component query fetching mechanism. I was fetching data from a table having around 20 million records and writing those to a flat file. The process moves slowly on the designer showing 2000 records/sec are written to the file . Whether the toracleinput fetches records from DB as record by record or it fetches a complete set of data returned from the query at once and stores it into any cache or temporary memory . This is of a concern as if fetching is done record by record then we cannot control the data from the such a huge table because somebody might parallely update those records in the table from another session which have not been written in the file till now creating data discrepancy.So if anybody has clarification on this suggest me .Please reply asap.
Thanks,
hi,
check cursor option to able fetch array size in the JDBC driver.
It should optmize performance.
cf API JDBC The following two hints give the driver suggestions for improving performance: 1. The number of rows that should be fetched from the database each time new rows are needed The number of rows to be fetched is called the fetch size, and it can be set by two different methods: Statement.setFetchSize and ResultSet.setFetchSize. The statement that creates a ResultSet object sets the default fetch size for that ResultSet object, using the Statement method setFetchSize. The following code fragment sets the fetch size for the ResultSet object rs to 25. Until the fetch size is changed, any result set created by the Statement object stmt will automatically have a fetch size of 25. Statement stmt = con.createStatement(); stmt.setFetchSize(25); ResultSet rs = stmt.executeQuery(SELECT * FROM EMPLOYEES);
show code using by Talend when you check this option :
regards
laurent
Hi,
Actually my concern was to know the query fetching mechanism of toracleinput for a different reason. So please if anybody has details , please provide them.
Thanks,
