I am using the TOS big data 6.4 i have a job that contains the an excel sheet which contains approximately 10000 rows. i want to create a job where all my 10000 rows need to establish 10000 connections with the data base and want to insert into database parallel. That means i want to create 10000 connections how should i do this with the talend open studio...please help me...
From what you have said, it sounds like you want to do this on a single a machine and not in a Hadoop node cluster. If this is the case you need to think about what you are asking. Parallel processing is great, but it depends on how many cores you have available. If your machine has 10,000 cores, then you are in luck. Otherwise, you will probably end up massively slowing the processing of your data. When running parallel processing you should start by using the rule n-1 processes where n is the number of cores.
Here i want to check the data base capability for that i need to create approximately 10000 connections through talend from a single machine. i have a excel sheet having 10000 rows i want to insert all the rows parallel by creating that many number of connections to check my data base is stable or not. this is my requirement.
You are not going to be able to do this on one machine. As I said in my last post, a rule of thumb for parallel processing is number of cores-1 = number of parallel processes. Unless you have 10000 cores, you won't be able to achieve this on one machine
