https://community.qlik.com/t5/Talend-Studio/Spark-parallel-threading/m-p/2278519#M53974 <P>Hello,</P> <P>Could you please indicate which bigdata product version you're using?</P> <P>Best regards</P> <P>Sabrina</P> Fri, 30 Nov 2018 08:59:30 GMT Anonymous 2018-11-30T08:59:30Z https://community.qlik.com/t5/Talend-Studio/Spark-parallel-threading/m-p/2278518#M53973 <P>Hi All</P> <P>&nbsp;</P> <P>We have a standard job calling 60 Spark jobs using tRunJob component. we are not really able to launch 60 threads on EMR spark cluster. when we launch jobs we see that all 60 threads create 60 processes but shortly process count drops to 17 and only those 17 threads(tRunJObs) are passed to Spark EMR from job server. does anyone able to run 50+ Big data jobs from job server successfully? if yes kindly share some details.</P> <P>&nbsp;</P> <P>Below is our configuration.</P> <P>JObServer - 8CPU/32GB</P> <P>Data Node - 1 instance with 8CPU/32GB</P> <P>Worker Nodes - 4 nodes each with 8CPU/32GB RAM</P> Sat, 16 Nov 2024 07:11:51 GMT https://community.qlik.com/t5/Talend-Studio/Spark-parallel-threading/m-p/2278518#M53973 ankushd 2024-11-16T07:11:51Z https://community.qlik.com/t5/Talend-Studio/Spark-parallel-threading/m-p/2278519#M53974 <P>Hello,</P> <P>Could you please indicate which bigdata product version you're using?</P> <P>Best regards</P> <P>Sabrina</P> Fri, 30 Nov 2018 08:59:30 GMT https://community.qlik.com/t5/Talend-Studio/Spark-parallel-threading/m-p/2278519#M53974 Anonymous 2018-11-30T08:59:30Z